Boston Chapter of IEEE Computer Society, GBC/ACM and DAMA

7:00 PM, Tuesday, October 25, 2016

Broad Institute Auditorium (MIT building NE-30)

Database Decay and What To Do About It

Michael Stonebraker

M.I.T. Computer Science and Artificial Intelligence Laboratory

The traditional wisdom for designing database schemas is to use a design tool (typically based on a UML or ER model) to construct an initial data model for one's data and its instantiation as a collection of relational tables. Then applications are coded against this relational schema. When business circumstances change (as they do frequently) one should run the tool to produce a new data model and a new collection of tables. The new schema is populated from the old schema, and the applications are altered to work on the new schema, using relational views whenever possible to ease the migration. In this way, the database remains in 3rd normal form, which represents a "good" schema, as defined by DBMS researchers.

In a survey of 20 DBAs at three large companies in the Boston area, we found that this traditional wisdom is rarely-to-never followed for large, multi-department applications. Instead DBAs try very hard not to change the schema when business conditions change, preferring to "make things work" without schema changes. If they must change the schema, they work directly from the relational tables in place. Using these tactics, the ER or UML model (if it ever existed) diverges quickly from reality. Moreover, over time, the actual semantics of the data tend to drift farther and farther from a 3rd normal form data model.

We term this divergence of reality from 3rd normal form principles database decay. This talk explains why database decay occurs in large applications and presents a collection of ideas on how to fight it. These include defensive schemas, defensive applications, and a non-traditional model for application development.

Dr. Stonebraker has been a pioneer of data base research and technology for more than forty years. He was the main architect of the INGRES relational DBMS, and the object-relational DBMS, POSTGRES. These prototypes were developed at the University of California at Berkeley where Stonebraker was a Professor of Computer Science for twenty five years. More recently at M.I.T. he was a co-architect of the Aurora/Borealis stream processing engine, the C-Store column-oriented DBMS, the H-Store transaction processing engine, the SciDB array DBMS, and the Data Tamer data curation system. Presently he serves as Chief Technology Officer of Paradigm4 and Tamr, Inc.

Professor Stonebraker was awarded the ACM System Software Award in 1992 for his work on INGRES. Additionally, he was awarded the first annual SIGMOD Innovation award in 1994, and was elected to the National Academy of Engineering in 1997. He was awarded the IEEE John Von Neumann award in 2005 and the 2014 Turing Award, and is presently an Adjunct Professor of Computer Science at M.I.T, where he is co-director of the Intel Science and Technology Center focused on big data.

See https://amturing.acm.org/award_winners/stonebraker_1172121.cfm for more biographical details.

This joint meeting of the Boston Chapter of the IEEE Computer Society, GBC/ACM and DAMA will be held in the Broad Institute Auditorium (MIT building NE-30). The Broad Institute is on Main St between Vassar and Ames streets. You can see it on a map at this location. The auditorium is on the ground floor near the entrance.

Up-to-date information about this and other talks is available online at https://ewh.ieee.org/r1/boston/computer/. You can sign up to receive updated status information about this talk and informational emails about future talks at https://mailman.mit.edu/mailman/listinfo/ieee-cs, our self-administered mailing list.

For more information contact Peter Mager (p.mager at computer.org)

Updated: Sept 22, 2016.