Track your data across the fourth dimension
1 like•628 views
The document discusses tracking temporal data across time dimensions in databases. It describes how databases are good at capturing data at a single point in time but not across time. The document then covers the SQL:2011 standard for representing valid time, transaction time and bi-temporal data through the use of time periods and timestamps. Examples are provided of how to model temporal data and write queries involving time in a SQL database.
Track your data across the fourth dimension
- 1. Text Tracking your Data Across the Fourth Dimension Wellington Waterloo Web Makers Meetup
- 2. “In physics, spacetime is any mathematical model that combines space and time into a single interwoven continuum.” –Wikipedia
- 3. Databases are Good at ‘Now’ CRUD Create data Read data Update data Delete data
- 4. Bread and Butter Queries How many people work in department X? What’s the total amount paid in commissions by department? What was the top selling product in the last year?
- 5. The Fourth Dimension… Compare reporting relationships one month ago, today and in one months time Show me all changes made to this data over time As of one month ago, what did I believe to be the truth about this data yesterday and what did I actually believe was the truth yesterday?
- 6. “A temporal database is a database with built-in support for handling data involving time…” –Wikipedia ”
- 7. A Little History… Need for temporal support identified as early as 1992 Initial attempts to get support for this into the SQL standard were rejected Finally made it into the SQL:2011 standard
- 8. Some Sample Data Owner Property Jeremy ‘Some place’
- 9. What forms of temporal data are usually stored?
- 10. Temporal Aspects Decision Time Valid Time Transaction Time A table that implements one of these is temporal, two is bi-temporal, more is multi-temporal
- 11. Decision Time Records when a decision was taken Stored as a timestamp You may be doing this already…
- 12. Decision Time Example Owner Property Decision Time Jeremy ‘Some place’ 2012-09-28 Adam ‘Some place’ 2014-02-21
- 13. Valid Time The time during which a fact is true with respect to the real world Modelled as a range between two timestamps Closed at lower bound, can be open at upper bound
- 14. Valid Time Example Owner Property StartVT EndVT Jeremy ‘Some place’ 2012-09-28 2014-02-20 Adam ‘Some place’ 2014-02-21 ∞
- 15. Valid Time Example Owner Property StartVT EndVT Jeremy ‘Some place’ 2012-09-28 2014-03-20 Adam ‘Some place’ 2014-03-21 ∞
- 16. Transaction Time The time period during which a fact stored in the database is considered to be true Modelled as a range between two timestamps Closed at lower bound, can be open at upper bound
- 17. Transaction Time Example Owner Property StartVT EndVT StartTT EndTT Jeremy ‘Some place’ 2012-09-2 8 2014-02-2 0 2012-09-2 8 2014-09-0 9 Adam ‘Some place’ 2014-02-2 1 ∞ 2014-02-2 1 2014-09-0 9 Jeremy ‘Some place’ 2012-09-2 8 2014-03-2 0 2014-09-0 9 ∞ Adam ‘Some place’ 2014-03-2 1 ∞ 2014-09-0 9 ∞
- 18. Valid Time and Transaction Time are Orthogonal Valid Time Transaction Time
- 19. Enough theory! How do I add this stuff to my db schema?
- 20. SQL:2011 Temporal Additions ! PERIOD type for date ranges in table definitions Can query for data in a period using new predicates CONTAINS, OVERLAPS, EQUALS, PRECEDES and IMMEDIATELY SUCCEEDS Can also update or delete data matching a portion of a period using FOR PORTION OF {period_name} FROM {date} TO {date}
- 21. Implementing Valid Time Done by adding columns for start and end with a period constraint on the columns Period must also be included as part of the primary key
- 22. Table Definition using Valid Time CREATE TABLE Emp( ENo INTEGER, StartVT DATE, EndVT DATE, EDept INTEGER, PERIOD FOR EValidTime (StartVT, EndVT), PRIMARY KEY (ENo, EValidTime WITHOUT OVERLAPS), )
- 23. Foreign Key Problems… It’s possible that rows in child tables may reference rows in parent tables that are not currently valid and vice versa Therefore foreign key constraints have to be updated where needed to include period data
- 24. Valid time foreign key definition ALTER TABLE Emp ADD FOREIGN KEY (Edept, PERIOD EValidTime) REFERENCES Dept (DNo, PERIOD DValidTime)
- 25. Querying a Table using Valid Time SELECT Ename, Edept FROM Emp WHERE ENo = 22217 AND EValidTime CONTAINS DATE ‘2011-01-02’; ! SELECT Ename, Edept FROM Emp WHERE ENo = 22217 AND EValidTime OVERLAPS PERIOD (DATE ‘2010-01-01', DATE ‘2011-01-01');
- 26. Implementing Transaction Time RDBMS automatically creates a snapshot when data is updated or deleted Done by adding columns for start, end and a period for them Must also add WITH SYSTEM VERSIONING to create table statement
- 27. Table Definition using Transaction Time CREATE TABLE Emp ENo INTEGER, Sys_start TIMESTAMP(12) GENERATED ALWAYS AS ROW START, Sys_end TIMESTAMP(12) GENERATED ALWAYS AS ROW END, EName VARCHAR(30), PERIOD FOR SYSTEM_TIME (Sys_start, Sys_end) ) WITH SYSTEM VERSIONING
- 28. Querying a Table using Transaction Time SELECT ENo,EName,Sys_Start,Sys_End FROM Emp FOR SYSTEM_TIME AS OF TIMESTAMP '2011-01-02 00:00:00’; ! SELECT ENo,EName,Sys_Start,Sys_End FROM Emp FOR SYSTEM_TIME FROM TIMESTAMP '2011-01-02 00:00:00’TO TIMESTAMP '2011-12-31 00:00:00’;
- 29. Bi-Temporal table definition CREATE TABLE Emp( ENo INTEGER, StartVT DATE, EndVT DATE, EDept INTEGER, PERIOD FOR EValidTime (StartVT, EndVT), Sys_start TIMESTAMP(12) GENERATED ALWAYS AS ROW START, Sys_end TIMESTAMP(12) GENERATED ALWAYS AS ROW END, EName VARCHAR(30), PERIOD FOR SYSTEM_TIME (Sys_start, Sys_end), PRIMARY KEY (ENo, EValidTime WITHOUT OVERLAPS), FOREIGN KEY (Edept, PERIOD EValidTime) REFERENCES Dept (DNo, PERIOD DValidTime) ) WITH SYSTEM VERSIONING
- 30. Who supports SQL:2011 Temporal? IBM DB2. Called “Time Travel Queries”, although different syntax for FOR SYSTEM_TIME AS OF Oracle 12c, in compliance with SQL:2011 Others?
- 31. Further Reading ‘Developing Time Oriented Applications in SQL’ by Richard Snodgrass ‘Temporal Features in SQL:2011’ ‘Time and Relational Theory: Temporal Databases in the Relational Model and SQL'
- 32. Thanks for listening Any questions? Contact me: @JCook21 [email protected] I’d love some feedback!