Date rangestech15
- 1. Date Ranges: Time for Oracle 12c SQL "Ask not what Oracle can do for you, ask what you can do with Oracle." Stew Ashton 2.0 UKOUG Tech 15 Stew ASHTON UKOUG Tech 14
- 2. Agenda • Who am I? • Date/time data types • Date range concepts • 12c Temporal Validity • Date range DDL • Date range queries 2
- 3. Who am I? • 35 years as Developer / Technical Architect – Aeronautics, IBM, Finance – Mainframe, client-server, Web apps • 27 years as an American in Paris • 10 years using Oracle database – Performance analysis – Replace Java with SQL • 3 years as internal "Oracle Development Expert" • "Career 2.0" started last week 3
- 4. Review of date/time datatypes • There is no "date" type: always a time element – Closest thing to a date: dte DATE CHECK(dte = TRUNC(dte)) • There are no "formats" • Values from 4712-01-01 00:00:00 BC through 9999-12-31 23:59:59 AD • Any of the above can be used in ranges 4 Datatype Year Month Day Hour Minute Second Fraction Time zone Date Y Y Y Y Y Y Timestamp Y Y Y Y Y Y Y Timestamp with local time zone Y Y Y Y Y Y Y implicit Timestamp with time zone Y Y Y Y Y Y Y explicit
- 5. Date/time Ranges • Use same data_type for start and end (duh!) • To NULL or not to NULL? – SQL:2011 and 12c allow – Makes NULL mean something – Accounting for NULLs complicates queries and index design – Alternative: extreme values such as DATE '9999-21-31' • "Start" is included in the range, "end" is not – Works for date/time ranges, not just dates – See SQL:2011 standard, 12c Temporal Validity – Allows ranges to "meet" – SQL can't use BETWEEN "start" and "end" 5
- 6. Allen’s Time Interval Arithmetic 6 Day of month 1 2 3 4 A precedes B 1 2 B preceded by A 3 4 A meets B 1 2 B met by A 2 3 A overlaps B 1 3 B overlapped by A 2 4 A finished by B 1 3 B finishes A 2 3 A contains B 1 4 B during A 2 3 A starts B 1 2 B started by A 1 3 A equals B 1 2 B equals A 1 2 Meet Gap "Overlap" Day of month 1 2 3 4Day of month 1 2 3 4 A precedes B 1 2 B preceded by A 3 4 Day of month 1 2 3 4 A precedes B 1 2 B preceded by A 3 4 A meets B 1 2 B met by A 2 3
- 7. Temporal Validity: SQL:2011, 12c • Two temporal concepts – "Transaction time": when data committed (flashback) • "Oracle Flashback" Tues. 11:20 (Connor McDonald) – "Valid time": • start / end times determined by users • Past, present or future • "Period": date/time range – Closed-Open: includes start time but not end time – Constraint: start time < end time – NULL start time = valid any time before end – NULL end time = valid from start time on 7
- 8. What Oracle can do for you 8 create table valid_emp( ename varchar2(64), PERIOD FOR PRESENCE );
- 9. 9 select column_name, data_type, nullable, hidden_column, virtual_column from user_tab_cols where table_name = 'VALID_EMP' order by internal_column_id; Name Data Type Nullable Hidden Virtual PRESENCE_START TIMESTAMP(6) WITH TIME ZONE Y YES NO PRESENCE_END TIMESTAMP(6) WITH TIME ZONE Y YES NO PRESENCE NUMBER Y YES YES ENAME VARCHAR2 Y NO NO select search_condition from user_constraints where table_name = 'VALID_EMP'; SEARCH_CONDITION PRESENCE_START < PRESENCE_END
- 10. 10 insert all into valid_emp (ename, presence_start, presence_end) values('Stew', date '1998-03-01', date '2015-12-01') into valid_emp (ename, presence_start, presence_end) values('Stew', date '2016-01-01', date '2050-12-01') select null from dual; select * from valid_emp; ENAME Stew Stew Oops! Hidden columns…
- 11. 11 select ename, presence_start, presence_end from valid_emp; ENAME PRESENCE_START PRESENCE_END Stew 1998-03-01 2015-12-01 Stew 2016-01-01 2050-12-01
- 12. 12 select ename, presence_start, presence_end from valid_emp as of period for presence systimestamp; ENAME PRESENCE_START PRESENCE_END select * from table (dbms_xplan.display_cursor(format=>'+PREDICATE')); filter(( (T.PRESENCE_START IS NULL OR SYS_EXTRACT_UTC(T.PRESENCE_START)<=SYS_EXTRACT_UTC(SYSTIMESTAMP(6))) AND (T.PRESENCE_END IS NULL OR SYS_EXTRACT_UTC(T.PRESENCE_END) > SYS_EXTRACT_UTC(SYSTIMESTAMP(6))) ))
- 13. 13 select ename, presence_start, presence_end from valid_emp as of period for presence to_timestamp_tz('2015-11-01'); exec DBMS_FLASHBACK_ARCHIVE.ENABLE_AT_VALID_TIME ('CURRENT'); select ename, presence_start, presence_end from valid_emp; ENAME PRESENCE_START PRESENCE_END Stew 1998-03-01 2015-12-01 ENAME PRESENCE_START PRESENCE_END
- 14. Valid Time Range 14 select ename, presence_start, presence_end from valid_emp versions period for presence between to_timestamp_tz('2015-11-01') and to_timestamp_tz('2016-01-01'); ENAME PRESENCE_START PRESENCE_END Stew 1998-03-01 2015-12-01 Stew 2016-01-01 2050-12-01
- 15. • So what did Oracle do for us? – Notion of "period" – Automatic, hidden column definitions – New query syntax with automatic rewrite • And what did Oracle not do for us? – Query performance? – Temporal constraints? – Query for gaps or overlaps? – Temporal DML? • Now: what can we do with Oracle? 15
- 16. What we need to do: • Temporal constraints • Performance: indexing • Temporal queries • Temporal DML 16
- 17. Temporal Constraints • No gaps, no overlaps – Don't use ranges! – "effective date" • "start" = EFF_DATE , "end" = lead(EFF_DATE) over (EFF_DATE) • Temporal primary keys – Use ID + start_date (usually) – ORA-02329: …TIMESTAMP WITH TIME ZONE cannot be unique or primary key • Temporal foreign keys – Child range must be within parent range – "On commit" materialized views (hard to scale) – Triggers (hard to write correctly) 17
- 18. Temporal DDL & Indexing 18 create table valid_emp( ename varchar2(64), presence_start date, presence_end date not null, CHECK(presence_start < presence_end), period for presence(presence_start, presence_end), CONSTRAINT valid_emp_pk primary key (ename, presence_start) USING INDEX ( CREATE INDEX valid_emp_pk ON valid_emp (ename, presence_start, presence_end) ) );
- 19. Temporal Queries • Use Analytic functions or 12c MATCH_RECOGNIZE • Typical use cases – find gaps: "free time" in calendars – Merge ranges that "meet" – Merge ranges that "meet" or "overlap" – Join on intersecting date ranges • For simplicity, I assume "not NULL" 19
- 20. Find Gaps 20 START_DATE END_DATE ---------- ---------- 2007-01-12 2007-01-25 2007-01-20 2007-02-01 2007-02-05 2007-02-10 2007-02-05 2007-02-28 2007-02-10 2007-02-15 2007-03-01 2007-03-02 2007-03-03 2007-03-16 SELECT end_date start_gap, LEAD(start_date) OVER (ORDER BY start_date) end_gap FROM T; START_GAP END_GAP ---------- ---------- 2007-01-25 2007-01-20 2007-02-01 2007-02-05 2007-02-10 2007-02-05 2007-02-28 2007-02-10 2007-02-15 2007-03-01 2007-03-02 2007-03-03 2007-03-16
- 21. Find Gaps 21 START_DATE END_DATE ---------- ---------- 2007-01-12 2007-01-25 2007-01-20 2007-02-01 2007-02-05 2007-02-10 2007-02-05 2007-02-28 2007-02-10 2007-02-15 2007-03-01 2007-03-02 2007-03-03 2007-03-16 SELECT * FROM ( SELECT MAX(end_date) OVER (ORDER BY start_date) start_gap, LEAD(start_date) OVER (ORDER BY start_date) end_gap FROM T ) WHERE start_gap < end_gap; START_GAP END_GAP ---------- ---------- 2007-01-25 2007-01-20 2007-02-01 2007-02-05 2007-02-28 2007-02-05 2007-02-28 2007-02-10 2007-02-28 2007-03-01 2007-03-02 2007-03-03 2007-03-16
- 22. PACK: merge ranges that meet 22 ID START END 1 01-01 01-03 2 01-03 01-05 3 02-05 02-28 4 02-10 02-15 5 03-01 03-11 6 03-10 03-12 7 03-1103-21 select * from t match_recognize( order by start_date, end_date measures first(id) "first", last(id) "last", first(start_date) "start", last(end_date) "end" pattern(A* B) define A as end_date = next(start_date) ); first last start end ----- ---- ----- ----- 1 2 01-01 01-05 3 3 02-05 02-28 4 4 02-10 02-15 5 5 03-01 03-11 6 6 03-10 03-12 7 7 03-11 03-21 overlap intervening
- 23. Merge ranges that meet or overlap 23 ID START END 1 01-01 01-03 2 01-03 01-05 3 02-05 02-28 4 02-10 02-15 5 03-01 03-11 6 03-10 03-12 7 03-1103-21 select * from t match_recognize( order by start_date, end_date measures first(id) "first", last(id) "last", first(start_date) "start", MAX(end_date) "end" pattern(A* B) define A as MAX(end_date) >= next(start_date) ); first last start end ----- ---- ----- ----- 1 2 01-01 01-05 3 4 02-05 02-28 5 7 03-01 03-21 gap gap
- 24. Join on intersecting ranges • Prerequisite for temporal DML – Join input to table to find affected rows • Use UNION ALL to gather input and affected rows • Use UNPIVOT and DISTINCT to get range boundaries • Use LEAD() to create base ranges • DO LEFT JOIN from base ranges to table + input. 24 1 3 5 7 2 4 St 1 2 3 4 5 St End 1 2 2 3 3 4 4 5 St End Old New 1 2 1-3 2 3 1-3 2-4 3 4 3-5 2-4 4 5 3-5