Bring sanity back to sql (advance sql)
Download as PPTX, PDF2 likes813 views
The document discusses advanced SQL concepts, primarily focusing on managing null values and their peculiar behaviors, such as arithmetic operations and comparisons. It provides examples and solutions for SQL queries, including how to handle nulls when retrieving rows from tables and checking for data duplications and joinability. Additionally, it emphasizes the importance of avoiding complex queries (spaghetti SQL) by following conventions and utilizing tools for validation.
![Null Definition
● Null signifies an unknown value or that a
value does not exist.
● The predicate is null can be used to check
for Null values.
SELECT <column_list>
FROM [tablenames]
WHERE <column_a> IS NULL](https://image.slidesharecdn.com/bringsanitybacktosqladvancesql-180811090754/85/Bring-sanity-back-to-sql-advance-sql-10-320.jpg)
![Avoiding Spaghetti SQL
● Construct Rules and Conventions[example 1] [example 2]](https://image.slidesharecdn.com/bringsanitybacktosqladvancesql-180811090754/85/Bring-sanity-back-to-sql-advance-sql-41-320.jpg)
![Avoiding Spaghetti SQL
● Construct Rules and Conventions[example 1] [example 2]
● Automatically validate conventions in CI[tool 1]](https://image.slidesharecdn.com/bringsanitybacktosqladvancesql-180811090754/85/Bring-sanity-back-to-sql-advance-sql-42-320.jpg)
![Avoiding Spaghetti SQL
● Construct Rules and Conventions[example 1] [example 2]
● Automatically validate conventions in CI
● Break queries into multiple queries
[tool 1]](https://image.slidesharecdn.com/bringsanitybacktosqladvancesql-180811090754/85/Bring-sanity-back-to-sql-advance-sql-43-320.jpg)
![Avoiding Spaghetti SQL
● Construct Rules and Conventions[example 1] [example 2]
● Automatically validate conventions in CI
● Break queries into multiple queries
[tool 1]
WITH T2 AS
(subquery_statement_on_T1),
T3
AS(subquery_statement_on_T2)
SELECT *
FROM T3
CREATE TEMP TABLE T2 AS
(subquery_statement_on_T1);
CREATE TEMP TABLE T3 AS
(subquery_statement_on_T2);
SELECT *
FROM T3
With Statement: Create temp table Statement:](https://image.slidesharecdn.com/bringsanitybacktosqladvancesql-180811090754/85/Bring-sanity-back-to-sql-advance-sql-44-320.jpg)
Bring sanity back to sql (advance sql)
- 1. Advance SQL Bring Sanity Back to SQL Eyal Trabelsi
- 2. Agenda ● Execution Order ● Null ● SQL Sanity ● Spaghetti Query ● Additional Resources
- 5. Execution order More detailed execution order
- 8. Null Definition ● Null signifies an unknown value or that a value does not exist.
- 9. Null Definition ● Null signifies an unknown value or that a value does not exist. ● The predicate is null can be used to check for Null values.
- 10. Null Definition ● Null signifies an unknown value or that a value does not exist. ● The predicate is null can be used to check for Null values. SELECT <column_list> FROM [tablenames] WHERE <column_a> IS NULL
- 12. Null Weird Behavior ● 5 + Null returns ? ● 5 < Null returns ? ● Null = Null ? ● Null in (Null) returns ? ● Null and true? ● Null and false? ● Null or true ? ● Null or false?
- 13. Null Weird Behavior ● 5 + Null returns ? Null ● 5 < Null returns ? Null ● Null = Null ? Null != Null ● Null in (Null) returns ? Null because in use = operator ● Null and true? Null ● Null and false? False ● Null or true ? True ● Null or false? Null
- 14. Null Weird Behavior ● The result of any arithmetic expression involving null is null.
- 15. Null Weird Behavior ● The result of any arithmetic expression involving null is null. ● Any comparison with null returns unknown.
- 16. Null Weird Behavior ● The result of any arithmetic expression involving null is null. ● Any comparison with null returns unknown. ● All aggregate operations except count(*) ignore tuples with null values on the aggregated attributes.
- 17. Null Weird Behavior ● The result of any arithmetic expression involving null is null. ● Any comparison with null returns unknown. ● All aggregate operations except count(*) ignore tuples with null values on the aggregated attributes. ● logic inference
- 19. Null Weird Behavior- Problem Example
- 20. Null Weird Behavior- Problem Example a c c a d b a b NULL a c b T2T1
- 21. Null Weird Behavior- Problem Example a c c a d b a b NULL a c b T2T1 Retrieve rows of table table T1 which doesn’t exists in T2 ? (a is the fk)
- 22. Null Weird Behavior- Solution 1 a c c a d b a b NULL a c b T2T1 Retrieve rows of table table T1 which doesn’t exists in T2 ? (a is the fk)SELECT * FROM T1 WHERE a NOT IN (SELECT a FROM T2)
- 23. Null Weird Behavior- Solution 1 a c c a d b a b NULL a c b T2T1 Retrieve rows of table table T1 which doesn’t exists in T2 ? (a is the fk)SELECT * FROM T1 WHERE a NOT IN (SELECT a FROM T2) Is it correct ?
- 24. Null Weird Behavior- Solution 1 a c c a d b a b NULL a c b T2T1 Retrieve rows of table table T1 which doesn’t exists in T2 ? (a is the fk)SELECT * FROM T1 WHERE a NOT IN (SELECT a FROM T2) No it will retrieve no rows!!
- 25. Null Weird Behavior- Solution 2 a c c a d b a b NULL a c b T2T1 Retrieve rows of table table T1 which doesn’t exists in T2 ? (a is the fk)SELECT T1.* FROM T1 LEFT JOIN T2 ON T1.a = T2.a WHERE T2.a IS NULL
- 26. Null Weird Behavior- Solution 2 a c c a d b a b NULL a c b T2T1 Retrieve rows of table table T1 which doesn’t exists in T2 ? (a is the fk)SELECT T1.* FROM T1 LEFT JOIN T2 ON T1.a = T2.a WHERE T2.a IS NULL Is it correct ?
- 28. SQL Sanity- Duplication Check
- 29. SQL Sanity- Duplication Check When you want to check for duplications in T1
- 30. SQL Sanity- Duplication Check When you want to check for duplications in T1 SELECT pk FROM T1 GROUP BY pk HAVING COUNT(1)>1
- 32. SQL Sanity- Joinability When you want to check joinability ratios of T1 and T2
- 33. SQL Sanity- Joinability When you want to check joinability ratios of T1 and T2 SELECT SUM(CAST(T1.pk IS NOT NULL AS INT) AS t1_size, SUM(CAST(T2.pk IS NOT NULL AS INT) AS t2_size, SUM(CAST(T2.pk IS NOT NULL AND AS INT) AS t2_size, FROM T1 FULL OUTER JOIN T2 ON T2.pk = T2.pk
- 34. SQL Sanity- Joinability When you want to check joinability ratios of T1 and T2 WITH T3 AS (SELECT T1.pk IS NOT NULL AS t1_row, CAST(T2.pk IS NOT NULL AS INT) AS t2_row FROM T1 FULL OUTER JOIN T2 ON T2.pk = T2.fk) SELECT SUM(CAST(t1_row AS INT)) AS t1_size, SUM(CAST(t2_row AS INT)) AS t2_size, SUM(CAST(t1_row AND NOT t2_row AS INT)) AS only_t1_size, SUM(CAST(NOT t1_row AND t2_row AS INT)) AS only_t2_size, SUM(CAST(t1_row AND t2_row AS INT)) AS t1_t2_size, FROM T3
- 35. SQL Sanity- Outliers Understanding outlier detection in sql
- 36. SQL Sanity- Outliers Understanding outliers with skew and kurtosis
- 38. Spaghetti SQL
- 41. Avoiding Spaghetti SQL ● Construct Rules and Conventions[example 1] [example 2]
- 42. Avoiding Spaghetti SQL ● Construct Rules and Conventions[example 1] [example 2] ● Automatically validate conventions in CI[tool 1]
- 43. Avoiding Spaghetti SQL ● Construct Rules and Conventions[example 1] [example 2] ● Automatically validate conventions in CI ● Break queries into multiple queries [tool 1]
- 44. Avoiding Spaghetti SQL ● Construct Rules and Conventions[example 1] [example 2] ● Automatically validate conventions in CI ● Break queries into multiple queries [tool 1] WITH T2 AS (subquery_statement_on_T1), T3 AS(subquery_statement_on_T2) SELECT * FROM T3 CREATE TEMP TABLE T2 AS (subquery_statement_on_T1); CREATE TEMP TABLE T3 AS (subquery_statement_on_T2); SELECT * FROM T3 With Statement: Create temp table Statement:
- 46. Good Resources ● MUST SEE VIDEO ● Basic SQL reference ● Good SQL blog ● Another Good SQL blog ● http://www.silota.com/docs/recipes/
Editor's Notes
- #45: https://stackoverflow.caom/questions/46421743/what-would-be-the-difference-between-with-clause-temporary-table