sql_bootcamp.pdf
0 likes11 views
This document provides an introduction to SQL and relational databases. It discusses that SQL is the most widely used database language, used to manage data stored in relational databases. It also describes what a database and database management system are, how to connect to databases using clients, and how SQL can be used beyond just relational databases. The document then outlines categories of hands-on SQL exercises like retrieving, filtering, grouping, joining and summarizing data. It also discusses some advanced SQL topics like common table expressions and self-joins that were not covered.
sql_bootcamp.pdf
- 1. INTRO TO SQL Bootcamp (https://tdmdal.github.io/mma-sql/) September 21, 2020 Prepared by Jay / TDMDAL
- 2. What’s SQL (Structured Query Language) • Most widely used database (DB) language • a domain specific language (managing data stored in relational DB) • Not a proprietary language • Open specifications/standards • All major DBMS (DB Mgmt. System ) vendors implement ANSI Standard SQL • However, SQL Extensions are usually DB specific • Powerful despite simplicity ANSI - American National Standards Institute
- 3. What’s DB and DB Management System • What’s a database: A collection of data in an organized way • Relational DB • tables • columns/fields/variables and datatypes • rows/records/observations • primary key, foreign key, constraints and relationships (discuss later) • What is DBMS (DB Management System)? • A software system that manages/maintains relational DBs • e.g. MySQL, MariaDB, PostgreSQL, SQLite, Microsoft SQL Server, Oracle, etc.
- 4. Connect to a DB and use SQL – DB Client • DB specific management client • command-line console • GUI client (e.g. DB Browser for SQLite, MySQL Workbench, MS SSMS) • Generic DB client can connect to different DBs through connectors • GUI client (e.g. DBeaver, Navicat) • Programming language (e.g. Python + SQLAlchemy + DBAPI (e.g. SQLite, MySQL, PostgreSQL, etc.), R + dbplyr)
- 5. Beyond a relational DB language • SAS’s PROC SQL • Spark’s SparkSQL • Apache Spark is a big data computing framework • Hive’s HiveQL, an SQL-like query language • Apache Hive is a distributed data warehouse (data warehouse?) • Google BigQuery’s SQL • BigQuery is Google’s data warehouse (analyze petabytes of data at ease) ref. Database vs data warehouse; Data warehouse vs data lake note: NoSQL DB?
- 6. SQL Hands-on Exercises (Learning-by-doing) • Course website: https://tdmdal.github.io/mma-sql/ • Google Colab • Google’s Jupyter Notebook • A notebook can contain live code, equations, visualizations and narrative text • Why SQLite? • a small, fast, self-contained, high-reliability, full-featured, SQL DB engine • perfect for learning SQL
- 7. Preparation For RSM8411 (MMA, Fall 2020) • A different setup (a more advanced/powerful DBMS) • Microsoft SQL Server Express, a mini/desktop version of MS SQL Server • SQL Server Management Studio (SSMS), a GUI client for MS SQL Server • Get-started resources for this setup: see our bootcamp website • Please make sure you have the above setup installed • Set it up before the end of this bootcamp • Email me if you have trouble with installation • SQL syntax difference between SQLite and MS SQL • For 99% of what we will learn in this bootcamp, they are the same
- 9. Primary key, foreign key, constraints and relationships
- 10. Hands-on Part 1: Warm up • Retrieve data: SELECT...FROM... • Sort retrieved data: SELECT...FROM...ORDER BY... • Filter data: SELECT...FROM...WHERE... • IN, NOT, LIKE and % wildcard • Create calculated fields • mathematical calculations (e.g. +, -, *, /) • data manipulation functions (e.g. DATE(), ||)
- 11. Hands-on Part 2: Summarize and Group Data • Summarize data using aggregate functions (e.g. COUNT(), MIN(), MAX(), and AVG()). • Group data and filter groups: SELECT...FROM...GROUP BY...HAVING... • SELECT clause ordering: SELECT...FROM...WHERE...GROUP BY...HAVING...ORDER BY... • Filter data by subquery: SELECT...FROM...WHERE...(SELECT...FROM...)
- 12. Hands-on Part 2: Join Tables • Inner join: SELECT...FROM...INNER JOIN...ON... • Left join: SELECT...FROM...LEFT JOIN...ON... • Other join variations.
- 13. Join – Inner Join Table1 Table2 SELECT * FROM Table1 INNER JOIN Table2 ON Table1.pk = Table2.fk; pk t1c1 1 a 2 b fk t2c1 1 c 1 d 3 e pk t1c1 fk t2c1 1 a 1 c 1 a 1 d Table1 Table2
- 14. Join – Left (Outer) Join SELECT * FROM Table1 LEFT JOIN Table2 ON Table1.pk = Table2.fk; Table1 Table2 pk t1c1 1 a 2 b fk t2c1 1 c 1 d 3 e pk t1c1 fk t2c1 1 a 1 c 1 a 1 d 2 b null null Table1 Table2
- 15. Join - Left (Outer) Join With Exclusion Table1 Table2 pk t1c1 1 a 2 b fk t2c1 1 c 1 d 3 e Table1 Table2 SELECT * FROM Table1 LEFT JOIN Table2 ON Table1.pk = Table2.fk WHERE Table2.fk is NULL; pk t1c1 fk t2c1 2 b null null
- 16. Join – Right Outer Join* SELECT * FROM Table2 LEFT JOIN Table1 ON Table2.fk = Table1.pk --------------------------- SELECT * FROM Table1 RIGHT JOIN Table2 ON Table1.pk = Table2.fk; Table2 pk t1c1 1 a 2 b fk t2c1 1 c 1 d 3 e pk t1c1 fk t2c1 1 a 1 c 1 a 1 d null null 3 e Table1 Table2 Table1 SQLite doesn’t support this RIGHT JOIN key word, but some DBMSs do (e.g. MySQL).
- 17. Join - Right Outer Join With Exclusion* Table1 Table2 pk t1c1 1 a 2 b fk t2c1 1 c 1 d 3 e Table1 Table2 SELECT * FROM Table2 LEFT JOIN Table1 ON Table2.fk = Table1.pk WHERE Table1.pk is NULL; --------------------------- SELECT * FROM Table1 RIGHT JOIN Table2 ON Table1.pk = Table2.fk WHERE Table1.pk is NULL; pk t1c1 fk t2c1 2 b null null pk t1c1 fk t2c1 null null 3 e SQLite doesn’t support this RIGHT JOIN key word, but some DBMSs do (e.g. MySQL).
- 18. Join – Full Outer Join pk t1c1 1 a 2 b fk t2c1 1 c 1 d 3 e Table1 Table2 SELECT pk, t1c1, fk, t2c1 FROM Table1 LEFT JOIN Table2 ON Table1.pk = Table2.fk UNION SELECT pk, t1c1, fk, t2c1 FROM Table2 LEFT JOIN Table1 ON Table2.fk = Table1.pk; Table1 Table2 pk t1c1 fk t2c1 1 a 1 c 1 a 1 d 2 b null null null null 3 e Note: Some DBMS support FULL OUTER JOIN keyword (e.g. MS SQL) so you don’t need to do it the above way.
- 19. Join – Full Outer Join With Exclusion* pk t1c1 1 a 2 b fk t2c1 1 c 1 d 3 e Table1 Table2 SELECT pk, t1c1, fk, t2c1 FROM Table1 LEFT JOIN Table2 ON Table1.pk = Table2.fk WHERE Table2.fk is NULL UNION SELECT pk, t1c1, fk, t2c1 FROM Table2 LEFT JOIN Table1 ON Table2.fk = Table1.pk WHERE Table1.pk is NULL; Table1 Table2 pk t1c1 fk t2c1 2 b null null null null 3 e
- 20. Others • CTE and temporary table • Self-join • CASE keyword • UNION keyword
- 21. Many things we didn’t cover • Insert data (INSERT INTO…VALUES…; INSERT INTO…SELECT…FROM…) • Update data (UPDATE…SET…WHERE…) • Delete data (DELETE FROM…WHERE…) • Manipulate tables (CREATE TABLE…; ALTER TABLE…; DROP TABLE…) • Views (CREATE VIEW…AS…)
- 22. The list goes on and on • Stored procedures • Functions • Transaction processing • Cursors (going through table row by row) • WINDOW function • Query optimization • DB permissions & security • … Ref. A stack overflow discussion on What is “Advanced” SQL.