98765432345671223Intro-to-PostgreSQL.ppt

Intro to PostgreSQL
Mike Callahan
WFO Louisville
September 8, 2009

Topics
 Relational database theory
 PostgreSQL Reference
 psql
 Snoopy
 SQL syntax
 SELECT
 UPDATE
 INSERT INTO
 DELETE FROM
 CREATE TEMP TABLE
 COPY

Databases in AWIPS
PostgreSQL
Databases
Sources Data
D2D
WHFS
Other
Applications
SQL
psql
pg_...

Relational Theory
 Data are stored in groups called 'tables' or 'relations'
 Tables consist of 'rows' or 'records' and 'columns' or
'fields'
 Rows are usually identified by a unique 'key' which
may be a single column or a group of columns
 Columns can be linked to other tables with similar
columns
 Good design of the database structure or 'schema' is
critical.
 "All the data in a row should be dependant on the key, the
whole key, and nothing but the key."

Working with databases
 Most of the time, you will work with existing
databases
 However, you still need to know the schema
 We will concentrate on four tasks
 Extracting data
 Changing data
 Inserting data
 Deleting data

Why PostgreSQL
 Switching database server from HP to Linux
 PostgreSQL is free, Informix for Linux is not
 PostgreSQL is the most advanced open source
relational database, MySQL did not have
triggers, PostgreSQL did
 Version on AWIPS is 8.2, latest version is 8.4
 Documentation is excellent and is at
www.postgresql.org/docs/8.2/...
 We will use version 8.2 for our examples

PostgreSQL Documentation
 Preface
 Gives history, conventions, etc.
 Tutorial
 A quick introduction to SQL
 SQL Language
 A description of the standard and the PostgreSQL
additions
 Reference: SQL Commands
 Details of each command

psql
…/interactive/app-psql.html
 The standard PostgreSQL application to work
with databases
 It uses a superset of standard SQL
 It can be interactively like DBaccess, or in
scripts like sqlcmd

Common psql Command Line Options
 -A, --no-align: set output to non-aligned, no padding
 -c sql, --command sql: execute the sql command and
then exit
 -d name, --dbname name: name of database, same as
name as the first non-option argument
 -f name, --file name: use name as the source of
commands
 -o name, --output name: put the output in name
 -q, --quiet: suppress welcome messages
 -t, --tuples-only: suppress print column names, result
row counters, etc
 -?, --help: get command line help

Common psql Meta-commands
 a: toggle output format (aligned/unaligned)
 c name: connect to a database name
 copy table: copy a table to/from a file
 d : list all tables (display)
 d table: show information about a table
 e: edit the query buffer
 g: execute the query buffer (go)
 h command: display help on command
 i name: read name into query buffer (input)

Common psql Meta-commands
 o name: send output to name
 p: display the query buffer
 q: quit the program
 r: resets (clears) the query buffer
 t: toggle the output headers on/off
 w name: write the query buffer to name
 ! command: execute the Linux command
 ?: help on meta-commands

Interactive psql
 Typing in a query loads the query buffer
 The command will not run unless terminated
with a semicolon (;)
 Meta-commands all start with blackslash ()
 Exit with q

Snoopy
 Snoopy is an interactive, GUI front-end to psql
 It was designed for the hydro database, but
will work with any AWIPS database
 Queries are in a scrolling window, data is in a
separate scrolling window
 Help is available about tables, columns, rows,
SQL syntax, and hydro database data
dictionary
 Available on the AWIPS LAD

Snoopy Help
Available only for the
hydro database

SQL for PostgreSQL
…/interactive/sql-commands.html
 SQL, pronounced “sequel”, is the standard
way of working with relational databases
 The standard is well-supported but weak so
every database manager adds commands to the
standard
 In these examples, SQL keywords will be
CAPITALIZED
 However, case is NOT important except in
strings

Basic Data Types in SQL
 Character, Varchar: A sequence of alphanumeric
characters enclosed in single quotes; ‘a string 123’
 To include a single quote in a string double it;
'Mike''s string'
 Integers: A number without a decimal point; 125, -2
 Real: A number with a decimal point; 3.1416, 25.00
 Decimal, Numeric: Fixed decimal point, accurate but
slow math
 Boolean: TRUE or FALSE
 in psql TRUE is displayed as t, FALSE is displayed
as f

Date/Time Data Types in SQL
 Date: The word followed by a date enclosed in single
quotes; DATE ‘2006-2-18’
 Other formats are allowed but this is the standard.
 Time: The word followed by a time enclosed in
single quotes; TIME ‘02:34:00’
 Seconds and colons are optional.
 Timestamp: Both; TIMESTAMP ’2004-10-23
13:20:00’
 Interval: A time interval; INTERVAL ‘2 days’,
INTERVAL ‘12 hours’
 Many times you in PostGreSQL you don't need to
type the descriptor. For example: '2006-2-18',
'02:34', etc.

Special Values
 TIMESTAMP ‘NOW’
 The current time, date, or timestamp
 DATE ‘TODAY’
 The current date or midnight timestamp
 TIMESTAMP ‘TOMORROW’
 Tomorrow date or midnight timestamp
 DATE ‘YESTERDAY’
 Yesterday date or midnight timestamp
 TIME ‘ALLBALLS’
 Midnight 00:00:00 UTC time

Common SQL Commands
 SELECT
 Extract data from one or more tables
 UPDATE
 Change data in a table quickly
 INSERT INTO
 Add new data to a table
 DELETE FROM
 Remove data from a table
 CREATE TEMP TABLE
 Create a temporary table
 COPY
 Send the contents of a table to a file or vice versa

SELECT
 Most SQL work involves creating SELECT
statements
 You cannot harm data using a SELECT Command
 SELECT statements can be simple but can also be
almost incomprehensible
 SELECT statements are broken up into clauses
 FROM, WHERE, GROUP BY, HAVING, UNION,
INTERSECT, EXCEPT, ORDER BY, LIMIT, FOR
UPDATE
 We will only look at a few simple queries

Basic SELECT
 SELECT
 List of columns to extract
 FROM
 List of tables to work with
 WHERE
 Condition to limit selection
 GROUP BY
 Grouping expression
 HAVING
 Condition to limit grouping
 ORDER BY
 Sort order
 LIMIT
 Number of rows to extract

SELECT Command Structure
SELECT FROM WHERE GROUP BY
ORDER BY LIMIT
HAVING
List of
columns
List of
tables
Condition Expression
Condition Sort order Number

Building a SELECT Query
 You must know the schema (structure) of your
database to know the tables and the columns
you want to extract
 Know the condition you want to filter the
selection
 Know the sort order you want to see the data,
the order will be undefined if not specified

Example
 Tables and columns
 location: lid, name
 Table names need to be included only if column names are
ambiguous
 Condition
 location.state = 'KS'
 Order
 location.name
SELECT lid, name FROM location
WHERE state = 'KS'
ORDER BY name;
Find all IDs and names in Kansas sorted by name

Joining Two Tables
 Tables and columns
 location: lid, name
 observer: lastname
 Condition
 location.lid = observer.lid and location.state = 'KS'
 Sort order
 observer.lastname
Retrieve the list of locations, ids, and observers in KS sorted by
observer's last name
SELECT lid, name, lastname FROM location, observer
WHERE location.lid = observer.lid
AND location.state = 'KS'
ORDER BY lastname;

More on WHERE Clause
 Comparison operators
 =, != or <>, >, >=, <, <=
 Logical operators
 AND, OR, NOT
 String concatenation
 'a' || 'string'
 BETWEEN operator
 WHERE fs BETWEEN 10 AND 50
 IN operator
 WHERE state IN ('KY', 'IN', 'OH')
 IS NULL test
 WHERE fs IS NULL

Matching in Character Columns
 LIKE
 Used to set up search targets
 % - zero or more characters
 _ - a single character
 WHERE lid LIKE '___K1' or lid LIKE '___K2'
 ~
 Used to set up POSIX regular expression searches
 WHERE lid ~ '…K[12]'
 Use escape notation for backslash expressions
 '…wd' becomes e'…wd'

Substrings
 Use to extract part of a character column based
on position (count from 1)
 SUBSTRING(column FROM begin FOR length)
 Ex. show phone number of observers in the
Kansas City KS Metro area code
 SELECT name, hphone FROM observer WHERE
SUBSTRING(hphone FROM 1 FOR 3) = '913'

SELECT tricks
 Testing an expression
 SELECT SUBSTRING('ABCDE' FROM 1 FOR 3)
 Get all the fields in a table
 SELECT * FROM location
 Name shortcut
 SELECT l.lid, lastname FROM location l, observer o WHERE
l.lid=o.lid AND lastname LIKE 'a%'
 Rename columns
 SELECT lid AS Location_ID FROM location...
 Eliminate duplicate rows
 SELECT DISTINCT county FROM location
 Order shortcut
 SELECT lid, name FROM location ORDER BY 2
 Descending order
 ...ORDER BY lid DESC

Date/Time/Timestamp Functions
 AGE(timestamp, [timestamp])
 calculate interval between two timestamps, second
argument defaults to NOW
 …WHERE AGE(obstime) < INTERVAL '6 hours'
 EXTRACT(unit from timestamp)
 extract the unit from a timestamp
 common units: century, day, decade, dow (day of week
Sun=0, Mon=1, etc), doy (Julian date), hour, minute,
month, quarter, second, timezone, week, year
 …WHERE EXTRACT(dow from datcrst) = 3

Group Functions
 COUNT()
 Instead of displaying the rows, just count them
 SELECT COUNT(*) FROM observer
 AVG()
 Calculate the average of a column
 MAX()
 Calculate the maximum in a column
 MIN()
 Calculate the minimum in a column
 SUM()
 Calculate the sum of a column

The GROUP BY Clause
 When using the previous functions, the
GROUP BY clause tells what to group
 We want the highest crest at EVERY location,
sorted by ID
 If we don't use GROUP BY we will get the
highest crest of ALL IDs instead of the highest
crest at EVERY ID.
SELECT l.lid, MAX(stage) FROM location l, crest c
WHERE l.lid = c.lid GROUP BY l.lid ORDER BY l.lid;

The UPDATE Command
 Used to quickly change the values in a column
 This command can save a lot of work but can
also be dangerous
 Has SET, FROM, and WHERE clauses
 Here is a trick:
 Create the WHERE clause but use SELECT *
 You will see the rows you are going to change
 Build rest of query

The UPDATE Clauses
 SET
 The column and the value you want to assign it
 FROM
 A list of tables you used in the SET clause
 WHERE
 The conditions the data must match

UPDATE Command Structure
UPDATE FROM WHERE
Table
List of
tables
Condition
SET
Column
and value

Using UPDATE
 Ex. 913 area code changes to 914
 Table and column
 observer: hphone
 Value
 ‘914’ || SUBSTRING(hphone FROM 4)
 Condition
 SUBSTRING(hphone FOR 3) = '913'
UPDATE observer SET hphone = '914‘ || SUBSTRING(hphone FROM
4)
WHERE SUBSTRING(hphone FOR 3) = '913'
SELECT * FROM observer WHERE SUBSTRING(hphone FOR
3) = '913'
Check WHERE Clause
UPDATE rows

The INSERT INTO Command
 Insert data into a new row
 Has only VALUES clause
 Can also use a SELECT query

The INESRT Clause
 VALUES
 A list of data that you want to insert
 If the order of the columns is not listed, it must
match the order defined in the schema
 For safety sake, always list the columns
 SELECT
 A standard select query, but the schema of the
result must match the target

INSERT Command Structure
INSERT INTO VALUES
Table,
column list
List of
values
INSERT INTO SELECT
Table,
column list
Select
query
-or-

Using INSERT
 Want to add a new row in datum
 datum schema is
 lid - location id
 ddate - datum date
 elev - elevation
 Our values are ABEK1, 2002-3-23, 3056.34
INSERT INTO datum (lid, ddate, elev)
VALUES ('ABEK1', '2002-3-23', 3056.34);

Using INSERT (con't)
 Want to add data from a temporary table
named junk into the height table
 junk and height have the same schema
INSERT INTO height SELECT * FROM junk;

The DELETE FROM Command
 Remove rows
 Has only WHERE clause
 There is no quick undo so be careful
 If you accidentally leave off the WHERE
clause, ALL the rows will be deleted!
 Here is another trick:
 Create the WHERE clause but use SELECT *
 You will see the rows you are going to delete
 Change SELECT * to DELETE

The DELETE Clause
 WHERE
 The condition the rows must match

DELETE Command Structure
DELETE FROM WHERE
Table Condition

Using DELETE
 Want to remove an observer to make a location
inactive
 The location ID is CANK1
DELETE FROM observer WHERE lid = 'CANK1'
SELECT * FROM observer WHERE lid = 'CANK1'
Check WHERE Clause
DELETE row

The CREATE TEMP TABLE
Command
 Creates a temporary table that can be used in
future queries
 Table is automatically deleted once psql
session is completed.
 In Snoopy, table is deleted at end of last
statement in query.
 It has a simple structure

CREATE TEMP TABLE Command
Structure
CREATE TEMP TABLE AS
Table
Select
query

Using CREATE TEMP TABLE
 Want to copy gage readings for the last three days into a temp
table, junk
CREATE TEMP TABLE junk AS
SELECT * FROM height
WHERE AGE(obstime) < INTERVAL '3 days';

The COPY Command
 There is a quick and easy way to copy a table to a file
 You could process a table without working in SQL
 This is similar to the Informix UNLOAD command
 It has many options but we will only look at a few
 It combines well with the CREATE TEMP TABLE
command
 You must be logged on as postgres for this command
to work

COPY Command Structure
COPY TO WITH
File Options
Table
FROM

Example COPY Commands
 COPY observer TO 'observer.dat' WITH
DELIMITER '|';
 Same as Informix UNLOAD command
except default delimiter is a TAB
 COPY crest TO 'crest.csv' WITH CSV;
 Good way of moving data into Excel
 COPY observer FROM 'observer.dat'
WITH DELIMITER '|';
 Moving data into a table

Combining CREATE TEMP
TABLE and COPY
 We want to copy today's stages into a file,
stg_tdy.dat
CREATE TEMP TABLE junk AS
SELECT * FROM height
WHERE AGE(obstime) < INTERVAL '1 day';
COPY junk TO stg_tdy.dat;

In Conclusion
 PostgreSQL may not be as popular as MySQL, it will
do what we need it to do
 The PostgreSQL documentation is excellent and
available on-line as HTML or off-line as PDF
 psql is good to use in scripts
 Snoopy is good to use interactively
 The best way to learn SQL is to play with it
 You cannot harm data by SELECT queries

98765432345671223Intro-to-PostgreSQL.ppt

More Related Content

Similar to 98765432345671223Intro-to-PostgreSQL.ppt (20)

Recently uploaded (20)

98765432345671223Intro-to-PostgreSQL.ppt