SlideShare a Scribd company logo

Pro Postgres 9

Robert Treat
Robert Treat

The document discusses PostgreSQL version 9 and provides information about upgrading PostgreSQL. It recommends making backups before upgrading, reading release notes and commit logs, and testing upgrades. It describes using pg_dump/pg_restore for simple upgrades but notes potential issues with time and disk usage. It also mentions using a replication-based "Slony method" which is more complex and has potential compatibility issues.

Technology
1 of 90
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
Pro Postgres 9 Robert Treat, JDCon East 2011 Thursday, March 31, 2011
Who Am I? (Why Listen To Me) PostgreSQL User Since 6.5.x DBA of High Traffic / Large PostgreSQL Instances Long Time Contributor to PostgreSQL Project Contribute / Maintain Several Open Source Projects Co-Author Beginning PHP & PostgreSQL 8 (Apress) I like to call it “Postgres” Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Outline What you need to know about the project Backups Getting Started Replication Upgrading Query Tuning Configuring Your Server Other stuff you should know about Hardware Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Know Your Way Around The Project Thursday, March 31, 2011
Know Your Way Around The Project www.postgresql.org downloads security alerts documentation wiki bug reports support companies rss -> news - events - version Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Know Your Way Around The Project planet.postgresql.org project news helpful tips / examples community news Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Know Your Way Around The Project www.pgfoundry.org URI Type SkyTools modules Npgsql programs pg_bulkload resources plpgsql-debugger pagila, sample database Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Know Your Way Around The Project git.postgresql.org check_postgres modules edb-installers programs website code resources json type works in progress pg_top user repo’s see also github.com Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Know Your Way Around The Project archives.postgresql.org mailing list archives back to 1997 full text search via postgresql FTS keyword search suggestions lists for users, developers, regional, user groups Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Know Your Way Around The Project #postgresql irc.freenode.net real time help pg_docbot - ??help Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Know Your Way Around The Project project management core team committers -hackers road map web team Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Get Off To A Good Start Thursday, March 31, 2011
Get Off To A Good Start Use Package Management Consistent Standardized Simple Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Get Off To A Good Start Use Package Management Different across systems Upgrades can be a problem Trust your packager? Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Get Off To A Good Start Use Package Management Different across systems Upgrades can be a problem Trust your packager? Don’t Be Afraid To Roll Your Own Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Get Off To A Good Start A Note About Versions Just playing? Default packages probably OK, (min 8.2) Any production level work should start on 9.0 Any project not due to launch for 6 months, 9.1 Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Get Off To A Good Start So What Do You Get? pg_control version number: 903 Catalog version number: 201008051 Database system identifier: 5546136951888795324 Database cluster state: in archive recovery pg_control last modified: Tue Mar 22 11:50:24 2011 Latest checkpoint location: 7C/1D000058 Prior checkpoint location: 7C/1C000058 Latest checkpoint's REDO location: 7C/1D000020 Latest checkpoint's TimeLineID: 1 Latest checkpoint's NextXID: 0/4924 Latest checkpoint's NextOID: 16608 Latest checkpoint's NextMultiXactId: 1 Latest checkpoint's NextMultiOffset: 0 Latest checkpoint's oldestXID: 654 Latest checkpoint's oldestXID's DB: 1 pg_controldata Latest checkpoint's oldestActiveXID: Time of latest checkpoint: 4924 Tue Mar 22 11:46:19 2011 Minimum recovery ending location: 7C/16000000 Backup start location: 0/0 Current wal_level setting: hot_standby Current max_connections setting: 500 Current max_prepared_xacts setting: 0 Current max_locks_per_xact setting: 64 Maximum data alignment: 8 Database block size: 8192 Blocks per segment of large relation: 131072 WAL block size: 8192 Bytes per WAL segment: 16777216 Maximum length of identifiers: 64 Maximum columns in an index: 32 Maximum size of a TOAST chunk: 1996 Date/time type storage: 64-bit integers Float4 argument passing: by value Float8 argument passing: by value Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Get Off To A Good Start So What Do You Get? -rw------- 1 postgres postgres 4 Dec 2 12:23 PG_VERSION drwxr-xr-x 6 postgres postgres 6 Dec 2 13:41 base drwxr-xr-x 2 postgres postgres 42 Jan 28 17:49 global drwxr-xr-x 2 postgres postgres 3 Dec 2 12:23 pg_clog -rw------- 1 postgres postgres 3938 Dec 19 11:25 pg_hba.conf -rw------- 1 postgres postgres 1636 Dec 19 11:25 pg_ident.conf drwxr-xr-x 3 postgres postgres 113 Mar 22 00:00 pg_log drwxr-xr-x 4 postgres postgres 4 Dec 2 12:23 pg_multixact drwxr-xr-x 2 postgres postgres 3 Jan 12 14:23 pg_notify drwxr-xr-x 2 postgres postgres 3 Jan 12 14:23 pg_stat_tmp drwxr-xr-x 2 postgres postgres 3 Dec 2 12:23 pg_subtrans drwxr-xr-x 2 postgres postgres 2 Dec 2 12:23 pg_tblspc drwxr-xr-x 2 postgres postgres 2 Dec 2 12:23 pg_twophase lrwxrwxrwx 1 postgres postgres 30 Dec 19 11:13 pg_xlog -rw------- 1 postgres postgres 17915 Mar 3 21:52 postgresql.conf -rw------- 1 postgres postgres 63 Jan 12 14:23 postmaster.opts -rw------- 1 postgres postgres 54 Jan 12 14:23 postmaster.pid -rw-r--r-- 1 postgres postgres 336 Dec 20 16:14 recovery.conf Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Get Off To A Good Start So What Do You Get? -rw------- 1 postgres postgres 4 Dec 2 12:23 PG_VERSION drwxr-xr-x 6 postgres postgres 6 Dec 2 13:41 base drwxr-xr-x 2 postgres postgres 42 Jan 28 17:49 global drwxr-xr-x 2 postgres postgres 3 Dec 2 12:23 pg_clog -rw------- 1 postgres postgres 3938 Dec 19 11:25 pg_hba.conf -rw------- 1 postgres postgres 1636 Dec 19 11:25 pg_ident.conf drwxr-xr-x 3 postgres postgres 113 Mar 22 00:00 pg_log drwxr-xr-x 4 postgres postgres 4 Dec 2 12:23 pg_multixact drwxr-xr-x 2 postgres postgres 3 Jan 12 14:23 pg_notify drwxr-xr-x 2 postgres postgres 3 Jan 12 14:23 pg_stat_tmp drwxr-xr-x 2 postgres postgres 3 Dec 2 12:23 pg_subtrans drwxr-xr-x 2 postgres postgres 2 Dec 2 12:23 pg_tblspc drwxr-xr-x 2 postgres postgres 2 Dec 2 12:23 pg_twophase lrwxrwxrwx 1 postgres postgres 30 Dec 19 11:13 pg_xlog -rw------- 1 postgres postgres 17915 Mar 3 21:52 postgresql.conf -rw------- 1 postgres postgres 63 Jan 12 14:23 postmaster.opts -rw------- 1 postgres postgres 54 Jan 12 14:23 postmaster.pid -rw-r--r-- 1 postgres postgres 336 Dec 20 16:14 recovery.conf Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Get Off To A Good Start So What Do You Get? -rw------- 1 postgres postgres 4 Dec 2 12:23 PG_VERSION drwxr-xr-x 6 postgres postgres 6 Dec 2 13:41 base drwxr-xr-x 2 postgres postgres 42 Jan 28 17:49 global drwxr-xr-x 2 postgres postgres 3 Dec 2 12:23 pg_clog -rw------- 1 postgres postgres 3938 Dec 19 11:25 pg_hba.conf -rw------- 1 postgres postgres 1636 Dec 19 11:25 pg_ident.conf drwxr-xr-x 3 postgres postgres 113 Mar 22 00:00 pg_log drwxr-xr-x 4 postgres postgres 4 Dec 2 12:23 pg_multixact drwxr-xr-x 2 postgres postgres 3 Jan 12 14:23 pg_notify drwxr-xr-x 2 postgres postgres 3 Jan 12 14:23 pg_stat_tmp drwxr-xr-x 2 postgres postgres 3 Dec 2 12:23 pg_subtrans drwxr-xr-x 2 postgres postgres 2 Dec 2 12:23 pg_tblspc drwxr-xr-x 2 postgres postgres 2 Dec 2 12:23 pg_twophase lrwxrwxrwx 1 postgres postgres 30 Dec 19 11:13 pg_xlog -rw------- 1 postgres postgres 17915 Mar 3 21:52 postgresql.conf -rw------- 1 postgres postgres 63 Jan 12 14:23 postmaster.opts -rw------- 1 postgres postgres 54 Jan 12 14:23 postmaster.pid -rw-r--r-- 1 postgres postgres 336 Dec 20 16:14 recovery.conf Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Get Off To A Good Start Configure Your Logging $PGDATA/pg_log /var/log/postgresql when in doubt... (postgresql.conf) separate disk Logging is often overlooked, but it is the first step toward troubleshooting! Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Get Off To A Good Start Configure Authentication most systems have different defaults firewalls / selinux (FATAL?) rtfm (pg_hba.conf, grant, revoke) Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Get Off To A Good Start Authentication Methods TRUST md5 IDENT Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Get Off To A Good Start /contrib Trust these more than your own code package dependent use different schemas (when able) pgcrypto Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Get Off To A Good Start Additional Supplied Modules “contrib” Trust these more than your own code package dependent use different schemas (when able) pgcrypto, pg_upgrade http://www.postgresql.org/docs/current/interactive/contrib.html Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Let’s Talk About Upgrades Thursday, March 31, 2011
Let’s Talk About Upgrades Versioning First Digit (7.4.16 -> 8.2.0) Second Digit (8.3.6 -> 8.4.0) Third Digit (9.0.0 -> 9.0.2) Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Let’s Talk About Upgrades Versioning First Digit (7.4.16 -> 8.2.0) Second Digit (8.3.6 -> 8.4.0) Third Digit (9.0.0 -> 9.0.2) Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Let’s Talk About Upgrades Achtung!! Make Backups! Read the Release Notes! Read the Commit Logs! Test! Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Let’s Talk About Upgrades pg_dump/pg_restore simple -Fc is your friend dump with new version of pg_dump pitfalls: (time, hdd) Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Let’s Talk About Upgrades the “slony” method not simple create slave on new version switchover (switch back?) pitfalls: (initial synch, compatibility) Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Let’s Talk About Upgrades pg_upgrade in place upgrades rewrites system catalog info no way to go back (fs snapshots) first official release was 9.0 Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Let’s Talk About Upgrades upgrading legacy databases < 8.2 is no longer supported (upgrade now!) pg_upgrade only works on 8.3 or greater older slony can work back to 7.3ish pre-7.3 ? Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Let’s Talk About Upgrades is it worth it? any database with significant transaction workload (read or write) >= 8.3 (vxid) most systems will want at least 8.4 (fsm) replication is nice too though... Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Let’s Talk About Upgrades Versioning First Digit (7.4.16 -> 8.2.0) Second Digit (8.3.6 -> 8.4.0) Third Digit (9.0.0 -> 9.0.2) Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Let’s Talk About Upgrades minor release versioning policy drop in replacement no feature changes bug fixes only (includes security fixes) sometimes involves extra steps also referred to as service packs, hot fixes, fix packs, etc... It is considered more dangerous to not upgrade! Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Figure Your Configure Thursday, March 31, 2011
Figure Your Configure the basics: performance shared_buffers effective_cache_size work_mem checkpoint_segments Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Figure Your Configure the basics: logging stderr/pg_log vs. syslog/eventlog log_min_duration_statement log_line_prefix (%t, %p, %d, %u, %r) Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Figure Your Configure other stuff worth looking at maintenance_work_mem max_prepared_transactions wal_buffers checkpoint_completion_target autovacuum_freeze_max_age Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Figure Your Configure the basics: replication primary: wal_level = ‘hot_standby’ primary: max_wal_senders = n slave: hot_standby = on Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Hardware For Software Thursday, March 31, 2011
Hardware for Software People cpu and you connections, autovacuum, bgwriter, stats collector, replication Multiple CPU’s work wonders, up to 32 processors http://tweakers.net Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Hardware for Software People disk does and don’ts Put WAL on it’s own disk (RAID 1) Put DATA directory on it’s own disk (RAID 10) More Spindles is Good More Controllers Even Gooder (tablespaces) Battery Backed Controllers, Write Cache Enabled Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Hardware for Software People disk don’ts NFS == Not Fully Safe! RAID 5 Beware disks / controllers that lie Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Can You Back That Up? Thursday, March 31, 2011
Can You Back That Up? types of backups logical (sql export) physical (files on disk) Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Can You Back That Up? logical backups slow to create slow to restore “pure”, no system level corruption susceptible to database level corruption pg_dump is your friend pg_dumpall for globals Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Can You Back That Up? physical backups replication / failover machine “tarball” (pitr) filesystem snapshots (pitr) Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Can You Back That Up? replication / failover node good idea, but... offsite? fragile? we’ll talk more in replication section :-) Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Can You Back That Up? “tarball” (pitr) basic idea is to copy all database files and relevant xlogs use multiple machines if able use rsync if able copy the slave if able!! Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Can You Back That Up? filesystem snapshots (pitr) basic idea is to copy all database files and relevant xlogs use multiple machines if able use rsync if able copy the slave if able!! Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Replication, Replication Thursday, March 31, 2011
Replication, Replication 9.0 “Introduced” Replication In Postgres Built In Solution Based on WAL replay True Binary Replication 1st Cut, Still A Little Rough Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Replication, Replication However, There Are Lots Of Alternatives! Most Are Trigger Based: Slony, Bucardo, Londiste, RubyRep, etc... Statement Based (PGPoolII) Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Replication, Replication Quick Rundown: Slony Trigger Based Asynchronous Master -> (Multiple, Cascading) Slave Switchover “A Pain To Work With” Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Replication, Replication Quick Rundown: Londiste Trigger Based Asynchronous Master -> (Multiple) Slave Switchover “I Don’t Like Slony” Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Replication, Replication Quick Rundown: Bucardo Trigger Based Asynchronous Master -> (Multiple) Slave Master -> Master “Small Community” Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Replication, Replication Quick Rundown: Ruby Rep Trigger Based Asynchronous Master -> Slave Master -> Master “Huh?” Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Replication, Replication Determine Your Needs: Failover Simple 2 Server Setup? 9.0 Streaming Replication Multiple Slaves? Slony Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Replication, Replication Determine Your Needs: Switchover Master -> Slave Slony Londiste Master -> Master Bucardo Ruby Rep Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Replication, Replication Determine Your Needs: Single Read Slave Postgres Streaming Replication no changes to the slave Any of the Trigger Based Solutions Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Replication, Replication Determine Your Needs: Multiple Read Slaves Slony (Cascading) Bucardo (Split) Londiste (meh) Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Replication, Replication A Note On Scaling Writes Scaling Writes Is A Hard Problem Bucardo/RubyRep Help IF CPU Bound If you are bound on write i/o, you probably need to federate your database Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Query Your Queries Thursday, March 31, 2011
Query Your Queries finding slow queries: log_min_duration_statement -1, 0, n superuser only alter user LOG: duration: 5005.273ms statement: select pg_sleep(5); Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Query Your Queries finding slow queries: log analyzers command line tools i/o load generate reports cpu load archive reports Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Query Your Queries finding slow queries: log analyzers pqa pgfouine maatkit (percona) pgsi (endpoint) Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Query Your Queries fixing slow queries: explain analyze universal tool good for specific queries “explain” for large queries could be it’s own talk Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Query Your Queries fixing slow queries: explain analyze universal tool good for specific queries “explain” for large queries could be it’s own talk http://wiki.postgresql.org/Using_EXPLAIN Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Query Your Queries fixing slow queries: indexing (basic) use explain to find large sequential reads use pg_stat_* tables to find numerous reads use btree (gist/gin) dual column vs. single column Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Query Your Queries fixing slow queries: indexing (partial) restrain index to rows that matter can give significant speed improvements where clause of index should match where clause of query create index address_ba_part_idx on address (district) where district = ‘Buenos Aires’; Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Query Your Queries fixing slow queries: indexing (partial) restrain index to rows that matter can give significant speed improvements where clause of index should match where clause of query create index customer_active_part_idx on customer (customer_id) where active is true; Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Query Your Queries fixing slow queries: indexing (functional) some people prefer to call these expressional indexes Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Query Your Queries fixing slow queries: indexing (expressional) push expensive functions into your index system sees just WHERE indexedcolumn = ‘constant’ expression of index should match expression of queries narrow scope, but nice gains create unique index one_true_email_xidx on customer (lower(email)); Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Query Your Queries fixing slow queries: indexing (expressional) push expensive functions into your index system sees just WHERE indexedcolumn = ‘constant’ expression of index should match expression of queries narrow scope, but nice gains create index fullname_xidx on customer ((firstname||’ ‘||lastname)); Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Query Your Queries fixing slow queries: indexing (expressional) push expensive functions into your index system sees just WHERE indexedcolumn = ‘constant’ expression of index should match expression of queries narrow scope, but nice gains create index fullname_xidx on customer ((firstname||’ ‘||lastname)); Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Other Stuff I Should Mention Thursday, March 31, 2011
Other Stuff I Should Mention pgcrypto cryptography /contrib (export issues) md5, sha1, blowfish, many more Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Other Stuff I Should Mention dblink pgsql -> pgsql connections /contrib (still under development) can have performance issues on large queries make it live in it’s own schema may be replaced by foreign data wrappers Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Other Stuff I Should Mention *-link heterogenous connections for postgresql db specific and db independent options any pl/u language can implement this similar performance issue as dblink dblink-tds, dbi-link, oralink, odbclink Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Other Stuff I Should Mention autonomous logging tool persistent logging for postgresql functions built on top of dblink make it live in it’s own schema https://labs.omniti.com/trac/pgtreats Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Other Stuff I Should Mention check_postgres nagios oriented monitoring script comprehensive list of items for alterting adapted for other systems, some trending http://bucardo.org/check_postgres Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Other Stuff I Should Mention phpPgAdmin web based gui for postgresql remote administration of multiple servers implements much of postgresql functionality http://phppgadmin.sourceforge.net/ Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Other Stuff I Should Mention omnipitr advanced WAL file management scripts pitr archiving, pitr restore complete filesystem based backup can create backups from a slave server https://labs.omniti.com/trac/pgtreats Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Other Stuff I Should Mention books PostgreSQL 9.0 High Performance, by Greg Smith PostgreSQL 9.0 Admin Cookbook, by Simon Riggs Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Other Stuff I Should Mention books PostgreSQL 9.0 High Performance, by Greg Smith PostgreSQL 9.0 Admin Cookbook, by Simon Riggs Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Other Stuff I Should Mention we’re hiring :-) Database Operations Big Systems High Volume Workloads Keggerator! Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
Other Stuff I Should Mention we’re hiring :-) Database Operations Big Systems High Volume Workloads Keggerator! Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
El Fin Thanks For Coming! Thursday, March 31, 2011

More Related Content

What's hot (20)

PDF
Postgresql tutorial
Ashoka Vanjare
 
PPTX
Les Base de Données NOSQL -Presentation -
IliasAEA
 
PDF
5 Steps to PostgreSQL Performance
Command Prompt., Inc
 
PDF
Docker volume
MyoungSu Shin
 
PDF
MySQL: Indexing for Better Performance
jkeriaki
 
PPTX
Getting started with postgresql
botsplash.com
 
PDF
An introduction to MongoDB
Universidade de São Paulo
 
PDF
Introduction aux bases de données NoSQL
Antoine Augusti
 
PPTX
PostgreSQL- An Introduction
Smita Prasad
 
PDF
Hadoop Hbase - Introduction
Blandine Larbret
 
PDF
MongoDB et Elasticsearch, meilleurs ennemis ?
Sébastien Prunier
 
PDF
Linux tuning to improve PostgreSQL performance
PostgreSQL-Consulting
 
PPT
MySQL Atchitecture and Concepts
Tuyen Vuong
 
PPTX
Introduction to NoSQL
PolarSeven Pty Ltd
 
PPTX
DDD架構設計
國昭 張
 
PPTX
PL/pgSQL - An Introduction on Using Imperative Programming in PostgreSQL
Reactive.IO
 
PPTX
Introduction to PostgreSQL
Joel Brewer
 
PDF
PostgreSQL Tutorial for Beginners | Edureka
Edureka!
 
PDF
İleri Seviye T-SQL Programlama - Chapter 12
Cihan Özhan
 
PDF
Apache Storm
Edureka!
 
Postgresql tutorial
Ashoka Vanjare
 
Les Base de Données NOSQL -Presentation -
IliasAEA
 
5 Steps to PostgreSQL Performance
Command Prompt., Inc
 
Docker volume
MyoungSu Shin
 
MySQL: Indexing for Better Performance
jkeriaki
 
Getting started with postgresql
botsplash.com
 
An introduction to MongoDB
Universidade de São Paulo
 
Introduction aux bases de données NoSQL
Antoine Augusti
 
PostgreSQL- An Introduction
Smita Prasad
 
Hadoop Hbase - Introduction
Blandine Larbret
 
MongoDB et Elasticsearch, meilleurs ennemis ?
Sébastien Prunier
 
Linux tuning to improve PostgreSQL performance
PostgreSQL-Consulting
 
MySQL Atchitecture and Concepts
Tuyen Vuong
 
Introduction to NoSQL
PolarSeven Pty Ltd
 
DDD架構設計
國昭 張
 
PL/pgSQL - An Introduction on Using Imperative Programming in PostgreSQL
Reactive.IO
 
Introduction to PostgreSQL
Joel Brewer
 
PostgreSQL Tutorial for Beginners | Edureka
Edureka!
 
İleri Seviye T-SQL Programlama - Chapter 12
Cihan Özhan
 
Apache Storm
Edureka!
 

Viewers also liked (20)

PDF
The Essential PostgreSQL.conf
Robert Treat
 
PDF
Out of the Box Replication in Postgres 9.4(PgCon)
Denish Patel
 
PDF
PostgreSQL Disaster Recovery with Barman
Gabriele Bartolini
 
PDF
Scaling postgres
Denish Patel
 
PDF
Inaugural Addresses
Booz Allen Hamilton
 
PDF
Teaching Students with Emojis, Emoticons, & Textspeak
Shelly Sanchez Terrell
 
PDF
UX, ethnography and possibilities: for Libraries, Museums and Archives
Ned Potter
 
PDF
Hype vs. Reality: The AI Explainer
Luminary Labs
 
PDF
Designing Teams for Emerging Challenges
Aaron Irizarry
 
PDF
Visual Design with Data
Seth Familian
 
PDF
3 Things Every Sales Team Needs to Be Thinking About in 2017
Drift
 
PDF
How to Become a Thought Leader in Your Niche
Leslie Samuel
 
PDF
Vert.x
Matt Stine
 
KEY
High performance network programming on the jvm oscon 2012
Erik Onnen
 
PDF
Advanced WAL File Management With OmniPITR
Robert Treat
 
PDF
Think_your_Postgres_backups_and_recovery_are_safe_lets_talk.pptx
Payal Singh
 
PDF
Managing PostgreSQL with PgCenter
Alexey Lesovsky
 
PDF
Nine Pages You Should Optimize on Your Blog and How
Leslie Samuel
 
PPTX
Backups
Payal Singh
 
PDF
Analytics Trends 2016: The next evolution
Deloitte United States
 
The Essential PostgreSQL.conf
Robert Treat
 
Out of the Box Replication in Postgres 9.4(PgCon)
Denish Patel
 
PostgreSQL Disaster Recovery with Barman
Gabriele Bartolini
 
Scaling postgres
Denish Patel
 
Inaugural Addresses
Booz Allen Hamilton
 
Teaching Students with Emojis, Emoticons, & Textspeak
Shelly Sanchez Terrell
 
UX, ethnography and possibilities: for Libraries, Museums and Archives
Ned Potter
 
Hype vs. Reality: The AI Explainer
Luminary Labs
 
Designing Teams for Emerging Challenges
Aaron Irizarry
 
Visual Design with Data
Seth Familian
 
3 Things Every Sales Team Needs to Be Thinking About in 2017
Drift
 
How to Become a Thought Leader in Your Niche
Leslie Samuel
 
Vert.x
Matt Stine
 
High performance network programming on the jvm oscon 2012
Erik Onnen
 
Advanced WAL File Management With OmniPITR
Robert Treat
 
Think_your_Postgres_backups_and_recovery_are_safe_lets_talk.pptx
Payal Singh
 
Managing PostgreSQL with PgCenter
Alexey Lesovsky
 
Nine Pages You Should Optimize on Your Blog and How
Leslie Samuel
 
Backups
Payal Singh
 
Analytics Trends 2016: The next evolution
Deloitte United States
 
Ad

Similar to Pro Postgres 9 (20)

PDF
PostgreSQL Development Today: 9.0
PostgreSQL Experts, Inc.
 
PDF
PostgreSQL Conference: West 08
Joshua Drake
 
ODP
Pro PostgreSQL, OSCon 2008
Robert Treat
 
PDF
Get to know PostgreSQL!
Oddbjørn Steffensen
 
PDF
Whats wrong with postgres | PGConf EU 2019 | Craig Kerstiens
Citus Data
 
PDF
Intro to Postgres 9 Tutorial
Robert Treat
 
PDF
Quality Assurance in PostgreSQL
Aleksander Alekseev
 
PDF
Exploring Postgres with Bruce Momjian
EDB
 
PDF
Letters from the open source trenches - Postgres community
Selena Deckelmann
 
PDF
The Accidental DBA
PostgreSQL Experts, Inc.
 
PDF
How to contribute PostgreSQL
Hari Babu kommi
 
PDF
PostgreSQL_ Up and Running_ A Practical Guide to the Advanced Open Source Dat...
MinhLeNguyenAnh2
 
PDF
A Guide To PostgreSQL 9.0
Robert Treat
 
PDF
Check Please!
Robert Treat
 
PDF
0292-introduction-postgresql.pdf
Mustafa Keskin
 
PDF
Mastering PostgreSQL Administration
Command Prompt., Inc
 
PDF
Mastering PostgreSQL Administration
EDB
 
PDF
Scaling with Postgres (Robert Treat)
Ontico
 
PDF
Open Source Contributions to Postgres: The Basics POSETTE 2024
ElizabethGarrettChri
 
PDF
PostgreSQL Server Programming Second Edition Usama Dar Hannu Krosing Jim Mlod...
trddarvai
 
PostgreSQL Development Today: 9.0
PostgreSQL Experts, Inc.
 
PostgreSQL Conference: West 08
Joshua Drake
 
Pro PostgreSQL, OSCon 2008
Robert Treat
 
Get to know PostgreSQL!
Oddbjørn Steffensen
 
Whats wrong with postgres | PGConf EU 2019 | Craig Kerstiens
Citus Data
 
Intro to Postgres 9 Tutorial
Robert Treat
 
Quality Assurance in PostgreSQL
Aleksander Alekseev
 
Exploring Postgres with Bruce Momjian
EDB
 
Letters from the open source trenches - Postgres community
Selena Deckelmann
 
The Accidental DBA
PostgreSQL Experts, Inc.
 
How to contribute PostgreSQL
Hari Babu kommi
 
PostgreSQL_ Up and Running_ A Practical Guide to the Advanced Open Source Dat...
MinhLeNguyenAnh2
 
A Guide To PostgreSQL 9.0
Robert Treat
 
Check Please!
Robert Treat
 
0292-introduction-postgresql.pdf
Mustafa Keskin
 
Mastering PostgreSQL Administration
Command Prompt., Inc
 
Mastering PostgreSQL Administration
EDB
 
Scaling with Postgres (Robert Treat)
Ontico
 
Open Source Contributions to Postgres: The Basics POSETTE 2024
ElizabethGarrettChri
 
PostgreSQL Server Programming Second Edition Usama Dar Hannu Krosing Jim Mlod...
trddarvai
 
Ad

More from Robert Treat (20)

PDF
Advanced Int->Bigint Conversions
Robert Treat
 
PDF
Explaining Explain
Robert Treat
 
PDF
the-lost-art-of-plpgsql
Robert Treat
 
PDF
Managing Chaos In Production: Testing vs Monitoring
Robert Treat
 
PDF
Managing Databases In A DevOps Environment 2016
Robert Treat
 
PDF
Less Alarming Alerts - SRECon 2016
Robert Treat
 
PDF
What Ops Can Learn From Design
Robert Treat
 
PDF
Postgres 9.4 First Look
Robert Treat
 
PDF
Less Alarming Alerts!
Robert Treat
 
PDF
Past, Present, and Pachyderm - All Things Open - 2013
Robert Treat
 
PDF
Big Bad "Upgraded" Postgres
Robert Treat
 
PDF
Managing Databases In A DevOps Environment
Robert Treat
 
PDF
Scaling with Postgres (Highload++ 2010)
Robert Treat
 
PDF
Database Scalability Patterns
Robert Treat
 
PPT
Scaling With Postgres
Robert Treat
 
PDF
Intro to Postgres 8.4 Tutorial
Robert Treat
 
PDF
Intro to pl/PHP Oscon2007
Robert Treat
 
PDF
The Essential postgresql.conf
Robert Treat
 
PDF
PostgreSQL Partitioning, PGCon 2007
Robert Treat
 
PDF
Database Anti Patterns
Robert Treat
 
Advanced Int->Bigint Conversions
Robert Treat
 
Explaining Explain
Robert Treat
 
the-lost-art-of-plpgsql
Robert Treat
 
Managing Chaos In Production: Testing vs Monitoring
Robert Treat
 
Managing Databases In A DevOps Environment 2016
Robert Treat
 
Less Alarming Alerts - SRECon 2016
Robert Treat
 
What Ops Can Learn From Design
Robert Treat
 
Postgres 9.4 First Look
Robert Treat
 
Less Alarming Alerts!
Robert Treat
 
Past, Present, and Pachyderm - All Things Open - 2013
Robert Treat
 
Big Bad "Upgraded" Postgres
Robert Treat
 
Managing Databases In A DevOps Environment
Robert Treat
 
Scaling with Postgres (Highload++ 2010)
Robert Treat
 
Database Scalability Patterns
Robert Treat
 
Scaling With Postgres
Robert Treat
 
Intro to Postgres 8.4 Tutorial
Robert Treat
 
Intro to pl/PHP Oscon2007
Robert Treat
 
The Essential postgresql.conf
Robert Treat
 
PostgreSQL Partitioning, PGCon 2007
Robert Treat
 
Database Anti Patterns
Robert Treat
 

Recently uploaded (20)

PPT
Ericsson LTE presentation SEMINAR 2010.ppt
npat3
 
PDF
“NPU IP Hardware Shaped Through Software and Use-case Analysis,” a Presentati...
Edge AI and Vision Alliance
 
PDF
CIFDAQ Market Wrap for the week of 4th July 2025
CIFDAQ
 
PDF
Future-Proof or Fall Behind? 10 Tech Trends You Can’t Afford to Ignore in 2025
DIGITALCONFEX
 
PDF
Transforming Utility Networks: Large-scale Data Migrations with FME
Safe Software
 
PDF
“Squinting Vision Pipelines: Detecting and Correcting Errors in Vision Models...
Edge AI and Vision Alliance
 
PDF
POV_ Why Enterprises Need to Find Value in ZERO.pdf
darshakparmar
 
PDF
Go Concurrency Real-World Patterns, Pitfalls, and Playground Battles.pdf
Emily Achieng
 
PDF
UPDF - AI PDF Editor & Converter Key Features
DealFuel
 
PDF
Agentic AI lifecycle for Enterprise Hyper-Automation
Debmalya Biswas
 
PDF
SIZING YOUR AIR CONDITIONER---A PRACTICAL GUIDE.pdf
Muhammad Rizwan Akram
 
PPTX
Seamless Tech Experiences Showcasing Cross-Platform App Design.pptx
presentifyai
 
PDF
How do you fast track Agentic automation use cases discovery?
DianaGray10
 
PDF
The 2025 InfraRed Report - Redpoint Ventures
Razin Mustafiz
 
PPTX
Designing_the_Future_AI_Driven_Product_Experiences_Across_Devices.pptx
presentifyai
 
PPTX
COMPARISON OF RASTER ANALYSIS TOOLS OF QGIS AND ARCGIS
Sharanya Sarkar
 
DOCX
Cryptography Quiz: test your knowledge of this important security concept.
Rajni Bhardwaj Grover
 
PDF
Newgen 2022-Forrester Newgen TEI_13 05 2022-The-Total-Economic-Impact-Newgen-...
darshakparmar
 
PDF
Transcript: Book industry state of the nation 2025 - Tech Forum 2025
BookNet Canada
 
PDF
LOOPS in C Programming Language - Technology
RishabhDwivedi43
 
Ericsson LTE presentation SEMINAR 2010.ppt
npat3
 
“NPU IP Hardware Shaped Through Software and Use-case Analysis,” a Presentati...
Edge AI and Vision Alliance
 
CIFDAQ Market Wrap for the week of 4th July 2025
CIFDAQ
 
Future-Proof or Fall Behind? 10 Tech Trends You Can’t Afford to Ignore in 2025
DIGITALCONFEX
 
Transforming Utility Networks: Large-scale Data Migrations with FME
Safe Software
 
“Squinting Vision Pipelines: Detecting and Correcting Errors in Vision Models...
Edge AI and Vision Alliance
 
POV_ Why Enterprises Need to Find Value in ZERO.pdf
darshakparmar
 
Go Concurrency Real-World Patterns, Pitfalls, and Playground Battles.pdf
Emily Achieng
 
UPDF - AI PDF Editor & Converter Key Features
DealFuel
 
Agentic AI lifecycle for Enterprise Hyper-Automation
Debmalya Biswas
 
SIZING YOUR AIR CONDITIONER---A PRACTICAL GUIDE.pdf
Muhammad Rizwan Akram
 
Seamless Tech Experiences Showcasing Cross-Platform App Design.pptx
presentifyai
 
How do you fast track Agentic automation use cases discovery?
DianaGray10
 
The 2025 InfraRed Report - Redpoint Ventures
Razin Mustafiz
 
Designing_the_Future_AI_Driven_Product_Experiences_Across_Devices.pptx
presentifyai
 
COMPARISON OF RASTER ANALYSIS TOOLS OF QGIS AND ARCGIS
Sharanya Sarkar
 
Cryptography Quiz: test your knowledge of this important security concept.
Rajni Bhardwaj Grover
 
Newgen 2022-Forrester Newgen TEI_13 05 2022-The-Total-Economic-Impact-Newgen-...
darshakparmar
 
Transcript: Book industry state of the nation 2025 - Tech Forum 2025
BookNet Canada
 
LOOPS in C Programming Language - Technology
RishabhDwivedi43
 

Pro Postgres 9

  • 1. Pro Postgres 9 Robert Treat, JDCon East 2011 Thursday, March 31, 2011
  • 2. Who Am I? (Why Listen To Me) PostgreSQL User Since 6.5.x DBA of High Traffic / Large PostgreSQL Instances Long Time Contributor to PostgreSQL Project Contribute / Maintain Several Open Source Projects Co-Author Beginning PHP & PostgreSQL 8 (Apress) I like to call it “Postgres” Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 3. Outline What you need to know about the project Backups Getting Started Replication Upgrading Query Tuning Configuring Your Server Other stuff you should know about Hardware Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 4. Know Your Way Around The Project Thursday, March 31, 2011
  • 5. Know Your Way Around The Project www.postgresql.org downloads security alerts documentation wiki bug reports support companies rss -> news - events - version Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 6. Know Your Way Around The Project planet.postgresql.org project news helpful tips / examples community news Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 7. Know Your Way Around The Project www.pgfoundry.org URI Type SkyTools modules Npgsql programs pg_bulkload resources plpgsql-debugger pagila, sample database Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 8. Know Your Way Around The Project git.postgresql.org check_postgres modules edb-installers programs website code resources json type works in progress pg_top user repo’s see also github.com Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 9. Know Your Way Around The Project archives.postgresql.org mailing list archives back to 1997 full text search via postgresql FTS keyword search suggestions lists for users, developers, regional, user groups Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 10. Know Your Way Around The Project #postgresql irc.freenode.net real time help pg_docbot - ??help Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 11. Know Your Way Around The Project project management core team committers -hackers road map web team Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 12. Get Off To A Good Start Thursday, March 31, 2011
  • 13. Get Off To A Good Start Use Package Management Consistent Standardized Simple Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 14. Get Off To A Good Start Use Package Management Different across systems Upgrades can be a problem Trust your packager? Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 15. Get Off To A Good Start Use Package Management Different across systems Upgrades can be a problem Trust your packager? Don’t Be Afraid To Roll Your Own Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 16. Get Off To A Good Start A Note About Versions Just playing? Default packages probably OK, (min 8.2) Any production level work should start on 9.0 Any project not due to launch for 6 months, 9.1 Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 17. Get Off To A Good Start So What Do You Get? pg_control version number: 903 Catalog version number: 201008051 Database system identifier: 5546136951888795324 Database cluster state: in archive recovery pg_control last modified: Tue Mar 22 11:50:24 2011 Latest checkpoint location: 7C/1D000058 Prior checkpoint location: 7C/1C000058 Latest checkpoint's REDO location: 7C/1D000020 Latest checkpoint's TimeLineID: 1 Latest checkpoint's NextXID: 0/4924 Latest checkpoint's NextOID: 16608 Latest checkpoint's NextMultiXactId: 1 Latest checkpoint's NextMultiOffset: 0 Latest checkpoint's oldestXID: 654 Latest checkpoint's oldestXID's DB: 1 pg_controldata Latest checkpoint's oldestActiveXID: Time of latest checkpoint: 4924 Tue Mar 22 11:46:19 2011 Minimum recovery ending location: 7C/16000000 Backup start location: 0/0 Current wal_level setting: hot_standby Current max_connections setting: 500 Current max_prepared_xacts setting: 0 Current max_locks_per_xact setting: 64 Maximum data alignment: 8 Database block size: 8192 Blocks per segment of large relation: 131072 WAL block size: 8192 Bytes per WAL segment: 16777216 Maximum length of identifiers: 64 Maximum columns in an index: 32 Maximum size of a TOAST chunk: 1996 Date/time type storage: 64-bit integers Float4 argument passing: by value Float8 argument passing: by value Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 18. Get Off To A Good Start So What Do You Get? -rw------- 1 postgres postgres 4 Dec 2 12:23 PG_VERSION drwxr-xr-x 6 postgres postgres 6 Dec 2 13:41 base drwxr-xr-x 2 postgres postgres 42 Jan 28 17:49 global drwxr-xr-x 2 postgres postgres 3 Dec 2 12:23 pg_clog -rw------- 1 postgres postgres 3938 Dec 19 11:25 pg_hba.conf -rw------- 1 postgres postgres 1636 Dec 19 11:25 pg_ident.conf drwxr-xr-x 3 postgres postgres 113 Mar 22 00:00 pg_log drwxr-xr-x 4 postgres postgres 4 Dec 2 12:23 pg_multixact drwxr-xr-x 2 postgres postgres 3 Jan 12 14:23 pg_notify drwxr-xr-x 2 postgres postgres 3 Jan 12 14:23 pg_stat_tmp drwxr-xr-x 2 postgres postgres 3 Dec 2 12:23 pg_subtrans drwxr-xr-x 2 postgres postgres 2 Dec 2 12:23 pg_tblspc drwxr-xr-x 2 postgres postgres 2 Dec 2 12:23 pg_twophase lrwxrwxrwx 1 postgres postgres 30 Dec 19 11:13 pg_xlog -rw------- 1 postgres postgres 17915 Mar 3 21:52 postgresql.conf -rw------- 1 postgres postgres 63 Jan 12 14:23 postmaster.opts -rw------- 1 postgres postgres 54 Jan 12 14:23 postmaster.pid -rw-r--r-- 1 postgres postgres 336 Dec 20 16:14 recovery.conf Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 19. Get Off To A Good Start So What Do You Get? -rw------- 1 postgres postgres 4 Dec 2 12:23 PG_VERSION drwxr-xr-x 6 postgres postgres 6 Dec 2 13:41 base drwxr-xr-x 2 postgres postgres 42 Jan 28 17:49 global drwxr-xr-x 2 postgres postgres 3 Dec 2 12:23 pg_clog -rw------- 1 postgres postgres 3938 Dec 19 11:25 pg_hba.conf -rw------- 1 postgres postgres 1636 Dec 19 11:25 pg_ident.conf drwxr-xr-x 3 postgres postgres 113 Mar 22 00:00 pg_log drwxr-xr-x 4 postgres postgres 4 Dec 2 12:23 pg_multixact drwxr-xr-x 2 postgres postgres 3 Jan 12 14:23 pg_notify drwxr-xr-x 2 postgres postgres 3 Jan 12 14:23 pg_stat_tmp drwxr-xr-x 2 postgres postgres 3 Dec 2 12:23 pg_subtrans drwxr-xr-x 2 postgres postgres 2 Dec 2 12:23 pg_tblspc drwxr-xr-x 2 postgres postgres 2 Dec 2 12:23 pg_twophase lrwxrwxrwx 1 postgres postgres 30 Dec 19 11:13 pg_xlog -rw------- 1 postgres postgres 17915 Mar 3 21:52 postgresql.conf -rw------- 1 postgres postgres 63 Jan 12 14:23 postmaster.opts -rw------- 1 postgres postgres 54 Jan 12 14:23 postmaster.pid -rw-r--r-- 1 postgres postgres 336 Dec 20 16:14 recovery.conf Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 20. Get Off To A Good Start So What Do You Get? -rw------- 1 postgres postgres 4 Dec 2 12:23 PG_VERSION drwxr-xr-x 6 postgres postgres 6 Dec 2 13:41 base drwxr-xr-x 2 postgres postgres 42 Jan 28 17:49 global drwxr-xr-x 2 postgres postgres 3 Dec 2 12:23 pg_clog -rw------- 1 postgres postgres 3938 Dec 19 11:25 pg_hba.conf -rw------- 1 postgres postgres 1636 Dec 19 11:25 pg_ident.conf drwxr-xr-x 3 postgres postgres 113 Mar 22 00:00 pg_log drwxr-xr-x 4 postgres postgres 4 Dec 2 12:23 pg_multixact drwxr-xr-x 2 postgres postgres 3 Jan 12 14:23 pg_notify drwxr-xr-x 2 postgres postgres 3 Jan 12 14:23 pg_stat_tmp drwxr-xr-x 2 postgres postgres 3 Dec 2 12:23 pg_subtrans drwxr-xr-x 2 postgres postgres 2 Dec 2 12:23 pg_tblspc drwxr-xr-x 2 postgres postgres 2 Dec 2 12:23 pg_twophase lrwxrwxrwx 1 postgres postgres 30 Dec 19 11:13 pg_xlog -rw------- 1 postgres postgres 17915 Mar 3 21:52 postgresql.conf -rw------- 1 postgres postgres 63 Jan 12 14:23 postmaster.opts -rw------- 1 postgres postgres 54 Jan 12 14:23 postmaster.pid -rw-r--r-- 1 postgres postgres 336 Dec 20 16:14 recovery.conf Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 21. Get Off To A Good Start Configure Your Logging $PGDATA/pg_log /var/log/postgresql when in doubt... (postgresql.conf) separate disk Logging is often overlooked, but it is the first step toward troubleshooting! Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 22. Get Off To A Good Start Configure Authentication most systems have different defaults firewalls / selinux (FATAL?) rtfm (pg_hba.conf, grant, revoke) Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 23. Get Off To A Good Start Authentication Methods TRUST md5 IDENT Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 24. Get Off To A Good Start /contrib Trust these more than your own code package dependent use different schemas (when able) pgcrypto Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 25. Get Off To A Good Start Additional Supplied Modules “contrib” Trust these more than your own code package dependent use different schemas (when able) pgcrypto, pg_upgrade http://www.postgresql.org/docs/current/interactive/contrib.html Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 26. Let’s Talk About Upgrades Thursday, March 31, 2011
  • 27. Let’s Talk About Upgrades Versioning First Digit (7.4.16 -> 8.2.0) Second Digit (8.3.6 -> 8.4.0) Third Digit (9.0.0 -> 9.0.2) Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 28. Let’s Talk About Upgrades Versioning First Digit (7.4.16 -> 8.2.0) Second Digit (8.3.6 -> 8.4.0) Third Digit (9.0.0 -> 9.0.2) Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 29. Let’s Talk About Upgrades Achtung!! Make Backups! Read the Release Notes! Read the Commit Logs! Test! Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 30. Let’s Talk About Upgrades pg_dump/pg_restore simple -Fc is your friend dump with new version of pg_dump pitfalls: (time, hdd) Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 31. Let’s Talk About Upgrades the “slony” method not simple create slave on new version switchover (switch back?) pitfalls: (initial synch, compatibility) Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 32. Let’s Talk About Upgrades pg_upgrade in place upgrades rewrites system catalog info no way to go back (fs snapshots) first official release was 9.0 Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 33. Let’s Talk About Upgrades upgrading legacy databases < 8.2 is no longer supported (upgrade now!) pg_upgrade only works on 8.3 or greater older slony can work back to 7.3ish pre-7.3 ? Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 34. Let’s Talk About Upgrades is it worth it? any database with significant transaction workload (read or write) >= 8.3 (vxid) most systems will want at least 8.4 (fsm) replication is nice too though... Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 35. Let’s Talk About Upgrades Versioning First Digit (7.4.16 -> 8.2.0) Second Digit (8.3.6 -> 8.4.0) Third Digit (9.0.0 -> 9.0.2) Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 36. Let’s Talk About Upgrades minor release versioning policy drop in replacement no feature changes bug fixes only (includes security fixes) sometimes involves extra steps also referred to as service packs, hot fixes, fix packs, etc... It is considered more dangerous to not upgrade! Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 38. Figure Your Configure the basics: performance shared_buffers effective_cache_size work_mem checkpoint_segments Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 39. Figure Your Configure the basics: logging stderr/pg_log vs. syslog/eventlog log_min_duration_statement log_line_prefix (%t, %p, %d, %u, %r) Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 40. Figure Your Configure other stuff worth looking at maintenance_work_mem max_prepared_transactions wal_buffers checkpoint_completion_target autovacuum_freeze_max_age Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 41. Figure Your Configure the basics: replication primary: wal_level = ‘hot_standby’ primary: max_wal_senders = n slave: hot_standby = on Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 43. Hardware for Software People cpu and you connections, autovacuum, bgwriter, stats collector, replication Multiple CPU’s work wonders, up to 32 processors http://tweakers.net Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 44. Hardware for Software People disk does and don’ts Put WAL on it’s own disk (RAID 1) Put DATA directory on it’s own disk (RAID 10) More Spindles is Good More Controllers Even Gooder (tablespaces) Battery Backed Controllers, Write Cache Enabled Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 45. Hardware for Software People disk don’ts NFS == Not Fully Safe! RAID 5 Beware disks / controllers that lie Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 46. Can You Back That Up? Thursday, March 31, 2011
  • 47. Can You Back That Up? types of backups logical (sql export) physical (files on disk) Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 48. Can You Back That Up? logical backups slow to create slow to restore “pure”, no system level corruption susceptible to database level corruption pg_dump is your friend pg_dumpall for globals Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 49. Can You Back That Up? physical backups replication / failover machine “tarball” (pitr) filesystem snapshots (pitr) Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 50. Can You Back That Up? replication / failover node good idea, but... offsite? fragile? we’ll talk more in replication section :-) Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 51. Can You Back That Up? “tarball” (pitr) basic idea is to copy all database files and relevant xlogs use multiple machines if able use rsync if able copy the slave if able!! Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 52. Can You Back That Up? filesystem snapshots (pitr) basic idea is to copy all database files and relevant xlogs use multiple machines if able use rsync if able copy the slave if able!! Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 54. Replication, Replication 9.0 “Introduced” Replication In Postgres Built In Solution Based on WAL replay True Binary Replication 1st Cut, Still A Little Rough Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 55. Replication, Replication However, There Are Lots Of Alternatives! Most Are Trigger Based: Slony, Bucardo, Londiste, RubyRep, etc... Statement Based (PGPoolII) Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 56. Replication, Replication Quick Rundown: Slony Trigger Based Asynchronous Master -> (Multiple, Cascading) Slave Switchover “A Pain To Work With” Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 57. Replication, Replication Quick Rundown: Londiste Trigger Based Asynchronous Master -> (Multiple) Slave Switchover “I Don’t Like Slony” Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 58. Replication, Replication Quick Rundown: Bucardo Trigger Based Asynchronous Master -> (Multiple) Slave Master -> Master “Small Community” Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 59. Replication, Replication Quick Rundown: Ruby Rep Trigger Based Asynchronous Master -> Slave Master -> Master “Huh?” Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 60. Replication, Replication Determine Your Needs: Failover Simple 2 Server Setup? 9.0 Streaming Replication Multiple Slaves? Slony Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 61. Replication, Replication Determine Your Needs: Switchover Master -> Slave Slony Londiste Master -> Master Bucardo Ruby Rep Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 62. Replication, Replication Determine Your Needs: Single Read Slave Postgres Streaming Replication no changes to the slave Any of the Trigger Based Solutions Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 63. Replication, Replication Determine Your Needs: Multiple Read Slaves Slony (Cascading) Bucardo (Split) Londiste (meh) Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 64. Replication, Replication A Note On Scaling Writes Scaling Writes Is A Hard Problem Bucardo/RubyRep Help IF CPU Bound If you are bound on write i/o, you probably need to federate your database Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 66. Query Your Queries finding slow queries: log_min_duration_statement -1, 0, n superuser only alter user LOG: duration: 5005.273ms statement: select pg_sleep(5); Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 67. Query Your Queries finding slow queries: log analyzers command line tools i/o load generate reports cpu load archive reports Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 68. Query Your Queries finding slow queries: log analyzers pqa pgfouine maatkit (percona) pgsi (endpoint) Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 69. Query Your Queries fixing slow queries: explain analyze universal tool good for specific queries “explain” for large queries could be it’s own talk Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 70. Query Your Queries fixing slow queries: explain analyze universal tool good for specific queries “explain” for large queries could be it’s own talk http://wiki.postgresql.org/Using_EXPLAIN Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 71. Query Your Queries fixing slow queries: indexing (basic) use explain to find large sequential reads use pg_stat_* tables to find numerous reads use btree (gist/gin) dual column vs. single column Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 72. Query Your Queries fixing slow queries: indexing (partial) restrain index to rows that matter can give significant speed improvements where clause of index should match where clause of query create index address_ba_part_idx on address (district) where district = ‘Buenos Aires’; Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 73. Query Your Queries fixing slow queries: indexing (partial) restrain index to rows that matter can give significant speed improvements where clause of index should match where clause of query create index customer_active_part_idx on customer (customer_id) where active is true; Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 74. Query Your Queries fixing slow queries: indexing (functional) some people prefer to call these expressional indexes Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 75. Query Your Queries fixing slow queries: indexing (expressional) push expensive functions into your index system sees just WHERE indexedcolumn = ‘constant’ expression of index should match expression of queries narrow scope, but nice gains create unique index one_true_email_xidx on customer (lower(email)); Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 76. Query Your Queries fixing slow queries: indexing (expressional) push expensive functions into your index system sees just WHERE indexedcolumn = ‘constant’ expression of index should match expression of queries narrow scope, but nice gains create index fullname_xidx on customer ((firstname||’ ‘||lastname)); Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 77. Query Your Queries fixing slow queries: indexing (expressional) push expensive functions into your index system sees just WHERE indexedcolumn = ‘constant’ expression of index should match expression of queries narrow scope, but nice gains create index fullname_xidx on customer ((firstname||’ ‘||lastname)); Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 78. Other Stuff I Should Mention Thursday, March 31, 2011
  • 79. Other Stuff I Should Mention pgcrypto cryptography /contrib (export issues) md5, sha1, blowfish, many more Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 80. Other Stuff I Should Mention dblink pgsql -> pgsql connections /contrib (still under development) can have performance issues on large queries make it live in it’s own schema may be replaced by foreign data wrappers Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 81. Other Stuff I Should Mention *-link heterogenous connections for postgresql db specific and db independent options any pl/u language can implement this similar performance issue as dblink dblink-tds, dbi-link, oralink, odbclink Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 82. Other Stuff I Should Mention autonomous logging tool persistent logging for postgresql functions built on top of dblink make it live in it’s own schema https://labs.omniti.com/trac/pgtreats Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 83. Other Stuff I Should Mention check_postgres nagios oriented monitoring script comprehensive list of items for alterting adapted for other systems, some trending http://bucardo.org/check_postgres Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 84. Other Stuff I Should Mention phpPgAdmin web based gui for postgresql remote administration of multiple servers implements much of postgresql functionality http://phppgadmin.sourceforge.net/ Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 85. Other Stuff I Should Mention omnipitr advanced WAL file management scripts pitr archiving, pitr restore complete filesystem based backup can create backups from a slave server https://labs.omniti.com/trac/pgtreats Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 86. Other Stuff I Should Mention books PostgreSQL 9.0 High Performance, by Greg Smith PostgreSQL 9.0 Admin Cookbook, by Simon Riggs Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 87. Other Stuff I Should Mention books PostgreSQL 9.0 High Performance, by Greg Smith PostgreSQL 9.0 Admin Cookbook, by Simon Riggs Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 88. Other Stuff I Should Mention we’re hiring :-) Database Operations Big Systems High Volume Workloads Keggerator! Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 89. Other Stuff I Should Mention we’re hiring :-) Database Operations Big Systems High Volume Workloads Keggerator! Robert Treat | Pro Postgres 9 Thursday, March 31, 2011
  • 90. El Fin Thanks For Coming! Thursday, March 31, 2011