Как PostgreSQL работает с диском
8 likes•1,982 views
The document discusses various PostgreSQL operations, including write-ahead logging (WAL) and checkpoint management. It addresses hardware considerations, RAID configurations, and settings for performance optimization related to background writer and autovacuum processes. Additionally, it covers methods for file synchronization and tuning parameters to enhance database performance.
![pg_stat_bgwriter
pgbench=# select * from pg_stat_bgwriter ;
-[ RECORD 1 ]---------+------------------------------
checkpoints_timed | 29
checkpoints_req | 13
checkpoint_write_time | 206345
checkpoint_sync_time | 9989
buffers_checkpoint | 67720
buffers_clean | 1046
maxwritten_clean | 0
buffers_backend | 48142
buffers_backend_fsync | 0
buffers_alloc | 30137
stats_reset | 2014-10-24 17:59:15.812002-04
postgres=# select pg_stat_reset_shared(’bgwriter’);
-[ RECORD 1 ]--------+-
pg_stat_reset_shared |](https://image.slidesharecdn.com/hl2014kosmodemiansky-141101010749-conversion-gate02/85/PostgreSQL-9-320.jpg)
Как PostgreSQL работает с диском
- 1. Êàê PostgreSQL ðàáîòàåò ñ äèñêîì Èëüÿ Êîñìîäåìüÿíñêèé [email protected]
- 2. Ïëàí Çà÷åì áàçå äèñê? ×òî îñîáåííîãî â ðàáîòå ñ äèñêîì ó PostgreSQL? Ïîòåíöèàëüíûå óçêèå ìåñòà Ìîíèòîðèíã äèñêîâîé íàãðóçêè Âûáîð àïïàðàòíîãî îáåñïå÷åíèÿ äëÿ ñåðâåðà ñ PostgreSQL Íàñòðîéêà
- 3. Çà÷åì áàçå äèñê? ×èòàòü ñòðàíèöû ñ äèñêà Ïèñàòü Write Ahead Log (WAL) Ñèíõðîíèçèðîâàòü WAL ñ õðàíèëèùåì (CHECKPOINT)
- 4. Çà÷åì áàçå äèñê? ×èòàòü ñòðàíèöû ñ äèñêà Ïèñàòü Write Ahead Log (WAL) Ñèíõðîíèçèðîâàòü WAL ñ õðàíèëèùåì (CHECKPOINT) Ñïåöèôèêà PostgreSQL autovacuum pg_clog tmp, äèñêîâûå ñîðòèðîâêè, õýøèðîâàíèå
- 5. Çà÷åì áàçå äèñê? ×èòàòü ñòðàíèöû ñ äèñêà Ïèñàòü Write Ahead Log (WAL) Ñèíõðîíèçèðîâàòü WAL ñ õðàíèëèùåì (CHECKPOINT) Ñïåöèôèêà PostgreSQL autovacuum pg_clog tmp, äèñêîâûå ñîðòèðîâêè, õýøèðîâàíèå
- 6. Æèçíåííûé öèêë ñòðàíèöû â PostgreSQL shared_buers l êýø ÎÑ l äèñêè
- 7. Checkpoint Äëÿ ÷åãî íóæåí? ×èñòûåñòðàíèöû ïîäíèìàþòñÿâ shared_buers, åñëè õîòü îäèí tuple èçìåíèëñÿ, ñòðàíèöà ïîìå÷àåòñÿ êàê ãðÿçíàÿ COMMIT; íå âîçâðàùàåò óïðàâëåíèÿ, ïîêà ñòðàíèöà íå çàïèñàíà â WAL Âðåìÿ îò âðåìåíè ïðîèñõîäèò CHECKPOINT: ãðÿçíûå ñòðàíèöû èç shared_buers ïèøóòñÿ íà äèñê (fsync) Ïðè áîëüøèõ çíà÷åíèÿõ shared_buers âîçíèêàþò ïðîáëåìû
- 8. Checkpoint Äèàãíîñòèêà Âñïëåñêè íà ìîíèòîðèíãå äèñêîâîé óòèëèçàöèè (iostat -d -x 1, ïîñëåäíÿÿ êîëîíêà, %util) pg_stat_bgwriter
- 9. pg_stat_bgwriter pgbench=# select * from pg_stat_bgwriter ; -[ RECORD 1 ]---------+------------------------------ checkpoints_timed | 29 checkpoints_req | 13 checkpoint_write_time | 206345 checkpoint_sync_time | 9989 buffers_checkpoint | 67720 buffers_clean | 1046 maxwritten_clean | 0 buffers_backend | 48142 buffers_backend_fsync | 0 buffers_alloc | 30137 stats_reset | 2014-10-24 17:59:15.812002-04 postgres=# select pg_stat_reset_shared(’bgwriter’); -[ RECORD 1 ]--------+- pg_stat_reset_shared |
- 10. ×òî äåëàòü? Hardware: RAID Äåøåâûé RAID êîíòðîëëåð - çëî, óæ ëó÷øå software RAID RAID êîíòðîëëåð äîëæåí áûòü ñ áàòàðåéêîé Ïðîèçâîäèòåëè: megaraid èëè perc - OK, à íàïðèìåð HP èëè ARECA - íå âñåãäà Áàòàðåéêà - ïðèñóòñòâóåò è èñïðàâíà cache mode ! write back io mode ! direct Disk Write Cache Mode ! disabled
- 11. ×òî äåëàòü? Hardware: Äèñêè 2,5SAS (âïîëíå áûâàåò 15K) - seek â 2-3 ðàçà áûñòðåå ÷åì 3,5 Íå âñå SSD îäèíàêîâî ïîëåçíû: enterprise level Intel p3700 vs desktop-level Samsung Èñïîëüçîâàòü òîëüêî SSD äëÿ PostgreSQL îçíà÷àåò áûòü ãîòîâûìè ê ðÿäó ïðîáëåì RAID 1+0 Eñëè íåò õîðîøèõ äèñêîâ èëè êîíòðîëëåðà - synchronous_commit ! o
- 12. ×òî äåëàòü? Ôàéëîâàÿ ñèñòåìà xfs èëè ext4 - ÎÊ, zfs è ïðî÷èå lvm - íå ïðî ïðîèçâîäèòåëüíîñòü barrier=0, noatime
- 13. ×òî äåëàòü? Îïåðàöèîííàÿ ñèñòåìà Ïî óìîë÷àíèþ vm:dirty_ratio = 20 vm:dirty_background_ratio = 10 - ïëîõî Áîëåå ðàçóìíûå çíà÷åíèÿ vm:dirty_background_bytes = 67108864 vm:dirty_bytes = 536870912 (512Mb êýø íà RAID) Åñëè íåò êýøà íà RAID - ðàçäåëèòü íà 4
- 14. ×òî äåëàòü? postgresql.conf wal_buers (768kB ! 16Mb) checkpoint_segments (3 - ÷åêïîéíò êàæäûå 48Mb ! 256 - 4Gb) checkpoint_timeout = 60 (÷òî ïåðâîå íàñòóïèò) checkpoint_completion_target = 0:9 (äëÿ ðàñïðåäåëåíèÿ ÷åêïîéíòîâ)
- 15. Êàê ñåáÿ ïðîâåðèòü? pgdev@pg-dev-deb:~$ tt_pg/bin/pg_test_fsync 5 seconds per test O_DIRECT supported on this platform for open_datasync and open_sync. Compare file sync methods using one 8kB write: (in wal_sync_method preference order, except fdatasync is Linux’s default) open_datasync 11396.056 ops/sec 88 usecs/op fdatasync 11054.894 ops/sec 90 usecs/op fsync 10692.608 ops/sec 94 usecs/op fsync_writethrough n/a open_sync 67.045 ops/sec 14915 usecs/op Compare file sync methods using two 8kB writes: (in wal_sync_method preference order, except fdatasync
- 16. Íåáîëüøàÿ õèòðîñòü: ïóñòü bgwriter ðàáîòàåò postgres=# select name, setting, context, max_val, min_val from pg_settings where name ~ ’bgwr’; name | setting | context | max_val | min_val -------------------------+---------+---------+---------+--------- bgwriter_delay | 200 | sighup | 10000 | 10 bgwriter_lru_maxpages | 100 | sighup | 1000 | 0 bgwriter_lru_multiplier | 2 | sighup | 10 | 0 (3 rows)
- 17. Ïðî ÷òî íå çàáûòü: autovacuum postgres=# select name, setting, context from pg_settings where category ~ ’Autovacuum’; name | setting | context -------------------------------------+-----------+------------ autovacuum | on | sighup autovacuum_analyze_scale_factor | 0.02 | sighup autovacuum_analyze_threshold | 50 | sighup autovacuum_freeze_max_age | 200000000 | postmaster autovacuum_max_workers | 3 | postmaster autovacuum_multixact_freeze_max_age | 400000000 | postmaster autovacuum_naptime | 60 | sighup autovacuum_vacuum_cost_delay | 20 | sighup autovacuum_vacuum_cost_limit | -1 | sighup autovacuum_vacuum_scale_factor | 0.001 | sighup autovacuum_vacuum_threshold | 50 | sighup (11 rows)