SQL server Backup Restore Revealed
0 likes608 views
Antonios Chatzipavlis presented on SQL Server backup and restore. The presentation covered database architecture basics including data files, transaction log files, and the buffer cache. It also discussed backup types like full, differential, transaction log, copy only and partial backups. Backup strategies and restore processes were explained, including restoring to a point in time and restoring system databases. The internals of how SQL Server performs backups using buffers and I/O threads was also summarized.
![Restore to a point in time
(t)
DB
Full Recovery
Model
FULL
LOG LOG DIFF LOG LOG
t6’
Some crucial
data deleted
LOG
RESTORE RESTORE RESTORE RESTORE RESTORE
t7’
Discover
RESTORE LOG ‘<dbname>’
FROM backupdevice
WITH STOPAT = ‘YYYY/MM/DD HH:MM:SS’
STOPAT = { 'datetime'| @datetime_var }
STOPATMARK = { 'mark_name' | 'lsn:lsn_number’ }
STOPBEFOREMARK = { 'mark_name' | 'lsn:lsn_number' } [ AFTER 'datetime']
DATA
LOSS](https://image.slidesharecdn.com/sqlserverbackuprestorerevealed-201004001223/85/SQL-server-Backup-Restore-Revealed-47-320.jpg)
SQL server Backup Restore Revealed
- 1. Antonios Chatzipavlis DATA SOLUTIONS CONSULTANT & TRAINER SQL Server Backup/Restore Revealed SQL Night #43 Sep 26, 2020 20:30 SQL Nights
- 4. 1988 2000 Founderv 6.0 60 +2010 Antonios Chatzipavlis Data Solutions Consultant & Trainer
- 5. V.102010 - 2020 A community for Greek professionals who use the Microsoft Data Platform Connect / Explore / Learn WebminarsArticles Videos News EventsResources @sqlschool@antoniosch Sqlschoolgr Page Sqlschool User Group Sqlschool.gr Group SQLschoolGR channel
- 7. Take Your Career to New Heights Save $50 USD Register for PASS Summit 2020 and as a participant in SQLschool.gr, you can save $50 USD the registration price. To claim use code* LGDISET1I. * Unique discount code cannot be applied retroactively and cannot be combined with any other offers.
- 8. Presentation Content SQL Server Backup/Restore Revealed • Database Architecture Basics • Backup / Restore • Backup / Restore Internals
- 10. Connect / Explore / Learn Database Architecture Data file(s) 8KB Pages VLFs T-log file Database
- 11. Connect / Explore / Learn Buffer Cache usage Data file(s) 8KB Pages VLFs T-log file Database Buffer cache Memory
- 12. Connect / Explore / Learn Transaction log usage Data file(s) 8KB Pages VLFs T-log file Database Buffer cache Memory Commit returned Pages marked as dirty
- 13. Connect / Explore / Learn Checkpoint Data file(s) 8KB Pages VLFs T-log file Database Buffer cache Memory Pages marked as dirty checkpoint
- 14. Connect / Explore / Learn Checkpoint frequency Data file(s) 8KB Pages VLFs T-log file Database Buffer cache Memory DD D D D D
- 15. Connect / Explore / Learn Automatic Indirect Manual Internal CHECKPOINT statement configures the target recovery duration Checkpoint Options
- 16. Backup / Restore
- 17. Connect / Explore / Learn Backup / Restore Strategy Understanding RPO and RTO (t) DISASTER RPO RTO backup Restore end Lost data DB Down t1 t2 t3
- 18. Connect / Explore / Learn Retention & Testing Policy Backup Integrity •Insufficient Copies of Backups •Insufficient Data on the Backups •Unreadable Backups •Unavailable Hardware •Old Hardware •Misaligned Hardware •Mirrored Backups •Backup Verification •Backup storage size •Backup security Managing Backups
- 19. Connect / Explore / Learn Backup Device(s) Backup Device Slot 1 – Backup A Slot 2 – Backup B Slot 3 – Backup C Slot 4 – Backup D Slot 5 – Backup E Slot N – Backup … Backup devices types • Physical • Logical • Azure Blob storage
- 20. Connect / Explore / Learn Media Set Media Family Backup Sets, Media Sets, Media Families Backup Device A Slot 1 – Backup Database A Slot 2 – Backup Database B Slot 3 – Backup Database C Backup Set Backup Set Backup Set Media Header
- 21. Connect / Explore / Learn Media Set Media Family 2/2Media Family 1/2 Backup Sets, Media Sets, Media Families Backup Device A Slot 1 – Backup Database A (50%) Slot 2 – Backup Database B (50%) Slot 3 – Backup Database C (50%) Media Header Backup Device B Slot 1 – Backup Database A (50%) Slot 2 – Backup Database B (50%) Slot 3 – Backup Database C (50%) Backup Set Backup Set Backup Set Media Header
- 22. Connect / Explore / Learn Recovery Models Simple No T-log backups Auto truncate T-log Full T-log backups required Avoids data loss due to a damaged or missing data file Recovery at specific point in time Bulk logged T-log backups required Enhance the performance of bulk copy operation Reduce log space by using minimal logging for many bulk operations
- 23. Connect / Explore / Learn Backup Types Full Differential Transaction Log Tail Log Copy Only Partial Filegroup Snapshot
- 24. Connect / Explore / Learn BACKUP TYPES SIMPLE FULL BULK LOGGED Full ● ● ● Differential ● ● ● Transaction Log ● ● Tail Log ● Copy Only ● ● ● Partial ● ● ● Filegroup ● ● ● Snapshot ● Backup Types / Recovery Model
- 25. Connect / Explore / Learn Full Backup (t) New DB FULL FULL FULL
- 26. Connect / Explore / Learn Restore Full Backup (t) New DB FULL FULL FULL t4 DAMAGE RESTORE
- 27. Connect / Explore / Learn Differential Backup (t) New DB FULL DIFF DIFF DIFF
- 28. Connect / Explore / Learn Restore Differential Backup (t) New DB FULL DIFF DIFF DIFF t5 DAMAGE RESTORE RESTORE
- 29. Connect / Explore / Learn Transaction Log Backup (t) New DB FULL LOG LOG DIFF LOG LOG
- 30. Connect / Explore / Learn Restore Transaction Log Backup (t) New DB FULL LOG LOG DIFF LOG LOG t7 DAMAGE RESTORE RESTORE RESTORE RESTORE
- 31. Connect / Explore / Learn Tail Log Backup (t) New DB FULL LOG LOG DIFF LOG LOG DAMAGE TAIL LOG RPO A tail-log backup captures any log records which has not yet been backed up by the last transactional log backup.
- 32. Connect / Explore / Learn Copy Only Backup (t) New DB FULL LOG LOG DIFF LOG LOG t2-3 FULL COPY ONLY
- 33. Connect / Explore / Learn Partial Backup (t) New DB FULL Primary filegroup Read only Filegroup Read/Write Filegroup FULL READ/WRITE PARTIAL FULL READ ONLY PARTIAL DIFFERENTIAL READ/WRITE PARTIAL DIFFERENTIAL READ/WRITE PARTIAL All SQL Server recovery models support partial backups Partial backups are designed for use under the simple recovery model to improve flexibility for backing up very large databases that contain one or more read-only filegroups.
- 34. Connect / Explore / Learn Restore Partial Backup (t) New DB FULL Primary filegroup Read only Filegroup Read/Write Filegroup FULL READ/WRITE PARTIAL FULL READ ONLY PARTIAL DIFFERENTIAL READ/WRITE PARTIAL DIFFERENTIAL READ/WRITE PARTIAL t6 DAMAGE RESTORE RESTORERESTORE
- 35. Connect / Explore / Learn Filegroup Backup (t) New DB FULL Primary filegroup Other Filegroups FILEGROUP A FILEGROUP B FILEGROUP C FILEGROUP A FILEGROUP B FILEGROUP C t0A LOG t1A LOG t1B LOG t2A LOG t2B LOG t3A LOG t3B LOG t4A LOG t4B LOG t5A LOG t5B LOG t6A LOG t6B LOG t7A LOG t7B LOG
- 36. Connect / Explore / Learn Restore Filegroup Backup (t) New DB FULL Primary filegroup Other Filegroups FILEGROUP A FILEGROUP B FILEGROUP C FILEGROUP A FILEGROUP B FILEGROUP C t0A LOG t1A LOG t1B LOG t2A LOG t2B LOG t3A LOG t3B LOG t4A LOG t4B LOG t5A LOG t5B LOG t6A LOG t6B LOG t7A LOG t7B LOG t8 DAMAGE FILEGROUP B t8’ TAIL LOG
- 37. Connect / Explore / Learn 8KB Pages Database Snapshot Data file(s) Database Snapshot SELECT on Snapshot We can restore this snapshot, but it should not be considered as a backup
- 38. Connect / Explore / Learn File-snapshot backup SQL Server instance on Azure Virtual Machine ( SQL Server 2016 + ) SQL Server Instance On Azure VM DB Azure Blob Storage (t) FULL LOG LOG LOG LOG LOG LOG LOG t9 DAMAGETAIL LOG RESTORE ALL (SEQUENCIAL) Azure Blob Storage TO URL https://<storagename>.blob.core.windows.net/<container>/xxx.bak FILE_SNAPSHOT RESTORE
- 39. Connect / Explore / Learn When already running FULL DIFFERENTIAL LOG Full Backup Blocked Blocked Allowed ( >SQL 2005) Differential Backup Blocked Blocked Allowed ( >SQL 2005) Log Backup Allowed ( >SQL 2005) Allowed ( >SQL 2005) Blocked Backup Concurrency in the same database
- 40. Connect / Explore / Learn Data Copy Creates files and copies data to the files Redo Phase Applies committed transactions from restored log entries Undo Phase Rolls back transactions that were uncommitted at the recovery point Phases of the Restore Process
- 41. Connect / Explore / Learn RESTORE LABELONLY Returns information about the backup media on a specified backup device. RESTORE HEADERONLY Returns all the backup header information for all backup sets on a particular backup device. RESTORE FILELISTONLY Returns a list of data and log files contained in a backup set. RESTORE VERIFYONLY Verifies the backup but does not restore it and checks to see that the backup set is complete, and the entire backup is readable. Retrieving Backup Metadata
- 42. Backup / Restore
- 44. Page(s) Restore Online Restore – Enterprise Edition (t) DB Full Recovery Model FULL LOG LOG DIFF LOG LOG t7 Page corruption 823, 824 PAGE(s) RESTORE RESTORE RESTORE RESTORE TAIL LOG RESTORE msdb dbo suspect_pages;
- 45. Page(s) Restore Offline Restore – Enterprise and Standard Editions (t) DB Full Recovery Model FULL LOG LOG DIFF LOG LOG t7 Page corruption 823, 824 PAGE(s) RESTORE RESTORE RESTORE RESTORE TAIL LOG RESTORE msdb dbo suspect_pages;
- 46. Page Restore
- 47. Restore to a point in time (t) DB Full Recovery Model FULL LOG LOG DIFF LOG LOG t6’ Some crucial data deleted LOG RESTORE RESTORE RESTORE RESTORE RESTORE t7’ Discover RESTORE LOG ‘<dbname>’ FROM backupdevice WITH STOPAT = ‘YYYY/MM/DD HH:MM:SS’ STOPAT = { 'datetime'| @datetime_var } STOPATMARK = { 'mark_name' | 'lsn:lsn_number’ } STOPBEFOREMARK = { 'mark_name' | 'lsn:lsn_number' } [ AFTER 'datetime'] DATA LOSS
- 48. Connect / Explore / Learn master • Start the server instance in single-user mode by using the –m startup option. • Use a RESTORE DATABASE statement to restore a full database backup of the master database by using the sqlcmd utility. • Terminate your sqlcmd connection. • Remove the single-user startup parameter. • Restart SQL Server. model • Start the server instance with TF3608. • Use a RESTORE DATABASE statement to restore a full database backup of the master database by using the sqlcmd utility. • Terminate your sqlcmd connection. • Remove the single-user startup parameter. • Restart SQL Server. msdb • You can restore it by using the RESTORE DATABASE statement as you would a user db Restore System databases
- 49. Connect / Explore / Learn Transaction Log Truncation T-logfile Start End End File growth
- 50. Connect / Explore / Learn Transaction Log Truncation T-logfile Start End Start End
- 52. Smart Backups
- 53. Connect / Explore / Learn Differential Backup Commonly used case Restore chain too long impacting RTO
- 54. Connect / Explore / Learn Smart differential backup New approach ( SQL Server 2016 + ) modified_extent_page_count -> sys.dm_db_file_space_usage Total number of pages modified in allocated extents of the file since last full database backup. The modified page count can be used to track differential changes in the database since last full backup to decide if differential backup is beneficial SELECT file_id, total_page_count, modified_extent_page_count, cast (( modified_extent_page_count * 1.0 / total_page_count * 1.0 ) * 100.0 as decimal(5,2)) as pctchanges FROM sys.dm_db_file_space_usage
- 55. Connect / Explore / Learn Transaction Log Backup Commonly used case Not Adaptive • Frequent Auto grows • VLF Fragmentation • Inconsistent backup size
- 56. Connect / Explore / Learn Smart transaction log backup New approach ( SQL Server 2016 + ) select from sys.dm_db_log_stats DB_ID Adaptive • Minimal Auto grows • No VLF Fragmentation • Consistent backup size
- 57. Smart Backups
- 58. Backup Internals
- 59. Connect / Explore / Learn Memory out of buffer cache How SQL Server Backup Works DB 1 Backup command executed 2 Checkpoints (few times) Checkpoints 3 Workers creation Controller Read thread (one per volume) Write thread (one per backup device) 4 Start buffering data (async I/O) backup device BUFFER BUFFER BUFFER BUFFER BUFFER BUFFER BUFFER BUFFER 5 Writing buffered data 6 Exchange the buffers
- 60. Connect / Explore / Learn BufferCount • Specifies the total number of I/O buffers to be used for the backup operation. • You can specify any positive integer • Large numbers of buffers might cause “out of memory” errors because of inadequate virtual address space in the Sqlservr.exe process. MaxTransferSize • Specifies the largest unit of transfer in bytes to be used between SQL Server and the backup media. • The possible values are multiples of 65536 bytes (64 KB) ranging up to 4194304 bytes (4 MB). Important Backup Options Calculate the Buffer Count Formula
- 61. Backup Internals
- 62. Connect / Explore / Learn Backup related wait types BACKUPBUFFER Occurs when a backup task is waiting for data or is waiting for a buffer in which to store data. BACKUPIO Occurs when a backup task is waiting for data or is waiting for a buffer in which to store data. BACKUPTHREAD Occurs when a task is waiting for a backup task to finish.
- 64. Connect / Explore / Learn SQL Server tracks all backup activity in the following tables in the msdb database: • msdb.dbo.backupfile • msdb.dbo.backupfilegroup • msdb.dbo.backupmediafamily • msdb.dbo.backupmediaset • msdb.dbo.backupset Viewing Backup History https://bit.ly/365LosB
- 65. Connect / Explore / Learn Deleting Backup History sp_delete_backuphistory @oldest_date sp_delete_database_backuphistory @database_name
- 66. Connect / Explore / Learn Backup/Restore operation execution info How to know the execution percentage and completion time for a backup / restore operation SELECT SUBSTRING case when then else end DATEADD FROM sys.dm_exec_requests CROSS APPLY sys.dm_exec_sql_text INNER JOIN sys.dm_exec_sessions INNER JOIN sys.dm_exec_connections WHERE https://www.sqlschool.gr/blog/how-to-know-the-execution-percentage-and-completion-time-for-a-task-1054.aspx
- 67. Any questions
- 68. Thank you! @antoniosch - @sqlschool ./sqlschoolgr - ./groups/sqlschool ./c/SqlschoolGr SQLschool.gr Group Antonios Chatzipavlis Data Solutions Consultant & Trainer
- 69. A community for Greek professionals who use the Microsoft Data Platform Copyright © 2010-2020 SQLschool.gr. All right reserved. PRESENTER MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION