SQLBits XIV - The Big Backup Theory

1 SQLBits XIV - The Big Backup Theory
I know, I know,
who schedules a 75
minute session on
Backups!?!
In your shoes my
reaction would be…

“Oh what
fresh hell is
this?”
The Dumpling Paradox
S1 E7

The Big Backup Theory
Richard Douglas

The goal of
this
presentation
The Cooper-Hofstadter Polarization - S1 E9

I do work for a vendor,
there may be some
subliminal messages in
this presentation

Don’t have a
“Backup
Strategy”

Agenda
• Why do we need backups?
• Backups available in SQL Server
• Understanding the Transaction Log
• Backup Strategy vs. Restore Strategy
• How to make your backups faster
• How to make your restores faster

Have a
“Restore
Strategy”

If we haven’t met yet…
• Richard Douglas
• Sales Engineer / SQL Evangelist
• SQL Server MCITPro
• SQL User Group Leader
• Blog: http://SQL.RichardDouglas.co.uk
• Twitter: @SQLRich
• Email: Richard.Douglas@Software.Dell.com
• Slides: http://www.Slideshare.net/SQLRich

Seriously,
think about it.
A “Restore
Strategy” makes
so much sense!

This time
Sheldon is
wrong…
Backups are
Essential!
The Cooper-Hofstadter
Polarization S1 E9

Why do we need backups?
• Protect valuable data
• Provides the potential to return to a “point in time”
• Confusion between Disaster Recovery and High Availability
• Backup misconceptions
Image backups are sufficient
Snapshots are sufficient
Differential backups are incremental
System databases only need to be backed up once a week
Only need to keep my last backup
Buffer pool usage

Have a
“Restore
Strategy”

Recovery Models
• Simple
• Full
• Bulk Logged
Data
File
Log
File
SQL
Database
Automated administration Manual Administration
Note: Recovery model will be defaulted to the one in the Model database

Backup Types
• Full
• Differential
• Log
• File / Filegroup
• Tail of the log
• Database Snapshot

Have a
“Restore
Strategy”

Transaction Log Internals
• The Transaction Log is segmented into pieces called
Virtual Log Files (VLF’s)
• There will always be at least one “active” VLF
• The number of VLF’s is determined by the size at creation / growth time.
– Under 64MB 4VLF’s
– >=64 and > 1GB 8 VLF’s
– >= 1GB 16 VLF’s
• But wait, there’s more. VLF’s aren’t the smallest segment of the Transaction
Log …
Active
VLF
Inactive
VLF
Inactive
VLF
Inactive
VLF
Inactive
VLF

Log Blocks
• Each VLF will have a number of “Log Blocks” inside it
• These blocks vary in size:
– Anywhere between 512bytes and 60KB
• A new block is created when:
– The data inside the block hits the 60KB limit
– A commit or abort occurs within a transaction
• Each log block contains a parity bit of 64 or 128 which is flipped every time it
is used.
• Inside each log block we have…

Explaining the
transaction log
is a bit like
nesting dolls

Log Records
• Log records are identified by Log Sequence Numbers or LSN’s
– VLF Sequence nbr : Log block nbr : Log record nbr
• Each data page contains the LSN of the last record that made changes to
that page.
• Log records are written in the order they occur.
i.e. SQL Server will not wait for a transaction to finish before writing away
another concurrent transaction.
• Log records often store before and after images of changes.
• They contain a list of pages that this change has affected.
• Log records are stored in the buffer pool (log cache) until flushed to disk
Look for WAITLOG wait type.
This will become clear when we
cover the restore process

Checkpoint
• Data pages are marked as “dirty” when they are changed.
• The checkpoint process writes all dirty pages to disk and marks them as
cleared.
• Checkpoint is an optimisation process, it reduces the amount of writes to
disk to keep data up to date.

Have a
“Restore
Strategy”

Restore Strategy main considerations
• Time taken to restore data
aka Recovery Time Objective (RTO)
• Data loss acceptability
aka Recovery Point Objective (RPO)
• Cost associated with system
unavailable
Backup Strategy main considerations
• Time taken to perform a backup
• Cost associated with longer
maintenance window
• Cost associated with storing
backup files
Backup Strategy vs Restore Strategy

Showing you
why you need to
have a “Restore
Strategy”

Full backup
0
0
0
1
0
2
0
3
0
4
0
5
0
6
0
7
0
8
0
9
1
0
1
1
1
2
1
3
1
4
1
5
1
6
1
7
1
8
1
9
2
0
2
1
2
2
2
3
0
0
0
1
0
2
0
3
0
4
0
5
0
6
0
7
0
8
0
9
1
0
1
1
1
2
1
3
1
4
1
5
1
6
1
7
1
8
1
9
2
0
2
1
2
2
2
3
0
0
0
1
0
2
0
3
0
4
0
5
0
6
0
7
0
8
0
9
1
0
1
1
1
2
1
3
1
4
1
5
1
6
1
7
1
8
1
9
2
0
2
1
2
2
2
3
0
0
0
1
0
2
0
3
0
4
0
5
0
6
0
7
0
8
0
9
1
0
1
1
1
2
1
3
1
4
1
5
1
6
1
7
1
8
1
9
2
0
2
1
2
2
2
3
0
0
0
1
0
2
0
3
0
4
0
5
0
6
0
7
0
8
0
9
1
0
1
1
1
2
1
3
1
4
1
5
1
6
1
7
1
8
1
9
2
0
2
1
2
2
2
3
0
0
0
1
0
2
0
3
0
4
0
5
0
6
0
7
0
8
0
9
1
0
1
1
1
2
1
3
1
4
1
5
1
6
1
7
1
8
1
9
2
0
2
1
2
2
2
3
0
0
0
1
0
2
0
3
0
4
0
5
0
6
0
7
0
8
0
9
1
0
1
1
1
2
1
3
1
4
1
5
1
6
1
7
1
8
1
9
2
0
2
1
2
2
2
3
M
T
W
T
F
S
S

Differential
0
0
0
1
0
2
0
3
0
4
0
5
0
6
0
7
0
8
0
9
1
0
1
1
1
2
1
3
1
4
1
5
1
6
1
7
1
8
1
9
2
0
2
1
2
2
2
3
0
0
0
1
0
2
0
3
0
4
0
5
0
6
0
7
0
8
0
9
1
0
1
1
1
2
1
3
1
4
1
5
1
6
1
7
1
8
1
9
2
0
2
1
2
2
2
3
0
0
0
1
0
2
0
3
0
4
0
5
0
6
0
7
0
8
0
9
1
0
1
1
1
2
1
3
1
4
1
5
1
6
1
7
1
8
1
9
2
0
2
1
2
2
2
3
0
0
0
1
0
2
0
3
0
4
0
5
0
6
0
7
0
8
0
9
1
0
1
1
1
2
1
3
1
4
1
5
1
6
1
7
1
8
1
9
2
0
2
1
2
2
2
3
0
0
0
1
0
2
0
3
0
4
0
5
0
6
0
7
0
8
0
9
1
0
1
1
1
2
1
3
1
4
1
5
1
6
1
7
1
8
1
9
2
0
2
1
2
2
2
3
0
0
0
1
0
2
0
3
0
4
0
5
0
6
0
7
0
8
0
9
1
0
1
1
1
2
1
3
1
4
1
5
1
6
1
7
1
8
1
9
2
0
2
1
2
2
2
3
0
0
0
1
0
2
0
3
0
4
0
5
0
6
0
7
0
8
0
9
1
0
1
1
1
2
1
3
1
4
1
5
1
6
1
7
1
8
1
9
2
0
2
1
2
2
2
3
M
T
W
T
F
S
S

Transaction Log Backup
0
0
0
1
0
2
0
3
0
4
0
5
0
6
0
7
0
8
0
9
1
0
1
1
1
2
1
3
1
4
1
5
1
6
1
7
1
8
1
9
2
0
2
1
2
2
2
3
0
0
0
1
0
2
0
3
0
4
0
5
0
6
0
7
0
8
0
9
1
0
1
1
1
2
1
3
1
4
1
5
1
6
1
7
1
8
1
9
2
0
2
1
2
2
2
3
0
0
0
1
0
2
0
3
0
4
0
5
0
6
0
7
0
8
0
9
1
0
1
1
1
2
1
3
1
4
1
5
1
6
1
7
1
8
1
9
2
0
2
1
2
2
2
3
0
0
0
1
0
2
0
3
0
4
0
5
0
6
0
7
0
8
0
9
1
0
1
1
1
2
1
3
1
4
1
5
1
6
1
7
1
8
1
9
2
0
2
1
2
2
2
3
0
0
0
1
0
2
0
3
0
4
0
5
0
6
0
7
0
8
0
9
1
0
1
1
1
2
1
3
1
4
1
5
1
6
1
7
1
8
1
9
2
0
2
1
2
2
2
3
0
0
0
1
0
2
0
3
0
4
0
5
0
6
0
7
0
8
0
9
1
0
1
1
1
2
1
3
1
4
1
5
1
6
1
7
1
8
1
9
2
0
2
1
2
2
2
3
0
0
0
1
0
2
0
3
0
4
0
5
0
6
0
7
0
8
0
9
1
0
1
1
1
2
1
3
1
4
1
5
1
6
1
7
1
8
1
9
2
0
2
1
2
2
2
3
M
T
W
T
F
S
S

Figured out it’s
safer to have a
“Restore
Strategy” yet?

Optimizing backups
Server
Architecture
Database
Architecture
Backup
Options

Have a
“Restore
Strategy”

Server
Architecture

What
computer do
you have? And
please don’t
say “a white
one”.
The Peanut Reaction – S1 E16

• IO Subsystem
– RAID Levels
– Disk Partition Alignment
– Dedicated drives
• Network cards
– Add more NICs
– Use different subnets
– Use different switches
Optimizing backups – Server Architecture
The DBA team start negotiations
with the Infrastructure team
http://www.flickr.com/photos/jakesutton/859193891/in/photostream

Have a
“Restore
Strategy”

Optimizing backups – Database Architecture
• Table design
– See Natural Born Killers webcast November 2012 (http://dell.to/15RwEtz)
– Blogged at http://bit.ly/GHuUrH
– Space that a table uses is not completely released after you use a DELETE
statement to delete data from the table in SQL Server:
http://support.microsoft.com/kb/913399
• Index Design
– See indexing webcasts October 2012 (http://dell.to/19urm8s)
• File placement
• Compression

End of
subliminal
messages

Actually, I
wouldn’t mind
a caramel
macchiato

Optimizing Backups
• Compression
• BufferCount
• MaxTransferSize
• BlockSize
• Multiple backup devices
Other Backup Options of note
• Init / NoInit
• Format / NoFormat
• Copy Only
• Checksum
• Continue After Error
Backup Options
For a full list of all the backup options refer to Books Online (BOL)
http://msdn.microsoft.com/en-us/library/ms186865.aspx

Backup Options Continued
• Memory used is accessed from outside of the bufferpool
• You can calculate memory usage:
Buffercount * MaxTransferSize
Setting these values too high may cause out of memory conditions
Buffercount calculations for Full / Diff backups:
• 2000
– (NumberofBackupDevices * GetSuggestedIoDepth)
+ NumberofBackupDevices + (DatabaseDeviceCount)
• 2005 and above
– (NumberofBackupDevices * GetSuggestedIoDepth)
+ NumberofBackupDevices + (2*DatabaseDeviceCount)

Calculating BufferCount
• GetSuggestedIODepth
– Disk = 3
– Tape = 3
– VDI = 1 default and 4 max based on device spanning needs http://bit.ly/1i69GUG
• DatabaseDeviceCount = The number distinct drives the database files reside
on.
• I think the formula for 2008+ is wrong. Waiting for CSS to return my email.
The following seems to work:
(NumberofBackupDevices * GetSuggestedIoDepth [4])
+ NumberofBackupDevices + (2*DatabaseDeviceCount)

Demo

51 The Big Backup Theory
Things that
break the log
chain

Restore Problems
• Broken Transaction Log Chain
– Clearing the transaction log (using WITH TRUNCATE ONLY or WITH NO LOG)
– Changing the database to SIMPLE recovery model (checkpoint will clear log))
– Reverting from a database snapshot
– Deleting a log file in the chain
– Backups to NUL devices

Backups
are like
Schrodinger’s
cat

Restore Myths
• Downgrading a backup
• No need to test backups
• Restore time will be the same as the backup time
• Table restores

The Restore Process
A restore operation can have up to four phases:
1. File creation and initialization
2. Data and/or log copy
3. REDO phase of recovery
4. UNDO phase of recovery

The Restore Process
File creation and initialization
• In the beginning there was file creation.
• Then there was the zeroing of the files.
• If the database already exists overwrite using the “WITH REPLACE” clause,
do not drop and restore.
• Use “Instant File Initialization” (IFI) if possible.
Remember the log file still needs to be zeroed out.
• There is a security risk.
• TF 3004 will provide more information about IFI
• TF 1806 will disable IFI

The Restore Process
Data and/or log copy
• In this stage SQL Server:
– Reads the data from the backup file(s)
– Writes the data to the database files
• Having multiple data files and backup files can improve I/O throughput.
• Using native or third party compression reduces the amount of data to be
read (and potentially copied across the network).
Compression vs DeDuplication
• Whilst deduplication does reduce storage costs it will be rehydrated before
copying back across the network and so will not aid the speed of a restore.

The Restore Process
REDO phase of recovery (1)
• The database being recovered has it’s boot page (page 9) checked.
• This page will provide the details of the most recent checkpoint.
• Checkpoint log records are examined to see if there is a list of uncommitted
transactions
• Recovery will start at the oldest uncommitted transaction for the most recent
checkpoint

The Restore Process
• Each log record is tied to an LSN and contains page numbers of records it has altered
• Each data page’s page header contains the last LSN which made a change to it
• The redo phase compares the two LSN’s
• The redo phase uses something akin to the following formula:
IF PageHeaderLSN < LogRecordLSN
BEGIN
Replay Log Record –Record has not been updated
END
ELSE
DoNothing –Record has this change and possibly some others

The Restore Process
Finding the end of the log
There are two ways:
1. If the log has not been wrapped then it checks for the occurrence of 512 byte
block full of zeros.
2. If it has been wrapped then it reads through all the active VLF’s in the order of
their sequence number and reads through each log block until it finds one with
the wrong flipped parity bit

The Restore Process
UNDO phase of recovery
• The redo phase keeps track of uncommitted transactions. If there are
uncommitted transactions during recovery then the undo phase kicks in
• As mentioned in the redo section the data page page header contains an
LSN. This is compared to the Log Record LSN
IF PageHeaderLSN >= LogRecordLSN
BEGIN
Undo Log Record –Record requires rollback
Create a “compensation” log record
Update PageHeader LSN with compensation log record LSN
END

Further ways to optimize restores
• Using “Instant File Initialization”
• Using the correct restore strategy
• Server Architecture
• Database Architecture
• Backup compression
• Restore options
– Buffercount
– MaxTransferSize

65 Global MarketingThe Big Backup Theory
Any questions?

Thank you for attending
Richard Douglas
Richard.Douglas@Software.Dell.com
@SQLRich

SQLBits XIV - The Big Backup Theory

More Related Content

What's hot (19)

Similar to SQLBits XIV - The Big Backup Theory (20)

Recently uploaded (20)

SQLBits XIV - The Big Backup Theory

Editor's Notes