Tivoli Storage, IBM Software Group
Potpourri du Backup
Andy RaibeckOxford University TSM Symposium 2007St. Catherine’s College, 25 – 27 September
© 2007 IBM Corporation
Tivoli Storage, IBM Software Group
TrademarksTrademarks
The following terms are trademarks of International Business gMachines Corporation in the United States, other countries, or both:
• AIX• IBM• Tivoli
Other company, product, and service names may be trademarks or service marks of othersservice marks of others.
Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both.
UNIX is a registered trademark of The Open Group in the United St t d th t iStates and other countries.
© 2007 IBM Corporation2 Potpourri du Backup
Tivoli Storage, IBM Software Group
IntroductionIntroduction
There is a potpourri of backup techniques available in e e s a potpou o bac up tec ques a a ab eTSM
Probably more to come in the futureProbably more to come in the future
One size does not fit all
Why can’t TSM decide for me?
© 2007 IBM Corporation3 Potpourri du Backup
Tivoli Storage, IBM Software Group
What is covered in this presentationWhat is covered in this presentation
The various backup techniques offered by TSM Backup-Archive p q y pClient:– What you get vs. what you give up with different technologies and
variationsvariations
– Decision points
Provide qualitative advice on evaluating different technologies andProvide qualitative advice on evaluating different technologies and variations– What tools are available today to help with these decisions
© 2007 IBM Corporation4 Potpourri du Backup
Tivoli Storage, IBM Software Group
What is NOT covered in this presentationWhat is NOT covered in this presentation
How to implement:o to p e e t– Not a tutorial in option syntax
– No dsm.opt or dsm.sys examples!No dsm.opt or dsm.sys examples!
Other backup or archive considerations
Recovery or retrieve considerations
© 2007 IBM Corporation5 Potpourri du Backup
Tivoli Storage, IBM Software Group
AgendaAgenda
Backup techniques– What was available before ADSM/TSM came on the scene– TSM Progressive incremental backup model and its derivatives– Image backupImage backup
How to determine which backup technique best fits– What problem are you trying to solve– Windows– Other operating systems
© 2007 IBM Corporation6 Potpourri du Backup
Tivoli Storage, IBM Software Group
AgendaAgenda
Backup techniques– What was available before ADSM/TSM came on the scene– TSM Progressive incremental backup model and its derivatives– Image backupImage backup
How to determine which backup technique best fits– What problem are you trying to solve– Windows– Other operating systems
© 2007 IBM Corporation7 Potpourri du Backup
Tivoli Storage, IBM Software Group
Other Backup Products TSM
Long restore timeP di tili ti
No unnecessary backups Long restore timeP di tili ti
Other Backup Products TSM
Poor media utilizationLots of copies of unchanged dataHigh network utilizationHigher impact to application server
Client data consolidated on a few tapesLower network utilizationLow impact to application
Poor media utilizationLots of copies of unchanged dataHigh network utilizationHigher impact to application server
Full + Incremental
Full Backup
Progressive incrementalBase BackupFull Backup
Full + Incremental Full + Differential
Incremental
Differential
Incremental
Incremental
BAIncremental
Incremental
Differential AutomatedTapereclamation
Full Backup
Incremental
Full Backupreclamation
IncrementalIncremental
Incremental Tape reclamationCollocation
© 2007 IBM Corporation8 Potpourri du Backup
Collocation
Tivoli Storage, IBM Software Group
AgendaAgenda
Backup techniques– What was available before ADSM/TSM came on the scene– TSM Progressive incremental backup model and its derivatives– Image backupImage backup
How to determine which backup technique best fits– What problem are you trying to solve– Windows– Other operating systems
© 2007 IBM Corporation9 Potpourri du Backup
Tivoli Storage, IBM Software Group
Progressive Incremental BackupProgressive Incremental Backup
1 23
DB
1
2
2
1 Cli t i f t i f fil t
4
1. Client queries server for current view of file system
2. Server returns list of active versions for entire file system
3 Client scans and compares with local file system3. Client scans and compares with local file system
4. Client backs up new and changed files
© 2007 IBM Corporation10 Potpourri du Backup
Tivoli Storage, IBM Software Group
Progressive IncrementalProgressive Incremental
a.k.a Full Progressive Incremental or Full Incremental or g“Incremental Forever”
What you gety g– Single-instance store
• Technically progressive incremental backup is a form of SIS (not a de-dup though)dup though)
– Easier restore• No concept of applying incremental or differential changes to full
© 2007 IBM Corporation11 Potpourri du Backup
Tivoli Storage, IBM Software Group
Progressive IncrementalProgressive Incremental
When to use thise to use t s– Default technology in TSM
– Hot spots:Hot spots:• Memory
– Number of active backup versions– Length of file nameg
• Run time– Biggest factor is file system scan time
© 2007 IBM Corporation12 Potpourri du Backup
Tivoli Storage, IBM Software Group
Progressive Incremental DerivativesProgressive Incremental Derivatives
Memory efficient backupe o y e c e t bac up
Memory efficient disk caching
Virtual mount points
Incremental by date
File lists
J l b d b kJournal-based backups
© 2007 IBM Corporation13 Potpourri du Backup
Tivoli Storage, IBM Software Group
Memory EfficientMemory Efficient
1 23 5
DB
1
2
25
1 Cli t i f t i f fi t di t
41. Client queries server for current view of first directory
2. Server returns list of files in directory (db query)
3. Client scans and compares with local file system
4. Client backs up new and changed files
5. Client queries server for current view of next directory, etc.
© 2007 IBM Corporation14 Potpourri du Backup
Tivoli Storage, IBM Software Group
Memory Efficient BackupMemory Efficient Backup
What you get– Smaller backup memory footprint
What you don’t getMemory savings in directories with large number of objects– Memory savings in directories with large number of objects
What you give up– More network chit-chat– Increased backup run time
When to use thisIncremental backups run out of memory– Incremental backups run out of memory
– Can get rough estimate on memory usage
© 2007 IBM Corporation15 Potpourri du Backup
Tivoli Storage, IBM Software Group
Memory Efficient Disk CachingMemory Efficient Disk Caching
1 23
DB
1
2
2
4
1. Client queries server for current view of file system
2. Server returns list of files for entire file system; client stores inventory on disk rather than in memoryy
3. Client scans and compares with local file system
4. Client backs up files
© 2007 IBM Corporation16 Potpourri du Backup
Tivoli Storage, IBM Software Group
Memory Efficient Disk CachingMemory Efficient Disk Caching
How it works– Same as progressive incremental except …– B-A client can store TSM Server db queries on disk instead of in memory
What you getWhat you get– Even smaller memory footprint if regular memoryefficient backup doesn’t
reduce memory
Wh t iWhat you give up– Processing time (takes a bit longer to store and retrieve information from
disk)
When to use this– Incremental backup running out of memory and memoryefficient backup
not sufficient Need individual file restore– Journal based backup technology is not available on your platform (note
you might need this just to get the first backup done for journal based backups).
© 2007 IBM Corporation17 Potpourri du Backup
Tivoli Storage, IBM Software Group
Virtual Mount PointVirtual Mount Point
/home
/home/1 /home/2
/home/2
Large file system logically partitioned
Managed as separate file spaces on TSM Server
© 2007 IBM Corporation18 Potpourri du Backup
Tivoli Storage, IBM Software Group
Virtual Mount PointVirtual Mount Point
How this works– Instead of mapping entire file system to single namespace on TSM
Server (file space), user can define– What you get– Progressive incremental only has to work against subset of file system
thus reducing memory requirements
What you don’t gety g– Relief from directories with large number of objects
What you give up– Centralized management of file system on TSM Server– Virtual mount points are static and can’t be migrated
When to use thisWhen to use this– Large, balanced (number of directories and objects) UNIX and Linux file
systems can be efficiently divided
© 2007 IBM Corporation19 Potpourri du Backup
Tivoli Storage, IBM Software Group
Incremental by DateIncremental by Date
13
DB
1
2
4
1. Client queries server last backup of entire file system
2. Server returns timestamp of last backup (end) of entire file system
3 Client scans and compares with local file system3. Client scans and compares with local file system
4. Client backs up files
© 2007 IBM Corporation20 Potpourri du Backup
Tivoli Storage, IBM Software Group
Incremental By DateIncremental By DateWhat you get– Reduced time determining which files have changed– Removing processing time on server to query database – Removing network traffic to communicate results
What you give upFl ibilit f b k ti t b d i b k f th ti fil t– Flexibility on scope of backup operation; you must be doing backups of the entire file system
– File backups where changes do not affect date (attribute, mode, ACL)• Rename, copy, move, security change
– Expiration of deleted files– Policy rebinding– Entire file system still needs to be scanned– No free pass for having client and server clocks out of synch
May not be able to use when client and server are in different time zones– May not be able to use when client and server are in different time zones.
When to use this– Not meeting backup windows and …
File system changes purely additive or changes e g code repository or data that is basically– File system changes purely additive or changes, e.g., code repository or data that is basically “additive” (think of data like your family photos)
Can be effective when doing weekly (or periodic) full incremental backupsRead Chapter 9 in the User’s Guide for more critical information on this option
© 2007 IBM Corporation21 Potpourri du Backup
p p
Tivoli Storage, IBM Software Group
File List BackupFile List Backup
2application
1DB3
1
1 A li ti t li t f fil f b k1. Application creates list of files for backup
2. Application passes list of files to client
3. Client backs up files (SELECTIVE backup)
© 2007 IBM Corporation22 Potpourri du Backup
Tivoli Storage, IBM Software Group
File listsFile lists
What you gety g– Selective backup eliminates the query of the TSM Server db and
scan of local file system
What you give up– Time – you have to find some mechanism to feed the file list– Implicit file specifications – file list will not accept wild cards or
directory recursion
When to use thisWhen to use this– Not meeting backup windows and …
File system changes are predictable e g an application is– File system changes are predictable, e.g., an application is creating the changes and it would be easy to interface into the application to get that list of changes
© 2007 IBM Corporation23 Potpourri du Backup
Tivoli Storage, IBM Software Group
Journal Based BackupJournal Based Backup
2journal
1DB3
1
1 Journal monitors file system for changes1. Journal monitors file system for changes
2. Client queries journal for file system changes
3 Cli t b k fil3. Client backs up files
© 2007 IBM Corporation24 Potpourri du Backup
Tivoli Storage, IBM Software Group
Journal Based BackupJournal Based Backup
What you get– Time to determine which files have changed is greatly reduced
What you don’t get– Relief from doing a full progressive (full) incremental backup against a file system on someRelief from doing a full progressive (full) incremental backup against a file system on some
periodic basis:• Initial backup to enable the journal• If a discrepancy is found between journal and TSM Server database• If the velocity of changing files is high enough to cause a notification buffer overrun• If the journal service is stopped and restarted• In general, it is a good idea to do periodic progressive incremental backups
When to use thisN t ti b k i d d– Not meeting backup window and …
– Small number of files (< 1,000,000) and small number of changes between backups (<1,000,000)
– Large number of objects (<10 000 000) with 10-15% change rate try JBB (should emphasize a– Large number of objects (<10,000,000) with 10-15% change rate try JBB (should emphasize a low change velocity, too… large numbers of changes over a short timeframe can cause notification buffer overflows)
– Large number of objects + large number of changes will stress JBB … not a good fit
© 2007 IBM Corporation25 Potpourri du Backup
Tivoli Storage, IBM Software Group
Adaptive Sub-file BackupAdaptive Sub file Backup
1cache
3
DB1
1cache
2 3
1. Client determines the file is a candidate for backup
2. Client backs up files
3 During data movement client determines if there is enough information stored in3. During data movement, client determines if there is enough information stored in the local cache to determine which sub-file parts have changed
© 2007 IBM Corporation26 Potpourri du Backup
Tivoli Storage, IBM Software Group
Adaptive Sub-file BackupAdaptive Sub file Backup
What you get– Faster throughput– Reduced storage pool consumption
What you give upWhat you give up– Local cache space (in the tens of megabytes)– A bit of CPU– Restore time (due to restore of base + delta)– Possible out-of-space conditions, if disk space is tight during restore, due to how
file is reconstructed from base + delta
When to use this– Network constrained– Small file profiles (limited to file size < 2 GB)Small file profiles (limited to file size 2 GB)
© 2007 IBM Corporation27 Potpourri du Backup
Tivoli Storage, IBM Software Group
AgendaAgenda
Backup techniques– What was available before ADSM/TSM came on the scene– TSM Progressive incremental backup model and its derivatives– Image backupImage backup
How to determine which backup technique best fits– What problem are you trying to solve– Windows– Other operating systems
© 2007 IBM Corporation28 Potpourri du Backup
Tivoli Storage, IBM Software Group
Image BackupImage Backup
DB1
1. Client sends logical block image of file system to server
© 2007 IBM Corporation29 Potpourri du Backup
Tivoli Storage, IBM Software Group
Image BackupImage Backup
No mapping between file and block
Off-line and static backups vs. on-line backups
What you getF t b k– Faster backups
– No scan time to determine what has changed– Faster overall data movement
What you give up– Ability to restore individual files directly from TSM Server (we know, we know…)
• You would need to restore image to alternate location and pull data directly using Explorer g p y g por equivalent.
© 2007 IBM Corporation30 Potpourri du Backup
Tivoli Storage, IBM Software Group
Image BackupImage Backup
Image backup– Backup at logical volume level– Ability to take image + incremental backups to provide single file restoreOff-line image backupg p– Volume mounted read-only– Available for AIX, Windows, Linux x86, Solaris and HP– Exploited best by flashcopy operationsp y py pOn-line image (dynamic) backup– Volume left on-line
Fuzzy backup of changing data– Fuzzy backup of changing dataOn-line image backup using snapshot– Volume left on-line– Image backup taken at single point-in-time (“crash consistent”)– Available for Windows and Linux x86
© 2007 IBM Corporation31 Potpourri du Backup
Tivoli Storage, IBM Software Group
Image + Incremental by DateImage Incremental by Date
421
DB3
2
f f
51. dsmc backup image sends logical block image of file system to server2. Upon subsequent dsmc backup image –mode=incremental, client queries server
last backup of entire file system3 S t ti t f l t b k f ti fil t3. Server returns timestamp of last backup of entire file system4. Client scans and compares with local file system5. Client creates transactions of files for backup
© 2007 IBM Corporation32 Potpourri du Backup
Tivoli Storage, IBM Software Group
Image + Incremental by Date Restore ScenarioImage Incremental by Date Restore Scenario
1DB
2
1
31. Client request incremental image restore
2. Server returns the base image
3 Server returns additional files which need to be applied to the base image to3. Server returns additional files which need to be applied to the base image to satisfy the recovery point
© 2007 IBM Corporation33 Potpourri du Backup
Tivoli Storage, IBM Software Group
Image + Incremental By DateImage Incremental By Date
What you getat you get– Benefit of image backup plus some individual file protection of
files that are changing
What you give up– See Incremental by DateSee Incremental by Date
– No reconciliation of deleted files
© 2007 IBM Corporation34 Potpourri du Backup
Tivoli Storage, IBM Software Group
Image + Progressive Incremental BackupImage Progressive Incremental Backup
2 341
DB
2
3
3
51. dsmc backup image sends logical block image of file system to server2. Upon subsequent dsmc incremental, client queries server for current view of file system3. Server returns list of files for entire file systemy4. Client scans and compares with local file system5. Client backs up changed files
© 2007 IBM Corporation35 Potpourri du Backup
Tivoli Storage, IBM Software Group
Image + Progressive Incremental Restore ScenarioImage Progressive Incremental Restore Scenario
1
DB2
33
4x
1. Client requests incremental image restore
2. Server returns the base image
3 Server returns additional files which need to be applied to the base image to3. Server returns additional files which need to be applied to the base image to satisfy the recovery point
4. Server optionally returns the list of files which need to be deleted from the base image to satisfy the recovery point
© 2007 IBM Corporation36 Potpourri du Backup
g y y p
Tivoli Storage, IBM Software Group
Image + Progressive IncrementalImage Progressive Incremental
What you gety g– Benefit of image backup restore for improving RTO
– Benefit of progressive incremental backups for better RPO
What you give up– Additional time to take periodic image backups
– Storage space at TSM Server
© 2007 IBM Corporation37 Potpourri du Backup
Tivoli Storage, IBM Software Group
DisclaimerDisclaimer
The performance data contained in the following slides e pe o a ce data co ta ed t e o o g s deswas measured in a controlled environment. Results obtained in other operating environments can vary significantly, depending on factors such as system workload and configuration. Accordingly, this data does not constitute a performance guarantee or warrantynot constitute a performance guarantee or warranty.
© 2007 IBM Corporation38 Potpourri du Backup
Tivoli Storage, IBM Software Group
Improved incremental backup memory usageImproved incremental backup memory usage
Incremental backup, 1 million files workload, Win2003 server, 1 WinXP client, LAN, TCP/IP, Disk storage poolIBM x360, 4-way 1.6GHz Xeon MP server, 6 GB, IBM PC 1-way 3GHz Pentium 4 client, 512 MB, Compression No,
Encryption No Storage pool CRC No Bufpoolsize 131072 Txngroupmax 256 TXNBytelimit 2097152 NTFS filespace
DescriptionElapsed time (seconds)
0% changed data 5% changed data 20% changed data
Full incremental backup, memoryeff diskcachem, default db key size 3138 4102 5742
Encryption No, Storage pool CRC No, Bufpoolsize 131072, Txngroupmax 256, TXNBytelimit 2097152, NTFS filespace
Full incremental backup, memoryeff diskcachem, optimal db key size 2002 2850 4631
Full incremental backup, memoryefficient no 939 1620 4320
Full incremental backup, memoryefficient yes 813 1329 2479
Incremental-by-date incremental backup 707 1384 4699
Journal-based incremental backup 2 1995 6133
The disk cache method writes server inventory data to a client disk file
Tivoli Storage Manager Version 5.4.0 Incremental Backup PerformanceWindows XP Client (3GHz, 512 MB RAM)
1 million 1KB files workload
The disk cache method uses less than 20 MB of memoryThe disk cache method is generally slower than other methods
3000
4000
5000
6000
7000
ed ti
me
(sec
)
Full incremental backup,memoryefficientbackup diskcachemethod,default db key size
Full incremental backup,memoryefficientbackup diskcachemethod,optimal db key size
Full incremental backup,memoryefficientbackup no
0
1000
2000
3000
0% 5% 20%
Percent changed dataEl
apse Full incremental backup,
memoryefficientbackup yes
Incremental-by-date incremental backup
Journal-based incremental backup
© 2007 IBM Corporation39 Potpourri du Backup
Tivoli Storage, IBM Software Group
Improved incremental backup memory usageImproved incremental backup memory usage
Incremental backup, 1 million files workload, Win2003 server, 8 WinXP clients, LAN, TCP/IP, Disk storage poolIBM x360, 4-way 1.6GHz Xeon MP server, 6 GB, IBM PC 1-way 3GHz Pentium 4 clients, 512 MB, Compression No,
Encryption No Storage pool CRC No Bufpoolsize 131072 Txngroupmax 256 TXNBytelimit 2097152 NTFS filespace
DescriptionElapsed time (seconds)
0% changed data 5% changed data 20% changed data
Full incremental backup, memoryeff diskcachem, default db key size 4142 5520 8551
Encryption No, Storage pool CRC No, Bufpoolsize 131072, Txngroupmax 256, TXNBytelimit 2097152, NTFS filespace
Full incremental backup, memoryeff diskcachem, optimal db key size 2969 4168 8120
Full incremental backup, memoryefficient no 1975 3436 7241
Full incremental backup, memoryefficient yes 1086 2177 4763
Incremental-by-date incremental backup 812 2189 5420
Journal-based incremental backup 4 2074 5746
For all tests measured with 8 WinXP clients, the disk cache method was slower than the ' ffi i tb k ' th d
Tivoli Storage Manager Version 5.4.0 Incremental Backup Performance
8 Windows XP Clients (3GHz, 512 MB RAM)1 million 1KB files workload'memoryefficientbackup yes' method
For 5% changed data, this elapsed time difference was measured at +1991 seconds, or 91% longer
4000
5000
6000
7000
8000
9000
ed ti
me
(sec
)
Full incremental backup,memoryefficientbackup diskcachemethod,default db key size
Full incremental backup,memoryefficientbackup diskcachemethod,optimal db key size
Full incremental backup,memoryefficientbackup no
0
1000
2000
3000
4000
0% 5% 20%
Percent changed dataEl
apse Full incremental backup,
memoryefficientbackup yes
Incremental-by-date incremental backup
Journal-based incremental backup
© 2007 IBM Corporation40 Potpourri du Backup
Tivoli Storage, IBM Software Group
Improved incremental backup memory usageImproved incremental backup memory usage
Incremental backup, 21 million files workload, Win2003 server, Win2003 client, LAN, TCP/IP, Disk stgpoolIBM x360, 4-way 1.6GHz Xeon MP server, 6 GB, IBM x346, 2-way 3.4GHz client, 4GB, Compression No, Encryption
No Storage pool CRC No Bufpoolsize 131072 Txngroupmax 256 TXNBytelimit 2097152 NTFS filespace
DescriptionElapsed time (seconds)
0% changed data 5% changed data 20% changed data
Full incremental backup, memoryeff diskcachem, default db key size n/a 33699 35536
No, Storage pool CRC No, Bufpoolsize 131072, Txngroupmax 256, TXNBytelimit 2097152, NTFS filespace
Full incremental backup, memoryeff diskcachem, optimal db key size 18654 21678 24680
Full incremental backup, memoryefficient no Failed Failed Failed
Full incremental backup, memoryefficient yes 13760 18394 23310
Incremental-by-date incremental backup 11262 14679 20902
Journal-based incremental backup 2 10916 Failed
The 'memoryefficient no' method fails for this workload because it needs to use more virtual memory than is
il bl t 32 bit li t
Tivoli Storage Manager Version 5.4.0 Incremental Backup Performance
Windows 2003 (32bit) Client (3.4GHz, 4GB RAM)21 million 1KB files workloadavailable to a 32 bit client
The journal-based backup method fails when there are lots of changed filesThe disk cache method with optimal key size is only 18% slower than the 'memoryefficient yes' method at
20000
25000
30000
35000
40000
ed ti
me
(sec
)
Full incremental backup,memoryefficientbackup diskcachemethod,default db key size
Full incremental backup,memoryefficientbackup diskcachemethod,optimal db key size
Full incremental backup,memoryefficientbackup no
18% slower than the memoryefficient yes method at 5%
0
5000
10000
15000
0% 5% 20%
Percent changed dataEl
apse Full incremental backup,
memoryefficientbackup yes
Incremental-by-date incremental backup
Journal-based incremental backup
© 2007 IBM Corporation41 Potpourri du Backup
Tivoli Storage, IBM Software Group
Improved incremental backup memory usageImproved incremental backup memory usage
Incremental backup, 21 million files workload, Win2003 server, Win2003 x64 client, LAN, TCP/IP, Disk stgpoolIntel 4-way 3GHz (Paxville dual core, hyperthread) server, IBM x346, 2-way 3.4GHz client, 4GB, Compression No,
Encryption No Storage pool CRC No Bufpoolsize 131072 Txngroupmax 256 TXNBytelimit 2097152 NTFS filespace
DescriptionElapsed time (seconds)
0% changed data 5% changed data 20% changed data
Full incremental backup, memoryeff diskcachem, default db key size n/a 42496 45117
Encryption No, Storage pool CRC No, Bufpoolsize 131072, Txngroupmax 256, TXNBytelimit 2097152, NTFS filespace
Full incremental backup, memoryeff diskcachem, optimal db key size n/a 21327 24350
Full incremental backup, memoryefficient no 23652 27176 30088
Full incremental backup, memoryefficient yes 12724 16244 20714
Incremental-by-date incremental backup n/a 13681 17974
Journal-based incremental backup 2 12075 Failed
The disk cache method was faster than 'memoryefficientbackup no' method on Windows x64
Tivoli Storage Manager Version 5.4.0 Incremental Backup Performance
Windows 2003 x64 Client (3.4GHz, 4GB RAM)21 million 1KB files workload
64bit OS allows larger process virtual memory, but large real memory demand cause pagingThe disk cache method disk I/O is more efficient than paging disk I/O
25000
30000
35000
40000
45000
50000
ed ti
me
(sec
)
Full incremental backup,memoryefficientbackup diskcachemethod,default db key size
Full incremental backup,memoryefficientbackup diskcachemethod,optimal db key size
Full incremental backup,memoryefficientbackup no
0
5000
10000
15000
20000
0% 5% 20%
Percent changed dataEl
apse Full incremental backup,
memoryefficientbackup yes
Incremental-by-date incremental backup
Journal-based incremental backup
© 2007 IBM Corporation42 Potpourri du Backup
Tivoli Storage, IBM Software Group
AgendaAgenda
Backup techniques– What was available before ADSM/TSM came on the scene– TSM Progressive incremental backup model and its derivatives– Image backupImage backup
How to determine which backup technique best fits– What problem are you trying to solve– Windows– Other operating systems
© 2007 IBM Corporation43 Potpourri du Backup
Tivoli Storage, IBM Software Group
What Problem Are You Trying to Solve?What Problem Are You Trying to Solve?
RTO and RPO
Progressive Incremental backup is still most comprehensive backup– Best method to detect any type of file change– Individual file restore
$64,000 question is “Why can’t you use progressive incremental backup?”backup?– Memory– Backup window
© 2007 IBM Corporation44 Potpourri du Backup
Tivoli Storage, IBM Software Group
AgendaAgenda
Backup techniques– What was available before ADSM/TSM came on the scene– TSM Progressive incremental backup model and its derivatives– Image backupImage backup
How to determine which backup technique best fits– What problem are you trying to solve– Windows– Other operating systems
© 2007 IBM Corporation45 Potpourri du Backup
Tivoli Storage, IBM Software Group
toolstools
tsmstats.inits stats– Introduced in TSM 5.4.0 Backup-Archive Client
direx exedirex.exe– Coming soon from TSM
others
© 2007 IBM Corporation46 Potpourri du Backup
Tivoli Storage, IBM Software Group
tsmstats initsmstats.ini
[fileSystemStatistics.\\patmos\m$]
maxPath=68
totalLocalFiles=1381
totalLocalDirs=552totalLocalDirs=552
maxDirCount=30
maxDirName=M:\test\Hiragana
© 2007 IBM Corporation47 Potpourri du Backup
Tivoli Storage, IBM Software Group
Windows – In Your ToolkitWindows In Your Toolkit
Progressive incremental backup
Journal based backup
Virtual mount points
Adaptive sub-file backup
Image backup (on-line and “used block only”)
© 2007 IBM Corporation48 Potpourri du Backup
Tivoli Storage, IBM Software Group
Windows – Attacking Memory ProblemsWindows Attacking Memory Problems
You need to solve the memory problem first– JBB needs progressive incremental before the journal is enabled– JBB will fall-back to a progressive incremental at some point– Incremental by date needs to use a periodic full progressive
incremental at some point
Is memoryefficient yes going to workIs memoryefficient yes going to work– Run direx.exe to find out biggest directory bottleneck
• Most objects in single directory * 300 < 2 GB ?– Then use memoryefficient yesThen use memoryefficient yes
• Most objects in single directory * 300 > 2 GB ?– Memoryefficient yes not going to solve the problem
> Use memoryefficient diskcache or> Break-up file system orBreak up file system or> Use image incremental
If memoryefficient backup is going to help, you can now consider JBB to further reduce backup window
© 2007 IBM Corporation49 Potpourri du Backup
consider JBB to further reduce backup window
Tivoli Storage, IBM Software Group
Windows – Attacking the Backup WindowWindows Attacking the Backup Window
Consider JBBCo s de J– JBB will have to fall back to progressive incremental model at
some point• Notification buffer overrun (this is a “safety valve”)
– How often is this going to happen in your environment
H ft t l t thi– How often can you tolerate this
© 2007 IBM Corporation50 Potpourri du Backup
Tivoli Storage, IBM Software Group
JBB – Tracking Down the Buffer Overrun Safety ValveJBB Tracking Down the Buffer Overrun Safety Valve
Run TSM journal daemon stand-alone (i.e., not u S jou a dae o sta d a o e ( e , otcommunicating with the B-A client)– Look for notification buffer overflow errors in error logg
New TSM 5.4.1.2 trace flag helps you see activity in journal daemonj– Can use to tune journal daemon exclude list
© 2007 IBM Corporation51 Potpourri du Backup
Tivoli Storage, IBM Software Group
Creating Stand-Alone JBB DaemonCreating Stand Alone JBB Daemon
Modify journal .ini file to journal one or more file systems– Can use configuration wizard (just don’t start the service).
Modify the .ini file– Add trace semantics (see next page)– Modify communications (named pipe) so that the B-A client can’t talk to it
Start the journal serviceStart the journal service– Alternatively run in foreground: tsmjbbd.exe i
© 2007 IBM Corporation52 Potpourri du Backup
Tivoli Storage, IBM Software Group
tsmjbbd initsmjbbd.ini
…[JournalSettings]
fl jbbfilTraceflags=jbbfilemoncsvTracefile=M:\jbbtraceJournalPipe=\\.\pipe\bogusPipeE l jbb lErrorlog=jbberror.logJournalDir=C:\Program Files\Tivoli\TSM\baclient…
© 2007 IBM Corporation53 Potpourri du Backup
Tivoli Storage, IBM Software Group
Buffer Overrun Safety Valve (jbberror log)Buffer Overrun Safety Valve (jbberror.log)
……06/04/2007 15:48:02 fifoQInsert(00A4ED04): Queue threshold reached, thread 960 returned from wait.06/04/2007 15:48:02 fifoQInsert(00A4ED04): Queue threshold reached, thread 960 will block until queue is under threshold.06/04/2007 15:48:02 fifoQInsert(00A4ED04): Queue threshold reached thread 960 returned06/04/2007 15:48:02 fifoQInsert(00A4ED04): Queue threshold reached, thread 960 returned from wait.06/04/2007 15:48:05 psFsMonitorThread(tid 960): Notification buffer overrun for monitored FS 'C:\', journal will be reset.06/04/2007 15:48:05 jnlDbCntrl(): Resetting journal for fs 'C:'.…
© 2007 IBM Corporation54 Potpourri du Backup
Tivoli Storage, IBM Software Group
Now What?Now What?
How often are you seeing buffer overruns?o o te a e you see g bu e o e u s– More often then backup window?
• you will not be able to use JBB effectively– Sometimes?
• You have to decide if it is acceptableNe er?– Never?• WOOHOO!
JBB is a good fit• JBB is a good fit
Don’t forget you can tune JBB’s exclude list and eliminate unnecessary journal entries
© 2007 IBM Corporation55 Potpourri du Backup
eliminate unnecessary journal entries
Tivoli Storage, IBM Software Group
Image BackupImage Backup
Consider using imageg g– Can’t resolve the memory problems
– Too many changes (> 30%) for JBB or progressive incremental backup
– Most of the file system is small files (avg. size < 1 MB)
– Need faster recovery time then file-level restore can provide
Try to keep the file change rate < 30% when using image and incremental combinations
© 2007 IBM Corporation56 Potpourri du Backup
Tivoli Storage, IBM Software Group
WindowsWindows
Other thoughtsg– Compression … if you have relatively new CPU and no hardware
compression… why not?
T k f l d li t– Take care of your exclude lists• Aggressive use of EXCLUDE.DIR where possible
– Open File Support• Are there files that are open that you need protected, e.g., ntuser.dat on
workstations?• … and can you not get the files off-line in some other manner, e.g., presched
command?command?
© 2007 IBM Corporation57 Potpourri du Backup
Tivoli Storage, IBM Software Group
AgendaAgenda
Backup techniques– What was available before ADSM/TSM came on the scene– TSM Progressive incremental backup model and its derivatives– Image backupImage backup
How to determine which backup technique best fits– What problem are you trying to solve– Windows– Other operating systems
© 2007 IBM Corporation58 Potpourri du Backup
Tivoli Storage, IBM Software Group
AIX – In Your ToolkitAIX In Your Toolkit
Progressive incremental backupProgressive incremental backup
Journal based backupJournal based backup
Virtual mount pointsVirtual mount points
Adaptive sub-file backupdapt e sub e bac up
Image backupg p
© 2007 IBM Corporation59 Potpourri du Backup
Tivoli Storage, IBM Software Group
Linux – In Your ToolkitLinux In Your Toolkit
Progressive incremental backupg p
Journal based backupJournal based backup
Virtual mount pointsVirtual mount points
Ad ti b fil b kAdaptive sub-file backup
Image backup (on-line)
© 2007 IBM Corporation60 Potpourri du Backup
Tivoli Storage, IBM Software Group
UNIX – In Your ToolkitUNIX In Your Toolkit
Progressive incremental backupg p
Journal based backupJournal based backup
Virtual mount pointsVirtual mount points
Ad ti b fil b kAdaptive sub-file backup
Image backup (off-line)
© 2007 IBM Corporation61 Potpourri du Backup
Tivoli Storage, IBM Software Group
Macintosh and NetWare – In Your ToolkitMacintosh and NetWare In Your Toolkit
Progressive incremental backupg p
Journal based backupJournal based backup
Virtual mount pointsVirtual mount points
Ad ti b fil b kAdaptive sub-file backup
Image backup
© 2007 IBM Corporation62 Potpourri du Backup
Tivoli Storage, IBM Software Group
Other Platforms – Attacking Memory ProblemsOther Platforms Attacking Memory Problems
You need to solve the memory problem first– JBB (AIX) will have to be enabled; and it will fall back to a
progressive incremental model at some point– Incremental by date needs to use a periodic full progressive– Incremental by date needs to use a periodic full progressive
incremental at some point
Is memoryefficient yes going to work– Run direx.exe to find out biggest directory bottleneck
• Most objects in single directory * 300 < 2 GB ?– Then use memoryefficient yesMost objects in single director * 300 > 2 GB ?• Most objects in single directory * 300 > 2 GB ?– Memoryefficient yes not going to solve the problem
> Use memoryefficient diskcache or> Break-up files system or
U i i t l> Use image incremental
If memoryefficient backup is going to help, now consider JBB (AIX) to further reduce backup window
© 2007 IBM Corporation63 Potpourri du Backup
( )
Tivoli Storage, IBM Software Group
AIX – Attacking the Backup WindowAIX Attacking the Backup Window
Consider JBBCo s de J– JBB will have to fall back to progressive incremental model at
some point• Notification buffer overrun (this is a “safety valve”)
– How often this will happen in your environment
H ft t l t thi– How often can you tolerate this
© 2007 IBM Corporation64 Potpourri du Backup
Tivoli Storage, IBM Software Group
JBB – Tracking Down the Buffer Overrun Safety ValveJBB Tracking Down the Buffer Overrun Safety Valve
Run TSM journal daemon stand-alone (i.e., not u S jou a dae o sta d a o e ( e , otcommunicating with the B-A client)– Look for notification buffer overflow errors in error logg
New TSM 5.4.1.2 trace flag helps you see activity in journal daemonj– Can use to tune journal daemon exclude list
© 2007 IBM Corporation65 Potpourri du Backup
Tivoli Storage, IBM Software Group
AIX - Creating Stand-Alone JBB DaemonAIX Creating Stand Alone JBB Daemon
Modify journal .ini file to journal one or more file systemsod y jou a e to jou a o e o o e e syste s
Modify the .ini fileAdd t ti ( t )– Add trace semantics (see next page)
– Rename daemon so that the B-A client can’t talk to it
Start the journal daemon
© 2007 IBM Corporation66 Potpourri du Backup
Tivoli Storage, IBM Software Group
tsmjbbd initsmjbbd.ini
…[JournalSettings]
fl jbbfilTraceflags=jbbfilemoncsvTracefile=M:\jbbtraceErrorlog=jbberror.log…
© 2007 IBM Corporation67 Potpourri du Backup
Tivoli Storage, IBM Software Group
Now What?Now What?
How often are you seeing buffer overruns?o o te a e you see g bu e o e u s– More often then backup window? You will not be able to use JBB
effectively
– Sometimes? You have to decide if it is acceptable
– Never? WOOHOO! JBB is a good fit
Don’t forget you can tune JBB’s exclude list and li i t j l t ieliminate unnecessary journal entries
© 2007 IBM Corporation68 Potpourri du Backup
Tivoli Storage, IBM Software Group
Image Backup – non-WindowsImage Backup non Windows
Consider using image– Can’t resolve the memory problems– Too many changes (> 30%) for JBB or progressive incremental backup– File system is at least 60% fullFile system is at least 60% full
• UNIX and Linux paying for bandwidth by backing-up unused blocks– Where no on-line image support is available, you can unmount the file system– Most of the file system is small files (ave. < 1 mb) y ( )– Need faster recovery time then file-level restore can provide
Try to keep the file change rate < 30% when using image and incremental combinationscombinations
© 2007 IBM Corporation69 Potpourri du Backup
Tivoli Storage, IBM Software Group
Other PlatformsOther Platforms
Other thoughts– Compression … if you have relatively new CPU and no hardware
compression… why not?– Take care of your exclude lists
• Aggressive use of EXCLUDE.DIR where possible– Snapshot-based file backup or using snapshotroot option with 3rd party
snapshot provider• Are there files that are open that you need protected?• Are there files that are open that you need protected? • … and can you not get the files off-line in some other manner, e.g., presched
command
© 2007 IBM Corporation70 Potpourri du Backup
Tivoli Storage, IBM Software Group
ConclusionConclusion
Myriad of backup techniques in TSM and more to comey ad o bac up tec ques S a d o e to co e
One size does not fit all
Start with progressive incremental backup at it is still most comprehensive backup
Move to other progressive incremental or image techniques as the need arises
© 2007 IBM Corporation71 Potpourri du Backup