Welcome to the ILTA OSSPG Presentation:

Open SourceOpen Source Backup Tools

with

Nate SmithNate Smith&

D l Q llDale Qualls

Open Source Backupith B lwith Bacula

Bacula Rocks•Free and Open Source Software (FOSS)Software (FOSS)•Bacula•Case Study•ConclusionConclusion

FOSSFOSS

• Free and Open SourceFree and Open Source• Free as in “Free Speech” or “Free as in

beer”beer• English doesn’t have a good word for this.

“Software Libre”

Nate’s PhilFOSSiphyNate s PhilFOSSiphy

• Proprietary software: TheProprietary software: The glass is half empty: “I better get what I paid for”

• FOSS: The glass is half full: “you get what you get and you don’t throw a fit”.

The RealityThe Reality

• Proprietary

• FOSS

FO$$ and your budgetFO$$ and your budget

• It can be a bit of a socialist trade-off TheIt can be a bit of a socialist trade off. The software may be free but it may take more time to install configure and testtime to install, configure, and test.

• There is a trade-off between time and money that balances out in the endmoney that balances out in the end.

• Or does it…..?

FO$$ and your budgetFO$$ and your budget

BaculaBacula

• Open source, network-based, backup and restore software.

• Backup+Dracula“It comes by night and sucks the vital y gessence from your computers.“

BaculaBacula

• Mainly written by Kern SibbaldMainly written by Kern Sibbald• Contributions by others

S f th b t d t d• Some of the best-documented open source software I have seen.

• http://www.bacula.org

BaculaBacula

• Consists of three main ProgramsConsists of three main Programs– File Daemon– Director Daemon ++Director Daemon – Storage Daemon

• These programs can exist together on theThese programs can exist together on the same machine and/or be on separate machines.

BaculaBacula• Jobs are the basic unifying structurey g• Name of Job – must be unique• Type – to do: backup, Backup, Migrate, Admin, Restore

L l d t il f j b t F ll Diff ti l I t l• Level – detail of job type: Full, Differential, Incremental• FileSet – what to files to backup• Client – where to get the files (machine name)Client where to get the files (machine name)• Storage – where to put the files (which hardware) • Pool – which set of Volumes (tapes, disk) to use• Schedule – when to do it

BaculaBacula

• Uses your choice of open sourceUses your choice of open source databases to record files.

Postgres– Postgres– MySQL

SQLite– SQLite • Or not

– Can also use Oracle and probably MS SQL Server.

BaculaBacula

• Additional UtilitiesAdditional Utilities• Graphical Interfaces

W b– Web– wxWidgets

Bacula Sends emailBacula Sends email

Case StudyCase Study

• McKee, Voorhees & Sease, P.L.C.McKee, Voorhees & Sease, P.L.C.20 attorneys, about 55 people totaltotalIntellectual Property “Boutique” firmfirm

Case StudyCase Study

• Superbowl XLll Sunday 2-3-2008Superbowl XLll Sunday 2 3 2008

February 3, 2008 “Superbowl S d ”Sunday”

• “what’s that clicking sound?”D th b RAID• Death by RAID

Case StudyCase Study

• What crashed:What crashed:– NT 4 HP Proliant 3000 File Server

About 300 GB RAID 5 Storage– About 300 GB RAID 5 Storage– 2 DLT backup Drives

7 Mb I t t C ti– 7 Mbps Internet Connection• Issues

– Aging Hardware and software (ArcServe) – Slow Irregular Backup

Case StudyCase Study

• Installed Bacula Late Summer/Fall of 2007Installed Bacula Late Summer/Fall of 2007• Local Bacula Server; 1 U Via Epia running

LinuxLinux.• Seven+ Terabytes of storage on a generic

2 U ff it i Li d SD2 U server; offsite, running Linux and SD• Bacula Client (FD) on servers• Full Backup every Friday Night +

Weekend. Incremental M-F.

Th3s3 Things com3 in Thr33sTh3s3 Things com3 in Thr33s

• Spring Floods of 2008Spring Floods of 2008• An unfortunate decision.

The Bacula ExperienceThe Bacula Experience

The Perfect is the enemy of the goodThe Perfect is the enemy of the good-Voltaire

Parting AdviceParting Advice

• Test (test test) your restoreTest (test, test) your restore• Document your system

D t f k RAID f t• Do not use funky RAID features, per Ontrack data recovery.

• Do not depend solely on RAID• Back up your system, the job you save p y y , j y

may be your own.

Part 2

Open Source MisnomersOpen Source Misnomers

• Open Source is written by amateurs!Open Source is written by amateurs!– In some cases, maybe, but for the most part

you won’t find a more talented group ofyou won t find a more talented group of people. Many Open Source developers are employed by companies such as Sun, Novell, p y y pEMC, Google and numerous other industry leaders for the sole purpose of giving back to th itthe community.

Open Source MisnomersOpen Source Misnomers

• Open Source software is not as secure asOpen Source software is not as secure as commercial software!– Although possible, not likely. One of the great g p , y g

things about open source is that it is exactly that, open. Most established projects have numerous developers (and users) eyeballingnumerous developers (and users) eyeballing the code to make sure that it’s not going to harm you or your environment. It is still good y y gdue diligence to do a bit of research before you implement any software in a production environmentenvironment.

Open Source MisnomersOpen Source Misnomers

• Some Open Source hasn’t been updatedSome Open Source hasn t been updated in years and is at a low release level (like 1.5.22!) )– When software is written it is possible that it is

complete and may not need enhancements d t th hit t f th ddue to the architecture of the code.

– True, release levels are squirrely in Open Source but in most cases a pre 1 0 (or beta)Source but in most cases a pre 1.0 (or beta) release of Open Source is far superior to a 1.0 release of commercial software as the Open SSource developers aren’t driven by market demand.

Pattishall environment• 6 HP DL 380 Servers (SLES 10SP1)

– 8GB RAM– 1TB DAS– 4 W2K servers– 10 XP desktops (for investigative use)0 des tops ( o est gat e use)– 14 SLES 10SP1 guests (file servers and utility servers

(MX, DNS, Nagios, DimDim, Moodle)) • 3 DL 140 Servers (SLES 10SP1)• 3 DL 140 Servers (SLES 10SP1)

– 2 with 4 SLES 10SP1 guests handling email backups and archiving1 ith SLES 10SP1 i d l 750GB d i– 1 with SLES 10SP1 running dual 750GB drives (litigation data fileserver, rsyncing data to the secondary drive, not backed up beyond this.)

Pattishall environmentPattishall environment• 2 HP EVO 510SFF Desktops (CENTOS 5)2 HP EVO 510SFF Desktops (CENTOS 5)

– 384MB RAM750GB HD– 750GB HD

– running rdiff-backup2 C DL (RH 8)• 2 Compaq DL (RH 8)– 256MB RAM– 40GB HD (boot drive), 750GB HD external data

drive– running rsync

Pattishall environment (DR)Pattishall environment (DR)• 4 HP ML 370 Servers (SLES10 SP1)

– 4GB RAM– 300 GB DAS– Dual duty VM hosts and rsync’ing data from

office) • 1 HP PC (SLES10 SP1) used as iSCSI

Host– AMD QuadCore Phenom– 8GB RAM– 4 -1TB SATA Drives (3 TB RAID5)

Why open source?Why open source?

• Fully configurable and/or customizable byFully configurable and/or customizable by on-site talent.

• Community based support keeps it fresh• Community based support keeps it fresh and up-to-date.Fl ibl• Flexible

• Dependability• Cost

rsync and rdiff-backup1 rsync and rdiff backup• Open source backup solutions that utilize

delta (file differential) copying to save time

1

delta (file differential) copying to save time and bandwidthFle ible sol tions that can be tili ed on a• Flexible solutions that can be utilized on a variety of platforms

• “Free” as in beer and as in speech

rsyncy• http://samba.anu.edu.au/rsync/

Maintained by Wayne Davison• Maintained by Wayne Davison • Current release is 3.0.3 (bug fix release) • Provides fast incremental file transfer

across LANs and WANs. Ability to limit bandwidth is built in Can be directed to anbandwidth is built in. Can be directed to an offsite target.Freely available under the GNU GPL• Freely available under the GNU GPL

• Perfect for duping data off-site or to removable storageremovable storage.

rsyncy

• Can be tunneled via SSH for additionalCan be tunneled via SSH for additional security

• Backup entire directory trees or file• Backup entire directory trees or file systemsR t i hi i i d• Retain ownership, permissions and symbolic links on *nix file systems

• Can run on Windows under Cygwin (linux-like environment for Windows)

rdiff-backupp• http://www.nongnu.org/rdiff-backup/• Maintained by Andrew FergusonMaintained by Andrew Ferguson• Current release 1.0.5 (testing 1.2.1) • Backups up only incremental (after initial• Backups up only incremental (after initial

backup) but keeps incremental copies separate Backups are purge able basedseparate. Backups are purge able based on date (to comply with retention policies)

• Can restore from any point in time viaCan restore from any point in time via simple command line

• X-PlatformX Platform

rdiff-backupp• Written using python scripts (easy to

customize)customize).• Can be tunneled via SSH and can use a

CIFS SMBFS or SSHFS targetCIFS, SMBFS or SSHFS target• Backup entire directory trees or file systems

Retain ownership permissions and symbolic• Retain ownership, permissions and symbolic links on *nix file systemsVery fast and reliable set it and forget it!• Very fast and reliable – set it and forget it!

• Good community support, the maintainer releases enhancements and patches quicklyreleases enhancements and patches quickly.