Open Source Backup Conference 2014: Workshop bareos introduction, by Philipp Storz
0 likes490 views
The document is a workshop guide for the Bareos backup software, detailing installation, configuration, and usage. It covers topics such as virtual machine setup, communication architecture, and configuration file syntax, while also providing hands-on exercises for backup and restore operations. The guide includes examples of common tasks, commands, and configuration adjustments to optimize backup processes.
![Job: Definition of a Job
combines the other resources to a runnable backup job
Job {
[...]
Client = bareos-fd # what client to backup?
FileSet = "Full Set" # which files to backup?
Schedule = "WeeklyCycle" # when to backup?
Storage = File # where to backup?
Messages = Standard # where to send messages?
Pool = File # what target pool?
}](https://image.slidesharecdn.com/workshopbareosintroductionbyphilippstorz-141024051627-conversion-gate02/85/Open-Source-Backup-Conference-2014-Workshop-bareos-introduction-by-Philipp-Storz-27-320.jpg)
![mhvtl
$ lsscsi -g
[0:0:0:0] disk VBOX HARDDISK 1.0 /dev/sda /dev/sg0
[3:0:0:0] mediumx STK L700 0105 /dev/sch0 /dev/sg2
[3:0:1:0] tape IBM ULT3580-TD5 0105 /dev/st0 /dev/sg1
$ mtx -f /dev/tape/by-id/scsi-SSTK_L700_XYZZY_A status
Storage Changer /dev/tape/by-id/scsi-SSTK_L700_XYZZY_A:1 Drives, 43 Slots ( 4 Import
Data Transfer Element 0:Empty
Storage Element 1:Full :VolumeTag=E01001L5
Storage Element 2:Full :VolumeTag=E01002L5
[...]
Storage Element 37:Full :VolumeTag=F01037L5
Storage Element 38:Full :VolumeTag=F01038L5
Storage Element 39:Full :VolumeTag=F01039L5
Storage Element 40 IMPORT/EXPORT:Empty
Storage Element 41 IMPORT/EXPORT:Empty
Storage Element 42 IMPORT/EXPORT:Empty
Storage Element 43 IMPORT/EXPORT:Empty](https://image.slidesharecdn.com/workshopbareosintroductionbyphilippstorz-141024051627-conversion-gate02/85/Open-Source-Backup-Conference-2014-Workshop-bareos-introduction-by-Philipp-Storz-34-320.jpg)
![estimate listing
*estimate listing
[..]
Using Catalog "MyCatalog"
Connecting to Client bareos-fd at bareos:9102
-rwxr-xr-x 1 root root 13973 2014-09-17 15:15:46 /usr/sbin/grub-install
-rw-r--r-- 1 root root 987 2014-09-17 15:15:46 /usr/sbin/envvars
lrwxrwxrwx 1 root root 24 2014-09-18 14:22:29 /usr/sbin/httpd2 -
[..]
-rwxr-xr-x 1 root root 14768 2014-09-17 15:15:46 /usr/sbin/switch_root
-rwxr-xr-x 1 root root 1593 2014-09-17 15:15:46 /usr/sbin/grubonce
drwxr-xr-x 2 root root 12288 2014-09-18 14:22:29 /usr/sbin
2000 OK estimate files=310 bytes=18,987,521](https://image.slidesharecdn.com/workshopbareosintroductionbyphilippstorz-141024051627-conversion-gate02/85/Open-Source-Backup-Conference-2014-Workshop-bareos-introduction-by-Philipp-Storz-37-320.jpg)
![restore files
*restore
To select the JobIds, you have the following choices:
1: List last 20 Jobs run
2: List Jobs where a given File is saved
3: Enter list of comma separated JobIds to select
4: Enter SQL list command
5: Select the most recent backup for a client
[..]
5
[...]
You have selected the following JobId: 1
Building directory tree for JobId(s) 1 ... ++++++++++++++++++++++++++++++++++++++++
309 files inserted into the tree.
cwd is: /
$ find *
[...]
$ mark /usr/sbin/mtx
1 file marked.
$ done
[...]
OK to run? (yes/mod/no): yes](https://image.slidesharecdn.com/workshopbareosintroductionbyphilippstorz-141024051627-conversion-gate02/85/Open-Source-Backup-Conference-2014-Workshop-bareos-introduction-by-Philipp-Storz-40-320.jpg)
![status scheduler
shows what jobs are triggered by which schedule
Schedule Jobs Triggered
===========================================================
WeeklyCycle
BackupClient1
CopyToTape
WeeklyCycleAfterBackup
BackupCatalog
shows a preview for 7 days
Scheduler Preview for 7 days:
Date Schedule Overrides
==============================================================
Fri 19-Sep-2014 23:05 WeeklyCycle Level=Incremental
Fri 19-Sep-2014 23:10 WeeklyCycleAfterBackup Level=Full
Sat 20-Sep-2014 23:05 WeeklyCycle Level=Incremental
Sat 20-Sep-2014 23:10 WeeklyCycleAfterBackup Level=Full
[...]](https://image.slidesharecdn.com/workshopbareosintroductionbyphilippstorz-141024051627-conversion-gate02/85/Open-Source-Backup-Conference-2014-Workshop-bareos-introduction-by-Philipp-Storz-49-320.jpg)
Open Source Backup Conference 2014: Workshop bareos introduction, by Philipp Storz
- 1. Welcome to the Workshop: Bareos Introduction Introduction Workshop
- 2. Start Make sure you have VirtualBox installed Connect to the network get DHCP address
- 3. Download and unpack Appliance download http://192.168.1.1/bareos_64_Bit_osbconf.x86_64-2014.09.17.ovf.tar.gz unpack appliance tar xzvf bareos_64_Bit_osbconf.x86_64-2014.09.17.ovf.tar.gz bareos_64_Bit_osbconf-2014.09.17/ bareos_64_Bit_osbconf-2014.09.17/bareos_64_Bit_osbconf.x86_64-2014.09.17.ovf bareos_64_Bit_osbconf-2014.09.17/bareos_64_Bit_osbconf.x86_64-2014.09.17.mf bareos_64_Bit_osbconf-2014.09.17/bareos_64_Bit_osbconf.x86_64-2014.09.17-disk1.vmdk
- 4. Import appliance 1. Start Virtualbox 2. File .. Import Appliance 3. Choose bareos_64_Bit_osbconf.x86_642014.09.17. ovf
- 5. Take a snapshot If things go wrong, you can always go back CtrlShifts configure Network: bridged Network to eth0/wlan0 start virtual machine
- 6. Language Configuration Choose Language : German/English Choose Keyboard : AltN Choose Timezone : AltN
- 8. Introduction of the hosts Marco van Wieringen longtime contributor to bacula code founder of bareos main programmer Philipp Storz author of bacula book at open source press founder of bareos programming and coordination
- 9. Introduction of attendees Please tell us your name organization experience with backup software experience with bareos/bacula do you have a config problem to be solved?
- 10. Create teams two persons can solve tasks together
- 11. Bareos Architecture Communication via TCP/IP defined ports are used communication can use TLS
- 12. FileDaemon Runs on Client Computer read, write, verify files read, write ACLs, attributes make VSS snapshots checksum calculation compression/encryption run scripts
- 13. Storage Daemon device access (disk, tape) media changer control read barcodes labels write logical labels run copy and migration jobs handle media errors
- 14. Director handle catalog media and pool handling scheduling determine what to backup backup level does message, statistics and reports run scripts
- 15. Catalog store info about all files, media, jobs PostgreSQL/MySQL/SQLite DB
- 16. bconsole UI for restores query status catalog queries run jobs
- 19. Architecture Test which bareos daemon schedules the backups? where is the administrative data stored? which bareos daemon reads and writes files on the client? which bareos daemon handles media?
- 20. Bareos Configuration Configuration is done in config files Each daemon has its own config file usually in /etc/bareos bareosdir. conf bareosfd. conf bareossd. conf bconsole.conf
- 21. Bareos configuration syntax Configuration files consist of resources directives resources can have subresources resourcename { directive = value directive = value SubResource { directive = value directive = value } }
- 22. Most important Director Resources: Director Fileset Schedule Client Job
- 23. Director Resource: Definition of Directors' properties Director { # define myself Name = bareos-dir QueryFile = "/usr/lib/bareos/scripts/query.sql" Maximum Concurrent Jobs = 1 Password = "UbCeWuuamTN/FNQhNH7rbdmx8X+ra9j0UkMMrSdUmkO4" # Console password Messages = Daemon # remove comment in next line to load plugins from specified directory # Plugin Directory = /usr/lib64/bareos/plugins }
- 24. FileSet: Definition what to backup FileSet { Name = "Full Set" Include { Options { signature = MD5 } File = /usr/sbin }
- 25. Schedule: Definition when to run a backup Schedule { Name = "WeeklyCycle" Run = Full 1st sun at 23:05 Run = Differential 2nd-5th sun at 23:05 Run = Incremental mon-sat at 23:05 }
- 26. Client: Definition of a Client Client { Name = bareos-fd Address = bareos Password = "lecCqzgBjxgM0J3+1adiuLzhy0cPGIHrdYMdtGHMbvKX" # password for File Retention = 30 days # 30 days Job Retention = 6 months # six months AutoPrune = no # Prune expired Jobs/Files }
- 27. Job: Definition of a Job combines the other resources to a runnable backup job Job { [...] Client = bareos-fd # what client to backup? FileSet = "Full Set" # which files to backup? Schedule = "WeeklyCycle" # when to backup? Storage = File # where to backup? Messages = Standard # where to send messages? Pool = File # what target pool? }
- 28. Configuration Test which resource configures when a job is scheduled? which resource configures what files are backed up? which resource configures what client to backup? which resource combines the other resources?
- 29. Full and differential backups Full backup everything differential backup what has changed since last full timestamps are used restore needs diff and full media
- 30. incremental and virtual backups incremental backup what has changed since last backup of any level restore needs all media virtual create new full backup from existing backup data client not involved
- 31. Backup Level Test How is determined what has to be backed up during diff and incr. backups? Which backup level needs the least amount of data on the backup media? Why is differential backup probably more relyable than incremental backup?
- 32. Backup Appliance OpenSUSE 13.1 users root admin password is bareos
- 33. Backup Appliance Full Bareos installation Director Postgresql Catalog DB Filedaemon Storage Daemon File Storage Device writes to /var/lib/bareos/storage Virtual Autochanger Mhvtl Simulates StorageTek L700 One LTO5 drive 40 Slots 4 I/O Ports / Mail Slots Bareos WebUI DisktoDisktoTape configured
- 34. mhvtl $ lsscsi -g [0:0:0:0] disk VBOX HARDDISK 1.0 /dev/sda /dev/sg0 [3:0:0:0] mediumx STK L700 0105 /dev/sch0 /dev/sg2 [3:0:1:0] tape IBM ULT3580-TD5 0105 /dev/st0 /dev/sg1 $ mtx -f /dev/tape/by-id/scsi-SSTK_L700_XYZZY_A status Storage Changer /dev/tape/by-id/scsi-SSTK_L700_XYZZY_A:1 Drives, 43 Slots ( 4 Import Data Transfer Element 0:Empty Storage Element 1:Full :VolumeTag=E01001L5 Storage Element 2:Full :VolumeTag=E01002L5 [...] Storage Element 37:Full :VolumeTag=F01037L5 Storage Element 38:Full :VolumeTag=F01038L5 Storage Element 39:Full :VolumeTag=F01039L5 Storage Element 40 IMPORT/EXPORT:Empty Storage Element 41 IMPORT/EXPORT:Empty Storage Element 42 IMPORT/EXPORT:Empty Storage Element 43 IMPORT/EXPORT:Empty
- 35. Handson start console Connecting to Director bareos:9101 1000 OK: bareos-dir Version: 14.3.0 (21 August 2014) Enter a period to cancel a command. * execute help
- 36. estimate command *estimate The defined Job resources are: 1: BackupClient1 2: CopyToTape 3: BackupCatalog 4: RestoreFiles Select Job resource (1-4): 1 Using Catalog "MyCatalog" Connecting to Client bareos-fd at bareos:9102 2000 OK estimate files=310 bytes=18,987,521
- 37. estimate listing *estimate listing [..] Using Catalog "MyCatalog" Connecting to Client bareos-fd at bareos:9102 -rwxr-xr-x 1 root root 13973 2014-09-17 15:15:46 /usr/sbin/grub-install -rw-r--r-- 1 root root 987 2014-09-17 15:15:46 /usr/sbin/envvars lrwxrwxrwx 1 root root 24 2014-09-18 14:22:29 /usr/sbin/httpd2 - [..] -rwxr-xr-x 1 root root 14768 2014-09-17 15:15:46 /usr/sbin/switch_root -rwxr-xr-x 1 root root 1593 2014-09-17 15:15:46 /usr/sbin/grubonce drwxr-xr-x 2 root root 12288 2014-09-18 14:22:29 /usr/sbin 2000 OK estimate files=310 bytes=18,987,521
- 38. run job BackupClient1 *run Automatically selected Catalog: MyCatalog Using Catalog "MyCatalog" A job name must be specified. The defined Job resources are: 1: BackupClient1 2: CopyToTape 3: BackupCatalog 4: RestoreFiles Select Job resource (1-4): 1 Run Backup job JobName: BackupClient1 Level: Incremental Client: bareos-fd Format: Native FileSet: Full Set Pool: File (From Job resource) Storage: File (From Job resource) When: 2014-09-18 17:16:15 Priority: 10 OK to run? (yes/mod/no): yes Job queued. JobId=1
- 39. check status by looking for messages *messages 18-Sep 17:16 bareos-dir JobId 1: No prior Full backup Job record found. 18-Sep 17:16 bareos-dir JobId 1: No prior or suitable Full backup found in catalog. 18-Sep 17:16 bareos-dir JobId 1: Start Backup JobId 1, Job=BackupClient1.2014-09-18_18-Sep 17:16 bareos-dir JobId 1: Created new Volume "File-0001" in catalog. 18-Sep 17:16 bareos-dir JobId 1: Using Device "FileStorage" to write. 18-Sep 17:16 bareos-sd JobId 1: Labeled new Volume "File-0001" on device "FileStorage" 18-Sep 17:16 bareos-sd JobId 1: Wrote label to prelabeled Volume "File-0001" on device 18-Sep 17:17 bareos-sd JobId 1: Elapsed time=00:00:15, Transfer rate=1.267 M Bytes/second 18-Sep 17:17 bareos-dir JobId 1: Bareos bareos-dir 14.3.0 (21Aug14): Build OS: x86_64-suse-linux-gnu suse openSUSE 13.1 (Bottle) (x86_64) JobId: 1 Job: BackupClient1.2014-09-18_17.16.49_08 Backup Level: Full (upgraded from Incremental) Client: "bareos-fd" 14.3.0 (21Aug14) x86_64-suse-linux-gnu,suse,openSUSE FileSet: "Full Set" 2014-09-18 17:16:52 Pool: "File" (From Job resource) Catalog: "MyCatalog" (From Client resource) Storage: "File" (From Job resource) Scheduled time: 18-Sep-2014 17:16:15 Start time: 18-Sep-2014 17:16:54 End time: 18-Sep-2014 17:17:09 Elapsed time: 15 secs Priority: 10 FD Files Written: 310 SD Files Written: 310 FD Bytes Written: 18,987,521 (18.98 MB) SD Bytes Written: 19,019,534 (19.01 MB) Rate: 1265.8 KB/s Software Compression: None VSS: no
- 40. restore files *restore To select the JobIds, you have the following choices: 1: List last 20 Jobs run 2: List Jobs where a given File is saved 3: Enter list of comma separated JobIds to select 4: Enter SQL list command 5: Select the most recent backup for a client [..] 5 [...] You have selected the following JobId: 1 Building directory tree for JobId(s) 1 ... ++++++++++++++++++++++++++++++++++++++++ 309 files inserted into the tree. cwd is: / $ find * [...] $ mark /usr/sbin/mtx 1 file marked. $ done [...] OK to run? (yes/mod/no): yes
- 41. check restore *mess 18-Sep 17:29 bareos-dir JobId 2: Start Restore Job RestoreFiles.2014-09-18_17.29.41_18-Sep 17:29 bareos-dir JobId 2: Using Device "FileStorage" to read. 18-Sep 17:29 bareos-sd JobId 2: Ready to read from volume "File-0001" on device "FileStorage" 18-Sep 17:29 bareos-sd JobId 2: Forward spacing Volume "File-0001" to file:block 0:196. 18-Sep 17:29 bareos-dir JobId 2: Bareos bareos-dir 14.3.0 (21Aug14): Build OS: x86_64-suse-linux-gnu suse openSUSE 13.1 (Bottle) (x86_64) JobId: 2 Job: RestoreFiles.2014-09-18_17.29.41_09 Restore Client: bareos-fd Start time: 18-Sep-2014 17:29:43 End time: 18-Sep-2014 17:29:45 Elapsed time: 2 secs Files Expected: 1 Files Restored: 1 Bytes Restored: 49,296 Rate: 24.6 KB/s FD Errors: 0 FD termination status: OK SD termination status: check OK in system: Termination: Restore OK $ find /tmp/bareos-restores/ /tmp/bareos-restores/ /tmp/bareos-restores/usr /tmp/bareos-restores/usr/sbin /tmp/bareos-restores/usr/sbin/mtx
- 42. bconsole test What command shows possible available commands? What command is used to show how much data a backup will contain, and how can the exact files be shown? What command is used to run a backup? What command is used to do a recover files?
- 43. Exercise 1 BackupClient1 should additionally backup /etc Hint: reload command can be used to update dir configuration without new start
- 44. Solution for Exercise 1 1. Add line "File = /etc" to FileSet "Full Set" in /etc/bareosdir. conf 2. open bconsole and type "reload" 3. run "estimate listing" to see if /etc would be backed up 4. alternatively, run job "BackupClient1"
- 45. status command shows status of system components
- 46. status director shows next scheduled jobs Scheduled Jobs: Level Type Pri Scheduled Name Volume =================================================================================== Incremental Backup 10 19-Sep-14 23:05 BackupClient1 File-0001 Incremental Copy 10 19-Sep-14 23:05 CopyToTape Full Backup 11 19-Sep-14 23:10 BackupCatalog File-0001 shows running jobs Running Jobs: Console connected at 19-Sep-14 13:51 No Jobs running. show terminated jobs Terminated Jobs: JobId Level Files Bytes Status Finished Name ==================================================================== 1 Full 310 18.98 M OK 18-Sep-14 17:17 BackupClient1 2 1 49.29 K OK 18-Sep-14 17:29 RestoreFiles
- 47. status client shows runnig jobs on client Running Jobs: Director connected at: 19-Sep-14 13:54 No Jobs running. ==== shows terminated jobs on client Terminated Jobs: JobId Level Files Bytes Status Finished Name ====================================================================== 1 Full 310 18.98 M OK 18-Sep-14 17:17 BackupClient1 2 1 49.29 K OK 18-Sep-14 17:29 RestoreFiles 3 Incr 0 0 OK 18-Sep-14 23:05 BackupClient1 ==== info comes from local status file can differ from director view as other director can also connect this client
- 48. status storage shows running jobs shows waiting jobs shows terminated jobs shows device status Autochanger "LTO-Changer" with devices: "Drive-1" (/dev/nst0) Device "FileStorage" (/var/lib/bareos/storage) is not open. Device "Drive-1" (/dev/nst0) is not open. Drive 0 is not loaded. shows volume status Used Volume status: ==== ====
- 49. status scheduler shows what jobs are triggered by which schedule Schedule Jobs Triggered =========================================================== WeeklyCycle BackupClient1 CopyToTape WeeklyCycleAfterBackup BackupCatalog shows a preview for 7 days Scheduler Preview for 7 days: Date Schedule Overrides ============================================================== Fri 19-Sep-2014 23:05 WeeklyCycle Level=Incremental Fri 19-Sep-2014 23:10 WeeklyCycleAfterBackup Level=Full Sat 20-Sep-2014 23:05 WeeklyCycle Level=Incremental Sat 20-Sep-2014 23:10 WeeklyCycleAfterBackup Level=Full [...]
- 50. Exercise 2 Full Backup of Job BackupClient1 should be scheduled in 5 minutes
- 51. Solution for Exercise 2 1. Add line "Run = Full at 15:05" to Schedule "WeeklyCycle" in /etc/bareosdir. conf 2. open bconsole and type "reload" 3. run "status schedule schedule=WeeklyCycle" and check if schedule is updated 4. wait 5 minutes and check if backup starts
- 52. Exercise 3: backup partner's client Part 1: Configure link between director and client
- 53. Solution for Exercise 3 p.1 1. Add a client ressource to your director configuration pointing to your partner's client 2. Add a director ressource to your client's config pointing to your partner's director 3. Restart director and filedaemon 4. run status client to see if your partner's client is accessible
- 54. Exercise 3: backup partner's client Part 2: configure a backup job for partner's client
- 55. Solution for Exercise 3 p.2 1. Create a fileset to backup your partner's client 2. Create a job ressource to backup partner's client 3. Run backup of your partner's client 4. Run restore to your partner's client
- 56. Open talk your questions? other ideas: Disaster recovery usage of bls/bextract check copy to tape setup in vm Philipp Storz, Bareos GmbH & Co. KG Open Source Backup Conference 2014