Single-binary self-hosted Markdown blog on Rust axum: no database, live search, Typst PDF export, and strong SEO.
axumblogdockermarkdownminijinjarustself-hostedtypstvite
1---
2title: Set up automated server backups with Borg
3slug: set-up-automated-server-backups-with-borg
4date: 2022-06-25
5publish_date: 2022-06-25
6tags: sysadmin
7description: The Borg deduplicating backup program can automate daily, weekly, and monthly backups with a single script saving space and keeping data safe from mistakes.
8cover_image: borg-logo.webp
9---
10
11Borg is a backup program written in Python that has excellent deduplication functionality. For my blog server backups you can see just how much smaller the deduplicated repository size is.
12
13```shell
14--------------------------------------------------------------------------------------
15 Original size Compressed size Deduplicated size
16All archives: 780.76 MB 588.23 MB 141.46 MB
17
18 Unique chunks Total chunks
19Chunk index: 2780 24029
20--------------------------------------------------------------------------------------
21```
22
23If you run your websites on small servers like I do then this comes in handy to save some money. I currently have two layers of backups on my servers:
24
251. Linode provides their "Linode Backup" system. This is a complete backup of the entire disk your server is running on for a reasonable price.
262. Borg backup for daily automated backups that allow for quick "oops" restores.
27
28These backups serve two different purposes. I can do an emergency full system restore using Linode, and a quick Borg restore if I happen to do something stupid, like accidentally delete a blog post.
29
30I currently use a slightly modified script in my `/etc/periodic/daily` folder based on [Borg's quick start automation suggestion](https://borgbackup.readthedocs.io/en/stable/quickstart.html#automating-backups).
31
32```shell
33#!/bin/sh
34
35# Setting this, so the repo does not need to be given on the commandline:
36export BORG_REPO=/srv/backup
37
38# some helpers and error handling:
39info() { printf "\n%s %s\n\n" "$( date )" "$*" >&2; }
40trap 'echo $( date ) Backup interrupted >&2; exit 2' INT TERM
41
42info "Starting backup"
43
44# Backup the most important directories into an archive named after
45# the machine this script is currently running on:
46
47borg create \
48 --verbose \
49 --filter AME \
50 --list \
51 --stats \
52 --show-rc \
53 --compression lz4 \
54 --exclude-caches \
55 \
56 ::'{now}' \
57 /srv/git \
58 /srv/docker \
59 /srv/data \
60 /etc/caddy \
61
62backup_exit=$?
63
64info "Pruning repository"
65
66# Use the `prune` subcommand to maintain 7 daily, 4 weekly and 6 monthly
67# archives of THIS machine.
68
69borg prune \
70 --list \
71 --show-rc \
72 --keep-daily 7 \
73 --keep-weekly 4 \
74 --keep-monthly 6 \
75
76prune_exit=$?
77
78# actually free repo disk space by compacting segments
79
80info "Compacting repository"
81
82borg compact
83
84compact_exit=$?
85
86# use highest exit code as global exit code
87global_exit=$(( backup_exit > prune_exit ? backup_exit : prune_exit ))
88global_exit=$(( compact_exit > global_exit ? compact_exit : global_exit ))
89
90if [ ${global_exit} -eq 0 ]; then
91 info "Backup, Prune, and Compact finished successfully"
92elif [ ${global_exit} -eq 1 ]; then
93 info "Backup, Prune, and/or Compact finished with warnings"
94else
95 info "Backup, Prune, and/or Compact finished with errors"
96fi
97
98exit ${global_exit}
99```
100
101Borg's documentation can explain this better than I can but the gist is:
102
103* I run daily backups on all my unique server files in `/srv` and `/etc`. I don't backup anything that is a system default.
104* Then prune my backups to only keep 7 daily, 4 weekly, and 6 monthly backups at any given time.
105* Then we compact the repository to save space and exit.
106
107A better option for storing Borg backups would be to set up a Borg repo on another server or a platform like [BorgBase](https://www.borgbase.com/). BorgBase is nice since they will notify you by email if a backup doesn't happen or fails however, as I said before, I only use Borg for "oops" backups so if I were to miss a couple I wouldn't stress out about it.
108
109As an extra to this post, I have on my servers a `server-health-check.sh` script that allows me to quickly check things like auto-updates, Borg backups, free memory, and used disk space! I run it every so often just for peace of mind.
110
111```shell
112#!/bin/sh
113
114echo -e "\napk upgrades ------------------------------------------------------------------"
115tail /var/log/apk-autoupgrade.log
116
117echo -e "\nborg backups ------------------------------------------------------------------"
118borg list /srv/backup
119
120echo -e "\nfree memory ------------------------------------------------------------------"
121free -h | head -n2
122
123echo -e "\nfree space ------------------------------------------------------------------"
124df -h | head -n1 && df -h | grep "/dev/sda" | head -n1
125```
126
127The output of this script is pretty well formatted without much effort and looks a little something like this:
128
129```shell
130apk upgrades ------------------------------------------------------------------
131[Sat Jun 25 02:00:00 UTC 2022] fetch http://dl-cdn.alpinelinux.org/alpine/latest-stable/main/x86_64/APKINDEX.tar.gz
132[Sat Jun 25 02:00:00 UTC 2022] fetch http://dl-cdn.alpinelinux.org/alpine/latest-stable/community/x86_64/APKINDEX.tar.gz
133[Sat Jun 25 02:00:00 UTC 2022] OK: 512 MiB in 253 packages
134
135borg backups ------------------------------------------------------------------
1362022-06-19T02:00:00 Sun, 2022-06-19 02:00:01 [1c4a6d43b6ad80b3ee8e890ca5d9978ce60247a9a7a667614764b722f95a4d20]
1372022-06-20T02:00:01 Mon, 2022-06-20 02:00:01 [cf62b24465eb0d9b6296f6a7a752fa905b428b53a086adc5ca70c4d458570934]
1382022-06-21T02:00:07 Tue, 2022-06-21 02:00:08 [450016022ad57454d73815350f70f51c41bb3beda1c10405f41671947dd4fcfd]
1392022-06-22T02:00:01 Wed, 2022-06-22 02:00:01 [85aa754d92e5c33cd1f13abb5aff57a9092e197bf386eb73e60e26663d6c9b7b]
1402022-06-23T02:00:01 Thu, 2022-06-23 02:00:01 [6014ed6f2388302d6b09aecde605e4708b191e805190d154ae7bd7abf7d300c6]
1412022-06-24T02:00:01 Fri, 2022-06-24 02:00:01 [5676379d5d5b234446a438a624ab5e380f61c050d06a5f068191822ff1e8a495]
1422022-06-25T02:00:00 Sat, 2022-06-25 02:00:01 [8a5c05fcd142d48bb799cf13a64ecc22045d87b1d7239e76fa082fb5ef4f875f]
143
144free memory ------------------------------------------------------------------
145 total used free shared buff/cache available
146Mem: 983.8M 626.2M 97.1M 516.0K 260.5M 343.8M
147
148free space ------------------------------------------------------------------
149Filesystem Size Used Available Use% Mounted on
150/dev/sda 24.1G 7.5G 15.4G 33% /
151```
152
153If you're interested in Borg you can read more about it [on Borg's website](https://www.borgbackup.org/), they have extensive and well written documentation.