collective.recipe.backup version 4
There are lots of changes since version 3.1.
Since the end of 2017, there is a new version 4.0 of collective.recipe.backup. There are lots of changes since version 3.1. Let's see some of the highlights.
Safety and exactness of restore
- When restoring, first run checks for all filestorages and blobstorages. When one of the backups is missing, we quit with an error. This avoids restoring a filestorage and then getting into trouble due to a missing blobstorage backup.
- When restoring to a specific date, find the first blob backup at or before the specified date. Otherwise fail. The repozo script does the same. We used to pick the first blob backup after the specified date, because we assumed that the user would specify the exact date that is in the filestorage backup. Note that the timestamp of the filestorage and blobstorage backups may be a few seconds or minutes apart. So now the user should pick the date of the blob backup or slightly later. This date will give the same result with 3.1 and 4.0. But: when you use the new blob_timestamps == true option, these dates are the same.
Blob timestamps
- Added blob_timestamps option. Default is false. By default we create blobstorage.0. The next time, we rotate this to blobstorage.1 and create a new blobstorage.0. With blob_timestamps = true, we create stable directory names that we do not rotate. They get a timestamp, just like the repozo backup. For example: blobstorage.1972-12-25-01-02-03.
- When backing up a blobstorage, use the timestamp of the latest filestorage backup. If a blob backup with that name is already there, then there were no database changes, so we do not make a backup.
- Automatically remove old blobs backups that have no corresponding filestorage backup. We compare the timestamp of the oldest filestorage backup with the timestamps of the blob backups. This can be the name, if you use blob_timestamps = true, or the modification date of the blob backup. This means that the keep_blob_days option is ignored, unless you use only_blobs = true.
- Note: it is fine to switch to blob_timestamps even when you already have 'old' backups. Restoring those will still work.
- blob_timestamps = true may become the new default later (maybe 4.1). This may even become the only valid value later (maybe 5.0), removing the creation of blobstorage.0. This would simplify the code. If you don't like this, please speak up and create an issue.
Archiving and compressing blobs
- Renamed gzip_blob option to archive_blob. Kept the old name as alias for backwards compatibility. This makes room for letting this create an archive without zipping it.
- Added compress_blob option. Default is false. This is only used when the archive_blob option is true. When switched on, it will compress the archive, resulting in a .tar.gz instead of a tar file. When restoring, we always look for both compressed and normal archives. We used to always compress them, but in most cases it hardly decreases the size and it takes a long time anyway. I have seen archiving take 15 seconds, and compressing take an additional 45 seconds. The result was an archive of 5.0 GB instead of 5.1 GB.
- Note that with both archive_blob and blob_timestamps set to true, you get filenames like blobstorage.1972-12-25-01-02-03.tar.
- Added incremental_blobs option. This creates tarballs with only the changes compared to the previous blob backups. This option is ignored when the archive_blob option is false.
Various
- No longer create the fullbackup script by default. You can still enable it by setting enable_fullbackup to true.
- Added Python 3 support. The integration with plone.recipe.zope2instance is not tested there, because there is no Python 3 compatible release of it yet.
Upgrading
- In most cases you can simply use the new version without changes.
- Adding blob_timestamps = true is highly recommended. If you do this, you can remove the keep_blob_days option, unless you use only_blobs = true.
- If you want the fullbackup script, enable it by setting enable_fullbackup to true.
- When you used the gzip_blob option, you should rename this to archive_blob. Maybe enable the compress_blob option, but you are probably better off without this.