Ask HN: Versioning Backup

1 point

4 years ago

Hello HN!

I currently have a backup/archival setup which works like this:

  Local Machine --> Syncthing --> cluster of boxes regionally separated --> zfs shares --> restic --> Amazon S3 (soon to be replaced)

As it stands, I have restic running on two servers, however they operate in a sort of primary/secondary mode (of my own design), where restic will only run on the secondary server if it cannot reach the primary server on a designated port and receive a correct message.

So far, this set up has worked well, I have hourly/daily/weekly snapshots on zfs, and restic runs daily archiving all the information.

I've been pretty happy with this setup, it's mostly been designed to protect against loss of data on my main machine (via PEBAK or ransomware type issues).

One noticeable gap I've found is the gap of versioning backup. Similar to something like Dropbox where it tracks the versions of files (up to a certain number), it would be nice if the first layer (i.e. the Syncthing replacement) could handle file versions. Obviously, this wouldn't be useful for items for which versioning isn't required (e.g. video/audio files) or where versioning is implicit (e.g. source code repos), but I found I like having versioning for things like my documents which do not fall into that category. The ideal state would be a program which could be configured for multiple profiles, so I could have video/audio backed up in one way, source code/artifacts backed up another, and finally documents backed up yet another way. I would then be able to view the revisions and restore.

I'm not looking for it to understand the structure of file and the changes (a la Office365 document changes) but at least show me the change times and the file size etc, and I could restore accordingly.

Some solutions I've tried:

- https://bvckup2.com/ (windows-only) -- this one came really close in that it had almost all the features, however it lacked a robust versioning feature. As it stands, it supports a file based feature which moves the file to an "archive" location. There isn't an in-app way to do this, although reviewing the archive folder is helpful.

- git-annex -- this one looked like it would work, but as I understand it, it does not actually store the differences of file versions. If you manually create tags, it will store those, but it won't by default (or atleast I couldn't find it in the git-annex-assistant stuff).

- Syncthing -- while there is a versioning capability as part of Syncthing, it is for inbound changes only, it won't version an outbound change.

- VSC/Time Machine -- these are close, but ultimately seem to be similar versions of creating snapshot at time intervals.

I'll be honest, I haven't spent too much time with git annex, it feels like it could work with a special remote of bup, but I just couldn't figure it out and I also wanted to avoid using bup until it was a bit more stable.

I also tried using NILFS2 (https://www.kernel.org/doc/html/latest/filesystems/nilfs2.html) on one of my boxes to see if I could use that in a similar fashion. While it worked it a fairly interesting manner, I didn't see a lot of broad support and didn't want to go down that route.

Does anyone have an ideas?

HN threads I've read:

- https://news.ycombinator.com/item?id=18692844

- https://news.ycombinator.com/item?id=28758415

- https://news.ycombinator.com/item?id=25758675

- https://news.ycombinator.com/item?id=18689269

- https://news.ycombinator.com/item?id=21211859

- https://news.ycombinator.com/item?id=22855755

- https://news.ycombinator.com/item?id=23281880

- https://news.ycombinator.com/item?id=29863822

- https://news.ycombinator.com/item?id=22253981

- https://news.ycombinator.com/item?id=30481935