2

I want to build a home server for my media data without wasting too much disk space for redundancy.
Redundancy is not the same as backup and as I plan to do daily backups to a second pool of disks, I accept the risk of a disk failure.
I will use smart monitoring for predictive action though. Even if a disk eventually fails without premonition I will not lose more than a day of changes on my rather static archive.
I like the possibility to pool disks with different sizes and mergerfs looks very suitable for this.
In the tutorials which I found, mergerfs is often used together with snapraid but in my eyes snapraid is rather slow.

Btrfs on the other hand has copy on write and a built-in snapshot functionality like NTFS or ZFS.
I know that btrfs can also pool disks but from what I read, in "single" jbod mode the whole pool will be corrupted if one disk fails whereas with mergerfs only the data of the failed disk needs to be recovered.
This can make a big difference in recovery time.
Also mergerfs can spin down inactive disks and thus will possibly extend the lifespan of my disks.

So my question is:
Will a bunch of btrfs disks work together with mergerfs and can I still use the btrfs snapshot functionality on the single disks?

user333869
  • 549
  • 1
  • 4
  • 13

3 Answers3

1

It varies per your requirements. But for things like media which doesn't often change - mergerfs isn't the "worst" idea possible. Especially if it's only going to be a handful of users at a time. I've personally built my own NAS systems since 2005, and have gone through all the various things from every RAID setup I can shake a stick at (hardware, software or on motherboard, 0, 1, 5, 6, 10), as well as going the ZFS route and a few years ago BTRFS.

From my personal stuff (anecdotal so take some salt) the worst of the lot is RAID. Its redundancy is a complete joke in comparison to the other stuff.

If you want some in-between idea with less serious hardware requirements than ZFS, but better "safety" than RAID, then BTRFS is a usable option. If you do want some redundancy and the ability to heal corrupted files (they do happen and you only notice it too late).

But if all you're doing is serving media files, this feels a waste. And things like a union file system starts to make a lot of sense. It's definitely better than going for RAID 0.

For my own home media server, I've been running on some Union FS or other for more than 10 years. Originally using AUFS, but finding it less than stable. Moving to mhddfs and still not seeing a huge improvement. Both had this tendency of just stopping working and needing to be restarted. Around 5 years ago I moved to MergerFS - have yet to experience any issue with it.

Of course, you will have to make sure you either have good backups, or that all the files are easily replaceable. If something goes wrong with a disk, its data is gone (at least not like RAID 0 where all the data for everything is now in the void).

But I prefer having some means to recover / replace disks. It has happened every 2 to 3 years that I had to fix something. And in the MergerFS (and the others) idea I do add SnapRAID parity on as well - it has some benefits beyond just a RAID 5 or 6. Though do note, it is NOT recommended for a server where the files are going to change often. This also does add a "snapshot" idea as SnapRAID can recover files if they were deleted or changed - similar to what you get from the CoW in things like BTRFS - just periodic instead of real-time.

As a side note, I'm rather lazy. Have setup stuff from a command line before. But really, that's not my cup of tea. Of all the NAS-specific operating systems I've tried out to make my life "easier", I prefer Open Media Vault as it includes all the options listed above with a reasonably simple web interface.

As for if BTRFS works underneath MergerFS ... yes it does. MergerFS doesn't care what the underlying file system is. As long as it handles access control and attributes it works as designed. It may be a bit of a waster placing singular BTRFS disks underneath, but that's an option also - just that you lose most of BTRFS's benefits by not using some of its RAID-like features, meaning you might as well just use EXT4 instead.

user180833
  • 11
  • 2
1

When using btrfs with mergerfs and will warn you do not use drive multiplexers with it. Find a motherboard with enough drive ports or real sata controllers to run all the drives. On each drive make a media share folder join them with mergerfs for the shows if you want a drive for redundancy their is a raid you can use called snapraid that works good. this works good for low number of users.

if you want to use btrfs for snaps you need to go all in and start with empty drives and join them all into a btrfs raid of some sort. I have no suggestions on how to do this.

1

Yes, you can use mergerfs with basically any filesystem underneath, even mixing them. Note that if you do intend to use SnapRAID with it, there are some limitations/caveats on filesystems but btrfs is usually fine.

Using something like btrfs with SnapRAID is a bit redundant because in this setup you won't really need/use a couple of the most common features of btrfs which are checksums/scrubs and multiple device support since you're replacing those with SnapRAID and mergerfs.

You'll still be able to use the following special features of btrfs:

  • Send/receive
  • Snapshots
  • Compression
  • Deduplication

Note that for deduplication and compression, SnapRAID will only see the apparent sizes of the files so it will need space for the parity of the duplicated and uncompressed data.

You can use snapshots on each disk/subvolume but SnapRAID will only operate on the main filesystem. You can still access the snapshots though and you can use snapraid-btrfs which is a script that uses read-only snapshots for additional protection around each SnapRAID sync. That way if you delete files they're still in the snapshot until the next sync and if you lose a disk SnapRAID still has access to the full set of data it used for parity in the snapshot, even if some files were since deleted.

Nattgew
  • 1,869
  • 2
  • 13
  • 16