So on ZFS with four disks it’s better to do RAIDZ2 than RAID10, right? Hmm
@wxcafe That depends on whether you want better redundancy or better performance. raid10 will be faster, but if both drives in one of the mirrors fail, all of your data is gone. Z2 will still have your data if any two drives fail. I have an 8 drive Z2 array in my server, with a ZFS mirror on two SSDs for the boot drive.
@wxcafe raid-z2 should provide the same capacity as raid10, but it allows any 2 disks to fail - with raid10, you can lose 2 disks, but only if you lose one disk from each raid1 pair (in practice, treat it as max 1 disk lost).
raid-z2 should be basically the same as raid6 in terms of redundancy, but without as much performance downside as raid6.
@kepstin well the problem with raid5/6 is that if one drive fails another one is more likely to fail while you’re rebuilding, on common RAID implems. Afaiui ZFS negates this because it validates data continuously so drive faults would be caught early.
@wxcafe well, since raid6 and raid-z2 have 2-drive redundancy, having a second drive fail during recovery is not a problem. With raid10, if the *one* drive storing the duplicate data fails during recovery, you're out of luck.
Regardless of raid implementation, you should scrub the array regularly. ZFS automates that. Linux mdraid you should use a cron job.
(edit: fix typo "raid1" → "raid10")
@kepstin but yeah my current md raid5 is scrubbing regularly... I'm just not totally confident in the whole stack that's md/lvm/...
@wxcafe the particular improvement with raid-z in zfs (and btrfs's builtin raid, fwiw) is that it adds data checksumming, which means that not only can it detect silent corruption (bit flips), it also can fall back and read from all available alternate mirrors/parity in that case to find a good copy of the data.
Standard raid5/6 don't know which copy of the data is good/bad, they can only recover from a drive that is dead or *reporting* io errors.
@kepstin yeah that's what I meant, exactly. So I'd trust raidz better than raid5 and raidz2 better than raid6 etc
@wxcafe btw, *regardless* of filesystem, it's important with any raid method that you don't have a timeout mismatch, see https://raid.wiki.kernel.org/index.php/Timeout_Mismatch
If your drive supports erc (check with smartctl -l scterc /dev/sdx), set it to 70 (7 seconds). Otherwise you need to increase the linux block layer timeout to something ridiculous like several minutes.
@wxcafe (drive manufactures annoyingly use erc control support as an upsell feature on "raid" or "nas" branded drives, it's often not available on consumer models targetting windows)
@kepstin yeah, I heard about that and my arrays only use nas drives. not like they're really that much more expensive anyway so...
@wxcafe you can run that "smartctl -l scterc /dev/sdx" command to verify they're set up right then, if it says it's disabled or >7.0 seconds, you'll want to configure it. (Probably add a boot script to run smartctl -l scterc,70,70 /dev/sdx on each drive)
@kepstin yeah they are by default, np :)
@wxcafe If you're only going to have four disks and never add more, and they're SSDs or you don't care about random read speed, yes, pretty much (although there are weird complexities if you have lots of very small files where RAIDZ will take up extra space).