Unfortunately I have a lot of experience with this: attaching permanent array members via USB is a bad idea. OP, if it’s not too late, and assuming you haven’t already and decided to double down on yolo, I’d recommend reading about the downsides of this approach. It is easy to find relevant discussions (and catastrophes) in r/zfs.
Thunderbolt enclosures are a bit more expensive, but they won’t periodically fuck up your shit just because.
This isn’t my first rodeo with ZFS on USB. I’ve been running USB for a few years now. Recently I ran this particular box through a battery of tests and I’m reasonably confident that with my particular set of hardware it’ll be fine. It passed everything I threw at it, once connected to a good port on my machine. But you’re generally right and as you can see I discussed that in the testing thread, and I encountered some issues that I managed to solve. If you think I’ve missed something specific - let me know! 😊
Bus issues usually. Having a disk (or 4) drop out of a ZFS filesystem regularly isn’t a good time.
If you can find a combination of enclosure, driver/firmware and USB port that provides you with a reliable connection to the drive then USB is just another storage bus. It’s generally not recommended because that combination (enclosure, chipset, firmware, driver, port) is so variable from situation to situation but if you know how to address the pitfalls it can usually work fine.
You want ASMedia ASM1351 (heatsinked) or ASM235CM on the device side 🥹
This box has 4x ASM235CM and from the testing I’ve conducted over the last week it seems rock solid, so long as it’s not connected to the Ryzen’s built-in USB controller. It’s been flawless on the B350 chipset’s USB controller.
I think I’ve seen this hypothesis too and it makes sense to me.
If I’m building a new AMD system today, I’d look for a board that exposes more of the chipset-provided USB ports. Otherwise I’d budget for a high quality 4-port PCIe USB controller, if I’m planning to rely a lot on USB on that system.
This article provides some context. Now I do have the latest firmware which should have these fixes but they don’t seem to be foolproof. I’ve seen reports around the web that the firmware improves things but doesn’t completely eliminate them.
If you’ve seen devices disconnecting and reconnecting on occasion, it could be it.
I imagine if someone had to do it this way for whatever reason, Thunderbolt would be more reliable? Assuming it’s true Thunderbolt using a SATA bridge?
Oh no.
Unfortunately I have a lot of experience with this: attaching permanent array members via USB is a bad idea. OP, if it’s not too late, and assuming you haven’t already and decided to double down on yolo, I’d recommend reading about the downsides of this approach. It is easy to find relevant discussions (and catastrophes) in r/zfs.
Thunderbolt enclosures are a bit more expensive, but they won’t periodically fuck up your shit just because.
Thanks for the warning ⚠️🙏
This isn’t my first rodeo with ZFS on USB. I’ve been running USB for a few years now. Recently I ran this particular box through a battery of tests and I’m reasonably confident that with my particular set of hardware it’ll be fine. It passed everything I threw at it, once connected to a good port on my machine. But you’re generally right and as you can see I discussed that in the testing thread, and I encountered some issues that I managed to solve. If you think I’ve missed something specific - let me know! 😊
Is it because USB doesn’t expose proper UUIDs or something?
Bus issues usually. Having a disk (or 4) drop out of a ZFS filesystem regularly isn’t a good time.
If you can find a combination of enclosure, driver/firmware and USB port that provides you with a reliable connection to the drive then USB is just another storage bus. It’s generally not recommended because that combination (enclosure, chipset, firmware, driver, port) is so variable from situation to situation but if you know how to address the pitfalls it can usually work fine.
You want ASMedia ASM1351 (heatsinked) or ASM235CM on the device side 🥹
This box has 4x ASM235CM and from the testing I’ve conducted over the last week it seems rock solid, so long as it’s not connected to the Ryzen’s built-in USB controller. It’s been flawless on the B350 chipset’s USB controller.
Could you elaborate? Do you happen to have any theories as to why?
I have a zen 3 chip and while I’m not planning anything like this, I’m just curious.
I don’t know, but I’d guess the buffered chipset controller has more stability during certain power state transitions.
I think I’ve seen this hypothesis too and it makes sense to me.
If I’m building a new AMD system today, I’d look for a board that exposes more of the chipset-provided USB ports. Otherwise I’d budget for a high quality 4-port PCIe USB controller, if I’m planning to rely a lot on USB on that system.
This article provides some context. Now I do have the latest firmware which should have these fixes but they don’t seem to be foolproof. I’ve seen reports around the web that the firmware improves things but doesn’t completely eliminate them.
If you’ve seen devices disconnecting and reconnecting on occasion, it could be it.
I imagine if someone had to do it this way for whatever reason, Thunderbolt would be more reliable? Assuming it’s true Thunderbolt using a SATA bridge?
Yes - and unless you treat each enclosure as its own failure domain, it will still be a compromise, but it’s a lot better.
deleted by creator