Hi everyone, I've been working on my homelab for a year and a half now, and I've tested several approaches to managing NAS and selfhosted applications. My current setup is an old desktop computer that boots into Proxmox, which has two VMs:
- TrueNAS Scale: manages storage, shares and replication.
- Debian 12 w/ docker: for all of my selfhosted applications.
The applications connect to the TrueNAS' storage via NFS. I have two identical HDDs as a mirror, another one that has no failsafe (but it's fine, because the data it contains is non-critical), and an external HDD that I want to use for replication, or some other use I still haven't decided.
Now, the issue is the following. I've noticed that TrueNAS complains that the HDDs are Unhealthy and has complained about checksum errors. It also turns out that it can't run S.M.A.R.T. checks, because instead of using an HBA, I'm directly passing the entire HDDs by ID to the VM. I've read recently that it's discouraged to pass virtualized disks to TrueNAS, as data corruption can occur. And lately I was having trouble with a selfhosted instance of gitea, where data (apparently) got corrupted, and git was throwing errors when you tried to fetch or pull. I don't know if this is related or not.
Now the thing is, I have a very limited budget, so I'm not keen on buying a dedicated HBA just out of a hunch. Is it really needed?
I mean, I know I could run TrueNAS directly, instead of using Proxmox, but I've found TrueNAS to be a pretty crappy Hypervisor (IMHO) in the past.
My main goal is to be able to manage the data that is used in selfhosted applications separately. For example, I want to be able to access Nextcloud's files, even if the docker instance is broken. But maybe this is just an irrational fear, and I should instead backup the entire docker instances and hope for the best, or maybe I'm just misunderstanding how this works.
In any case, I have some data that I want to store and want to reliably archive, and I don't want the docker apps to have too much control over it. That's why I went with the current approach. It has also allowed for very granular control. But it's also a bit more cumbersome, as everytime I want to selfhost a new app, I need to configure datasets, permissions and mounting of NFS shares.
Is there a simpler approach to all this? Or should I just buy an HBA and continue with things as they are? If so, which one should I buy (considering a very limited budget)?
I'm thankful for any advice you can give and for your time. Have a nice day!
You should have all your data separately stored, it shouldn't be locked inside containers, and using a VM hosted on a device to serve the data is a little convoluted
I personally don't like TrueNAS - I'm not a hater, it just doesn't float my boat (but I suspect someone will rage-downvote me π)
So, as an alternative approach, have a look at OpenMediaVault
It's basically a Debian based NAS designed for DIY systems, which serves the local drives but it also has docker on, so feels like it might be a better fit for you.
I tried OMV in the past, but I found TrueNAS to be more intuitive... but that's just personal preference I guess, and I'm not opposed to using OMV. Are you suggesting, then, that I run OMV on bare metal, and use it for everything? Or should it be inside a VM? If it's the former, how easy is it to setup docker, because I'm not that familiar with OMV (it's been a long time since I last checked it out). Is it like installing it in Debian directly? How does it handle the storage?
I always prefer bare metal for the core NAS functionality. There's no benefit in adding a hypervisor layer just to create an NFS / SMB / iSCSI share
OMV comes with it's own bare metal installer, based on Debian, so it's as stable as a rock.
If you've used it before, you're probably aware that it needs it's own drive to install on, then everything else is the bulk storage pool.. I've used various USB / mSATA / M.2 drives over the years and found it's a really good way to segregate things.
I stopped using OMV when - IMO - "core" functions I was using (ie syncthing) became containers, because I have no use for that level of abstraction (but it's less work for the OMV dev to maintain addons, so fair enough)
So, you don't have to install docker, OMV automatically handles it for you.
How much OMV's moved on, I don't know, but I thought it would simplify your setup.
Okay, thank you, that's good to know. However, I don't have two separate devices that I can use to separate the NAS functionality from the Docker functionality, that's why I was using Proxmox in the first place. And, I'm not sure how well Docker can run in OMV. But I'll still keep it in mind as an option, thank you!
I think you've misunderstood
Ok, OMV needs a separate (small) boot drive to install on (ie consider a M.2 / SSD on a USB adapter)
But, then all your (large) storage is used for the NAS.
OMV will run Docker containers, but their data would also be pointed to the large NAS storage.
You're right, I misunderstood. I understand now, thank you for replying with detail. I'm currently still not over my "I like Hypervisors now" phase. If I go back to bare metal, I will most probably use the setup you described. Still, thank you very much, and I'm keeping this thread for future reference.