57
Move UnRaid from metal to Proxmox (lemmy.ml)
submitted 6 months ago by jjlinux@lemmy.ml to c/selfhosted@lemmy.world
62 comments fedilink hide all child comments

I have a trusty UnRaid server that has been running great for almost 3 years now, with some kinks and headaches here and there, but mostly very stable. Now I'm entertaining the idea of setting that box up with ProxMox, and running UnRaid virtualized. The reason being that I want to use UnRaid exclusively as a NAS and then run all dockers and VMs on ProxMox (at least that's how I'm picturing it). I would like to know your opinion on this idea. All I have is Nextcloud, Immich, Vaultwarden, Jellyfin, Calibre, Kavita and a Windows VM I use to update some hardware every now and then. I mainly want to do that for the backup capabilities in ProxMox for each instance. Storage is not a concern, and I have 64GB of ECC Ram running in that box. What are the Pros and Cons, or is it even worth it to move all this to ProxMox?

top 50 comments
sorted by: hot top controversial new old
[-] Ransack@lemmy.dbzer0.com 34 points 6 months ago

Use either proxmox or unraid. Don't stack.

They are both great in their own respects but you need to choose what works for you and your hardware.

Up until recently I liked unraid due to being able to use multiple disks with different capacities. You don't really have that freedom with proxmox.

[-] pyrosis@lemmy.world 6 points 6 months ago

To most of your comment I completely agree minus the freedom for choosing different disk sizes. You absolutely can do that with btrfs or just throwing a virtual layer on top of some disks with something like mergerfs.

[-] Ransack@lemmy.dbzer0.com 3 points 6 months ago

You're correct, with a bit more know how and knowledge it's completely doable. Quick question maybe, once you create a pool and are utilizing it, are you able to add/remove drives as needed or does that require additional work to be completed? I am under the impression that the pools can be created with a variety of drives but making any physical adjustments are a bit of trouble.

However, I do appreciate you posting about this, maybe it'll help someone else that might be browsing through here. Thank you.

[-] pyrosis@lemmy.world 3 points 6 months ago

It depends on your needs. It's entirely possible to just format a bunch of disks as xfs and setup some mount points you hand to a union filesystem like mergerfs or whatever. Then you would just hand that to proxmox directly as a storage location. Management can absolutely vary depending how you do this.

At its heart it's just Debian so it has all those abilities of Debian. The web UI is more tuned to vm/lxc management operations. I don't really like the default lvm/ext4 but they do that to give access to snapshots.

I personally just imported an existing zfs pool into proxmox and configured it to my liking. I discovered options like directly passing datasets into lxc containers with lxc options like lxc.mount.entry

I recently finished optimizing my proxmox for performance in regards to disk io. It's modified with things like log2ram, tmpfs in fstab for /tmp and /var/tmp, tcp congestion control set to cubic, a virtual opnsense heavily modified for 10gb performance, a bunch of zfs media datasets migrated to one media dataset and optimized for performance. Just so many tweaks and knobs to turn in proxmox that can increase performance. Folks even mention docker I've got it contained in an lxc. My active ram usage for all my services down to 7 gigs and disk io jumping .9 - 8%. That's crazy but it just works.

[-] jjlinux@lemmy.ml 3 points 6 months ago

And that's why I chose to ask here. More heads put together come up with better choices. Watching this TechHut video (https://www.youtube.com/watch?v=ahOXQM4416Q) and another one from Christian Lempa (https://www.youtube.com/watch?v=M3pKprTdNqQ) is what led me to think it could be an idea.

I guess it's the "add another server" to route for me.

Thanks so much.

[-] Ransack@lemmy.dbzer0.com 4 points 6 months ago

Best of luck jjlinux. I hope you have a ton of fun getting your system up and running.

load more comments (1 replies)
load more comments (1 replies)
[-] roofuskit@lemmy.world 3 points 6 months ago

If you have a disk controller you can pass through.

[-] NullGator@lemmy.ca 11 points 6 months ago

If you virtualize unraid, unraid wont have direct drive access - you can get around this by getting an HBA card and forwarding that to the unraid VM. Others have mentioned that proxmox doesn't have docker support, I personally run docker containers within lxc boxes on proxmox. There are solutions to make managing containers easier, like portainer, if you want to go down that route.

[-] jjlinux@lemmy.ml 2 points 6 months ago

After Ai watched Lempa's video virtualizing TrueNAS passing through all drives on ProxMox, I started searching to see if anyone had tried the same with UnRaid, and TechHut actually did it.

https://www.youtube.com/watch?v=ahOXQM4416Q

However, my use case is somewhat different than his, and he's just a hobbyist like me, so I'm much more comfortable asking in this community where it's highly likely that someone already crashed and burned before me, lol.

I'm thinking I'll take the advise of just building a new server for ProxMox, and then use my current UnRaid box exclusively for storage. That should be somewhat safer, right?

[-] NullGator@lemmy.ca 2 points 6 months ago

That's my current configuration, it works well. Put your storage on a separate network. I use smb shares so my data is password protected, even on that separate network.

Main downside of this is there's more places for failure to occur. If your NAS goes down, there's no storage access for proxmox which may cause service downtime. Alternatively if proxmox goes down, this also causes service downtime. For me this is fine, but something to keep in mind. Ideal solution would be 2 HA clusters for storage and compute, but thats expensive haha.

load more comments (1 replies)
load more comments (3 replies)
[-] just_another_person@lemmy.world 7 points 6 months ago* (last edited 6 months ago)

Oof. No.

Wouldn't do it for a litany of reasons, but the main being that it's not meant for such things. You want it to be as close to the OS and drivers as possible. Anything getting between Unraid managing the disks is overly complex, and asking for trouble. What happens if the container dies? What happens if the container gets OOMkill'd?

If you're not going to use it to manage your disks, then I guess no issues, but there's better suited software for such things.

Isn't Unraid also a VM host of sorts?

[-] jjlinux@lemmy.ml 3 points 6 months ago

Yeah, UnRaid does all of that, but from my very basic testing of ProxMox in an old computer, the VM management is much better than in UnRaid. The same goes for VLAN awareness with just 1 nic.

I'm in no way unsatisfied with UnRaid, but I watched a video by Christian Lempa doing something similar, only with TrueNAS instead of UnRaid, which is what got my brain thinking about all these potential options.

https://www.youtube.com/watch?v=M3pKprTdNqQ

[-] just_another_person@lemmy.world 3 points 6 months ago

There's the question of "CAN I do this?" vs "SHOULD I do this?". I don't think abstracting your main storage handling software away from where it definitely needs to be is going to net you anything positive, but add more issues and complications.

I'm sure you can find videos of people running drivers out of containers just because it's possible. Should you though? Nope.

[-] jjlinux@lemmy.ml 2 points 6 months ago

I do have the advantage of having a mirror of my server 2.5K miles away in my brother's house. That's probably why I'm thinking about being so candidly careless.

I appreciate the great advise. But now I'm willing to take one for the team and come back with either am horror story or an epic win.

BRB.

[-] just_another_person@lemmy.world 3 points 6 months ago

You're thinking about this wrong way though. Why are trying to abstract the thing that keeps your disks working properly? What's your gain here?

load more comments (3 replies)
[-] pyrosis@lemmy.world 2 points 6 months ago

Have you considered the increase in disk io and that hypervisor prefer to be in control of all hardware? Including disks...

If you are set on proxmox consider that it can directly share your data itself. This could be made easy with cockpit and the zfs plugin. The plugin helps if you have existing pools. Both can be installed directly on proxmox and present a separate web UI with different options for system management.

The safe things here to use are the filesharing and pool management operations. Basically use the proxmox webui for everything it permits first.

Either way have fun.

[-] jjlinux@lemmy.ml 1 points 6 months ago

I actually never considered this. And if I'm understanding you correctly, this would render using UnRaid unnecessary.

This is great info. I'm going to fit my current ProxMox test rig with a few disks I have (old small disks I have replaced over the years that still work) and test this option first. This might make this easier.

If this works out, I can still keep the server I set up off-site to mirror my storage, right? Even if that is still UnRaid? I need more coffee.

[-] pyrosis@lemmy.world 2 points 6 months ago

Yup you can. In fact you likely should and will probably find yourself improving disk io dramatically compared to your original thoughts doing this. It's better in my opinion to let the hypervisor manage disks operations. That means in my opinion it should also share files with smb and NFS especially if you are already considering nas type operations.

Since proxmox supports zfs out of the box along with btrfs and even XFS you have a myriad of options. You combine that with cockpit and you have a nice management interface.

I went the zfs route because I'm familiar with it and I appreciate it's native sharing options built into the filesystem. It's cool to have the option to create a new dataset off the pool and directly pass it into a new lxc container.

[-] jjlinux@lemmy.ml 1 points 6 months ago

I'm very inclined to use this method instead.

I would like to ask for some suggestions on the initial process to migrate the data from UnRaid.

Considering that:

  • My disk pool is made out of 2 10TB disks, for a total of 20TB
  • It also has a 10TB parity disk
  • The pool is using just -6TB of the storage

The option I see is:

  • Get another 10TB disk
  • I can clear the parity drive and copy my data from the pool to that disk for migrating
  • Configure the pool disks to RaidZ and once I complete that, use the other 2 disks as parity pool

Or, I bite the bullet, get brand new 10TB disks, 12 to make it Raidz2 and have a storage pool of 40TB (35 usable?). I'm thinking 4 groups of 3 disks each should do the trick. Then use the same method to migrate my data.

With 64GB of ECC RAM, I should have a pretty swift storage IOPS that way.

[-] pyrosis@lemmy.world 2 points 6 months ago

Another thing to keep in mind with zfs is underlying vm disks will perform better if the zfs pool is a type of mirror or stripe of mirrors. Z1 Z2 type pools are better for media and files. Cm disk io will improve on the mirror type style dramatically. Just passing what I've learned over time in optimizing systems.

[-] jjlinux@lemmy.ml 2 points 6 months ago

I'll be studying that link you sent me deeply before I start my adventure here.

I didn't know this rabbit hole was so deep. Love it!

[-] pyrosis@lemmy.world 2 points 6 months ago

Bookmark this if you utilize zfs at all. It will serve you well.

https://jrs-s.net/2018/08/17/zfs-tuning-cheat-sheet/

You will be amused with zfs performance in proxmox due to all the tuning that is possible. If this is going to be an existing zfs pool keep in mind it's easier to just install proxmox with the zfs option and let it create a zfs rpool during setup. For the rpool tweak a couple options. Make sure ashift is at least 12 during the install or 13 if you are using some crazy fast SSD as proxdisk for the rpool.

It needs to be 12 if it's a modern day spinner and probably a good setting for most ssds. Do not go over 12 if it's a spinning disk.

Now beyond that you can directly import your existing zfs pool into proxmox with a single import command. Assuming you have an existing zfs pool.

In this scenario zfs would be fully maintaining disk operations for both an rpool and a media pool.

You should consider tweaking a couple things to really improve performance via the guide de I linked.

Proxmox vms/zvols live in their own dataset. Before you start getting to crazy creating vms make sure you are taking advantage of all the performance tweaks you can. By default proxmox sets a default record size for all datasets to 128k. qcow2, raw, and even zvols will benefit from record size of 64k because it tends to improve the underlying filesystem performance of things like ext4, XFS, even UFS. Imo it's silly to create vm filesystems like btrfs if you're vm is sitting on top of a cow filesystem.

Another huge improvement is tweaking the compression algorithm. lz4 is blazing fast and should be your default go to for zfs. The new one is pretty good but can slow things down a bit for active operations like active vm disks. So make sure your default compression is lz4 for datasets with vm disks. Honestly it's just a good default to specify for the entire pool. You can select other compressions for datasets with more static data.

If you have a media dataset full of files like music, vids, pics. Setting a record size of 1mb will heavily improve disk io operations.

In proxmox it will default to grabbing half of your memory for arc. Make sure you change that after install. It's a file that defines arc_max in byte number format. Set the max to something more reasonable if you have 64 gigs of memory. You can also define the arc_min

Some other huge improvements? If you are using an SSD for your proxmox install I highly recommend you install log2ram on your hypervisor. It will stop all those constant log writes on your SSD. It will also sync them to disk on a timer and shutdown/reboot. It's also a huge performance and SSD lifespan improvement to migrate /tmp and /var/tmp to tmpfs

So many knobs to turn. I hope you have fun playing with this.

[-] jjlinux@lemmy.ml 2 points 6 months ago

Thanks so much.

All this info brought me back to the drawing board.

This led me to start searching for new components, as I'm pretty sure that I will want to build a new rig and just probably donate my current box.

Thank you, I really appreciate it. My bank account, not so much 🤣🤣

load more comments (2 replies)
load more comments (1 replies)
[-] desentizised@lemm.ee 6 points 6 months ago

It's understandable that you want to take your virtualization-capabilities to the next level but I also don't see the appeal of containerizing unraid like many others here. I started using unraid last autumn and to me it really is about being able to mix drive sizes. It's a backup to my main server's ZFS pool so (fingers crossed) I don't even really worry about drive failures on unraid. (I have double parity on ZFS and single parity on unraid.)

Anyways my point is I started out with 8 SATA slots plus an old USB-based enclosure with i set to JBOD mode and that was a pretty stupid idea. unraid couldn't read SMART data from those USB drives. Every once in a while one of the drives would suddenly show up as having an unsupported partition layout. Couple weeks ago all 5 drives in the enclosure started showing up as unusable. So as you can imagine I dropped that enclosure and now am working solely off the 8 internal slots. I'd imagine that virtualizing unraid's disk access might potentially yield similar issues. At least the comments of people here remind me of my own janky setup.

[-] jjlinux@lemmy.ml 2 points 6 months ago

You do make a great point. I really am feeling more inclined to spinning up a new rig for ProxMox, and leave my UnRaid to do what it's good at in it's bare metal state as it is today.

This self hosting rabbit hole runs scarily deep.

load more comments (3 replies)
[-] Mixel@feddit.de 3 points 6 months ago* (last edited 6 months ago)

I am in the same boat currently and thinking about how I can migrate my stuff over without having a 1 month downtime EDIT: after reading all the comments I'm still not sure if I should do it or like I said even how. I love my unraid it fits me well however I think I also have fallen in love with proxmox

[-] pyrosis@lemmy.world 2 points 6 months ago

It's the production vs development issue. My advice is the old tech advice. "If it's not broken don't try to fix it"

Modified into a separate proxmox development environment. Btw proxmox is perfect for this with vm and container snapshots.

When you get a vm or container in a more production ready state then you can attempt migrations. That way the users don't kill you :)

[-] Mixel@feddit.de 1 points 6 months ago

That is true, I mean I mostly only use my homelab except some game servers that I am running. And you are totally right. Only reason why I want to run proxmox or in general why I have a homelab is to learn more about servers and self hosting. I am currently in the first year of my apprenticeship and I have learned so much since I got my server up and running 😄 and I think I can learn a lot more when I am using proxmox

[-] jjlinux@lemmy.ml 2 points 6 months ago

Yup. I think I'm going to go the 2 servers way after all, but not before I try doing it in one, because, we'll, why not? Isn't that what home labs are about? 🤣🤣🤣

[-] Mixel@feddit.de 2 points 6 months ago

Please keep me up to date what you try and how you are trying to migrate it over! :D and obviously good luck

[-] jjlinux@lemmy.ml 2 points 6 months ago

Absolutely. This is why I love Lemmy as a whole, and my wife hates it.

The combined amount of wisdom I've found here interacting with so many smart individuals is a serious treasure of knowledge and a powerful drive to keep exploring and learning.

[-] savjee@lemmy.world 3 points 5 months ago

I've been running Unraid on top of Proxmox for over 3 years. No problems whatsoever. I initially bought a RAID controller to directly pass the drives to the UnRAID VM. Another option is to passthrough the SATA controller of your motherboard (only possible if you don't use them on the host).

I documented the process on my blog (it's quite straightforward): https://simplyexplained.com/blog/howto-virtualize-unraid-on-proxmox-host/

load more comments (3 replies)
[-] catloaf@lemm.ee 3 points 6 months ago

Proxmox doesn't run docker containers. You can probably install docker to make it run them, but it's not supported.

I also wouldn't run unraid on a virtual disk just to provide storage. Personally, I have one almalinux VM running on Proxmox that runs all my containers and has a big virtual disk to store my media.

[-] HybridSarcasm@lemmy.world 3 points 6 months ago* (last edited 6 months ago)

Proxmox is Debian at its core, which is supported by Docker. There’s no good reason to not run Docker on the bare metal in a homelab. I’d be curious to know what statement Proxmox has made about supporting Docker. I’ve found nothing.

[-] catloaf@lemm.ee 4 points 6 months ago
[-] HybridSarcasm@lemmy.world 3 points 6 months ago

That’s not a definitive support statement about Docker being unsupported. In fact, even in the Admin Guide, it only provides recommendations. The comment I replied said Docker is unsupported by Proxmox. I maintain that there is no such statement from Proxmox.

[-] catloaf@lemm.ee 2 points 6 months ago

Maybe not explicitly unsupported, but I think "it interferes with some mechanisms on which we rely" should be more discouraging than a policy statement.

[-] wreckedcarzz@lemmy.world 2 points 6 months ago

I bought a used machine a couple weeks ago and am setting it up (1st bare metal build), prox with debian vm running docker. I found it annoying that pm doesn't support it natively but the ability to do snapshots through pm is nice, and let's me fuck around more than I would otherwise, slowly build up a machine.

But almost all of the stuff I have running on other machines is just docker containers, so it would be nice if pm just added a checkbox during install or something. (I want to poke at and learn pm, plus mess around with other vms, that's why I didn't do straight Debian)

load more comments (2 replies)
[-] aodhsishaj@lemmy.world 2 points 6 months ago

You can however run any LXC which you can definitely do natively.

[-] brygphilomena@lemmy.world 2 points 6 months ago

I wouldn't, you'll lose a lot not having it manage the disks such as using dissimilar disks for the array and having it spin down unused disks. You might be able to pass disks through so the unraid VM can manage them directly, but it might be harder than I'd personally want to deal with.

If you aren't running VMs much. Truenas scale I believe can do docker well. I've seen a lot of people put that in a VM on proxmox with disks passed through to be used as the NAS portion.

[-] Moonrise2473@feddit.it 2 points 6 months ago

If you want to just use it exclusively as a Nas, then why not truenas?

I have a unRAID server but the nas part is nowhere as good as truenas (slower, worse ad integration)

Main issue with virtualization is the bootable USB with the serial number that's used as DRM

[-] Lifebandit666@feddit.uk 2 points 6 months ago

If you want to move to Proxmox then I say give it a go.

Maybe just keep what you have running and set up another machine to have a play. If you like it, then stick it on your main machine and work out how to replace everything, could be a fun project for you.

I use Proxmox and have Open Media Vault as my NAS. I use SMB/CIFS to share the drives and have a share that Proxmox can use for daily backups, as well as having backups on the main SSD every week. I need to off-site backups but I haven't researched that yet.

I have a Debian VM that runs Docker and have everything running on that except OMV and Home Assistant. I have another Debian VM that I spin up to try things out.

RAM-wise I'm hitting about 12gb so if you have something with 16 lying around you can easily try out most of what you have running already, and if you don't have anything to run it on you're talking under £100 for a mini PC.

Give it a go, I'm sure you can come up with something to run on a mini pc anyway

[-] jjlinux@lemmy.ml 3 points 6 months ago

This is actually good advise. I am running ProxMox on a very old PC (3rd Gen i3 with 8GB of ram), and I really like it, which is why I wanted to move it to my server Box instead.

But now that you mention it, I may be better off keeping my UnRaid box as it is and use is as NAS/Storage exclusively and then build a good box to run everything else from ProxMox.

My wife is going to kill me in my sleep, lol.

load more comments
view more: next ›
this post was submitted on 10 May 2024
57 points (91.3% liked)

Selfhosted

39700 readers
563 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

  1. Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.

  2. No spam posting.

  3. Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.

  4. Don't duplicate the full text of your blog or github here. Just post the link for folks to click.

  5. Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).

  6. No trolling.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 1 year ago
MODERATORS
ruud@lemmy.world
CannaVet@lemmy.world
Loki@lemmy.world
HybridSarcasm@lemmy.world
devve@lemmy.world
HybridSarcasm@lemmy.hybridsarcasm.xyz