I'm using an old Synology for backups. But for a NAS, you could build a DIY machine and put TrueNAS on it.
Get an ATX case with lots of drive bays, a SAS HBA (and SAS expander if you want lots of drives), cheap mobo/CPU/RAM and OS of your choice. Spin it up, play with it, settle on an OS you like, and upgrade components as needed.
If you don't like paying for software you should probably avoid Unraid too.
TrueNas for hosting an SMB, PiHole container and a couple VMs. My proper home lab is on another Proxmox server.
Unraid, and TrueNAS are both solid options. You also have Proxmox you can run. It is important to understand though that Unraid and Proxmox are more than just a NAS. They also host VMs and Docker containers. You do have to pay for a license for Unraid though. It is a one time purchase.
The big thing with Unraid compared to the others is that you don't really have to worry about the size of disks to get the protection of a parity drive. As long as your largest drive in the array is the parity drive you are good.
TrueNAS also has paid options but they do have the free version.
Proxmox is also open source. Which may be something that you like.
Personally I use Unraid. It is solid and I have had zero issues.
I use TrueNAS (ZFS) over Unraid because I wanted maximum data reliability. I needed a system with high integrity that I could deploy on multiple computers for backups. Unraid takes less planning and is more forgiving on hardware selection though.
TrueNAS, Unraid, and Synology DSM are all software RAID solutions.
It kind of depends on your use case.
If you're just using it purely for storage to serve up to clients on your network, I'd go with truenas. If you need the performance of raid, I'd also go with truenas.
If you're trying to build an all-in-one media server, unraid is a great choice. To be honest, though, I don't like it for much else, and you have to pay for it.
Synologies are a good option if you need a pretty simple solution, but like you said, you're definitely paying a premium for that simplicity.