There’s nothing really private about your ip. Whatever you need to do to secure your network you have to do in any solution for this problem.
If you had monitoring, you wouldn't have taken 6 hours to catch it.
I'd say learn HA anyway because it's a good skill, but that doesn't prevent you from having the other parts I mentioned. I say this because, again, unless you are experienced with HA, there will be edge cases where it's not going to do what you though it would do, and your service will be down all the same. Monitoring/alerting and one-click/shell script install will be much more valuable in the short-mid term.
HA involves many factors: service uptime, link uptime, db uptime, etc. I'd probably put a reverse proxy in front and use the servers as upstream. web servers tend to be more reliable, so in your case a single instance ought to suffice.
Aside from actual HA tools, your most important asset in this stage is a uptime check service that pings your server every n seconds, a reliable backup/restore procedure, and a one-button deployment strategy.
Shits can and will probably happen. What are you going to do when it does? And how fast can you respond? I say this because you most likely won't get HA right in the first, second, or third time, unless you already have tons of experience behind you. Embrace failure and plan accordingly.
Can't you setup a new array with your new 3 drives, copy the data over, then expand your array with your old drives? IIRC zfs added support for expanding pools last year or so?
If you want to use a cloud provider, then your main problem will be uploading this much data and then downloading it back again, both price and time wise.
An easier solution is in fact copying the data over to another storage. Maybe borrow some pendrives from friends/family?
You can buy a PCIe riser for your NIC. This could be a solution for the GPU taking a lot of space. Naturally you still need a slot on the case to attach the NIC to, but at least you will be able to put it into a PCIe slot.
It's fine to have high CPU usage when something is actually running. I meant that it's not normal to have background tasks (open programs that aren't actively being used) consistently consuming too much CPU. 0~5% would be healthy. Maybe even 10%. Most definitely not 75%+.
Wouldn’t we all be very pleased with 5k daily users? Lol
You will be fine. Any VPS/cloud will work for the database don’t forget your backups. In fact, this is where you should invest a bit more and have a managed database solution, so you can scale up if needed, unless you are very savvy with db administration
Airtable, notion, or any of the open source clones. Personally I don’t like any of the popular clones, but I’m picky.
If you don’t feel slowed down, I wouldn’t change a thing honestly. At most consider cabling your media devices instead of streaming over Wi-Fi.
2.5g works on cat5e for reasonable distance. I’d be surprised if anybody today would have something slower.
Is there a typo on the HDD prices? Looks super expensive. With that amount you could have 48TB of NVME drives (assuming you have a controller for that many sticks, but you probably could buy one with the leftover from the sticks)
In general you shouldn't spend more on newer/larger capacity HDDs unless you absolutely need that much storage per slot (that is, you are space constrained), as the prices invariably go down in less than an year. Smaller drives with 12~16TB are a much better TB/$. Check diskprices.com to see current trends.
If you care about speed, relying on HDDs for that is a bad proposition. You are better served by a SSD cache in front of the HDDs. In case of SSD failure, it's much cheaper/faster to replace that SSD than to rebuild your strained RAID.
As for CPU performance, pretty much any newer Intel QuickSync will handle your needs. If money isn't an issue, I'd rather go with Intel NUC 12th gen or newer. I have one and it handles anything I throw at it, including the most demanding 4k transcoding. The NUC and a 4-disk Synology uses less than 50w with the disks being used.