[-] pissy_corn_flakes@alien.top 1 points 11 months ago

I max out my pppoe FTTH install on pfsense with no issues

0

So I've been performing upgrades on my homelab over the past few days. I consolidated several Xeon servers into a single large server. When I switched over to the new rig, I treated myself to the new Gen5 Hyper M.2 card .

Card arrived today from Amazon. Before I move forward, I should highlight some of my jankiness in my setup: I run 2x EVGA 1000W power supplies daisy chained together using an add2psu module. (Looking back, it was a horrible idea..)

My setup was as follows: PSU1 powers my MB, CPU, 2x CPU 8 pins and a GPU. PSU2 provides power to about 24 drives.

This new Gen5 Hyper M.2 card requires 6x PCI power, like a GPU. This is where things go down hill. I provided this power from the 2nd PSU. I guess I never realized, but since the HDDs are 'isolated' within the second PSU, I guess things were safe. But once I added a new lead from PSU2 to power a card that likely is also receiving some power from the motherboard, I probably created a grounding loop? (I'm just guessing).

1st power supply: I complete the upgrade, flick the power switch and go upstairs to my office. By the time I sit down, I'm able to ping the IPMI management interface...but then it goes offline. I start smelling a faint burning smell... I rush down to the basement. It smells NASTY, but no drama: Server is off. Some quick troubleshooting and I determine the 1st PSU is dead, the second one is fine. I swap in a spare 1000W PSUs (Old mining equipment FTW). I look around and don't see anything weird. I ASSumed I wasn't careful and shorted out the add2psu module on something. I fix this potential issue.

I power up the 'new' PSU and sit by and watch.. The computer powers up but the case LEDs are off. Before I can react there's a POP and TONS of smoke starts pouring out the PSU, being blown around by the fan. I quickly cut the power.

Anyways, it was the nastiest smell ever.. and funnily enough, the power supply had some sort of liquid coming out of it from whatever got melted. I was super lucky that none of my equipment got damaged in this process. I learned my lesson and consolidated to a single PSU. I also sent the Hyper M.2 card back as possibly defective.. I'm not going to try a 3rd time!

TLDR: My janky 2PSU system blew up dramatically. But luckily I didn't lose any gear besides the PSUs.

I needed to rant. Sorry for the long post. I have PTSD every time I turn on that computer now...

[-] pissy_corn_flakes@alien.top 0 points 11 months ago

I really get turned off when people post a picture and walk away with little to no detail. What are you trying to accomplish?

[-] pissy_corn_flakes@alien.top 1 points 11 months ago

I run two pfsense routers and use CARP. 1gbps/50 cable modem with 5 static ips available on both pfsenses, and a 1 gbps symmetrical FTTH with 1 static ip also available on both pfsenses (but uses CARP to know which is “live” - it uses pppoe, which makes this possible with 1 ip).

My current single point of failure is my single main switch. Works very well though. I’ve had 100% uptime for over a year!

1

Finally decommissioning two of my older Xeon servers. Going from Xeon E5 26XX machines (1x dual socket, 24/48, 1x SP 8/16) with 256GB ECC combined to 32/64 w/ 1TB ECC. Cant wait.

I'm currently using a Thermaltake server case W200 with pedestal. It's an awesome case, but since buying it I added a rack. I'd like to consolidate into the rack and get rid of the older stuff, taking up floor space.

Any recommendations for something in the 4-5U size with a good amount (24+) of drive bays that supports full height PCI cards? Ideally below $600 USD.

What are people using??

pissy_corn_flakes

joined 1 year ago