Hey all. I'm exploring the idea of building a desktop PC optimized for running LLMs locally. My two primary use cases are I'd like to be able to add local documents and then talk to my files; I'd also like to use it as a coding assistant. Lower priority use case but something I'm tangentially interested in is image generation using stable diffusion. I don't plan to do any model training, I'll leave that to the pros.
One of the decisions I'm currently working through is whether to create this as a desktop workstation (like a PC build) or as more of a homelab environment (like a "local cloud"). On one hand, I believe a desktop workstation would be easier for me to wrap my head around b/c I've built several gaming PCs, whereas I have no homelab or self hosting experience beyond running a local-only Jellyfin instance on an old laptop. On the other hand, I like the thought a separate, atomic AI hub as like a local cloud if you will, similar to how I think of the NAS as a separate thing. What I like about the separate local cloud thinking is In both cases, the AI hub and/or the NAS can be accessed from any device.
I would like to strike the right balance between budget, power efficiency, and speed. I don't need to set any land speed records, but I would also like to avoid waiting several minutes for responses. I can probably spend up to $2,000 on this project, and I'm located in the US.
My questions for those the community who've gone before me:
- Has anyone build built a desktop workstation and then wished they built it as a server?
- Is there actually much of a difference between a desktop workstation versus a homelab environment when it comes to hardware for AI tasks?
- What other questions I should be asking myself to decide which way to go?
Thanks!
I’m probably stating the obvious, but you can totally access your desktop PC remotely. Tools like SSH, TeamViewer, Moonlight, and others work pretty well for that. The biggest factor by far is whether you plan to game on it too. If so, a gaming PC usually makes way more sense than server hardware.
Otherwise, it really just comes down to raw specs and what's actually available to you locally. VRAM is the most important factor. You'll want plenty of it in your machine. Fast DDR5 RAM is nice to have too.
My personal take? When in doubt, go for the desktop PC. The setup’s easier, upgrades are simpler, and you’ll often find better deals especially if you’re buying used.
Okay, thanks for chiming in. Because I also game on PC, I think I'll scrap the AI server idea and stick with a desktop workstation that will do both gaming and AI tasks.
ETA good point about being able to remote into the desktop workstation from other devices.