1203
What is this? (Its OC!)
(lemmy.world)
List of icons/services suggested:
- Calibre
- Jitsi
- Kiwix
- Monero (Node)
- Nextcloud
- Pihole
- Ollama (Should at least be able to run tiny-llama 1.1B)
- Open Media Vault
- Syncthing
- VLC Media Player Media Server
Can you afford an Arc A770 or an old RTX 3060?
Used P100s are another good option. Even an RTX 2060 would help a ton.
27B is just really chunky on CPU, unfortunately. There's no way around it. But you may have better luck with MoE models like deepseek chat or Mixtral.