14
Beginner questions thread
(sh.itjust.works)
Community to discuss about LLaMA, the large language model created by Meta AI.
This is intended to be a replacement for r/LocalLLaMA on Reddit.
Late to the party, I never got FOSAI working until I found LMStudio, but I have 2 questions:
Is there any way I could utilize my GPU, a Radeon RX6800M (12GB VRAM)? I got Mistral-7B doing 5 tokens/s but it's all running on the CPU.
Is there any model specifically for programming questions? This could be of immense help to my projects without having to ask ChatGPT.
Have you tried the guide on AMD's site? It looks like it's for Windows, and I don't know what you're running. Plus, I use Ollama, so I probably can't be of much help.
For programing, my favorite is Dolphin-Mixtral, but I've had good results with Dolphin-Mistral and Llama2.