best method do use amd GPU for inference on linux (feddit.de)

submitted 1 year ago by Mixel@feddit.de to c/localllama@sh.itjust.works

0 comments fedilink hide all child comments

So what is currently the best and easiest way to use an AMD GPU for reference I own a rx6700xt and wanted to run 13B model maybe superhot but I'm not sure if my vram is enough for that Since now I always sticked with llamacpp since it's quiet easy to setup Does anyone have any suggestion?

no comments (yet)

sorted by: hot top controversial new old

there doesn't seem to be anything here

this post was submitted on 02 Jul 2023

1 points (100.0% liked)

LocalLLaMA

2244 readers

1 users here now

Community to discuss about LLaMA, the large language model created by Meta AI.

This is intended to be a replacement for r/LocalLLaMA on Reddit.

founded 1 year ago

MODERATORS

pax@sh.itjust.works

SkySyrup@sh.itjust.works

noneabove1182@sh.itjust.works