LocalLLaMA

2884 readers

36 users here now

Welcome to LocalLLaMA! Here we discuss running and developing machine learning models at home. Lets explore cutting edge open source neural network technology together.

Get support from the community! Ask questions, share prompts, discuss benchmarks, get hyped at the latest and greatest model releases! Enjoy talking about our awesome hobby.

As ambassadors of the self-hosting machine learning community, we strive to support each other and share our enthusiasm in a positive constructive way.

founded 2 years ago

MODERATORS

pax@sh.itjust.works

SkySyrup@sh.itjust.works

noneabove1182@sh.itjust.works

Smokeydope@lemmy.world

MonsterBug@sh.itjust.works

How do I get started with RAG (ideally with llama.cpp)? (lemmy.dbzer0.com)

submitted 1 day ago by hok@lemmy.dbzer0.com to c/localllama@sh.itjust.works

1 comments fedilink hide all child comments

I would like my model to know the code libraries I use and help me write code with them. I use llama.cpp's server and web UI for inference, but I have no clue how to get started with RAG, since it seems it is not natively supported with llama.cpp's server implementation. It almost looks like I would need to code my own agent.

I am not interested in commercial offerings or APIs. If you use RAG, how do you do it?

top 1 comments

sorted by: hot top controversial new old

[–] Sandbar_Trekker@lemmy.today 4 points 1 day ago

You can use something like Anything LLM for RAG:

https://github.com/Mintplex-Labs/anything-llm

It works with local models.

https://docs.anythingllm.com/agent/usage#what-is-rag-search-and-how-to-use-it