I'd suggest checking out WolframRavenwolf on raddit, he does regular LLM tests.
I'm looking at Beyonder 4x7B, Mistral Instruct 2x7B, Laser Dolphin 2x7B, and previously used Una Cybertron.
this post was submitted on 25 Jan 2024
22 points (100.0% liked)
LocalLLaMA
2884 readers
6 users here now
Welcome to LocalLLaMA! Here we discuss running and developing machine learning models at home. Lets explore cutting edge open source neural network technology together.
Get support from the community! Ask questions, share prompts, discuss benchmarks, get hyped at the latest and greatest model releases! Enjoy talking about our awesome hobby.
As ambassadors of the self-hosting machine learning community, we strive to support each other and share our enthusiasm in a positive constructive way.
founded 2 years ago
MODERATORS
Hey thanks ! I'll check these out.
I am using Mistral 7b Instruct for text summary and some light "assistant" type chatting for the last several months. I have been pleased at how accurate it is for my needs, especially given it's size.
I recall alot of trial and error to find models that were compatible with the version of llama-cpp-python that oobabooga uses (at any given time). GGUF should have made the model format (and therefore model selection) more simple, but i imagine there are still nuances that make it more difficult than it should be to find a working model for a noob.
Best of luck, let us know how it goes