this post was submitted on 05 Mar 2025
17 points (100.0% liked)

LocalLLaMA

2748 readers
14 users here now

Welcome to LocalLLama! This is a community to discuss local large language models such as LLama, Deepseek, Mistral, and Qwen.

Get support from the community! Ask questions, share prompts, discuss benchmarks, get hyped at the latest and greatest model releases! Enjoy talking about our awesome hobby.

As ambassadors of the self-hosting machine learning community, we strive to support eachother and share our enthusiasm in a positive constructive way.

founded 2 years ago
MODERATORS
pax@sh.itjust.works
SkySyrup@sh.itjust.works
noneabove1182@sh.itjust.works
Smokeydope@lemmy.world
MonsterBug@sh.itjust.works
17
Qwen/QwQ-32B · Hugging Face (huggingface.co)
submitted 2 weeks ago by Lantier@jlai.lu to c/localllama@sh.itjust.works
5 comments fedilink hide all child comments
all 6 comments
sorted by: hot top controversial new old
[–] Lantier@jlai.lu 2 points 2 weeks ago (1 children)

GGUF quants are already out: https://huggingface.co/bartowski/Qwen_QwQ-32B-GGUF

[–] JamonBear@sh.itjust.works 1 points 2 weeks ago

Yay! let's try

ollama run hf.co/bartowski/Qwen_QwQ-32B-GGUF:Q4_K_M

/set parameter num_ctx 32768

[–] morrowind@lemm.ee 2 points 2 weeks ago (1 children)

insane, absolutely insane

[–] Suoko@feddit.it 1 points 2 weeks ago (1 children)

Why insane? For quality, speed, size? I find the coder 1.5b and 3b light and good

[–] morrowind@lemm.ee 3 points 2 weeks ago

It matches R1 in the given benchmarks. R1 has 671B params (36 activated) while this only has 32