4
submitted 3 hours ago* (last edited 3 hours ago) by brucethemoose@lemmy.world to c/localllama@sh.itjust.works

https://huggingface.co/collections/Qwen/qwen25-66e81a666513e518adb90d9e

Qwen 2.5 0.5B, 1.5B, 3B, 7B, 14B, 32B, and 72B just came out, with some variants in some sizes just for math or coding, and base models too.

All Apache licensed, all 128K context, and the 128K seems legit (unlike Mistral).

And it's pretty sick, with a tokenizer that's more efficient than Mistral's or Cohere's and benchmark scores even better than llama 3.1 or mistral in similar sizes, especially with newer metrics like MMLU-Pro and GPQA.

I am running 34B locally, and it seems super smart!

As long as the benchmarks aren't straight up lies/trained, this is massive, and just made a whole bunch of models obsolete.

Get usable quants here:

GGUF: https://huggingface.co/bartowski?search_models=qwen2.5

EXL2: https://huggingface.co/models?sort=modified&search=exl2+qwen2.5

[-] brucethemoose@lemmy.world 1 points 3 hours ago

Oh, and you HAVE to try the new Qwen 2.5 14B.

The whole lineup is freaking sick, 34B it outscoring llama 3.1 70B in a lot of benchmarks, and in personal use it feels super smart.

[-] brucethemoose@lemmy.world 1 points 3 hours ago* (last edited 3 hours ago)

You can try a smaller IQ3 imatrix quantization to speed it up, but 22B is indeed tight for 8GB.

If someone comes out with an AQLM for it, it might completely fit in VRAM, but I'm not sure it would even work for a Pascal card TBH.

[-] brucethemoose@lemmy.world 3 points 1 day ago* (last edited 1 day ago)

I hate turn based combat too, but it was super enjoyable in coop. And it's quite good for being turn based.

It's also real-time outside of combat, FYI.

For solo, I'd probably get the mod that automates your companions, and reduce the difficulty to your taste to compensate.

[-] brucethemoose@lemmy.world 1 points 1 day ago

It should be the first, no question.

[-] brucethemoose@lemmy.world 1 points 2 days ago

I'm sorry, but Democratic leadership, the Washington Post, and the New York Times are not "right wing." The last Republican presidental candidate thje NYT endorsed was Dwight D. Eisenhower in 1956.

They're right wing to you and Lemmy, but Lemmy is not the center of America's political compass. And I'm speaking as a rabid DJT hater who votes straight ticket Democrat, bar one primary I registered republican in just so I could vote against DJT.

[-] brucethemoose@lemmy.world 4 points 2 days ago* (last edited 2 days ago)

Especially if you're mega rich.

[-] brucethemoose@lemmy.world 10 points 2 days ago* (last edited 2 days ago)

“Well, one lesson I’ve learned is that just because I say something to a group and they laugh doesn’t mean it’s going to be all that hilarious as a post on X,” he said in a follow-up post early Monday. “Turns out that jokes are WAY less funny if people don’t know the context and the delivery is plain text."

I knew people like this in real life, who'd say something horrible and follow it up with "It's just a joke," but only if they 'lose' and are called out on it.

They're slimey jerks, and it's utterly miserable to even be around them. And I don't understand why so many would worship/follow Elon and dwell on Twitter for it.

[-] brucethemoose@lemmy.world 2 points 3 days ago

It says center and center-right outlets have a left bias

Which outlets, specifically?

[-] brucethemoose@lemmy.world 23 points 3 days ago* (last edited 3 days ago)

Still an understatement, it deserves it and more.

I don't even like turned based games. I don't like most high fantasy. But holy moly, what a ride BG3 is.

I'm just gonna be pissed of their mixed support of modding (due to wotc) kills the modding community. If Skyrim and Rimworld can have a whole universe of fan content, BG3 should too.

[-] brucethemoose@lemmy.world 29 points 3 days ago

She endorsed Biden before. It wasn't really a surprise.

[-] brucethemoose@lemmy.world 38 points 3 days ago

It's still everywhere in my news/internet diet.

It's bleeding, for sure, but it's big. Its gone bad. But I think its premature to say its collapse is a good thing, because it just won't go away.

[-] brucethemoose@lemmy.world 80 points 3 days ago* (last edited 3 days ago)

It's not dead though, it's still linked to everywhere, from big news to niche communities because it still has that critical mass and inertia.

And I have to be cynical of the Fediverse, but realistically, what replaces it, at least here in the US? Discord? No, thanks, I'd at least rather have information be public.

I'm speaking as someone who has never used Twitter, but I can't ignore it, as much as I'd like to.

65
submitted 2 weeks ago* (last edited 2 weeks ago) by brucethemoose@lemmy.world to c/asklemmy@lemmy.world

Obviously there's not a lot of love for OpenAI and other corporate API generative AI here, but how does the community feel about self hosted models? Especially stuff like the Linux Foundation's Open Model Initiative?

I feel like a lot of people just don't know there are Apache/CC-BY-NC licensed "AI" they can run on sane desktops, right now, that are incredible. I'm thinking of the most recent Command-R, specifically. I can run it on one GPU, and it blows expensive API models away, and it's mine to use.

And there are efforts to kill the power cost of inference and training with stuff like matrix-multiplication free models, open source and legally licensed datasets, cheap training... and OpenAI and such want to shut down all of this because it breaks their monopoly, where they can just outspend everyone scaling , stealiing data and destroying the planet. And it's actually a threat to them.

Again, I feel like corporate social media vs fediverse is a good anology, where one is kinda destroying the planet and the other, while still niche, problematic and a WIP, kills a lot of the downsides.

10

cross-posted from: https://lemmy.world/post/19242887

I can run the full 131K context with a 3.75bpw quantization, and still a very long one at 4bpw. And it should barely be fine-tunable in unsloth as well.

It's pretty much perfect! Unlike the last iteration, they're using very aggressive GQA, which makes the context small, and it feels really smart at long context stuff like storytelling, RAG, document analysis and things like that (whereas Gemma 27B and Mistral Code 22B are probably better suited to short chats/code).

16
submitted 2 weeks ago* (last edited 2 weeks ago) by brucethemoose@lemmy.world to c/fosai@lemmy.world

I can run full 131K context with a 3.75bpw quantization, and still a very long one at 4bpw. And it should barely be fine-tunable in unsloth as well.

It's pretty much perfect! Unlike the last iteration, they're using very aggressive GQA, which makes the context small, and it feels really smart at long context stuff like storytelling, RAG, document analysis and things like that (whereas Gemma 27B and Mistral Code 22B are probably better suited to short chats/code).

29

Senior U.S., Qatari, Egyptian and Israeli officials will meet on Thursday under intense pressure to reach a breakthrough on the Gaza hostage and ceasefire deal.

he heads of the Israeli security and intelligence services told Netanyahu at the meeting on Wednesday that time is running out to reach a deal and emphasized that delay and insistence on certain positions in the negotiations could cost the lives of hostages, a senior Israeli official said.

85
36

HP is apparently testing these upcoming APUs in a single, 8-core configuration.

The Geekbench 5 ST score is around 2100, which is crazy... but not what I really care about. Strix Halo will have a 256 -bit memory bus and 40 CUs, which will make it a monster for local LLM inference.

I am praying AMD sells these things in embedded motherboards with a 128GB+ memory config. Especially in an 8-core config, as I'd rather not burn money and TDP on a 16 core version.

12

cross-posted from: https://lemmy.world/post/16629163

Supposedly for petty personal reasons:

The woman who controls the company, Shari Redstone, snatched defeat from the jaws of victory last week as she scuttled a planned merger with David Ellison's Skydance Media.

Redstone had spent six months negotiating a complicated deal that would have given control of Paramount to Ellison and RedBird Capital, only to call it off as it neared the finish line.

The chief reason for her decision: Her reluctance to let go of a family heirloom she fought very hard to get.

I cross posted this from c/Avatar, but I am a Trekkie too and don't like this one bit.

FYI previous articles seemed to imply the Sony deal is dead.

16

Supposedly for petty personal reasons:

The woman who controls the company, Shari Redstone, snatched defeat from the jaws of victory last week as she scuttled a planned merger with David Ellison's Skydance Media.

Redstone had spent six months negotiating a complicated deal that would have given control of Paramount to Ellison and RedBird Capital, only to call it off as it neared the finish line.

The chief reason for her decision: Her reluctance to let go of a family heirloom she fought very hard to get.

The fandom doesn't want to talk about it, but the Avatar franchise is in trouble.

7
submitted 3 months ago* (last edited 3 months ago) by brucethemoose@lemmy.world to c/avatar@lemmy.world

Avatar Studios seems to be part of Paramount Media, aka the "pay television channels" that I assume Sony is not interested in: https://en.wikipedia.org/wiki/Paramount_Global

And in light of this article: https://deadline.com/2024/05/paramount-sale-hollywood-studio-takeover-history-lessons-1235910245/

That doesn't look good for Avatar Studios. If they are left behind in a Sony sale, it seems the probability of them getting shut down (or just going down with whatever is left of Paramount) is very high.

10

The article is a very fast read because it's Axios, but in a nutshell, either:

  • Skydance gets Paramount intact, but possibly with financial trouble and selling some IP.

  • Sony gets Paramount, but restructures the company and also possibly sells some parts.

  • Nothing happens... and Paramount continues its downward spiral, probably accelerated by a failed sale.

The can of worms opened today, as now Paramount is officially open to a buyout from sony.

I don't like this at all. Avatar is a high budget IP, animesque fantasy, and not historically, proveably profitable like Star Trek/Spongebob. Avatar Studios is a real candidate to be chopped off.

8

As the title says. This includes any visual media, including all 7 Books and other stuff.

What kind screen do you watch it on? What sound setup? What source?

Screen poll: https://strawpoll.com/e6Z28M9aqnN

Source poll: https://strawpoll.com/Q0ZpRmzaVnM

I'm asking this because:

A: I'm curious how this fandom generally consumes the shows

B: I theorize this may have an impact on the experience. Avatar is an audiovisual feast, and I find I get caught up in the art/music more than many viewers seem to. LoK in particular is like a totally different show with high-bitrate HD vs. a bad stream.

view more: next ›

brucethemoose

joined 6 months ago