[-] hendrik@palaver.p3x.de 1 points 14 hours ago* (last edited 14 hours ago)

That was my question... How much on-chip memory do they have? And what are applications for that amount of memory? I think an image generator needs like 4-5GB and a LLM that's smart enough as a general porpose chatbot needs like 8-10GB. More will be better. And at that point you'd better make it unified memory like with the M-series Macs or other APUs? Or this isn't targeted at generative AI but some other applications. Hence my question.

[-] hendrik@palaver.p3x.de 2 points 18 hours ago* (last edited 18 hours ago)

Depending on the chip, they have somewhere from 100 to 400 GB/s. I'm not sure on the numbers on Intel processors. I think the consumer processors have about 50 - 80 GB/s. (~Alder Lake, dual channel DDR5) Mine seems to have way less. And a recent GPU will be somewhere in the range of 400 to 1000 GB/s. But consumer graphics cards stop at 24GB of VRAM and these flagship models are super expensive. Even compared to Apple products.

The people from the llama.cpp project did some measurements and I believe the Apple "Metal" framework seems to outperform the x86 computers by an order of magnitude or so. I'm not sure, it's been some time since i skimmed the discussions on their Github page.

[-] hendrik@palaver.p3x.de 6 points 18 hours ago* (last edited 18 hours ago)

Ich frage mich sowieso immer, warum Leute gegen Hilfe sind. Ich meine, klar ist es verwerflich mit Waffen zu handeln. Und teuer ist es wahrscheinlich auch. Aber die Alternative führt ja auch zu Konsequenzen, die wir dann morgen bezahlen müssen. Und Putin will ja bekanntermaßen die Sowjetunion wieder in altem Glanz erstrahlen lassen. Da geben wir mit dieser Entscheidung die baltischen Staaten et cetera auch mit auf. Oder wir verschieben das Problem nur um ein paar Jahre auf nachdem die Ukraine eingenommen ist. Letztendlich wird es aber dann ein größeres Problem geworden sein.

[-] hendrik@palaver.p3x.de -1 points 19 hours ago* (last edited 18 hours ago)

Uh, ist das EU Petitionsportal safe? Und möchte ich da all meine Daten, Adresse etc eintragen?

[-] hendrik@palaver.p3x.de 4 points 19 hours ago* (last edited 19 hours ago)

Get a different hobby. Find some activity you're interested in. Then focus on that and slowly let go. And keep in mind what matters to you.

[-] hendrik@palaver.p3x.de 9 points 19 hours ago

What kind of AI workloads are these NPUs good at? I mean it can't be most of generative AI like LLMs, since that's mainly limited by the memory bandwith and at this point it doesn't really matter if you have a NPU, GPU or CPU... You first need lots of fast RAM and a wide interface to it.

[-] hendrik@palaver.p3x.de 2 points 19 hours ago

The Apple chips also have a wide interface to the RAM. That means you can run chatbots (LLMs) and other AI workloads that are memory-bound at crazy speeds compared to an Intel (or AMD) computer.

[-] hendrik@palaver.p3x.de 3 points 19 hours ago

Ja, Englisch und Deutsch sind auch beides germanische Sprachen, sind also durchaus enger verwandt. Wobei ich finde, Englisch ist auch sowieso eine der einfachsten Sprachen zu erlernen. Man muss kein der/die/das mitlernen, die unregelmäßigen Verben sind finde ich ein Witz gegen Deutsch und Französisch (was ich mal in der Schule hatte), wo es ja zu jeder Regel ohnehin zig Ausnahmen gibt … Ich hab mal etwas im Internet herumgeschaut, die Leute sagen man kann B2 Deutsch so in circa 1-2 Jahren nebenher erlernen.

[-] hendrik@palaver.p3x.de 10 points 19 hours ago* (last edited 19 hours ago)

Software: https://github.com/awesome-selfhosted/awesome-selfhosted

Guide: https://github.com/mikeroyal/Self-Hosting-Guide

As a beginner you might want to start out with one of the all-in-one turnkey operating systems like yunohost.org , dietPi.com or unRaid or a bunch of others (see the awesome-selfhosted list)

[-] hendrik@palaver.p3x.de 15 points 1 day ago

Has that been tried since 1790 when the french decided to behead all the rich people?

[-] hendrik@palaver.p3x.de 64 points 1 day ago* (last edited 1 day ago)

It's shiny, they advertise, put in a money to spread the word. And the onboarding process probably is way easier?! Also back when Mastodon was in the media, it wasn't yet the right time. Now, especially with Musk, it is. And the attention is on Bluesky since that is newer and what's hyped right now.

81
submitted 1 month ago* (last edited 1 month ago) by hendrik@palaver.p3x.de to c/piracy@lemmy.dbzer0.com

Seems they recently changed something on Spotify and all the tools I've tried fail now. And DownOnSpot which seems promising has received a cease and desist letter and got taken down. What do you people use? I want something that actually fetches the audio from Spotify, not just rip it from YouTube. And it has to work as of now. Does the latest commit from DownOnSpot work? Back when I tested it a few weeks ago it failed due to some API changes. Are there other tools floating around?

6
submitted 2 months ago* (last edited 3 weeks ago) by hendrik@palaver.p3x.de to c/localllama@sh.itjust.works

I just found https://www.arliai.com/ who offer LLM inference for quite cheap. Without rate-limits and unlimited token generation. No-logging policy and they have an OpenAI compatible API.

I've been using runpod.io previously but that's a whole different service as they sell compute and the customers have to build their own Docker images and run them in their cloud, by the hour/second.

Should I switch to ArliAI? Does anyone have some experience with them? Or can recommend another nice inference service? I still refuse to pay $1.000 for a GPU and then also pay for electricity when I can use some $5/month cloud service and it'd last me 16 years before I reach the price of buying a decent GPU...

Edit: Saw their $5 tier only includes models up to 12B parameters, so I'm not sure anymore. For larger models I'd need to pay close to what other inference services cost.

Edit2: I discarded the idea. 7B parameter models and one 12B one is a bit small to pay for. I can do that at home thanks to llama.cpp

111
submitted 2 months ago* (last edited 2 months ago) by hendrik@palaver.p3x.de to c/fediverse@lemmy.world

tl;dr: Be excellent to each other, do something constructive here?

I'm not sure anymore where the Threadiverse is headed. (The Threadiverse being this threaded part of the Fediverse, i.e. Lemmy, MBin, PieFed, ...)
In my time here, I've met a lot of nice people and had meaningful conversations and learned lots of things. At the same time, it's always been a mixed bag. We've always had quite some argumentative people here, trolls, ... I've seen people hate on and yell at each other, and do all kinds of destructive things. My issue with that is: Negative behavior is disproportionately affecting the atmosphere. And I'd argue we have nowhere enough nice behavior to even that out.

I don't see Lemmy grow for quite some time now. Seems it's now leveling off at a bit less that 50k monthly active users. And I don't see how that'd change. I'm missing some clear vision/idea of where we want to be headed. And I miss an atmosphere that makes people want to join or stay here, of all of the places on the internet. The saying is: "If you don't go forwards you go backwards". I'm not sure if this applies... At least we're not shrinking anymore.

And I'm always unsure if the tone and atmosphere here changes subtly and gradually. I've always disagreed with a few dynamics here. But lately it feels like we're on the decline, at least to me. I occasionally keep an eye on the votes on my comments. And seems I'm getting fewer of them. Sometimes I reply to a post and not a single person interacts. Even OP seems to have abandoned their post moments after writing it. And also for nuanced and longer replies, I regularly don't get more than one or two upvotes. I think that used to be a bit better at some point. And I see the same thing happening with other peoples' comments. So it's not just me writing low-quality comments. What does work is stating simple truths. I regularly get some incoming votes with those. But my vision of this place isn't spreading simple truths, but have proper and meaningful discussions, learn things and new perspectives or just mingle with people or talk. But judging by the votes I observe, that isn't appreciated by the community here.

Another pet peeve of mine is the link aggregator aspect of Lemmy. I'd say at least 80% of Lemmy is about dumping some political (or tech) news articles. Lots of them don't generate any engagement. Lots of them are really low-effort. OP just dumps something somewhere, no body text added, no info about what's interesting about it. And people don't even read those articles. They just read the title and react (emotionally) to that. In the end probably neither OP nor the audience read the article and it's just littering the place. Burying and diminishing other, meaningful content. (With that said: There are also nice (news) discussions going on at the same time. And Lemmy is meant to be a link aggregator. It's just that my perception is: it's skewed towards low quality, low engagement and random noise.)

A few people here also don't really like political debate. And there's no escape from it here on Lemmy since so much revolves around that. And nowadays politics is about strong opinions, emotions and emotional reactions. And often limited to that. The dynamics of Lemmy reinforce the negative aspect of that, because the time when you're most incentivized to reply or react is, when it triggers some strong emotion in you, for example you strongly disagree with a comment and that makes you want to counter it and write your own opinion underneath. If you agree, you don't feel a strong emotion and you don't reply. And the majority of users seems to also forget to upvote in that case, as I lined out earlier. And we also don't write nuanced answers, dissect complex things and examine it from all angles. That's just effort and it's not as rewarding for the brain to do that as it is pointing out that someone is wrong. So it just fosters an atmosphere of being argumentative.

Prospect

I think we have several ways of steering the community:

  1. Technology: Features in the software, design choices that foster good behavior.
  2. Moderation: Give toxic people the boot, or delete content that drags down the place. Following: What remains is nice people and not adverse content.
  3. The community

I'd say 1 and 2 go without saying. (Not that everything is perfect with those...) But it really boils down to 3: The community. This is a fairly participatory place. We are the ones shaping the tone and atmosphere. And it's our place. It's kind of our obligation to care for it if we want to see it go somewhere. Isn't it?

So what's your vision of this place? Do you have some idea on where you'd like it to go? Practical ideas on how to achieve it?
Do you even agree with my perception of the dynamics here, and the implications and conclusions I came up with?

view more: next ›

hendrik

joined 4 months ago