Artificial Intelligence

1610 readers

6 users here now

Welcome to the AI Community!

Let's explore AI passionately, foster innovation, and learn together. Follow these guidelines for a vibrant and respectful community:

Be kind and respectful.
Share high-quality contributions.
Stay on-topic.
Enhance accessibility.
Verify information.
Encourage meaningful discussions.

You can access the AI Wiki at the following link: AI Wiki

Let's create a thriving AI community together!

founded 2 years ago

MODERATORS

ikidd@lemmy.world

How to Run Deepseek-R1-0528 Locally (unsloth.ai)

submitted 6 days ago by cm0002@lemmy.world to c/ai_@lemmy.world

2 comments fedilink hide all child comments

you are viewing a single comment's thread
view the rest of the comments

[–] besselj@lemmy.ca 3 points 6 days ago* (last edited 6 days ago) (1 children)

180GB of combined system RAM and VRAM recommended :(

Although it looks like the minimum of 60GB ram might be achievable for some

[–] fubarx@lemmy.world 1 points 6 days ago

M3 Max and M4 Max go up to 128GB combined memory. Only Macs that go higher are the Ultras that go to 192GB.

That limits how many users can run a local server on Apple Silicon. Even if they max out an Ultra, that would be 2 tokens/second. Wonder how much electricity that would burn?