20
lemmy high storage usage (lemmy.emphisia.nl)
submitted 11 months ago* (last edited 11 months ago) by youpie@lemmy.emphisia.nl to c/selfhosted@lemmy.world

My lemmy instance seems to be using a lot of storage, I am the only one using it and for like 10 minutes a day. it's currently using about 122 gb, after about 3 months of usage

 ~/l/volumes> sudo du -d 1 -h  .
47G	./pictrs
75G	./postgres
0	./lemmy-ui
122G	.

seeing the usage of other ppl this seems really big? Am i doing someting wrong? edit: it is also using a lot of memory, i dont know exactly how much but it seems to be most of my system's available ram (16 gb)

top 13 comments
sorted by: hot top controversial new old
[-] pnutzh4x0r@lemmy.ndlug.org 12 points 11 months ago

Could be what communities you are subscribed to. I run a small instance with about 3ish users, and here are my stats after about 3 months as well:

9.5G ./pictrs
12G	 ./postgres
8.0K ./lemmy-ui

What version of lemmy are you using? A recent update also introduced some space savings in the database (I think).

[-] youpie@lemmy.emphisia.nl 2 points 11 months ago

how do i check what version i am using?

[-] walden@sub.wetshaving.social 6 points 11 months ago* (last edited 11 months ago)

If viewing in a web browser, scroll to the bottom of the page. It'll say something like "BE: 0.18.5".

Edit: If you are hosting it yourself in docker, don't use the :latest tag. As of right now that will install 0.19 which is still in alpha. Use dessalines/lemmy:0.18.5

[-] poVoq@slrpnk.net 8 points 11 months ago

You used one of these subscribe to all community scripts, no? Otherwise it is hard to reach these figures as a single user instance.

However for the memory, I also have a hard time understanding why it eats up all ram. I am currently restarting the Lemmy backend twice a day to prevent the database from using all the memory and the system subsequently running out of memory. It definitely looks like some sort of memory leak, but not all instances seem effected. I already tried a few things to fine-tune the database and pool size etc. but it doesn't seem to help.

[-] youpie@lemmy.emphisia.nl 2 points 11 months ago

oh I used like a tool to speed up federation, maybe that caused it? it was called lcs maybe that caused it?

[-] poVoq@slrpnk.net 10 points 11 months ago

yes, mostly. This tool basically instructs your lemmy instance to try and fetch the entire Lemmyverse and mirror it on your server.

[-] hitagi@ani.social 5 points 11 months ago

I don't remember which update it was but newer versions of Lemmy use significantly less database storage.

[-] poVoq@slrpnk.net 7 points 11 months ago* (last edited 11 months ago)

It reduced it shortly, but the activity table is still (unecessarily) caching most of the lemmyverse for 3 months AFAIK.

Our database went from 30GB or so down to 7GB with that update, but now it is back to 30GB or so. I even did a full database vaccuum lately, but it only shaved off 2-3GB or so.

If my understanding is correct, then the activity table probably doesn't need to store more than a week or so, instead of the current 3 months. But this is something hard-coded in Lemmy and can't be easily configured.

[-] meldrik@lemmy.wtf 3 points 11 months ago

It’s all the images being cached from other servers. If you haven’t uploaded any images to your instance, you can just delete the content of the pictrs folder.

[-] poVoq@slrpnk.net 2 points 11 months ago

This will also delete all the user and community avatars though, as sadly Lemmy does not distinguish these from other images.

[-] meldrik@lemmy.wtf 2 points 11 months ago

Wasn’t it a single user instance?

[-] youpie@lemmy.emphisia.nl 1 points 11 months ago

And if i did upload images on my instance haha

[-] meldrik@lemmy.wtf 1 points 11 months ago

Then you would also have to delete those images. Unfortunately, there's no easy way to delete the cache of images that pictrs stores :(

this post was submitted on 23 Oct 2023
20 points (83.3% liked)

Selfhosted

39253 readers
475 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

  1. Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.

  2. No spam posting.

  3. Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.

  4. Don't duplicate the full text of your blog or github here. Just post the link for folks to click.

  5. Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).

  6. No trolling.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 1 year ago
MODERATORS