408

The Fediverse Desperately Needs Sustainable File Hosting (evergreenfiles.com)

submitted 1 week ago* (last edited 1 week ago) by sosodev@lemmy.world to c/fediverse@lemmy.world

92 comments fedilink hide all child comments

Disclaimer: I wrote this article and made this website.

There was some talk of this issue in the recent fediverse inefficiencies thread. I'm hopeful that in the future we'll have a decentralized solution for file hosting but for now I deeply believe that users should pay for their own file hosting.

you are viewing a single comment's thread
view the rest of the comments

[-] poVoq@slrpnk.net 20 points 1 week ago* (last edited 1 week ago)

Have you considered providing something like this: https://jortage.com/ and maybe contribute to their efforts to develop a specific API for that? Source code is here: https://github.com/jortage

[-] sosodev@lemmy.world 21 points 1 week ago

Jortage is a really interesting approach. It definitely helps reduce the impact of the file hosting problem but it doesn’t fully address the underlying cost issue. The cost of storing files grows every month indefinitely while donations typically don’t.

I would like to see a file hosting pool come to lemmy though. So I will look into it. :)

[-] poVoq@slrpnk.net 11 points 1 week ago* (last edited 6 days ago)

Pict-rs that is used by Lemmy to store images already supports S3 type storage, so in theory it should work with Jortage, but I don't think anybody has tested that yet. The people behind Feddit.org might have experimented with it as they expressed interest a while back.

[-] Deebster@lemmy.ml 3 points 1 week ago

I think the major advantage is the deduplication - when an image goes viral across Mastodon (or Lemmy) it's currently stored hundreds or thousands of times, each with its own cost. Do you dedupe (for either your customers' benefit or your own)?

[-] sosodev@lemmy.world 2 points 1 week ago* (last edited 1 week ago)

Are the images duplicated when shared? My understanding is that only a link to the file is replicated across servers and duplication comes from users manually uploading the same file to another server.

My website does not do any deduplication at this time.

[-] Deebster@lemmy.ml 4 points 1 week ago* (last edited 6 days ago)

Yes, for example go to https://infosec.exchange/explore

I see the top post as https://infosec.exchange/@nocontexttrek@mastodon.social/113433063621462027 and the image is https://media.infosec.exchange/infosec.exchange/cache/media_attachments/files/113/433/063/582/671/258/original/71da3801e4e4f08c.png

The link is to the original on https://files.mastodon.social/media_attachments/files/113/433/062/676/773/993/original/f828afef5cc7ed1c.png but when you click image the javascript loads a modal with the local cached version (same image as the thumbnail that infosec.exchange loads.

There's lots of different codebases across the fediverse so perhaps some hotlink, but local copies is the default.

[-] sosodev@lemmy.world 5 points 6 days ago* (last edited 6 days ago)

The Lemmy server config indicates that is an optional setting to improve user privacy so requests don't ever hit the original server from the client. Those cached files are only temporary and will be deleted after some time. So it's not really full blown duplication.

The default setting is to only generate the thumbnails and store those locally (indefinitely?) but even that can be turned off. I checked and it appears that lemmy.world has the thumbnail generation disabled so all images from other instances just link to the original on that instance.

[-] Deebster@lemmy.ml 2 points 6 days ago

Ok, so Lemmy doesn't cause the same amount of duplication, but I'd still argue that dedupe is valuable: it saves on hosting costs (your costs, in this case) and users will get a small advantage in having slightly higher cache hits.