42

Hello !piracy@lemmy.dbzer0.com

I intend to upload an archive of a website I scraped for the last few weeks. The torrent is about 4 TB large and contains 400000 folders which in turn contain 40 .jpg's on average (plus some metadata).

Should I just create the torrent with the files as they are right now, or should I put the individual folders in archives (or maybe even the entire torrent?)?

Thanks in advance

all 16 comments
sorted by: hot top controversial new old
[-] fkn@lemmy.world 40 points 1 year ago

People will want to download chicks or parts of the torrent, you should leave them separate.

[-] cesium@sh.itjust.works 6 points 1 year ago

Makes sense. What about individual folders? I'm concerned that the massive amount of small files could slow down the download quite a bit. Unfortunately, I'm not too familiar with the BitTorrent protocol, so I don't know if this would have an impact.

[-] zodo123@lemmy.world 14 points 1 year ago

Small files don’t impact BitTorrent transfers. They’re sent in pieces of a fixed size, and those pieces can contain multiple files.

[-] fkn@lemmy.world 2 points 1 year ago

Depends on final size of the folder zips. 20-30mb? Sure. 200-300? Nah.

[-] TheCraiggers@lemmy.dbzer0.com 15 points 1 year ago

Jpeg is already compressed, so compressing them again won't do anything but make it impossible for people to selectively download just the image/folder they desire.

Metadata on the other hand sounds like text files, which compress very well. Wether the space savings is worth it is hard to answer without more info. I'd personally lean towards not archiving it.

[-] neo@lemmy.comfysnug.space 11 points 1 year ago

Only if you can achieve significant compression. 15% or more.

[-] UntouchedWagons@lemmy.ca 2 points 1 year ago

Leave them as is. I hate having to deal with archives inside torrents unless there's a very good reason for them.

Why not both?

[-] Umbra@kbin.social -4 points 1 year ago

I'd put it all in a single archive.

this post was submitted on 26 Jul 2023
42 points (95.7% liked)

Piracy: ꜱᴀɪʟ ᴛʜᴇ ʜɪɢʜ ꜱᴇᴀꜱ

54420 readers
258 users here now

⚓ Dedicated to the discussion of digital piracy, including ethical problems and legal advancements.

Rules • Full Version

1. Posts must be related to the discussion of digital piracy

2. Don't request invites, trade, sell, or self-promote

3. Don't request or link to specific pirated titles, including DMs

4. Don't submit low-quality posts, be entitled, or harass others



Loot, Pillage, & Plunder

📜 c/Piracy Wiki (Community Edition):


💰 Please help cover server costs.

Ko-Fi Liberapay
Ko-fi Liberapay

founded 1 year ago
MODERATORS