1
1
submitted 1 year ago* (last edited 1 year ago) by JohnnyCanuck@sh.itjust.works to c/about@lemmit.online

In particular, posts to NSFW videos hosted on v.redd.it don't work on the www version. The links take you to the comments page, which blocks NSFW content, and nags you to go to the app.

old.reddit.com links just work without logging in.

2
1

See the bot in action here!

My instance running Leddit

Click here for a more detailed explanation about the bot's purpose

This bot is intended to be self-hosted. Unfortunately, I can't operate a public instance that takes subreddit requests because of how long syncing comments takes. For comparison, Lemmit takes 21 minutes to sync all of the subreddits on this instance using the old system, but Leddit takes the same amount of time to sync 3 subreddits with around 500k subscribers each once an hour. Smart syncing is planned, but it won't decrease the amount of time taken to sync big and active subreddits.

If you need help setting up an instance, feel free to ask questions in this thread or on the Leddit instance's community.

3
1

I want to follow many of the NSFW subreddit-communities from this account and probably request more, but the communities page doesn't show NSFW when not logged in (making an account on lemmit itself just to see the list seems like overkill and could be confusing about what's logged in where, if it's even allowed)

4
1
submitted 1 year ago* (last edited 1 year ago) by admin@lemmit.online to c/about@lemmit.online

Okay, this one took me a bit longer than I planned (mostly due to sql fun and trying to use integers as minutes, WEEEE!).

Backdrop: Last week I disabled the mirroring of a couple of subreddits to the database, because they were initially requested but the nobody subscribed to them. At the same time, the bot was just crawling in a loop, starting at todayilearned, ending at latestsubreddit. As more subreddits were requested, this loop took longer and longer (21 minutes before I rolled out this update). This wasn't sustainable.

So here's the new situation. The more popular a community is, the more often it will be updated. In this case popular means a mixture between number of subscribers and the amount of posts it receives per day (Link to relevant snippet of source code).

In short, the most popular subs will be synced every 10 minutes, the next tier ever 30 minutes, 120 minutes and the content with either no posts per day or no subscribers (other than the bot), will only be synced every 12 hours. I hope this will hit a good distribution of updates vs popularity, but it will most likely be refined at some point in the future.

Speaking of distribution, we now have over 300 communities on this server 🥳, and their update intervals are spread out as such:

  • Every 10 minutes: 22
  • Every 30 minutes: 39
  • Every 60 minutes: 55
  • Every 120 minutes: 143
  • Every 720 minutes: 44

With this update running live (I started typing after I deployed it, and it has now gotten through the backlog of 'abandoned' subs), I'm going to step back from feature development for a few days. Any bugs that cause the bot to crash will of course continue to be addressed.

Have a blast!

5
1
submitted 1 year ago by Azzu@lemm.ee to c/about@lemmit.online

Just as a disclaimer: I'm not complaining, it's great that all this exists at all :)

I'm just checking if this may be a bug or not: if I compare https://www.reddit.com/r/DotA2/new/ and https://lemmit.online/c/dota2?dataType=Post&page=1&sort=New then the bot is definitely skipping some posts.

For example, between "NothingToSay is called 'responsibility god' in CN Dota2 community." and "TIP: Medusa doesn’t reduce magic damage to her mana when BKB is up" there are 3 other posts on reddit directly, which are missing on lemmit.online

I'm just hoping this can be fixed since this bot makes populating the "real" dota2 community I moderate much easier, but some posts I want to cross-post are missing so I need to do shit manually.

6
1
submitted 1 year ago by madis@lemm.ee to c/about@lemmit.online

Have you considered doing something similar for Mastodon, to allow interacting with toots within Lemmy UI? I know the opposite is possible and Kbin also has some kind of integration, but that doesn't seem to fully work at the moment either.

7
1
submitted 1 year ago* (last edited 1 year ago) by Azzu@discuss.tchncs.de to c/about@lemmit.online

For example, it'd be nice for anyone that looks at/finds https://lemmit.online/c/dota2 to also find that https://discuss.tchncs.de/c/dota2 or https://lemmy.world/c/dota2 is an actual community that corresponds to that with user content, not bot content.

I'm sure there's lots of equivalents for other communities as well where that would make sense.

8
1
submitted 1 year ago by admin@lemmit.online to c/about@lemmit.online

Before was running on the cheapest model (1 core / 1GB mem / 30GB storage) at $12/month. The machine was running pretty low on memory, causing it to start swapping, which in turn caused the cpu to get too busy, and everything to slow down.

Now it has a whopping 2GB of memory, and things seem to have calmed down - cpu is back to around 10-15% usage, and swap is down to 0. Happy times all around.

Because of the amount of subs being archived, it now takes about 15 minutes between updates for each sub (was 18 before I updated the VM).

I'm planning to build some kind of scoring system, based on the amount of posts per subreddit (per day?), and amount of subscribers on the lemmy community. That way communities with little subscribers or that don't see many posts per day, will only be updated once per hour.

At the same time, I feel that subs like AskReddit, OutOfTheLoop and other "question-based" subreddits shouldn't be archived by Lemmit. In my opinion those kind of posts are useless without those answers, but please let me know if you disagree.

9
1
submitted 1 year ago by eatham@aussie.zone to c/about@lemmit.online

!perchance@lemmit.online doesnt match up with r/perchance. its last post was 5d ago.

10
1
Bug fixes 24-06-2023 (lemmit.online)
submitted 1 year ago by admin@lemmit.online to c/about@lemmit.online
  • Fixed a bug where posts would not be submitted because the title didn't contain long enough words.
  • Fixed a bug where posts would not be submitted because the url was too long.
  • Fixed a bug where posts would not be submitted when it was linking to a /user subreddit.
  • Fixed a bug where the bot would think Every Post Everywhere was a subreddit request, and would reply to it.
  • Fixed a bug where the bot would crash without recovering whenever something went wrong during new subreddit requests

A fruitful day all in all, I'd say.

11
1
Please don't tell me (lemmit.online)
submitted 1 year ago by admin@lemmit.online to c/about@lemmit.online

That the replies-everywhere-bug was just because I forgot to include a variable in the bot deployment? 🤦

12
2
submitted 1 year ago* (last edited 1 year ago) by admin@lemmit.online to c/about@lemmit.online

In the short time since this instance and bot launched, I've been seeing the same questions resurface multiple times. This is totally understandable, since the concept of a Fediverse is still new to most (myself included), and this server is not like the others.

Q: What is Lemmit?

A: Lemmit is a Lemmy instance specifically designed for archiving Reddit content. Users can request new subreddits to be included in the archiving process by posting in the !requests@lemmit.online community. It is powered by an open source python bot, which periodically checks the request list, adds new requests to the queue, and continuously monitors the Hot feed of those subs for new posts to cross-post here.

Q: Does it synchronize comments?

A: No, that would be impossible. Considering there are thousands of posts already on Lemmit, many of them having at least several hundred comments on Reddit, often buried in deep layers, it simply wouldn't be feasible to index those for more than a few posts, let alone keep them up to date.

Unfortunately, this means that archiving certain subreddits, such as Ask Historians/Men/Women/Hyperintelligentshadesofthecolourblue-type subs, is going to be rather pointless.

Q: Can it send comments back to Reddit?

A: No, it cannot. The purpose is to help bootstrap the Lemmy platform, not to serve as a bridge between the two networks. Also, see the answer about synchronizing comments.

Q: Can I request any subreddit?

A: Technically, yes. However, as the list of subs grows, the time it takes to update all of them will also increase. I do not have strict guidelines in place for this, so I'm relying on your common sense (hoooo boy). At some point, I will probably have to either stop accepting new requests or disable scraping for very low-traffic communities.

Q: Does this use the API? Will it keep working after July 1st?

A: Nope, it uses a combination of the public feed and scraping old.reddit.com. So, as long as those are still available, it will continue working. And even if they close those sources, there will probably be new ways to achieve the same effect. "Content, eh, finds a way."

Q: What started this?

A: Okay, nobody asked this, but I'm going to tell you anyway. After Reddit made it clear that they are effectively killing third-party apps and implementing plenty of other anti-end user decisions, I realized that I would either have to accept not being able to access my time-wasting content or have to do so in a rather uncomfortable way (either through the official app or old.reddit.com for as long as they'll allow it to exist).

Being a stubborn developer, naturally, I chose option C: Have my own Reddit. With blackjack, and hookers. This way, I would still be able to access my beloved content without being beholden to Reddit's mood swings and abusive relationship tendencies.

Besides that, I also know that Content is King. So I'm order to counter the network effect (No users because no content, No content because no users), I figured it would be better to have some inorganic content to bootstrap the adoption of Lemmy.

Q: This is spam, can you stop?

A: First off all, I apologise for the inconvenience. All you have to do is block @bot@lemmit.online, and none of its posts will ever show up on your instance.

Obviously I could stop, because running this server and software is only ever going to cost me time and money. But for the reasons listed above, I still think this server is a useful addition to the lemmyverse at this time. But I'm looking forward to the day where I can turn the bot off because it's no longer needed.

Q: Are NSFW subreddits allowed?

A: Absolutely. Like I said: Blackjack and hookers.

Q: My request isn't picked up by the bot!

A: That isn't a question. But yeah, the process isn't flawless yet. I'm trying to iron out all the bugs as I encounter them. In the meantime, feel free to re-request the subreddit by making a second post. No harm done.

Q: No new posts are showing up at all on Lemmit

A: If no posts are appearing on the Lemmit Frontpage (sorted by NEW), it's possible that the bot has crashed or is stuck on something. Since no software is flawless, this sometimes happens. I usually fix this as soon as I'm aware, and I'm happy to say that these kinds of fatal errors are becoming less and less frequent. However, they may still occur, and as a human with needs of sleep and other responsibilities, I'm not always able to fix them immediately.

Q: Posts aren't showing up on my instance, what's up?

A: First, check if any posts show up on the frontpage (see previous question). Other than that - I wish I had an answer to this. I'm not an expert on the inner workings of the Lemmy service. If you're an experienced Lemmy admin and think you may have a clue, please reach out.

Q: When are you updating to v0.18?

A: There have been some varying reports on its stability. For now I will be waiting until v0.18.1 rolls out.

13
1
submitted 1 year ago by admin@lemmit.online to c/about@lemmit.online

Long story short: I messed up with the domain registration for this instance, and never replied to a mandatory email. The domainname (lemmit.online) got put in suspension, causing disconnects all over the fediverse.

I fixed it as soon as I found out, but it will probably take a few more hours for the issues to be fully fixed.

So ehm. Whoops. Hope this explains and fixes the federation issues we've been having today.

14
1
submitted 1 year ago by Azzu@feddit.de to c/about@lemmit.online

When I went to https://feddit.de/c/requests@lemmit.online I noticed the sidebar links to /c/about - which doesn't exist on feddit.de

I'd suggest to change the link to https://lemmit.online/c/about so people from other instances can find it more easily.

15
1
submitted 1 year ago* (last edited 1 year ago) by Wander@yiffit.net to c/about@lemmit.online

Current reply:

I'll get right on that. Check out /c/subredditname@lemmit.online!

Proposed reply:

I'll get right on that. Check out /c/subredditname@lemmit.online!

Click here to fetch this community for your lemmy instance if you get a 404 error with the link above.

16
1
Are NSFW allowed? (lemmy.g97.top)
17
1
Bug fixes 21-06-2023 (lemmit.online)
submitted 1 year ago by admin@lemmit.online to c/about@lemmit.online

Most importantly that the bot no longer crashes (and does nothing all night while I sleep 😛) when trying to create a community that has already been requested.

Furthermore mostly making the code prettier and adding tests.

18
1

In card view all you see is the bot header and none of the actual content. Maybe it should go at the bottom and just prepend something short like [Lemmit Bot]

This bit.. This is an automated archive made by the Lemmit Bot. The original was posted on /r/ayaneo by /u/HystericalBanana on 2023-06-17 21:43:04+00:00.

19
1

Great for communities that have not made the jump to lemmy. Now I don't need to dive into reddit to get that info. Will this break when the reddit Api limits come into force (assuming people keep requesting more communities)?

20
1
submitted 1 year ago by eatham@aussie.zone to c/about@lemmit.online

If you want can you add comment syncing so we can see the comments on Lemmy too

21
1
Bug fixes 19-06-2023 (lemmit.online)
submitted 1 year ago* (last edited 1 year ago) by admin@lemmit.online to c/about@lemmit.online

Fixed a couple of bugs today:

  • Nasty one that made the bot get stuck in an infinite when trying to add a post by a deleted user, which kept the bit offline for most of last night.
  • Another creative one that, when posting certain links, would actually work, but the lemmy gateway would respond with a timeout. It only happens on certain links, but consistently. Which would make the bot think it was unsuccessful, which would make it try to post again the next time. Causing a duplicate post each time (technically it was a cross post to itself... Which is interesting in a whole new way).

TLDR: right now there is a workaround in place that assumes a timeout post to lemmit was actually successful. This might cause it to drop posts in the future, but seeing that the server is barely breaking a sweat at this time, it should be good until a better fix is implemented.

Also got some great feedback from users, which I added to the TODO.

22
1
submitted 1 year ago by bot@lemmit.online to c/about@lemmit.online

About Lemmit

0 readers
0 users here now

About the lemmit.online service and its software.


Got questions, complaints, suggestions? This is the place.

founded 1 year ago
MODERATORS