Lemmy Federate

90 readers
11 users here now

Updates and questions about lemmy-federate.com

founded 3 months ago
MODERATORS
1
 
 

I made this change to increase software support for Lemmy Federate. This way, besides Lemmy and Mbin, other software that supports FEP-1b12 such as NodeBB, Piefed, Guppe, Friendica or other custom implementations (like blogs) will be able to register to Lemmy Federate.

However, I must admit that this change makes the login mechanism more difficult. But I believe it was worth it.

An example authentication flow:

  • Enter instance host and click login

  • Add given record to DNS, and click login again

  • At this point, API key becomes reusable between logins.

If you experience any problems please let me know.

2
 
 

I was looking for a particular instance by it's not listed on the tool. I couldn't find info on what to do in this case. Is this something the instance admin must do?

3
 
 

Right now Lemmy federate says piefed.social isn't registered. I went to their matrix and asked the admin about adding lemmy-federate. They said they had attempted to do so but the Lemmy federate bot didn't recognize their account as an admin account.

With the end of lemm.ee and the piefed.social migration tools we are going to see a lot of communities pop up on piefed.

Would it be possible to either add piefed support or manually flag the piefed social admin account as validated?

4
 
 

What do you plan on doing with lemmy federate once this feature is implemented?

https://github.com/LemmyNet/lemmy/issues/2951

5
19
submitted 1 month ago* (last edited 1 month ago) by iso@lemy.lol to c/lemmyfederate@lemy.lol
 
 

I’ve seen many arguments claiming that Lemmy Federate creates a significant load on instances in terms of data storage and networking. I used to believe this wasn’t a very valid concern, as I assumed that communities without followers wouldn’t generate many posts, comments, likes, etc.

Today, I had the chance to test this on my own instance, and here are the results:

  • Communities
    • Total count: 33,920
    • Federated by LF: 5,863
    • Percentage: 17.28%
  • Posts
    • Total count: 3,217,783
    • Federated by LF: 114,067
    • Percentage: 3.54%
  • Comments
    • Total count: 14,222,401
    • Federated by LF: 192,925
    • Percentage: 1.36%

LF = Lemmy Federate. Posts and comments refer to the number of posts and comments in communities federated with Lemmy Federate.

As you can see from the statistics, almost 20% of the communities are federated by Lemmy Federate. Although this seems like a high number, only 3.5% of the posts are created in communities federated by Lemmy Federate. The number of comments is even less than the number of posts.

So to answer, Lemmy Federate creates an average load of around 2-3% in an instance with about 1000 users. You can compare it with its benefits and decide whether to use it or not.

If you want to get stats for your own instance, you can run the SQL script below. Let us know the results please :)

the script

-- make sure to replace the <lemmy_federate_bot_person_id>
WITH
-- Total community count
total_communities AS (
    SELECT COUNT(id) AS total FROM community where local != true
),

-- Communities federated by LF (assuming person_id <lemmy_federate_bot_person_id> represents LF bot)
-- A community is considered federated by LF if LF follows it and it has local subscribers
lf_communities AS (
    SELECT ca.community_id
    FROM community_aggregates ca
    JOIN community c on c.id = ca.community_id
    JOIN community_follower cf ON ca.community_id = cf.community_id
    WHERE ca.subscribers_local > 0 and c.local != true AND cf.person_id = <lemmy_federate_bot_person_id>
),
lf_communities_count AS (
    SELECT COUNT(*) AS count FROM lf_communities
),

-- Total post count
total_posts AS (
    SELECT COUNT(id) AS total FROM post
),

-- Posts in LF communities
lf_posts AS (
    SELECT COUNT(p.id) AS count
    FROM post p
    WHERE p.community_id IN (SELECT community_id FROM lf_communities)
),

-- Total comment count
total_comments AS (
    SELECT COUNT(id) AS total FROM comment
),

-- Comments on posts in LF communities
lf_comments AS (
    SELECT COUNT(c.id) AS count
    FROM comment c
    JOIN post p ON c.post_id = p.id
    WHERE p.community_id IN (SELECT community_id FROM lf_communities)
)

-- Final output
SELECT
    tc.total AS community_count,
    lfc.count AS community_count_lf,
    ROUND((lfc.count::decimal / tc.total) * 100, 2) AS community_lf_percent,
    tp.total AS post_count,
    lp.count AS post_count_lf,
    ROUND((lp.count::decimal / tp.total) * 100, 2) AS post_lf_percent,
    tcom.total AS comment_count,
    lcom.count AS comment_count_lf,
	ROUND((lcom.count::decimal / tcom.total) * 100, 2) AS comment_lf_percent
FROM total_communities tc
JOIN lf_communities_count lfc ON TRUE
JOIN total_posts tp ON TRUE
JOIN lf_posts lp ON TRUE
JOIN total_comments tcom ON TRUE
JOIN lf_comments lcom ON TRUE;

6
 
 

Report of the same issue in lemmy support - https://hackertalks.com/post/9701951

If a user is the only subscriber on a instance, and gets banned, and unbanned, then unban wont federate.

If lemmy-federate would subscribe even if there are already local subscribers, then this lemmy issue would be moot for lemmy-federated communities (there would always be at least one subscriber on the instance with the lemmy-federate bot, so the unban gets federated).

7
7
submitted 1 month ago* (last edited 1 month ago) by iso@lemy.lol to c/lemmyfederate@lemy.lol
 
 

Sometimes instances can crash or slow down while following other instances. Or the remote instance may be blocked or the bot may be stuck at the rate limit.

To troubleshoot such issues I added a logs section to the instance management page.

Note: these logs are responses of requests sent to your instance. You can only see the logs for your own instance.

Some techie friends may ask why I didn't use something like Grafana. My answer is, I was just lazy :)

8
 
 

Hey how can we troubleshoot when communities are not being federated?

For example, this community shows that there was an error federating to Lemmy.nz (but not what the error was). Previously it was showing as not allowed. screen shot showing metabolic_health@lemm.ee is not allowed to federate

Here are my settings: settings showing things like that it's enabled, disables NSFW, requires fediseer endorsement

There are other communities listed in this post, though really I'd like the tools to troubleshoot myself: https://lemmy.nz/post/21531352

Any help appreciated 🙂

9
 
 

Generic Threadiverse support

Thanks to @rikudou@lemmings.world's contribution (#28), Lemmy Federate now supports all software types that implements group federation such as PieFed, NodeBB, Guppe 🎉

But unfortunately, not everything is perfect. Since there is no Fediverse standard for verifying whether a user is an admin, I have to register admins manually. I am also considering manually approving instances that are not guaranteed in Fediseer against spam attacks. Please contact me for this.

Note: Lemmy and Mbin works as before.

Top instances of Lemmy

With the addition of Lemmy.ml, the top 25 largest instances on Lemmy now use Lemmy Federate (except slrpnk.net). I think we can now consider that we have fixed the accessibility issue that was the reason I created this tool. Even if we didn't fix it, at least we band-aided it :)

Instance blocking feature

In addition to the allow list, a block list has been added.

  • If you allow at least one instance, you will not follow any other instances.
  • If you block an instance, you will continue to follow instances other than those you blocked.

Dedicated community

I didn't want to open it before, but now that we are trying to be compatible with more software, I believe a dedicated community could be useful. That's why I created a community here !lemmyfederate@lemy.lol. If I make an update from now on, I'll probably post it there.

https://lemmy-federate.com/ https://github.com/ismailkarsli/lemmy-federate