1
submitted 4 months ago by auk@slrpnk.net to c/santabot@slrpnk.net

Hi everyone.

I am mostly pleased with how Santabot performed in its first live test today, but I also found issues which I am fixing. I made it a lot more strict and fixed a bug. Details to follow:

Mostly, I was surprised and pleased that the bot was coming up with right judgements for people I was talking with, but this thread has one comment that seems reasonable, and one comment that seemed like exactly the kind of inflammatory content I wanted the bot to eliminate. I looked into it for a while, and eventually had to revisit my attitude toward users that have a lot of "positive rank" counterbalancing also a lot of "negative rank." I think that the ratio of upvoted content to negative content that it's reasonable to ask someone to produce should be higher than 1:1. So I made it 2:1. That eliminated the user posting the offending comment while leaving the user posting the legitimate comment. I didn't have time to go into a full detailed analysis, but I did some spot checks on how that affected its other judgements, and generally I liked what I saw:

  • The percentage of total users banned has gone way up, from 0.4% to 2.8%.
  • @Kaboom@reddthat.com, @LibertyLizard@slrpnk.net, and other specific users that I looked into and thought should remain unbanned, are still unbanned by the bot's analysis.
  • Some users who weren't banned by this morning's configuration, but which seemed like they should be, are now banned again.

In addition to the algorithm issues, I found a bad bug. The bot is supposed to remove recent postings when banning someone, to stop throwaway accounts from coming in and posting and then being banned but with the content staying up. However, the bot was simply removing all recent content from all users whenever it banned anybody, and my test suite was too simple-minded to catch the problem until it was live and removed a random innocent user's comment and sent them a message that they were banned. Oops. I partly fixed it once I saw it. Maybe I shouldn't have used Futurama Santa as the avatar.

That's the update. More to come. If you have questions, comments, or concerns, please by all means say something and I'll do my best at a response.

no comments (yet)
sorted by: hot top controversial new old
there doesn't seem to be anything here
this post was submitted on 03 Jul 2024
1 points (100.0% liked)

Santabot

13 readers
2 users here now

Santabot is an automated moderation tool, designed to reduce moderation load and remove bad actors in a way that is transparent to all and mostly resistant to abuse and evasion.

This is a community devoted to meta discussion for the bot.

founded 4 months ago
MODERATORS