Dropsitenews published a list of websites Facebook uses to train its AI on. Multiple Lemmy instances are on the list as noticed by user BlueAEther
Hexbear is on there too. Also Facebook is very interested in people uploading their massive dongs to lemmynsfw.
Full article here.
Link to the full leaked list download: Meta leaked list pdf
Check out the robots.txt on any Lemmy instance....
Linked article in the body suggests that likely wouldn't have made a difference anyway
Yeah ive seen the argument in blog posts that since they are not search engines they dont need to respect robots.txt. Its really stupid.
"No no guys you don't understand, robots.txt actually means just search engines, it totally doesn't imply all automated systems!!!"
Scrapers ignore it
Thieves can smash a window to get into my house but I still lock my doors.
This is more like being there when they come to steal and you ask them to ignore some rooms please.
If they have a brain, and they do have the experience from Threads, they don't need to scrape Lemmy. They can just set up a shell instance, subscribe to Lemmy communities, and then use federation to get their data for free. That doesn't use robots.txt at all regardless.