Programming

I work in a lab, so yes, I understand how data science works. However, I think you have too much faith in the people running these scrapers.

I think it’s unlikely that ChatGPT would have had those early scandals of leaking people’s SSNs or other private information if the data was actually “cleared by a human team” The entire point of these big companies is laziness; I doubt they have someone looking over the thousands of sites worth of data they feed to their models.

Maybe they do quality checks on the data but even in that event, forcing them to toss out a large data set because some of it was poisoned is a loss for the company. And if enough people poison their work or are able to feed poison to the scrapers, it becomes much less profitable to scrape images automatically.

I previously mentioned methods for possibly slipping through automatic filters in the scraper (though maybe I mentioned that in a different comment chain).

As for a scraper acting like a human by use of an LLM, that sounds hella computationally expensive on the side of the scrapers. There would be few willing to put in that much effort, fewer scrapers makes DDOS like effect of scraping less likely. It would also take more time which means the scraper is spending less time harassing others.

But these are good suggestions. I suppose a drastic option for fighting a true AI mimicking a human would be to make all links have a random chance of sending any user to the tarpit. People would know to click back and try again, but the AI would at best have to render the site, process what it sees, decide it is in the tarpit, and then return. That would further slow down the scraper (or very likely stop/trap it) but that would make it slightly annoying for regular users.

In any case, at a certain point, trying to tailor an AI scraper to avoid a single specific website and navigate the traps for it would probably take more time and effort than sending a human to aggregate the content instead of an automated scraper