261
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
this post was submitted on 25 Jul 2024
261 points (96.1% liked)
Technology
59374 readers
3040 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each another!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
Approved Bots
founded 1 year ago
MODERATORS
Ddg shows bing results. Are we sure they won't regurgitate hallucinations too?
As does Ecosia.
Crawling the web is expensive, and plenty of bigger sites have proprietary deals (Reddit serving exclusive results to Google for instance). Also, since actively hosting data costs money, lots of sites have archived or compressed their offerings. Others have set up higher and higher paywalls, to limit what anyone without a subscription can see.
The end result is a treasure trove of data that is inaccessible to modern crawlers and scrapers. If you're not tapping into one of the big search engine catalogs, you're going to miss a lot of the more attractive results.
Then there's the problem of AI crap filling up lots of the spaces that used to be mineable for search results. This isn't a problem unique to Bing. AI contamination is everywhere and crawlers can't avoid it easily. What's a modern search engine to do?
Kagi & DDG already have their own ai stuff
i don't really have the technical knowledge to answer that, But I don't think that's how the Bing API works.
So do most "alternative" search engines, often with some of their own spice on top.
I know Startpage happens to use Google in their back-end, but Google's policy is a lot more restrictive than Microsoft's given their market position.