I have now read so many "ChatGPT can do X job better than workers" papers, and I don't think that I've ever found one that wasn't at least flawed if not complete bunk once I went through the actual paper. I wrote about this a year ago, and I've since done the occasional follow-up on specific articles, including an official response to one of the most dishonest published papers that I've ever read that just itself passed peer review and is awaiting publication.
That academics are still "bench-marking" ChatGPT like this, a full year after I wrote that, is genuinely astounding to me on so many levels. I don't even have anything left to say about it at this point. At least fewer of them are now purposefully designing their experiments to conclude that AI is awesome, and are coming to the obvious conclusion that ChatGPT cannot actually replace doctors, because of course it can't.
This is my favorite one of these ChatGPT-as-doctor studies to date. It concluded that "GPT-4 ranked higher than the majority of physicians" on their exams. In reality, it actually can't do the exam, so the researchers made a special, ChatGPT-friendly version of the exam for the sole purpose of concluding that ChatGPT is better than humans.
Because GPT models cannot interpret images, questions including imaging analysis, such as those related to ultrasound, electrocardiography, x-ray, magnetic resonance, computed tomography, and positron emission tomography/computed tomography imaging, were excluded.
Just a bunch of serious doctors at serious hospitals showing their whole ass.
I just want to emphasize that to set up a truly independent and unpaywalled piece of media, you probably need to abandon hope of it being even a viable side hustle. Quasi-independent media on, say, YouTube or Substack can make some money, but you're then stuck on those corporate platforms. If you want to do your own website or podcast or whatever, that's more independent, but you're still dependent on Google if you run ads, or on Patreon if you do that sort of thing. The lesson of Twitter should make pretty clear the danger inherent to that ecosystem. Even podcasts that seem independent can easily get into huge trouble if, say, Musk were to buy Patreon or iHeart.
I've been writing on my website for over two years now. My goal has always been to be completely independent of these kinds of platforms for the long term, no matter what, and the site's popularity has frankly exceeded my wildest dreams. For example, I'm the #1 google result for "anticapitalist tech:"
But I make no money. If I wanted this to be anything but a hobby, I'd have to sacrifice something that I think makes it valuable: I'd have to paywall something, or run ads, or have a paid discord server, or restrict the RSS feed. As things stand now, I don't know my exact conversion rate because I don't do any analytics and delete all web logs after a week, but I did keep the web logs from the most recent time that I went viral (top of hackernews and several big subreddits). I made something like 100 USD in tips, even though the web logs have millions of unique IPs. That's a conversion rate of something like 0.00002 USD per unique visitor.
Honestly, if I got paid even $15/hr, I would probably switch to doing it at least as a part time job, because I love it. Compare that to the right wing ecosystem, where there's fracking money and Thiel money just sloshing around, and it's very very obvious why Democrats are fucked, much less an actual, meaningful left. Even Thiel himself was a right wing weirdo before he was a tech investor, and a right wing think tank funded his anti-DEI book. He then went on to fund Vance. It's really hard to fight that propaganda machine part time.