18
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
this post was submitted on 30 Dec 2024
18 points (100.0% liked)
TechTakes
1512 readers
249 users here now
Big brain tech dude got yet another clueless take over at HackerNews etc? Here's the place to vent. Orange site, VC foolishness, all welcome.
This is not debate club. Unless it’s amusing debate.
For actually-good tech, you want our NotAwfulTech community
founded 2 years ago
MODERATORS
Nobody outside the company has been able to confirm whether the impressive benchmark performance of OpenAI's o3 model represents a significant leap in actual utility or just a significant gap in the value of those benchmarks. However, they have released information showing that the most ostensibly-powerful model costs orders of magnitude more. The lede is in that first graph, which shows that for whatever performance gain o3 costs over ~$10 per request with the headline-grabbing version costing ~$1500 per request.
I hope they've been able to identify a market willing to pay out the ass for performance that, even if it somehow isn't over hyped, is roughly equivalent to an average college graduate.
if all of that $1500 cost is electricity, and at arbitrarily chosen but probably high electricity price of $0.2/kWh, that's 7.5MWh per request. could be easily twice that. this is approx how much electricity four 4-person households consume in a year in poland. or about half of american one. six tons of TNT equivalent, or almost 2/3 ton of oil equivalent if you prefer
Actually wait I'm pretty sure it's even worse because I'm terrible at reading logarithmic scales. It's roughly halfway between $1,000 and $10,000 on their log scale, which if I do the math while actually awake works out closer to $3,000.
I'm wondering about the benchmark too. It's way above my level to figure out how it can be gamed. But, buried in the article:
The most expensive o3 version achieved 87.5%