159
The distributed replacement for Sci-Hub
(github.com)
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
this post was submitted on 24 Dec 2024
159 points (97.0% liked)
Piracy: ꜱᴀɪʟ ᴛʜᴇ ʜɪɢʜ ꜱᴇᴀꜱ
55056 readers
477 users here now
⚓ Dedicated to the discussion of digital piracy, including ethical problems and legal advancements.
Rules • Full Version
1. Posts must be related to the discussion of digital piracy
2. Don't request invites, trade, sell, or self-promote
3. Don't request or link to specific pirated titles, including DMs
4. Don't submit low-quality posts, be entitled, or harass others
Loot, Pillage, & Plunder
📜 c/Piracy Wiki (Community Edition):
💰 Please help cover server costs.
 |
 |
|---|---|
| Ko-fi | Liberapay |
founded 2 years ago
MODERATORS
AI can be good but I'd argue letting an LLM autonomously write a paper is not one of the ways. The risk of it writing factually wrong things is just too great.
To give you an example from astronomy: AI can help filter out "uninteresting" data, which encompasses a large majority of data coming in. It can also help by removing noise from imaging and by drastically speeding up lengthy physical simulations, at the cost of some accuracy.
None of those use cases use LLMs though.
Right, the public and journalists often lump everything together under the term "AI". When it's really a big difference between some domain specific pattern recognition task that can be done with machine learning and >99% accuracy... Or an ill-suited use-case where a LLM gets slapped on.
For example I frequently disagree with people using LLMs for summarization. That seems to be something a lot of people like. And I think they're particularly bad at it. All my results were riddled with inaccuracies, sometimes it'd miss the whole point of the input text. And it'd rarely summarize at all. It just picks a topic/paragraph here and there and writes some shorter version of that. Missing what a summary is about, providing me with the main points and conclusion, reducing the details and roughly outlining how the author got there. I think LLMs just can't do it.
I like them for other purposes, though.
Re LLM summaries: I've noticed that too. For some of my classes shortly after the ChatGPT boom we were allowed to bring along summaries. I tried to feed it input text and told it to break it down into a sentence or two. Often it would just give a short summary about that topic but not actually use the concepts described in the original text.
Also minor nitpick but be wary of the term "accuracy". It is a terrible metric for most use cases and when a company advertises their AI having a high accuracy they're likely hiding something. For example, let's say we wanted to develop a model that can detect cancer on medical images. If our test set consists of 1% cancer inages and 99% normal tissue the 99% accuracy is achieved trivially easy by a model just predicting "no cancer" every time. A lot of the more interesting problems have class imbalances far worse than this one too.
What's the correct term within casual language? "correctness"? But that has the same issue... I'm not a native speaker...
By the way, I forgot my main point. I think that paper generator was kind of a joke. At least the older one, which predates AI and uses "hand-written context-free grammar":
And there are projects like Papergen and several others. But I think what I was referring to was the AI scientist which does everything from brainstorming research ideas, to simulating experiments, writing reports etc. That's not meant to be taken seriously, in the sense that you'll publish the generated results. But seems pretty creative to me, to write a paper about an artificial scientist...