This is just a draft, best refrain from linking. (I hope we'll get this up tomorrow or Monday. edit: probably this week? edit 2: it's up!!) The [bracketed] stuff is links to cites.
Please critique!
A vision came to us in a dream — and certainly not from any nameable person — on the current state of the venture capital fueled AI and machine learning industry. We asked around and several in the field concurred.
AIs are famous for “hallucinating” made-up answers with wrong facts. The hallucinations are not decreasing. In fact, the hallucinations are getting worse.
If you know how large language models work, you will understand that all output from a LLM is a “hallucination” — it’s generated from the latent space and the training data. But if your input contains mostly facts, then the output has a better chance of not being nonsense.
Unfortunately, the VC-funded AI industry runs on the promise of replacing humans with a very large shell script. If the output is just generated nonsense, that’s a problem. There is a slight panic among AI company leadership about this.
Even more unfortunately, the AI industry has run out of untainted training data. So they’re seriously considering doing the stupidest thing possible: training AIs on the output of other AIs. This is already known to make the models collapse into gibberish. [WSJ, archive]
There is enough money floating around in tech VC to fuel this nonsense for another couple of years — there are hundreds of billions of dollars (family offices, sovereign wealth funds) desperate to find an investment. If ever there was an argument for swingeing taxation followed by massive government spending programs, this would be it.
Ed Zitron gives it three more quarters (nine months). The gossip concurs with Ed on this being likely to last for another three quarters. There should be at least one more wave of massive overhiring. [Ed Zitron]
The current workaround is to hire fresh Ph.Ds to fix the hallucinations and try to underpay them on the promise of future wealth. If you have a degree with machine learning in it, gouge them for every penny you can while the gouging is good.
AI is holding up the S&P 500. This means that when the AI VC bubble pops, tech will drop. Whenever the NASDAQ catches a cold, bitcoin catches COVID — so expect crypto to go through the floor in turn.
It is a more nuanced issue than this and a bit naive as well, if I can be blunt and still say; friend, I cared enough to read and comment.
The next most probable token is not related to hallucinations as described. This is like saying statistics is worthless because it doesn't give absolute answers.
AI can mean a great many different architectures and all have different strengths, weaknesses, and issues. The best way I can relate current LLM's is the early days of the microprocessor. There were a lot of issues and limitations to work through. Eventually companies like Sun made some really capable and powerful machines after the first few generations of microprocessors. These devices became much more complex with time; integrating many peripheral devices. Many systems used several microprocessors in a single machine when it wasn't cost prohibitive. If you have a computer in the last few generations, you have around twenty very similar microprocessors all working together on the same chip.
AI is presently at that early stage. It is a useful fundamental tool, but by itself it is not very remarkable. The innovations in the peripheral space are where things get interesting. The way these innovations get integrated and the way the complexity multiplies over time will follow a similar curve as the microprocessor.
There is and will be lots of misuse and failed companies over time, but the technology will continue. This is an inevitable future and it will never go away.
When someone talks about hallucinations, it means the model is outside of alignment. The complexity of the model and its bias is the largest factor. In many ways this is why the the proprietary AI models will fail eventually. Open Source, offline models are the future of AI for text. An 8×7B unfiltered research model hallucinates far less than others and does not involve the massive amount of data that can be collected an inferred by proprietary AI. The majority of hallucinations are due to user input errors that are not accounted for in the model tokenizer and loader code. This is just standard code errors. Processing every possible spelling, punctuation, and grammar error is a difficult task. The next probable token is not simply a matter of the probable vector in the dataset. If the input contains a rare error the output will be in the style of a foolish error. This is not a hallucination. It is responding in the style it was addressed. If the user does not have control of the entire text inside the present context, aka previous questions and answers, the style of "stupid error" will likely remain persistent. It is still not an error, it is just a mirror. Indeed this is the greatest analogy. AI is like a mirror of yourself upon the dataset. It can only reflect what is present in the dataset and only in a simulacrum of yourself through the prompts you generate. It will show you what you want to see. It is unrivaled access to information if you have the character to find yourself and what you are looking for in that reflection.
I have no idea what the future holds but I hope one day to obtain the title of "confusion monk".
spoiler
idk I've been playing Larn a lot and this is where my mind went.