it’s a good section! you can tell it’s effective when an AI fan spontaneously appears to show us his entire ass.

my one suggestion is to expand upon this paragraph:

The current workaround is to hire fresh Ph.Ds to fix the hallucinations and try to underpay them on the promise of future wealth. If you have a degree with machine learning in it, gouge them for every penny you can while the gouging is good.

I feel this could be followed up by a paragraph describing the extremely cultlike environment we know exists in damn near every serious AI company. for me, that’s the missing piece of why so many Ph.Ds are excited to be underpaid at OpenAI, a company with extremely questionable motives and practices in the academic space, and other AI companies with similar motives. through SneerClub we know the origins of that environment, but I haven’t seen a thorough analysis yet of the financial motives behind insisting your engineers are all members of ComStar, other than what we saw earlier this year after the schism at OpenAI. it’s very likely you have better sources for this stuff than I do though.

Yes.

Pessimistically, the world can stay irrational (heh) longer than we can stay solvent (alive and well enough to work wih this sneerious outlet)

The one major comment from me is I think you have a framing weakness centered on “if you know how LLMs work”, since that is a very load bearing point for much of what follows but holds no anchor point for those who do not know to also follow along (unless going on faith)

I realize you’re not writing this as an explainer blog, but a short sentence + link to elsewhere with an explanation (“for those who don’t know, has a decent explanation without being overly technical”) might be a good patch for that?

Is it worth also casting a light on the various vendor deals with e.g. Reddit (et al), in their search for structured and scoped training data?

Sorry I missed this, especially as a mod of this sub. It was a great post, so I'm assuming all the good feedback below was helpful

I'm not a stock person man, but didn't the hype from bitcoin last like a decade, despite not having a single widespread use case? Why wouldn't LLM hype last the same amount of time, when people actually use it for things?

something something Confessions of a VC-Funded Opium Eater

It is a more nuanced issue than this and a bit naive as well, if I can be blunt and still say; friend, I cared enough to read and comment.

The next most probable token is not related to hallucinations as described. This is like saying statistics is worthless because it doesn't give absolute answers.

AI can mean a great many different architectures and all have different strengths, weaknesses, and issues. The best way I can relate current LLM's is the early days of the microprocessor. There were a lot of issues and limitations to work through. Eventually companies like Sun made some really capable and powerful machines after the first few generations of microprocessors. These devices became much more complex with time; integrating many peripheral devices. Many systems used several microprocessors in a single machine when it wasn't cost prohibitive. If you have a computer in the last few generations, you have around twenty very similar microprocessors all working together on the same chip.

AI is presently at that early stage. It is a useful fundamental tool, but by itself it is not very remarkable. The innovations in the peripheral space are where things get interesting. The way these innovations get integrated and the way the complexity multiplies over time will follow a similar curve as the microprocessor.

There is and will be lots of misuse and failed companies over time, but the technology will continue. This is an inevitable future and it will never go away.

When someone talks about hallucinations, it means the model is outside of alignment. The complexity of the model and its bias is the largest factor. In many ways this is why the the proprietary AI models will fail eventually. Open Source, offline models are the future of AI for text. An 8×7B unfiltered research model hallucinates far less than others and does not involve the massive amount of data that can be collected an inferred by proprietary AI. The majority of hallucinations are due to user input errors that are not accounted for in the model tokenizer and loader code. This is just standard code errors. Processing every possible spelling, punctuation, and grammar error is a difficult task. The next probable token is not simply a matter of the probable vector in the dataset. If the input contains a rare error the output will be in the style of a foolish error. This is not a hallucination. It is responding in the style it was addressed. If the user does not have control of the entire text inside the present context, aka previous questions and answers, the style of "stupid error" will likely remain persistent. It is still not an error, it is just a mirror. Indeed this is the greatest analogy. AI is like a mirror of yourself upon the dataset. It can only reflect what is present in the dataset and only in a simulacrum of yourself through the prompts you generate. It will show you what you want to see. It is unrivaled access to information if you have the character to find yourself and what you are looking for in that reflection.