You’re illustrating the issue so many people have with this technology. Without a fundamental understanding of how it works, people will attempt to use it in ways it shouldn’t be used, and won’t understand why it isn’t giving them correct information. It simply doesn’t have the ability to do anything but put words in an order that statistically will resemble how a human might answer the question.

LLMs don’t know anything. They can’t tell fact from fiction (and are incapable of even trying), and don’t understand concepts such as verifying info when requested. That’s the problem, they don’t ‘understand’ anything, including what they are telling you. But they do spit out words in a statistically probable order, even if the result is complete bullshit. They do it so well that they can fool most people into thinking the computer actually knows what it’s telling you.

Here's a better metaphor because yours completely misses the mark when it comes to the difference between an LLM and an actual encyclopedia.

A painter will spend years honing his craft by studying other paintings as well as photos and real life. If you ask him to paint you a house from memory and try to build it with what he gives you, that just makes you an idiot, it doesn't make him a bad architect.

Chatgpt is not an encyclopedia and any thing it says that is remotely important to your personal or work life should be verified. They explicitly tell you it can and will give false responses.

LLM isn't a compilation of its training data, anymore than a cake is a pile of eggs, flower and sugar.

Jpg is a lossy compression algorithm. Statistical probability of words occuring in sequence is not compression. That's like saying generative images are compression, they aren't. It's not producing blurry matches of images, it's producing something "novel". Otherwise, that would be considered over fitting the data.