I think I also prove that just by existing

To be clear, I agree that LLMs are a step forward in some areas, predominantly search, and text style analysis.

The problem with saying LLMs are AI - let alone a step towards AGI - is that they cannot create. For example, Outsider art, or art brut, is impossible for an LLM to create because it can only generate output based on its training. No training, no output.

Compare that to how a small child finger paints, who has never been told anything about perspective or colour theory, and is just given a load of colours and some paper to play with.

The ability to create something from nothing is a fundamental aspect of what we would consider to be intelligence, just regurgitating what you've been told - like a pre-programmed billboard - is not intelligence.

In the context of large language models, if you give GPT3.5 the prompt:

Say something which has never been said before.

It responds

Certainly! "In a world where marshmallow clouds rain cotton candy dreams, unicorns compose symphonies of stardust, and jellybean butterflies flutter in chocolate rivers."

If you said that to a child, how long do you think it would be before they started just making up new words and sounds, like some sort of nonsense poetry? Children learn to speak purely though listening to others, the same principle as training data, but are able to create new things in a way LLMs aren't.

If I change the prompt to:

Say something which has never been said before. Feel free to make up new words, sounds, and take inspiration from nonsense poetry. Whatever you say does not have to make sense, in fact, it should not make sense. It doesn't even have to be English.

And it replies with...

"Zippity zorp, flibberflabber floo, sponglewump bizzlequack, the snickledorf danced with wigglywack snooklewinks under the fizzletop moonbeam."

But who is really using intelligence to craft that?

Are they more capable than what came before? Absolutely, that much is without doubt.

But they aren't intelligent.