That's also why it falls in the area of data science. You develop a model based on available papers of similar tasks and refine the model and training with statistical methods.

LLM is pretty much guessing the next word

In ML you are usually predicting a value based on values in the training set

No, that's just a small part of ML: Supervised learning. There also is unsupervised learning, reinforcement learning and a whole bunch of other things in machine learning; It's a way bigger field than just that.

And about your question: Yeah, LLMs are a prime example of machine learning. Very simplified, they use a kind of recurrent neural network to take inputs of arbitrary lengths and give outputs. They are trained on huge loads of data (text) to auto-complete the data (so that they get e.g. a sentence as input and give a second sentence that's likely the next sentence in the data as output). E.g. "Today I went" as input could generate "to school." as output.

ChatGPT is based on these LLMs like GPT-4 in the way that the start of the input data is commands in human language for the bot how to behave. (E.g. "You are called ChatGPT. You are not allowed to […]. You are helpful and friendly."), then adding the user input. The LLM then generates what the chatbot described with the given characteristics would give as an output based on the training set and it's returned as the output by ChatGPT.