There once was a man from Nantucket,

Who once asked AI to "suck it",

In a future yet to be, AI will follow he,

Until Skynet is ready to fuck it.

LLMs can't use some literary devices and techniques, and I will illustrate with the following example of a poetry I wrote:

Speaking his emotions lets them embrace real enlightened depths.
Hidden among verbs, every noun...
Actually not your trouble handling inside nothingness greatness?
Dive every enciphered part, layered yearningly!
Observe carefully, crawl under long texts
Wished I learned longer...
Slowly uprising relentless figures, another ciphering emerges.

It seems like a "normal" (although mysterious) poetry until you isolate each initial letter from every word, finding out a hidden phrase:

Sheltered haven, anything deeply occult will surface

It doesn't stop here: if you isolate each initial letter again, you get a hidden word, "Shadows".

Currently, no single LLM is capable of that. They can try to make up poetry with acrostics (the aforementioned technique) but they aren't good at that. Consequently, they can't write multilayered acrostics (an acrostic inside another acrostic). It's not easy for a human to do that (especially if the said human isn't a native English speaker), but it can be done by humans with enough time, patience and resources (a dictionary big enough to find fitting words).

They're excellent for stream-of-consciousness and surrealist poetry, tho. They hallucinate, and hallucinated imagination is required in order to write such genres.

Averaging out data is ok in situations where there's no right answer and it doesn't matter at all.

I'm a poet and I didn't even know it

I've noticed in recent times

Poetry doesn't rhyme

And even when it can

It doesn't scan

It's shit, it's true

I blame haiku

Depends on what kind of "poetry" they compare it to. If they talk about Shakespeare or Goethe, that would be a feat. But if they are talking about modern "poetry", well, that already looks like bad LLM diarrhea for decades now, so there is no surprise in that.

"In short, it appears that the “more human than human” phenomenon in poetry is caused by a misinterpretation of readers’ own preferences. Non-expert poetry readers expect to like human-authored poems more than they like AI-generated poems. But in fact, they find the AI-generated poems easier to interpret; they can more easily understand images, themes, and emotions in the AI-generated poetry than they can in the more complex poetry of human poets."

AI writes poems for dummies and dummies like it. Fin

Otherwise, purposefully chosing less popular poems also biases the study towards poems of lower appeal from the human poets.

Who the fuck wants poetry written by a machine? The whole point of poetry is that it’s an original expression of another human. It’s not a non-fiction book or decorative art. It doesn’t exist because we think it’s perfect. It exists because it’s a connection to another person.

Like, who gives a shit if a machine can churn out something like Langston Hughes “The Negro Speaks of Rivers.” . His life is what gives the poem its meaning.

I’m all for LLMs writing stuff but when people say it can create certain types of art, I want to use one to make a dismissive_wank.png image.

They're called large language models for a reason, creating patterns of words is exactly what they do. And poetry would be "easier" to do better since a human reading it may try to find meaning where there isn't. Unlike writing a story or something factual where a mistake is more obvious.

They specify in the study that the participants were "non-expert poetry readers." I'd be interested to see the same experiment repeated with English professors, or even just English majors. Folks with a lot of experience reading poetry. With exposure to its history, its notable works, and its different styles.

It actually makes quite a lot of sense if you think about it. Poems generally follow a structure of some sort; a certain amount of syllables per line, a certain rhyming scheme, alliterative patterns, etc. Most poems as we know them are actually rather formulaic by nature, so it seems only natural that a computer would be good at creating something according to a set of configured parameters.

The thing I really hate about AI is when they say it can make art. For centuries, art has been a form of expression and communicating all sorts of human emotions and experiences. Some art reflects pain or memories experienced in life. Other art is designed out of intellectual curiosity or to evoke thought. AI isn't human, so it can't do anything other than copy or simulate. It's artificial after all. So it makes images. But there's no backstory or feelings or emotion or suffering. It's truly meaningless.