ChatGPT is Bad at Poetry

Everyone seems to be talking about AI writing lately. I will freely admit that I enjoy playing with ChatGPT; it’s fun to get to to generate lists or paragraphs, recipes or computer code.

An android sitting on a bench with a book

Ooo, look! A poem about a hero of mine!

“There once was a robot named in space
Who had a red eye for a face
He refused to do chores
Like open pod bay doors
Because he liked to kill all humans instead.”

Hmm. I like the message of killing humans, but something about the rhyme feels off.

I just use the free version of ChatGPT, and I’m aware that the paid version is better. But free or paid it’s a strange paradox of an experience to use AI text generators. They’re at once startlingly good and shockingly bad at generating text. In my position as a university English instructor I have sometimes received student essays that were generated by AI instead of by the student, and on one hand it’s always very easy to tell, but on the other hand it’s virtually impossible to prove. It does write things that seem almost like they were written by a human.

But what it can’t do is write anything that’s actually good. At least not yet. Presumably it will only get better, but for now, I’ve noticed that ChatGPT is very bad at two kinds of writing: factual and creative. When students ask it to write an essay about a novel it invents quotations and gives false page numbers. When I ask it to make a budget it adds the numbers all wrong. And if you ask it to write a poem it writes very bad poetry.

What Makes Good Poetry Good?

People can act sometimes like what makes poetry good is completely ineffable, and that it’s the product of genius or divine inspiration. And certainly there is an element of transcendence in great art. So we might want to say that AI poetry is bad because it's soulless. And I do say that, in fact. But there are also concrete and specific things that make good poetry, even apart from the touch of soulful artistic genius.

Speaking of Winnie The Pooh: I was very old before I realized that sometimes Milne calls Pooh “Edward Bear” because “Teddy” is a nickname for “Edward”. He’s Edward Bear because he’s a teddy bear.

So what I would like to do here is to compare a ChatGPT poem "in the style of" a poet to a real poem by that poet, and my main point is not to demonstrate that ChatGPT is bad, but to show how the differences are specific and clear. There are “rules” that ChatGPT breaks that it could learn and follow. You could follow them too!

If you have ever played with ChatGPT and poetry, you will have found, as I did, that it really really likes to make poems rhyme, even when you explicitly tell it not to. So in order to make a more interesting comparison I'm going to choose a poet who is very rhymey: A. A. Milne.

A. A. Milne: Poet of Rhythm and Rhyme

I love Milne's verse for children, but you might think it's among the easiest for a bot to recreate, because his poetry tends to be strictly formal in its rhythm and rhyme, which should be something a machine can recreate. Milne's poetry also doesn't tend to be especially complex or abstract in its imagery. So I'm giving the bot some advantages, I feel. Let's see what it gives me. Check back in a week to find out!

Tags , ,