Study Reveals Readers Prefer AI-Generated Poetry Over Human Works, Sparking Debate on Creativity and Authenticity

• Bouwe van Straten

  Science Editor

Readers cannot determine whether a poem was written by a human or an artificial intelligence (AI). On average, they found poems written by AI even more beautiful, inspiring, rich in imagery and more meaningful than poetry written by humans. This is evident from a new scientific study, published in Scientific Reports.

A panel of 1,634 participants were given ten poems, either written by a human or by ChatGPT 3.5. These were human poets who are generally considered among the best in literary history, such as William Shakespeare, Lord Byron, Walt Whitman and Emily Dickinson.

The participants generally thought of the AI ​​poems as written by human beings. The five poems that they thought had not been written by anyone were all written by a real poet. The researchers believe that the subjects preferred the AI ​​poems because they were written in a simpler and more accessible way.

However, the test subjects’ preferences changed as soon as they were told that the poems were not written by humans, a second experiment showed. In it, 696 people had to judge the poems on qualities such as beauty, emotion, rhythm and originality.

Those who thought the poems were written by AI gave the poems a lower rating – regardless of whether they were written by AI or not. The group who did not know who wrote the poems rated the AI ​​poems higher.

“A well-done study,” said American psychologist Keith Holyoak, who was not involved in the research. He has long been concerned with the question of whether AI can create real poetry, including his book The Needle of the Hill.

Holyoak sees no evidence in this new research that AI is now a true poet. “The model can only write a poem in the style of Walt Whitman because he has been trained on the entire work of Walt Whitman,” he says. “If you were to train the model without these poems, the result would probably be sad.”

Perhaps more importantly, the model does not show any creativity. “If someone were to produce such symbols of Whitman, you could almost talk about plagiarism,” said Holyoak. “So we cannot compare this with the greatness of men. We can compare this to human plagiarism.”

So you could only talk about true creativity if AI wrote really good poems in the style of a new poet, whose poems were not included in the training data.

Furthermore, the question remains whether humans will be able to enjoy the fate of AI. In the study, the value for a poem decreased as soon as the researchers told them it was created by AI. “Readers also appreciate a poem because it is connected to the writer’s inner experience,” says Holyoak.

Overall, the research seems to say at least as much about readers as it does about AI skills. The subjects were not poetry experts, which could explain their preference for simple poems.

Plus, people prefer the average. For example, if you create a picture of an average face from hundreds of faces, most people will find that face very attractive.

Holyoak believes that poets could start using AI assistants to help them with the writing process. The question, however, is whether this will lead to better poems, or to more uniformity: “One risk is that AI support will stifle human creativity, causing poems to become increasingly similar. “