Man vs Machine: Can AI write as well as a human journalist?

It's one of the biggest questions of our time. Is AI really as creative as a human and can it write with the same skill as an experienced journalist or copywriter?

I started to worry about the future of writers after the service Jasper.ai launched in 2021 and threatened to put real-life Jaspers out of work permanently. The rise of ChatGPT and rival models has only compounded those concerns.

But is there anything to be worried about? To find out, I've decided to put my reputation on the line in a new series here on Machine that will directly compare the writing abilities of GenAI models and a human journalist (me) with 20 years of experience.

It's called Man vs Machine and will hopefully not be the worst idea of my two-decade-long career in the media.

Clearly, there are jobs that AI cannot do right now. I won't be asking it to interview tech pioneers and thought leaders, for instance. What I want to do is find out if it can perform the job of producing basic news stories. And if it can, I'll be happy to name AI as the first non-Jasper employee here at Machine.

So here goes. We'll start with a basic press release. Have a read of both articles and then vote for which you prefer in the survey at the bottom. I've removed obvious signs of AI like the dreaded em dash or phrase like "in the dynamic landscape of digital transformation".

And at the risk of biasing this experiment, we must issue the following warning.

Disclaimer: Some of the quotes in one of the articles below may not be accurate and the content could contain false or incorrect claims. Please check against the original release, which is linked at the bottom.

1) Human artists still outperform AI in creativity, study finds

Professional artists still have an edge over artificial intelligence in producing creative artwork, according to new research, but experts warn that the gap may soon close as AI continues to evolve.

A study published in Psychology of Aesthetics, Creativity, and the Arts found that images generated using prompts written by professional artists were rated as significantly more creative than those generated solely by AI or by novice artists.

Researchers at Duke University asked 15 professional artists and 15 untrained individuals to create text prompts for the AI image generator DALL-E 3. They also fed the same instructions into ChatGPT to generate additional AI-crafted prompts. A separate group of 299 participants then evaluated the resulting images on their creativity.

Dr Paul Seli, lead author and assistant professor of psychology and neuroscience at Duke, said the findings highlight the unique qualities of human creativity.

“Creativity has long been considered a uniquely human ability, tied to personal experiences, emotions and the drive to communicate meaning,” he said. “Now that AI can generate complex, aesthetically compelling artwork, we have to ask whether human creativity still holds a distinct edge, or if AI is fundamentally changing our understanding of creative expression.”

The study found that professional artists and ChatGPT both produced longer, more complex prompts, which led to more creative outputs. However, the professional artists used words with greater “semantic distance” - pairing less obviously related terms, a trait linked to originality.

For example, one artist’s prompt described “a madman trapped in a straitjacket made of toilet paper,” whereas a novice wrote “a frog using a leaf as an umbrella.”

Seli said the study suggests AI still struggles to replicate the depth and intent behind human creativity, but he warned that rapid advances in AI could soon challenge that advantage.

“If AI becomes better at understanding artistic intent or mimicking human-like intuition, the gap between AI and professional artists could narrow or even disappear,” he said. “However, human experiences and emotions may continue to set us apart—but whether that distinction matters in the long run remains an open question.”

2) Human artists are more creative than AI - for now

Professional artists are more creative than AI but may not be able to maintain their "edge" for much longer.

That's the warning from researchers at the American Psychological Association (APA), who found that human creatives come up with more novel and interesting ideas than machines.

The rapid rise of AI poses "existential questions about the nature of creativity", said lead researcher Paul Seli, PhD, assistant professor of psychology and neuroscience at Duke University. 

“Creativity has long been considered a uniquely human ability that is tied to personal experiences, emotions and the drive to communicate meaning,” he continued.

“But now that AI can generate complex, aesthetically compelling artwork, we have to ask if human creativity still holds a distinct edge or is AI fundamentally changing the way we think about creative expression.”

We're glad to report that Seli's research found that professional artists "still have the upper hand".

To prove this, researchers asked artists, non-artists and ChatGPT to write prompts for DALL-E 3, an image-generating AI model.

A group of 299 online participants then viewed the images and rated them on creativity.

Images created using professional artist's prompts were deemed to be "significantly more creative" than the AI-generated artwork.

However, AI performed better than non-artists, who finished in last place. 

Professional artists and ChatGPT used more words in their prompts, which may have helped to generate more creative artwork. Their language also showed more "semantic distance", which means the words found in prompts were not commonly related to each other.

For example, a professional artist's prompt said: "A madman trapped in a straitjacket made of toilet paper."

A non-artist wrote: "A frog using a leaf as an umbrella."

“Whether AI is truly ‘creative’ depends on how we define creativity,” Seli said. “AI undeniably generates images and other outputs that people perceive as creative. But if creativity is tied to human experience, emotions and intentionality, then AI appears to fall short.”  

Professional artists should not be complacent, as their advantage may be short-lived.

“If AI becomes better at understanding artistic intent or mimicking human-like intuition, the gap between AI and professional artists could narrow or even disappear,” Seli added. “However, it’s possible that the uniquely human aspects of creativity - such as emotional depth and lived experience - will continue to set human artists apart. Whether that distinction matters in the long run remains an open question. Only time will tell.” 

The study was published online in Psychology of Aesthetics, Creativity, and the Arts and is called Beyond the Brush: Human Versus Artificial Intelligence Creativity in the Realm of Generative Art.

Read the original release here and vote in our poll below to say which of these articles you prefer.