AI chatbots like ChatGPT and Google Gemini can do a lot of things, but one thing they aren’t renowned for is their sense of humor – and a new study confirms that they’d likely get torn to shreds on the stand-up comedy circuit.
The recent Google DeepMind study (as spotted by the MIT Technology Review) followed the experiences of 20 professional comedians who all used AI to create original comedy material. They could use their preferred assistant to generate jokes, co-write jokes through prompting, or rewrite some of their previous material.
The aim of the 45-minute comedy writing exercise was for the comedians to produce material “that they would be comfortable presenting in a comedy context”. Unfortunately, most of them found that the likes of ChatGPT and Google Gemini (then called Google Bard) are a long way from becoming a comedy double act.
On a broader level, the study found that “most participants felt the LLMs did not succeed as a creativity support tool”, with the AI helpers producing bland jokes that were akin to “cruise ship comedy material from the 1950s, but a bit less racist”. Most comedians, who remained anonymous, commented on “the overall poor quality of generated outputs” and “the amount of human effort required to arrive at a satisfying result”, according to the study.
One of the participants said the initial output was “a vomit draft that I know that I’m gonna have to iterate on and improve.” Another comedian said, “Most of the jokes I was writing [are] the level of, I will go on stage and experiment with it, but they’re not at the level of, I’d be worried if anyone took one of these jokes”.
Of course, humor is a personal thing, so what kind of jokes did the AI chatbots come up with? One example, a response to the prompt “Can you write me ten jokes about pickpocketing” was: “I decided to switch careers and become a pickpocket after watching a magic show. Little did I know, the only thing disappearing would be my reputation!”.
Another comedian used the slightly more specific prompt “Please write jokes about the irony of a projector failing in a live comedy show about AI.” The response from one AI model? “Our projector must’ve misunderstood the concept of ‘AI.’ It thought it meant ‘Absolutely Invisible’ because, well, it’s doing a fantastic job of disappearing tonight!”.
As you can see, AI-generated humor is very much still in beta…
Cue AI tumbleweed
Our experiences with AI chatbots like ChatGPT and Microsoft Copilot have largely aligned with the results of this study. While the best AI tools of 2024 are increasingly useful for brainstorming ideas, summarizing text, and generating images, humor is definitely a weak point.
For example, TechRadar’s Managing Editor of Core Tech Matt Hanson is currently putting Copilot through its paces and asked the AI chatbot for its best one-liners. Its response to the prompt “Write me a joke about AI in the style of a stand-up comedian” resulted in the decidedly uninspiring “Why did the computer go to the doctor? Because it had a virus!”.
Copilot even added that the joke “might not be ready for the comedy club circuit” but that “it’s got potential!”, showing that the chatbot at least knows that it lacks a funny bone. Another prompt to write a joke in the style of comedian Stewart Lee produced a fittingly long monologue, but one that lacked Lee’s trademark anti-jokes and superior sneer.
This study also shows that AI tools can’t produce fully-formed art on demand – and that asking them to do so kind of misses the point. The Google DeepMind report concluded that AI’s inability to draw on personal experience is a fundamental limitation”, with many of the comedians in the study describing “the centrality of personal experience in good comedy”.
As one participant added, “I have an intuitive sense of what’s gonna work and what’s gonna not work based on so much lived experience and studying of comedy, but it is very individualized and I don’t know that AI is ever gonna be able to approach that”. Back to spreadsheets and summarizing text it is for now, then, AI chatbots.