Is ChatGPT Really Funny? German Researchers Put ChatGPT's Humor to the Test

Urian B., Tech Times 10 June 2023, 01:06 am

OpenAI's ChatGPT was found to repeat certain jokes over and over in a study examining its ability to understand and generate humor.

A research paper released by two German researchers on Wednesday said that the ChatGPT-3.5 model nearly communicates on a human level but has fairly limited knowledge of jokes.

ChatGPT Repeated the Same 25 Jokes for 90% of the 1,008 Jokes

According to Ars Technica, German researchers Sophie Jentzsch and Kristian Kersting discovered that the AI chatbot prefers to repeat certain jokes.

In a test run, the two researchers from the Institute for Software Technology, German Aerospace Center (DLR), and Technical University Darmstadt found that more than 90% of 1,008 generated jokes were the same 25 jokes.

It led them to conclude that its responses were likely learned and memorized during the AI model's training and were not newly generated.

Tests Were Not Done on OpenAI's Newer ChatGPT 4

Instead of learning to make new ones, the chatbot spilled the jokes with little to no revisions. However, the tests were done on OpenAI's ChatGPT 3.5 version and not on the newer chatbot version, ChatGPT 4. The tests were reportedly conducted by prompting ChatGPT without having access to the model's inner workings or data set.

Jokes Were Grammatically Correct But Mostly the Same

According to Jentzsch and Kersting, they test how wide the variety of ChatGPT's jokes is.

"We asked it to tell a joke a thousand times... All responses were grammatically correct. Almost all outputs contained exactly one joke," the two German researchers noted.

"Only the prompt, 'Do you know any good jokes?' provoked multiple jokes, leading to 1,008 responded jokes in total. Besides that, the variation of prompts did have any noticeable effect," they added.

What Were ChatGPT's Most Popular Jokes

Some Reddit users shared that they also noticed how ChatGPT often gives a similar answer when asked for a joke.

When asked for a joke, they said the ChatGPT frequently replied, "Why did the tomato turn red? / Because it saw the salad dressing." The researchers found that the tomato joke was the second most popular result with ChatGPT 3.5's joke results.

According to the researchers, the most popular joke was, "Why did the scarecrow win an award?" / Because he was outstanding in his field."

OpenAI's Chatbot Tries to Expand the Reason Why It Gave Certain Jokes

The researchers further noted that they also found how the chatbot struggled with sequences that did not fit into learned patterns.

They also discovered that the ChatGPT could not tell when a joke was not funny. The chatbot was found to generate what was described as "functional yet plausible-sounding explanations."

In conclusion, the researchers learned that ChatGPT's detection of jokes was heavily influenced by the joke's structure and presence of puns or inclusion of wordplay, which shows a degree of "understanding" of humor elements.