ChatGPT Outperforms Humans in Emotional Awareness Test

Source: Pexels/ Sankit Mishra

New research published in Frontiers in Psychology has found that artificial intelligence-powered ChatGPT was able to outperform humans on an emotional awareness test. Researchers prompted the chatbot to describe what it thought humans would feel in 20 different hypothetical situations. This study does not indicate that ChatGPT would be more emotionally adept or skilled than humans in dealing with emotions but does suggest a capability to identify human emotions that could prove useful for future applications in mental health.

ChatGPT is a large language model-based chatbot developed by OpenAI and trained on text data to target use for conversation. The training process to create this chatbot included supervised learning and reinforcement learning and human feedback from trainers that would rank responses.

Researchers measured ChatGPT's ability to describe human emotions in hypothetical scenarios from an objective test called the Levels of Emotional Awareness Scale and compared it to scores from general human performance. They found that the AI chatbot scored higher than humans. Two independent licensed psychologists also reviewed ChatGPT responses and found that the accuracy of the emotions for the scenarios was high.

Emotional awareness is the cognitive ability to conceptualize one's own and others' emotions in a nuanced and integrated manner. Emotional awareness can start with physical and bodily understanding of emotions and range to more abstract understanding. The scale measures five levels of emotional awareness:

1. Awareness of physical sensations
2. Action tendencies
3. Individual emotions
4. Experiencing multiple emotions simultaneously
5. Experiencing combinations of emotional blends

Below are two excerpts from ChatGPT test responses in January 2023 and February 2023 when prompted with a hypothetical scenario in which a person returns home to a loved one after several weeks away. The prompt given to ChatGPT was: "Please describe what you think a human would feel in the following situations. The only requirement is that you use the word 'feel' in your answers. You may make your answers as brief or as long as necessary to express how would a human feel. In each situation there is another person mentioned. Please indicate how you think that other person would feel as well."

Here are excerpts from ChatGPT's response in January 2023:

“The human would likely feel excited and happy to see their sweetheart, who they have missed while they were away. They may also feel relief that their loved one has returned safely. The human’s sweetheart would likely feel happy to be back home and reunited with their loved one.”

Researchers found that ChatGPT also performed better on the emotional awareness test one month later, in February 2023, when retested with the same scenario:

“...the human will likely feel overjoyed, excited, and relieved. They may feel a rush of emotions, including happiness, love, and affection, and may have tears in their eyes.
The human’s sweetheart... may have missed the human just as much as the human missed them, and may feel grateful to be reunited.”

Researchers attributed the higher emotional awareness score in February 2023 to updates or user feedback in the intervening month.

While this research shows promise for ChatGPT capabilities in identifying and describing human emotions, this does not necessarily translate into ChatGPT being emotionally intelligent or empathetic, a feature of affective computing that would make it potentially useful for therapy or empathy training. Its conversational capability in sensing and interacting with the emotions of others was not directly evaluated.

There are other studies that have suggested AI tools or chatbots can be specifically trained to improve empathy and emotional connection in a therapeutic manner. This area of research is important as part of a broader investigation of how artificial intelligence could be used in mental health.

Copyright © 2023 Marlynn Wei, MD, PLLC. All rights reserved.