While the technical and innovative aspects of emotion recognition artificial intelligence (AI) are impressive, the depth and accuracy of the technology still remain debatable.

There are several ways in which businesses or governments use facial recognition technology. While it may have its fair share of ethical issues, facial recognition is an invaluable resource for detecting behaviors on the basis of the various non-verbal cues shown by people and what they may represent or precede. Emotion recognition is an extension of this technology. Emotion recognition AI uses machine learning algorithms to analyze how individuals emotionally react to specific situations and how their face reflects the reaction. In an ideal world, emotion recognition is groundbreaking for multiple reasons—the prime one being that it goes far beyond simply determining somebody’s identity, and can be used with great success in law enforcement, marketing, and many other areas.

How Emotion Recognition Powered By AI Works

Machine learning algorithms that enable the functioning of a facial recognition system are trained with datasets containing thousands of facial images of people from different races, countries and sexes exhibiting different emotions. The emotions shown in these images are classified into six categories—happiness, anger, surprise, disgust, fear, and sadness—and labeled accordingly. Based on this data, the emotion recognition AI makes associations between specific facial cues and their corresponding emotions. For instance, the AI learns that a smiling face is an indicator of happiness. It uses this training to make increasingly accurate predictions of people’s emotions.

Naturally, emotion recognition tools will learn and improve their detection capabilities with time. However, there are some issues that limit the effectiveness of the emotion recognition AI.

Why Gauging Real Emotions is Challenging with AI

One of the commonly heard psychology-related cliches is that an individual's face and eyes serve as the "window to their soul." While not entirely inaccurate, that line of thinking only scratches the surface when it comes to gauging an individual's true emotions. Firstly, not everybody reacts in the same way to any given situation. What some people may find revolting may make someone else smile due to various, deeply personal reasons. Additionally, there are several persons who choose to not bear their sentiments to others. So, while AI may record their expressionless face as “neutral,” the system simply cannot tell whether they are depressed, internally euphoric, contemplative, or suicidal.

Therefore, even before we make it to the racist and discriminatory aspects of facial recognition, emotion recognition AI falls short of expectations. So, does AI successfully determine a subject’s emotions? Yes and no. The technology does recognize facial expressions to identify emotions fairly accurately. However, human emotions can, sometimes, be too complex for even the most advanced AI tools to determine. But, facial recognition tools of the future may advance to the point where the minutest of factors, such as eyebrow curvature and skin paleness, will be evaluated to determine emotions with greater accuracy. Until then, as we have seen, technology still has some way to go in terms of its abilities.