Journal Article

DZNE-2026-00199

http://join2-wiki.gsi.de/foswiki/pub/Main/Artwork/join2_logo100x88.png Real-Time Emotion Recognition Performance of Mobile Devices: A Detailed Analysis of Camera and TrueDepth Sensors Using Apple's ARKit.

Aldenhoven, C. M. ; Nissen, L. ; Heinemann, M. ; Dogdu, C.DZNE* ; Hanke, A. ; Jonas, S. ; Reimer, L. M.Extern*

2026
MDPI Basel

This record in other databases:    

Please use a persistent id in citations: doi:10.3390/s26031060

Abstract: Facial features hold information about a person's emotions, motor function, or genetic defects. Since most current mobile devices are capable of real-time face detection using cameras and depth sensors, real-time facial analysis can be utilized in several mobile use cases. Understanding the real-time emotion recognition capabilities of device sensors and frameworks is vital for developing new, valid applications. Therefore, we evaluated on-device emotion recognition using Apple's ARKit on an iPhone 14 Pro. A native app elicited 36 blend shape-specific movements and 7 discrete emotions from N=31 healthy adults. Per frame, standardized ARKit blend shapes were classified using a prototype-based cosine similarity metric; performance was summarized as accuracy and area under the receiver operating characteristic curves. Cosine similarity achieved an overall accuracy of 68.3%, exceeding the mean of three human raters (58.9%; +9.4 percentage points, ≈16% relative). Per-emotion accuracy was highest for joy, fear, sadness, and surprise, and competitive for anger, disgust, and contempt. AUCs were ≥0.84 for all classes. The method runs in real time on-device using only vector operations, preserving privacy and minimizing compute. These results indicate that a simple, interpretable cosine-similarity classifier over ARKit blend shapes delivers human-comparable, real-time facial emotion recognition on commodity hardware, supporting privacy-preserving mobile applications.

Keyword(s): Humans (MeSH) ; Emotions: physiology (MeSH) ; Adult (MeSH) ; Male (MeSH) ; Female (MeSH) ; Mobile Applications (MeSH) ; Smartphone (MeSH) ; Young Adult (MeSH) ; Facial Expression (MeSH) ; ARKit ; emotion recognition ; face tracking ; real-time ; sensors

---

Contributing Institute(s):

1. Translational Dementia Research (Bonn) (AG Schneider)

Research Program(s):

1. 353 - Clinical and Health Care Research (POF4-353) (POF4-353)

---

Database coverage:
; ; ; Article Processing Charges ; Clarivate Analytics Master Journal List ; Current Contents - Engineering, Computing and Technology ; Current Contents - Physical, Chemical and Earth Sciences ; DOAJ Seal ; Ebsco Academic Search ; Essential Science Indicators ; Fees ; IF < 5 ; JCR ; PubMed Central ; SCOPUS ; Science Citation Index Expanded ; Web of Science Core Collection

---