---
title: "Empathic AuRea: Exploring the Effects of an Augmented Reality Cue for Emotional Sharing Across Three Face-to-Face Tasks"
authors:
  - Andreia Valente
  - Daniel Simoes Lopes
  - Nuno Jardim Nunes
  - Augusto Esteves
year: 2022
venue: "IEEE Conference on Virtual Reality and 3D User Interfaces (IEEE VR 2022)"
doi: "10.1109/VR51125.2022.00034"
url: "https://andreia-valente.com/publications/valente2022empathic.html"
pdf: "https://andreia-valente.com/pdfs/valente2022empathic.pdf"
topics:
  - Augmented Reality
  - Empathic Computing
  - Physiological Sensing
  - Emotional Sharing
  - ECG
---

# Empathic AuRea: Exploring the Effects of an Augmented Reality Cue for Emotional Sharing Across Three Face-to-Face Tasks

## Citation Metadata

- Authors: Andreia Valente, Daniel Simoes Lopes, Nuno Jardim Nunes, Augusto Esteves
- Venue: IEEE Conference on Virtual Reality and 3D User Interfaces (IEEE VR 2022)
- Year: 2022
- DOI: https://doi.org/10.1109/VR51125.2022.00034
- HTML: https://andreia-valente.com/publications/valente2022empathic.html
- PDF: https://andreia-valente.com/pdfs/valente2022empathic.pdf

## Plain-Language Summary

Empathic AuRea explores whether augmented reality can help people understand each other's emotional states during face-to-face interaction. The system senses physiological signals, infers emotional state, and displays an AR cue as a color-coded ripple around the person being sensed. The goal is to make otherwise hidden affective information visible in real time without replacing normal conversation, facial expression, or body language.

The paper studies AuRea across three tasks: a validation task, a collaborative pattern-block instruction task, and a storytelling task. The results show that emotional biofeedback can improve emotional understanding and support some collaborative outcomes, but its effect depends strongly on the task. It can help when a speaker needs to adapt instructions to a partner's state, but it can distract when the task depends on memory and focused attention.

## System and Study

AuRea uses physiological sensing and emotion recognition to generate an AR emotional sharing cue. One participant's inferred emotional state is visualized for the other participant through an augmented reality display. The study examines both the person viewing the cue and the person being sensed, which is important because emotional sharing systems can create asymmetric experiences.

The work evaluates performance, workload, usability, emotional understanding, worry, and interpersonal connection. It also examines how familiarity between participants affects comfort and interpretation.

## Key Findings

- AuRea improved the viewer's ability to infer a partner's emotional state in the validation task.
- In the collaborative pattern-block task, access to the emotional cue helped some participants adapt pacing and instructional style.
- In the storytelling task, the cue was less helpful and could distract from memorization.
- The person viewing the cue reported increased connection in some conditions, while the person being sensed could feel more exposed.
- Worry increased for the person whose emotions were being sensed, especially when paired with someone less familiar.
- Hardware constraints from the AR setup, including field of view and video passthrough quality, contributed to workload and discomfort.

## Design Implications

Emotional sharing cues should be matched to tasks where real-time interpersonal adaptation matters. They may be valuable for collaborative instruction, negotiation, or support tasks, but less suitable for memory-heavy or attention-intensive activities. The paper also highlights the importance of consent, reciprocity, and privacy: a system that empowers the viewer can simultaneously make the sensed person feel vulnerable.

## Emotion Recognition and Visualization

AuRea uses ECG-based emotion estimation to infer valence and arousal, then maps the inferred state to an augmented visual cue around the sensed person. The cue combines color and brightness in a ripple-like aura around the head, making affective state visible to a conversation partner while leaving the person physically present in the same face-to-face setting.

The emotion model was trained with physiological and self-report data collected while participants watched affective film clips. ECG was recorded with a BITalino setup, and valence and arousal were reported using the Self-Assessment Manikin. The model used ECG-derived features such as heart rate variability and ECG-derived respiration, after filtering and QRS detection, to predict a location in valence-arousal space. A separate color mapping associated warm colors with higher arousal and cool colors with lower arousal.

## Study Tasks and Results

The final study paired participants so that one person, the decoder, saw the AR emotional cue for the other person, the encoder. The study used three tasks: a validation task for emotional understanding, a Pattern Blocks task requiring instruction and collaboration, and a Storyteller task requiring memory and retelling.

The system improved emotional understanding in the validation task. In the Pattern Blocks task, decoders used the cue to adapt pacing and instruction style when the encoder appeared nervous or uncertain, improving collaborative performance. In the Storyteller task, however, the cue reduced memorization performance because participants had to divide attention between the story and the emotional visualization.

The paper reports an important asymmetry in social experience. Decoders often felt more connected because they gained access to additional emotional information, while encoders could feel more worried or exposed because their internal state was being represented to someone else. This effect was stronger when pairs were less familiar with each other.

## Limitations and Future Directions

The study was constrained by a small sample during COVID-era recruitment and by the hardware used for video see-through AR. The passthrough setup, field of view, camera offset, and display quality contributed to eyestrain, motion sickness, and physical demand. The authors suggest that optical see-through AR could reduce some of these issues.

Future work includes collecting physiological signals from both members of the pair, studying emotional contagion and synchrony, examining autism and social interaction support, and testing instructor-student or classroom scenarios where one-sided emotional awareness could either support adaptation or create new privacy concerns.

## Why This Paper Matters

This paper is useful for researchers studying empathic computing, physiological disclosure, AR collaboration, affective interfaces, and social biofeedback. It shows that emotional transparency is not automatically beneficial; it changes the social relationship between the person who sees the cue and the person whose body is being represented.

## Recommended Citation

Valente, A., Simoes Lopes, D., Nunes, N. J., & Esteves, A. (2022). Empathic AuRea: Exploring the effects of an augmented reality cue for emotional sharing across three face-to-face tasks. In 2022 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE. https://doi.org/10.1109/VR51125.2022.00034

```bibtex
@inproceedings{valente2022empathic,
  title = {Empathic {AuRea}: Exploring the Effects of an Augmented Reality Cue for Emotional Sharing Across Three Face-to-Face Tasks},
  author = {Valente, Andreia and Simoes Lopes, Daniel and Nunes, Nuno Jardim and Esteves, Augusto},
  booktitle = {2022 IEEE Conference on Virtual Reality and 3D User Interfaces (VR)},
  year = {2022},
  publisher = {IEEE},
  doi = {10.1109/VR51125.2022.00034},
  url = {https://doi.org/10.1109/VR51125.2022.00034}
}
```