Em-Object

Emotional I/O Interface for Sensing and Control based on human emotions 

What if inanimate objects could feel and express emotions? Or what if they could be a mirror of the present or past emotional state of the environment in which they are located, and convey that environmental emotional state in a very direct way? Could we encode other kinds of information into emotion? And can we control the devices and gadgest in our surroundings directly through our emotions? We developed EmObject while searching for an answer to those questions. 

EmObject is novel, interactive, tangible user interface that bridges the gap between the worlds of bits, atoms, and emotions. It is a physical manifestation of the familiar digital emoji, with rich input, output, and control capabilities. It allows (1) mapping of physical and digital info qrmation into emotional expressions, and (2) controlling external devices through change of the facial expressions of the device or the user. Emobject leverages the Affectiva APIs to detect human emotion from facial expressions, and can then control other devices in response to this emotional input. 

Application Scenarios

Social Mood Monitor and Display - Track the emotional atmosphere in the room through facial expression analysis, and display the average emotion for that environment based on a moving average. This application scenario allows a person who may just be entering the room to know that emotional atmoshpere they are entering and can act accordingly. 

Emotion Record / Reply - What if you could play back your facial expressions for the day in a time-lapsed fashion; would you be surprised at what you see? Would it provoke you to take a certain action or make behavioral change? 

Emotion Mirror Display - Emobject can be attached to a bathroom mirror to monitor your emotional state every time you face the mirror, and then display your average emotional expression during the past week, and the variation in emotions over time. When presented with this information, users would have the motivation to smile more often. 

Behavior Changing Device - Emobject could be used to encourage good habits such as proper hand washing by monitoring how many seconds people spend washing their hands. Users would be presented with a smiling face only if they spend 30+ seconds, and with an angry face if they took shorter time. 

Control Interface - Emobject can connect to IoT-enabled devices in the home, allowing users to control those devices simply by making particualr facial expession 

Development Platform - Users can map any input data-stream to any facial expression, and they can choose to control any IoT device through facial expressions. Moreover, they can even define entirely new kinds of facial expressions. 

My Contributions:

Collaborators:

Professors: