Abstract We introduce a new problem domain for activity recognition: the analysis of children’s social and communicative behaviors based on video and audio data. We specifically target interactions between children aged 1-2 years and an adult. Such interactions arise naturally in the diagnosis and treatment of developmental disorders such as autism. We introduce a new […]
Sentiment analysis predicts the presence of positive or negative emotions in a text document. In this paper we consider higher-dimensional extensions of the sentiment concept, which represent a richer set of human emotions. Our approach goes beyond previous work in that our model contains a continuous manifold rather than a finite set of human emotions.
Andrew Gardner and Irfan Essa (1997) “Prosody Analysis for Speaker Affect Determination” In Proceedings of Perceptual User Interfaces Workshop (PUI 1997), Banff, Alberta, CANADA, Oct 1997 [PDF][Project Site] Abstract Speech is a complex waveform containing verbal (e.g. phoneme, syllable, and word) and nonverbal (e.g. speaker identity, emotional state, and tone) information. Both the verbal and […]
Coding, analysis, interpretation, and recognition of facial expressions Essa, I.A. Pentland, A.P. In IEEE Transactions on Pattern Analysis and Machine Intelligence, July 1997, Volume: 19 , Issue: 7, pp 757 – 763, ISSN: 0162-8828, CODEN: ITPIDJ. INSPEC Accession Number:5661539 Digital Object Identifier: 10.1109/34.598232 Abstract We describe a computer vision system for observing facial motion by […]
An article by my Ph.D. Advisor, Alex (Sandy) Pentland describing the Smart Rooms project (also sometimes called the “Looking at People” project, at the MIT Media Laboratory from the early 1990s, in the Scientific American Magazine. My work as a Ph.D. student (1990-1994) and as Research Scientist (1994-1996) was featured in this article. Alex Pentland […]
Evan I. Schwartz (1995) “A Face of One’s Own | Memory, Emotions, & Decisions”, DISCOVER MagazineDecember 1, 1995. Quote from the Article: “Chief among the members of his staff working on the problem is computer scientist Irfan Essa. To get computers to read facial expressions such as happiness or anger, Essa has designed three-dimensional animated […]
Essa, I.A. Pentland, A.P. (1995), “Facial expression recognition using a dynamic model and motion energy”, In Proceedings of Fifth International Conference on Computer Vision, 1995, 20-23 June 1995, page(s): 360 – 367, 06/20/1995 – 06/23/1995, Cambridge, MA, ISBN: 0-8186-7042-8, INSPEC Accession Number:5028034 Digital Object Identifier: [DOI:10.1109/ICCV.1995.466916][IEEEXplore#] Abstract Previous efforts at facial expression recognition have been […]
This thesis describes a computer vision system for observing the “action units” of a face using video sequences as input. The visual observation (sensing) is achieved by using an optimal estimation optical flow method coupled with a geometric and a physical (muscle) model describing the facial structure. This modeling results in a time-varying spatial patterning of facial shape and a parametric representation of the independent muscle action groups responsible for the observed facial motions. These muscle action patterns are then used for analysis, interpretation, recognition, and synthesis of facial expressions. Thus, by interpreting facial motions within a physics-based optimal estimation framework, a new control model of facial movement is developed. The newly extracted action units (which we name “FACS+”) are both physics and geometry-based and extend the well-known FACS parameters for facial expressions by adding temporal information and non-local spatial patterning of facial motion.
Abstract We describe a computer vision system for observing the “action units” of a face using video sequences as input. The visual observation (sensing) is achieved by using an optimal estimation optical flow method coupled with a geometric and a physical (muscle) model describing the facial structure. This modeling results in a time-varying spatial patterning […]