Speech Communication Group

Gesture Research

Quick Access Links

Research

We are investigating the multimodal alignment across hand gestures, body movements, prosody, intonational phrases, morphosyntax, discourse, and pragmatics.

Part of this work involves creating a gesture labeling system that carefully identifies features that are mutually exclusive. Our system gives particular attention to the kinematics of gestures, separating the physical movement from the interpretation of that movement. As we focus on piecing apart the kinematic features, we bear in mind how this system would accommodate automatic feature tracking. As Artificial Intelligence and Machine Learning technologies advance, this system envisions what automatic gesture labeling would look like and how it would benefit researchers.

Collaborations

Working with Pilar Prieto, Patrick Rohrer, Nuria Gilbert, Ingrid Vila, Julia Florit (Universitat Pompeu Fabra in Barcelona) on M3D labeling system including a pragmatic labeling scheme

Publications

  • Shattuck-Hufnagel, S., Ren, A., Mathew, M., Yuen, I., Demuth, K. (2016) Non-referential gestures in adult and child speech: Are they prosodic?. Proc. Speech Prosody 2016, 836-839. (link)
  • Shattuck-Hufnagel, S., Ren, A., (2012) Preliminaries to a Kinematics of Gestural Accents. Presented at International Society for Gesture Studies 6 San Diego.
  • Shattuck-Hufnagel, S., Ren, A., Tauscher, E. (2010) Are torso movements during speech timed with intonational phrases? Presented at Speech Prosody 2010.

Team

Researchers
Stefanie Shattuck-Hufnagel
Ada Ren-Mitchell

Undergraduates
Sydne Ashford
Ashley Teng
Riley George

Alumni
Jennifer Xiong