Quick Access Links
We are investigating the multimodal alignment across hand gestures, body movements, prosody, intonational phrases, morphosyntax, discourse, and pragmatics.
Part of this work involves creating a gesture labeling system that carefully identifies features that are mutually exclusive. Our system gives particular attention to the kinematics of gestures, separating the physical movement from the interpretation of that movement. As we focus on piecing apart the kinematic features, we bear in mind how this system would accommodate automatic feature tracking. As Artificial Intelligence and Machine Learning technologies advance, this system envisions what automatic gesture labeling would look like and how it would benefit researchers.
Working with Pilar Prieto, Patrick Rohrer, Nuria Gilbert, Ingrid Vila, Julia Florit (Universitat Pompeu Fabra in Barcelona) on M3D labeling system including a pragmatic labeling scheme