Gestural Narrative Interactive Expression (GENIE) Project

2010-2012

Abstract

The Gestural Narrative Interactive Expression (GeNIE) Project (NEH Digital Humanities Start-Up Grant) aims to develop and better understanding the use of gestural interfaces for expressive works of interactive narrative. Gestural interfaces have become more popular with the increasing prevalence of systems such as the Nintendo Wii, Microsoft Kinect for Xbox 360, mobile phones with multi-touch screens and built-in gyroscopes and accelerometers, and laptop computers equipped with touchpads. Yet, most previous interactive narrative systems have used menu-driven or text-based interfaces for communication between characters. In contrast, the GeNIE project has resulted in a platform for building gesture-driven narratives: imagine responding to another character’s questions by continuously sliding your finger up and down, puppeteering your own character’s nodding head to indicate a “yes” response. Or you might use your finger to draw a ‘^’ shape, raising a character’s eyebrow to express skepticism, followed by your performance of a pinch-in motion using your fingers to result in your character’s mood becoming more withdrawn. The computer-controlled character’s facial expression continuously changes in response to your actions. The GeNIE platform enables this, and more, tied into a storytelling system for mobile devices based on my earlier work. Such interfaces have several advantages. They may be more immediately familiar for users, resulting in very intuitive input mechanisms. Beyond this, such interfaces allow for a diversity of forms of non-verbal input. Non-verbal, gestural input can be either universal (like smiling) or culturally-specific (like shoulder shrugging), but either way it is a type of communication that we all use everyday. Gestural interfaces have not yet been well explored in interactive narrative or creative computing more broadly, the GeNIE project is a step in that direction.

Video: