Prototype design: Prosodic Marionette


Images above show the redesign of the interface and elements (images 1 & 2),  a screenshot of the implemented version for testing (3), and the original interface before my redesign (last image).

The goal of the Prosodic Marionette interface is to allow children to manipulate word-icon blocks bearing the intonational components of an utterance in order to match a target production. After seeing the word-icon representations while listening to the utterance, the child reconstruct prosodic contrasts in graphical rather than spoken domain by means of manipulating phoneme blocks. To attend to potential different level skills of children, I conceived of a grid that can have its resolution changed by the instructor so as to allow manipulation of the phonemes within the grid with diverse levels of precision. Other design elements were devised to enhance manipulation of phoneme according to duration and pitch as well as the visual connection between word and icon.

I worked with Jon Brumberg (University of Kansas) and Rupal Patel (CadLab, Northeastern University) in developing the inerface design that was then implemented and tested by their group, a research supported by NIH grant.