Selected project · 2023

Mask Interface

Kamihikouki turns mouth movements into game input through a mask-shaped interface.

Co-leader · Hardware · Software · Interface design

Mask interface prototype with integrated sensing components

Question and concept

How can mouth movements become an engaging form of interaction? Our team explored a game in which a player controls a paper airplane using lip shapes, facial expressions, and chewing movements. The design was motivated by making oral-movement exercises more engaging through game interaction.

Early mask-interface concept sketch showing the infrared camera, muscle sensor, Raspberry Pi, and signal flow — Early concept sketch: sensor placement and the flow from mouth movements to interaction. View full-size concept.

Development timeline

The project progressed from ethnography and idea sketches to wearable prototypes, public demonstrations, and a later version without the fabric mask. The original timeline below shows how these iterations developed.

Project timeline from ethnography and idea sketches through prototypes 1.0 and 1.1, IVRC and U☆PoC presentations, and prototype 2.0 — Project timeline from 2023, showing prototype iterations and exhibition milestones. View full-size timeline.

My contribution

I co-led the project and directed development of the mask and hardware, contributed to programming and interface design, and coordinated the project. The sensing interface was integrated with a game developed by teammates.

Prototype

The mask uses cameras and accelerometers to capture lip shape and oral movement. We iterated on the wearable enclosure and sensing setup, then connected the detected movements to the paper-airplane game.

Two iterations of the wearable mask interface being worn — Iterations of the wearable interface.

Demonstration of the mouth-controlled interaction — Prototype demonstration.

Exhibition and recognition

The project was exhibited at IVRC 2023 and U☆PoC (UEC Idea Demonstration Contest). The U☆PoC project received the Aflac Inc. Award.

Project award presentation — Award presentation.

Visitors at the project exhibition — Exhibition setting.