Graption

Project Overview

Graption is multimodal AI for live captioning, built for the deaf and hard of hearing. Unlike existing tools that just transcribe speech, Graption shows who's speaking and how, positioning captions next to each speaker and flagging tone, ambient sounds, and moments when someone talks directly to you. Using a phone's camera and mic, Graption combines vision, audio, and language models to track speakers, detect context, and summarize anything missed while looking away. The project won at Hacklytics 2026 and is grounded in research from Georgia Tech Professor Thad Starner on head-worn caption displays, with wearable hardware as the long-term goal.