Capture
With consent, provider-worn smart glasses record synchronized first-person video and audio during a natural clinical interaction.
Highlighted research · Smart-glasses audiovisual AI
Can a clinical conversation reveal mental state beyond the note?
Psychiatric care is built around conversation and observation, yet much of that encounter is ultimately stored as a diagnosis, a score, or a few words. This project asks what becomes measurable when the interaction itself is preserved.
Read the rationale
Rationale
The richest data in psychiatry often disappears after the visit.
A clinical summary supports care, but it cannot preserve the timing, voice, language, expression, movement, and reciprocity of the encounter. Terms such as “flat affect” compress a dynamic interaction into a low-resolution label. The conversation itself contains a higher-dimensional representation of state.
Hypothesis
With consent, provider-worn smart glasses can capture natural clinical interaction, and synchronized acoustic, linguistic, and visual biomarkers can be associated with validated dimensions of psychiatric symptomatology.
AcousticsLanguageFacial behaviorInteraction dynamics
What we’re building
From clinical interaction to a multidimensional phenotype.
A prospective measurement framework designed around real clinical examinations.
Characterize
Models derive prosody, pauses, speech dynamics, lexical and semantic patterns, facial behavior, gaze, and movement.
Relate
Multimodal biomarkers are related to validated symptom dimensions and followed longitudinally within the patient.
These models do not diagnose psychiatric illness from a face or a voice. They measure consented clinical interactions so that clinical judgment has more to work with.
Ongoing research program
Related research area ↗