Audiovisual AI & Digital Phenotyping

Signature visualization

One encounter. Multiple measurable signals.

A synthetic interaction demonstrates how facial geometry, gaze, speech timing, prosody, and language can update together through time. Hover a signal layer to connect the model output back to the face.

Synthetic demonstration — not a diagnostic assessment.The model outputs illustrate a measurement framework; they do not infer a clinical diagnosis or ground-truth mental state.

Research scope

Capturing richer signals across the care continuum.

InpatientOutpatientAt home

Research portfolio

Selected Works

01Neurosurgery · 2026

Rui Feng et al.

Artificial Intelligence Monitoring of Neurological Status From Patient Videos in the Neuroscience Intensive Care Unit

Read the publication↗

InpatientPose AINeurotelemetry

Adult neurological monitoring

Turning routine ICU video into continuous neurotelemetry

Study workflow showing inpatient video collection, pose-estimation validation, movement-index construction, and comparison with neurological examinations — **Figure 1**The study workflow moves from secure inpatient video collection to pose-estimation validation, movement-index construction, and comparison with GCS and RASS. Figure reproduced from the publication.

Question

Can pose estimation transform routine inpatient video into a continuous, quantitative measure of neurological status?

Approach

ViTPose and Sapiens were evaluated across 998,520 video minutes from 119 patients. The stronger model generated a movement index compared with bedside GCS and RASS assessments.

Finding

Movement increased 21-fold from the lowest to highest GCS category and was approximately ten times higher in awake or agitated patients than in asleep or sedated patients.

Why it matters

Video could provide minimally invasive, interpretable neurotelemetry between intermittent bedside examinations.

02eClinicalMedicine · 2024

Alec Gleason et al.

Detection of neurologic changes in critically ill infants using deep learning on video data

Read the publication↗

InpatientNICUCerebral dysfunction

Neonatal neurotelemetry

Movement as a marker of neurologic change in critically ill infants.

ROC curves for sedation and cerebral dysfunction classifiers with feature-importance distributions by infant body region — **Figure 5**Classifiers trained on infant movement predict sedation and cerebral dysfunction; model interpretation highlights the most informative anatomic regions. Figure reproduced from the publication.

Question

Can movement extracted from routine NICU video detect clinically meaningful neurologic changes?

Approach

An infant-specific pose model was developed from 282,301 video minutes across 115 infants and linked to EEG-defined cerebral dysfunction and sedative exposure.

Finding

Movement was lower with sedation and cerebral dysfunction. On held-out infants, ROC-AUC was 0.87 for sedation and 0.76 for cerebral dysfunction.

Why it matters

Ordinary video may become a scalable, minimally invasive measurement system for infants whose neurologic status is otherwise difficult to assess continuously.

03SSRN preprint · 2026

Roshan S. Parikh et al.

Deep Learning for Pressure Injury Staging in Real-World Inpatient Photographs Evaluated across Diverse Skin Tones

Read the publication↗

InpatientPressure injuryDomain shift

Bedside visual biomarkers

Can bedside photographs support more consistent wound staging?

Heatmaps and bar charts comparing public, real-world, and combined pressure injury staging models across datasets and stages — **Figure 3**Selected panels show cross-dataset domain shift and the performance of real-world and combined training strategies. Figure reproduced from the preprint.

Question

Can deep learning stage pressure injuries reliably on real-world inpatient photographs and perform consistently across estimated skin-tone groups?

Approach

Public-dataset labels were audited before models were trained and evaluated using 33,342 wound photographs from 3,178 inpatients with note-derived and physician-adjudicated labels.

Finding

A public-trained model fell from 88–89% macro-F1 in-domain to 47.8% on bedside photographs; the combined real-world four-class model reached 76.7% macro-F1.

Why it matters

Clinical visual AI must be developed on representative bedside data, not just clean public benchmarks.

04Sensors · 2020

Fayzan F. Chaudhry et al.

Sleep in the Natural Environment: A Pilot Study

Read the publication↗

At homeWearablesProspective study

Natural-environment measurement

Do consumer sensors agree about how a person slept?

Seven-day natural-environment sleep study design and device-specific sleep staging timelines — **Figure 1**Prospective seven-day study design and one participant’s device-specific sleep staging across four sensing systems. Figure reproduced from the publication.

Question

Can commercially available sensors measure sleep meaningfully at home—and do their outputs agree with one another or established cognitive and psychological measures?

Approach

Twenty-one participants prospectively used Fitbit Surge, Withings Aura, Hexoskin, and Oura Ring in their natural environments. Device outputs were compared with self-report, PSQI, and cognitive testing.

Finding

Agreement between device-derived sleep metrics was generally low. Selected Oura and Withings measures were associated with PSQI or cognitive performance.

Why it matters

Digital biomarkers are device- and measurement-specific. Validation and harmonization are prerequisites for meaningful longitudinal use.

05Bioengineering · 2025

Huili Zheng et al.

Integration of Artificial Intelligence and Wearable Devices in Pediatric Clinical Care: A Review

Read the publication↗

PediatricsWearable AIEvidence synthesis

Field evidence

What will it take for pediatric wearables to become clinical tools?

Illustration of pediatric wearable device form factors including bands, adhesive patches, textiles, and an exoskeleton — **Figure 2**Four wearable-device form factors represented in pediatric clinical research. Figure reproduced from the publication.

Question

Where have wearable devices and AI already been tested in pediatric care, and what prevents wider clinical translation?

Approach

A structured review synthesized 36 pediatric clinical studies published from 2014 through 2025 across hospital, outpatient, rehabilitation, and home contexts.

Finding

Wearables span wrist devices, adhesive biosensors, textiles, and special-purpose systems, but evidence remains dominated by small, single-center studies focused on feasibility and signal validity.

Why it matters

Prospective multicenter studies, pediatric-specific design, robust signal quality, workflow integration, and meaningful clinical outcomes are needed before wearable data can support dependable care.

Audiovisual AI & Digital Phenotyping

Collect richer signals prospectively. Build more faithful digital biomarkers of human state.