Awardee Organization(s): University of California San Diego
Principal Investigator(s): Xinyu Zhang, PhD | Alison Moore, MD, MPH
Official Project Title: Non-Intrusive, Fine-Grained In-Home Daily Activity Transcription for Alzheimer’s Monitoring
AITC Partner: PennAITech
Website(s): http://xyzhang.ucsd.edu
Recent research identified a strong correlation between onset of Alzheimer’s disease (AD) and changes in fine physical activities, e.g., movement and dwelling time across locations, daily routines like medicine/water intake. Early detection of such indicators is crucial in compiling better treatment and slowing the progression. Conventional methods for monitoring the activities of daily living (ADL) rely on observation or self-report, which are time consuming, error-prone, and require strict patient compliance. This project aims to transcend such limitations and bridge the key technology gaps in bringing ADL sensing close to clinical practice. The project focuses on the development of EgoADL, a system that uses non-intrusive smartphone/smartwatch sensors to sense ADL. EgoADL builds on a novel self- supervised sensor fusion model that trains itself without user intervention. Instead of classifying among a small known set of ADLs, it directly transcribes raw multi-modal sensor signals into text logs of ADLs which can be interpreted by clinical practitioners or AI models. EgoADL will be the first to use non-visual sensors to transcribe fine ADLs (e.g., human-object interaction) with near-vision precision, in real-time and in a privacy-aware manner. The sensing data can facilitate follow-on clinical and AI analytics, potentially enabling early detection of chronic diseases and safe aging in place. Ubiquitous health monitoring is particularly important for rural and underserved communities, who either do not have access to or cannot afford prolonged hospitalization. EgoADL will be verified through a pilot study in UCSD’s and Upenn’s healthy aging facilities.
View Resource