PennAITech Awardees

Awardee Organization(s): Pennsylvania State University
Principal Investigator(s): Rui Zhang, PhD | Marie Boltz, PhD, GNP-BC
Official Project Title: Task-Oriented Multimodal Conversational AI for Assisting Seniors with Daily Tasks
AITC Partner: PennAITech
Website(s):
https://www.eecs.psu.edu/
https://ryanzhumich.github.io/

With a global population of over 1 billion people aged 60 and above, there is a rapidly increasing need for innovative age tech solutions to improve the quality of life of older adults. Conversational assistants, powered by cutting-edge technologies in Artificial Intelligence (AI), Natural Language Processing (NLP), and Large Language Models (LLM), are permeating into home care, assisted living, and nursing facilities for smart elderly care. One type of conversational assistant is task- oriented, which can significantly enhance the life experience for senior people by helping them with real-world complex daily tasks. A task-oriented virtual assistant facilitates daily tasks spanning diverse scenarios such as calling for help in response to emergencies, helping with online grocery shopping, recommending cooking recipes, managing smart home devices, and providing financial education and decision-making. It greatly promotes the life quality of older adults by improving their well-being, efficiency, safety, and independence. In this proposal, we design, develop, and deploy a task-oriented multimodal conversational assistant to help older adults with daily tasks. The innovation of this proposal lies in the fact that we will employ a human-centered participatory approach by emphasizing collaboration between designers and end-users through interviewing, prototyping, and testing to address their unique needs and preferences to improve their daily lives.

Awardee Organization(s): University of California San Diego
Principal Investigator(s): Xinyu Zhang, PhD | Alison Moore, MD, MPH
Official Project Title: Non-Intrusive, Fine-Grained In-Home Daily Activity Transcription for Alzheimer’s Monitoring
AITC Partner: PennAITech
Website(s): http://xyzhang.ucsd.edu

Recent research identified a strong correlation between onset of Alzheimer’s disease (AD) and changes in fine physical activities, e.g., movement and dwelling time across locations, daily routines like medicine/water intake. Early detection of such indicators is crucial in compiling better treatment and slowing the progression. Conventional methods for monitoring the activities of daily living (ADL) rely on observation or self-report, which are time consuming, error-prone, and require strict patient compliance. This project aims to transcend such limitations and bridge the key technology gaps in bringing ADL sensing close to clinical practice. The project focuses on the development of EgoADL, a system that uses non-intrusive smartphone/smartwatch sensors to sense ADL. EgoADL builds on a novel self- supervised sensor fusion model that trains itself without user intervention. Instead of classifying among a small known set of ADLs, it directly transcribes raw multi-modal sensor signals into text logs of ADLs which can be interpreted by clinical practitioners or AI models. EgoADL will be the first to use non-visual sensors to transcribe fine ADLs (e.g., human-object interaction) with near-vision precision, in real-time and in a privacy-aware manner. The sensing data can facilitate follow-on clinical and AI analytics, potentially enabling early detection of chronic diseases and safe aging in place. Ubiquitous health monitoring is particularly important for rural and underserved communities, who either do not have access to or cannot afford prolonged hospitalization. EgoADL will be verified through a pilot study in UCSD’s and Upenn’s healthy aging facilities.

Awardee Organization(s): Kennesaw State University
Principal Investigator(s): Maria Valero, PhD, MsC | Katherine Ingram, PhD
Official Project Title: GlucoCheck: A Non-Invasive & AI-Assisted Blood Glucose Monitoring Device for Older Adults
AITC Partner: PennAITech
Website(s):
https://ccse.kennesaw.edu
Iotas.kennesaw.edu

The GlucoCheck project presents a significant advancement in the realm of diabetes management, with a specific focus on enhancing the quality of life for older adults. In light of the escalating global prevalence of diabetes and its associated complications, the imperative for non-invasive, accurate, and user-friendly blood glucose monitoring has never been more pronounced. Diabetes poses severe health risks, particularly for older adults, making effective and comfortable glucose monitoring paramount. Existing methods, characterized by frequent finger-pricking and subcutaneous needle implants, entail discomfort, infection risks, and potential tissue damage, particularly in individuals with diminished skin elasticity and compromised immune responses. GlucoCheck emerges as a pioneering solution, harnessing near-infrared spectroscopy (NIR) technology augmented by AI. This device offers a non-invasive means of consistently monitoring blood glucose levels by simply wearing it on one’s finger. Importantly, GlucoCheck integrates AI algorithms that adapt to individual skin attributes, including color and texture, enhancing precision across diverse demographics. This project’s core objectives encompass rigorous validation of GlucoCheck’s efficacy, with a primary emphasis on older adult demographic. Comparative analyses will be conducted, aligning GlucoCheck’s measurements with conventional blood glucose monitors to ascertain accuracy and reliability. Our mission is underscored by the desire to deliver an efficacious, user-centric device tailored to the unique requirements of older adults. The project’s outcomes will furnish valuable insights that drive refinements in GlucoCheck, propelling us closer to positively impacting the livesof millions of individuals grappling with diabetes.

Awardee Organization(s): etectRx
Principal Investigator(s): Tony C. Carnes, PhD
Official Project Title: Real-Time Remote Monitoring of Confirmed Medication Adherence
AITC Partner: PennAITech
Website(s): https://etectrx.com

Medication non-adherence is responsible for up to $300 billion of avoidable healthcare costs in the United States with patients over 60 years of age consuming 50% of dispensed prescription drugs. This project enables real-time, remote monitoring of medication ingestions and enhances patient and caregiver feedback to help patients stay adherent and thus extend the time they are able to age grace fully at home.
The existing FDA-cleared IDCap system detects ingested medication signals using a watch or lanyard-style reader worn by the patient and forwards information to a server through an app on a patient’s smart phone. In this proposal we are looking to remove the individuals’ worn reader from the system and replace it with a series of readers placed in multiple locations in a person’s home to ensure ingestion detection without the user changing their usual behavior. Additionally, the new readers will interface with Alexa to facilitate audible and visual reminders and confirmations of medication ingestion. If the end user allows, remote care providers will be able to participate in the adherence journey and intervene when needed.

Awardee Organization(s): New York University Rory Meyers College of Nursing
Principal Investigator(s): Ab Brody, PhD, RN, FAAN
Official Project Title: Aliviado Dementia Care Machine Learning Algorithm Development for Caregiving
AITC Partner: PennAITech
Website(s):
https://www.aliviado.org
http://nursing.nyu.edu
https://www.hign.org

Care partners (CP) of persons living with dementia (PLWD) provide crucial support and find significant meaning in the care they provide. They show compassion to those they are caring for, and resilience in the face of adversity. Yet, many CP lack high-quality, evidence-based guidance for addressing care needs of PLWD. One key area that is often challenging to CP, yet where they have little support, is in addressing neuropsychiatric symptoms (NPS) such as agitation or wandering. Most PLWD experience more than one NPS at a time and thus not only do CP lack support in managing these symptoms, they don’t know which symptom to focus on first to reduce their burden and improve the quality of life of the PLWD. This is particularly true in underserved and marginalized communities who are less likely to have access to comprehensive dementia care or supportive services. Higher NPS, particularly in marginalized CP, greatly increases the risk of CP burden, physical and mental health challenges. To help CP make decisions about what NPS to prioritize, we will use artificial intelligence/machine learning (AI/ML) to develop a precision clinical decision support algorithm to assist CP in prioritizing which NPS to treat. The algorithm will be inserted into a user-friendly smartphone application which CP can download through the iOS or Android app store. The app will increase access to high-quality dementia support, empower CP to better manage NPS and improve the quality of life for both themselves and the PLWD.

Awardee Organization(s): VNS Health | Columbia University Irving Medical Center
Principal Investigator(s): Maryam Zolnoori, PhD
Official Project Title: A Speech-Processing Algorithm for Automatic Screening of African American Patients with Mild Cognitive Impairment and Early Dementia in Home Health Settings
AITC Partner: PennAITech
Website(s):
https://www.cuimc.columbia.edu
http://vnshealth.org

Mild cognitive impairment (MCI) and early-stage dementia (ED) are prevalent concerns, impacting one-in-five adults over age 60. Alarmingly, a significant percentage of these cases remain undiagnosed, leading to missed timely interventions. Our data emphasizes that African American seniors are particularly vulnerable, with existing disparities in healthcare access, biases, and varying health literacy levels exacerbating the situation. A novel observation we intend to leverage is the correlation between linguistic shifts and the onset of cognitive issues. Language, a foundational element of cognition, exhibits early perturbations during cognitive decline. The nuances of these changes can vary across racial boundaries, influenced by dialectic variations such as African American Vernacular English. In this pivotal study, our objective is to architect a diagnostic tool to detect nascent signs of MCI-ED by analyzing African American patients’ verbal communications during regular health consultations. By meticulously recording, processing, and extracting linguistic and phonetic features from these conversations, complemented by additional clinical data, we aim to devise a potent screening algorithm. This initiative aligns seamlessly with the National Institute on Aging’s focus on early identification of cognitive impairment in the elderly. The prospective outcome, an innovative algorithm, holds promise to enhance timely MCI-ED diagnosis efficacy, especially among African American individuals, thereby optimizing care quality and addressing longstanding disparities.