Intelligent Supportive Environments for Older Adults (COACH project)
Keywords: Cognitive device, cognitive orthosis, smart homes, assisted cognition, context-aware design, ADL prompting, ADL guidance.
Overview of Research
Dementia reduces a person's ability to perform activities of daily living (ADL) because of related difficulties in remembering the proper sequence of events that must occur and how to use the required tools. The current solution is to have a caregiver supervise activities and continually provide prompts as to the next step in the activity. Family caregivers can find assisting their loved ones to be particularly upsetting and embarrassing as it necessitates invasion of privacy and often role reversal. Research suggests that dependence on a caregiver might be improved using a supportive environment, or cognitive orthosis, that provides needed reminders and monitors progress.
The COACH (Cognitive Orthosis for Assisting with aCtivites in the Home) is a prototype of an intelligent supportive environment being developed to assist people with dementia complete ADLs with less dependence on a caregiver, representing one of the first clinically tested supportive devices to use artificial intelligence techniques. The device operates with a personal (desktop) computer and a single video camera to unobtrusively track a user during an ADL and provide pre-recorded (visual or video) prompts when necessary. To date, two successive prototypes of the COACH system have been through clinical trials based around the ADL of handwashing with subjects who had moderate-to-severe dementia. In both cases, these trials showed that the number of handwashing steps subjects were able to complete without assistance from the caregiver increased noticeably when the device was present. Statistically significant individual changes ranged from approximately 10 to 45 percent.
System Overview
COACH uses artificial intelligence (AI) algorithms and a single video camera to monitor progress, determine context, and provide pre-recorded prompts when necessary. For example, if the person forgets to turn off the water, the device prompts him/her to do so. The device has the capability of adjusting its parameters and cueing strategies to meet the changing needs and preferences of each user.
The only hardware visible to the user is the single video camera mounted over the sink (Figure 1) . The COACH system can be thought of as three systems working together: 1) the tracking system, 2) the planning system, and 3) the prompting system, as depicted in Figure 2. The tracking system uses a video camera to identify the position of the user's hands, as well as interactions with task objects (i.e. the soap, towel, tap, water, and sink). The latest version of COACH uses edge skin colour segmentation and flocking routines to track hand and object positions. There are several acceptable pathways to complete the handwashing task, as represented in Figure 3. The planning system is responsible for determining which step in the handwashing the user is completing and what the appropriate response (i.e. action) should be. The planning system is currently based on a partially observable Markov decision process (POMDP) model and is able to estimate unobservable states, such as the user's responsiveness and level of dementia. The planning system calculates the most probable state of the world, which is used with a pre-computed policy to determine what action COACH should take. Actions available to COACH are to do nothing (i.e. continue observing the user), select a prompt, or call the caregiver to intervene. The prompting system initiates the selected action.
If the user is prompted, the amount of detail that is provided in the prompt varies in specificity to match the user's abilities. For example, if the user is being asked to turn on the tap, a low specificity prompt would be audio only (e.g. "[Subject name], you are washing your hands. Turn on the tap.") while a high specificity would have a video demonstrating the step plus more detailed audio (e.g. "[Subject name], use the silver lever in front of you to turn the tap on"). If necessary, the device will repeat a cue or increase the level of specificity of cuing to promote the correct response from the user. If the user does not respond to any of the cues, the device calls for a caregiver to provide assistance.
Figure 1: Photo of the washroom setup
Figure 2: Outline of the systems of the COACH system
Figure 3: Flow diagram of steps involved in handwashing.
Future Work
Past clinical trials have guided our development of the prototype for the next set of clinical trials, which are to be community-based (i.e. within-home trials) set to start in late 2008:
- Tracking: The current colour segmentation/ flocking method will be combined with a steroevision camera to track the position of both hands, the position of task objects (e.g. soap and towel), and the interaction between the them in 3D.
- Planning: The POMDP has been refined using data from the previous set of clinical trials. This will allow the new system to more accurately determine the steps the user is completing, whether that sequence is acceptable, and whether a prompt should be issued.
- Prompting: The system will continue to play verbal or video (i.e. verbal and visual prompt over a LCD screen) prompts to the user when an error is made.
Funding Sources
Alzheimer Society of Canada (Operating Grant and Personnel Support Award)
Research Team
Alex Mihailidis, University of Toronto
Jesse Hoey, University of Dundee
Babak Taati, University of Toronto
Jennifer Boger, University of Toronto
Tammy Craig, University of Toronto
Craig Boutilier, University of Toronto
Geoff Fernie, Toronto Rehabilitation Institute
Pascal Poupart, University of Waterloo
