According to the Health Information National Trends survey, nearly one in three Americans wears some kind of wearable device to track their health and fitness. But how accurate are those devices?
The answer can vary quite a bit depending on the activity being tracked, and fitness devices can face particular challenges when assessing activities done indoors, since the devices can’t rely on GPS data.
These challenges make comparing devices and data collected by those devices a problem for researchers, especially when researchers are attempting to study members of an older population, who tend to spend more time inside and whose data may, therefore, not be as accurate.
In addition, assessing fitness trackers is challenging in indoor settings since there is no reliable gold standard data available.
But Sardar Ansari, assistant professor of emergency medicine at the University of Michigan Medical School and director of data science at the Weil Institute for Critical Care Research and Innovation, envisioned a solution.
Partnering with the University of Michigan HomeLab at the Institute for Social Research, Ansari devised a study to accurately assess the tracking capabilities of fitness devices for indoor activities, giving researchers a comprehensive look at what an average day of activity could include and establishing a gold standard baseline for future studies.
The HomeLab is a Core Facility that provides a residential-style simulation space to enhance ecological validity for studies of everyday activities and behaviors. Research cores are shared university facilities which offer specialized research services, equipment or expertise to U-M researchers, often at a recharge rate.
A 3D model of the HomeLab, top-down, used in the Virtual Reality tour.
A 3D model of the HomeLab – the view from the front door.
Ansari and his co-researchers combined data from the American Time Use Survey, which measures the amount of time people spend doing various activities, such as paid work, childcare, volunteering, and socializing, with intense observation at ISR’s HomeLab. People interviewed for the study identified activities that they would perform in the HomeLab environment, which was crucial for establishing the structure of the observation.
“A naive way of doing the study would be to have everyone do the same activities and fit as much as you can into every HomeLab session,” Ansari said. “That could give us a routine that was similar, but does not properly represent the participant’s own daily activities. In addition, cooking and cleaning the kitchen are very complex tasks and there’s a high level of variability between people who perform them. So you want to do more of those instead of more homogenous tasks like watching TV to capture the variability better.”
HomeLab supported the study by providing a high fidelity, true-to-life environment in which research participants moved through simulations of their day-to-day lives. The realism of the HomeLab space is a key part of its appeal — and a crucial part of its development, according to Alicia Carmichael, research process director for HomeLab.
“HomeLab actually started out as a break room,” Carmichael said. “But in a more natural environment, you get more natural behaviors. Now, when people enter the lab, people will say ‘Oh, wait a minute, I need to take my shoes off.’ They feel like they’re in a decidedly different space than the basement of a university building.”
Ansari, in turn, helped upgrade the observational capabilities within HomeLab with funds from a National Institutes of Health award used to complete the study.
“He actually added an ‘indoor GPS’ system to HomeLab to improve the tracking accuracy so that we could get people’s position down to plus or minus two centimeters,” Carmichael said. “That’s the level of accuracy and precision he needed in order to detect these errors that are in the devices that everybody’s using in their homes.”
A 3D model of the HomeLab – the view of the kitchen.
A 3D model of the HomeLab – the view of the bathroom and laundry room.
And there were errors. Ansari and his co-researchers found that previous studies underestimated the errors in fitness device measurements.
“The smartwatches have a substantial amount of errors when it comes to measuring distance indoors throughout the day,” he said. “We’ve also shown that these errors impact both step and distance measurements since step length is much more variable during indoor activities.”
These findings could have implications for researchers trying to extrapolate conclusions based on fitness device measurements, though Ansari was optimistic that they still could be used as a general intervention to encourage people to move more. But just as importantly, the study showed a new way the HomeLab space could be used to support research.
“Sardar’s study made the most use of the HomeLab’s features that I’ve ever seen of any investigator. The problems he outlined couldn’t be researched in someone’s actual home or in a blank white room,” Carmichael said. “This was a really beneficial relationship both ways. I’m hoping that this space can serve a lot of other investigators who are looking for an ecologically valid environment like this.”