Fitness Trackers Are Only 67% Accurate, New Research Finds

>I don't buy it, no brand can be accurate on energy expenditure because it depends on a lot more things than what the watch can monitor

To be fair, the comment you excerpted noted that the best performer was still just 71% accurate. So they’re saying they’re far from perfect.

>On watch, calorie counter is a function of time and heart rate.

Well, we just don’t know. But let’s take Apple as an example. Apple knows your workout type. And they know your weight if you’ve entered it. And lots of other sensors as well (O2, if possible, temp, accelerometers, etc).

They’ve put a crazy amount of money into their testing lab where they do have extensive metabolic rate testers, VO2 Max, etc.

Will they get as good as that equipment for a dinky watch? No. But I also am glad I don’t have to strap a face mask on during my daily bike rides.

Can they, with insane volumes of data and hard research, make pretty good directional inferences using models that match their watch sensors to gold standard tests? It seems the answer to this (both logically and in the research data) is yes.

This video is a decade old so they’ve only become more advanced and have more data since then, but there are lots of models you can build to power these estimates: https://youtu.be/BceaTNT14Ao

I’m glad you’re happy with your Casio. Keep rocking what works for you. But I’m not sure what’s so hard to believe here when other researchers have actually produced the data on their accuracy.

> I don't buy it, no brand can be accurate on energy expenditure because it depends on a lot more things than what the watch can monitor.

You could say the same about virtually every modeling problem, yet the world runs on models that work adequately in practice. You measure what you can, infer what you can, and fill in the gaps with priors.

At least in the case of Apple I have met a couple people who work on the fitness products and their accuracy. One of the people does real world testing so they are diving one day and coding the next. Great work if you can get it and your hobbies align.

The lab setup you describe is basically what they have to setup to correlate their proxies to energy. I’d expect the variability is going to come as a factor of body efficacy.

I used to be in much better shape (resting heart rate in the 50s) but I’m not much heavier and haven’t lost much of the muscle I gained (70ish percent of my former one rep max achievable). My day to day non-workout energy burn is measured higher now than it was when my heart rate was lower for all activity levels. I don’t think that it’s true I’m suddenly +15-20% energy consumption simply because I put on 10lbs but rather having a 10bpm increase in heart rate is probably what is being measured.

Short of that though it does a pretty good job. If I’m doing things that are strenuous but my arms are stationary it picks up the work. Very old wrist trackers needed to have an arm pumping or otherwise moving to do anything aside from a pre-calculated “you are alive” burn.

Given the level of effort for what we get, I’ll take it as a basic way to answer “did I actually get some exertion from watching my 2 year old today?”

> no brand can be accurate on energy expenditure because it depends on a lot more things than what the watch can monitor.

I was going to quip that I can probably look at a person and state their energy expenditure with 70 % accuracy (whatever that means -- I'm going to assume coefficient of determination.)

Just someone's weight alone ought to have nearly a 70 % r² with energy expenditure. (A quick internet search suggests 71 % though I have not bothered verifying their methodology.[1])

Of course, they could mean 70 % across days for the same person, which is significantly more impressive as it takes out the strongest signals, but it's not at all clear what they actually mean and that's sort of my point.

Edit: I read the article closer now and when they say 70 % they actually mean a Pearson r of 0.7, i.e. a coefficient of determination of 50 %. Reporting r as a percentage seems confused at best, or possibly dishonest.

[1]: https://www.researchgate.net/figure/The-relation-between-bod...

Exactly. It doesn't really matter.

If your tracker shows you walked 12000 steps today as opposed to 8000 yesterday, even if you actually walked 10500 today and 8750 yesterday, that's still progress.

For the heart such errors are problematic

That's not how error percentage works and why do we all just accept that everything has become mediocre?

Yeah. They may not be terribly accurate but if they are precise they can still be useful.

Yeah. I have noticed that the relative changes in numbers seem to be very correct. I can’t validate the absolute numbers, so I don’t really look at those so much. For the Apple Watch, I’ve noticed that the sleep tracker is quite accurate too, in absolute terms. That is, when I remember having long, wild dreams, I have a substantial amount of REM time. When I don’t remember my partner doing whatever at some hour, I was in deep sleep. If I toss and turn a lot, I see a lot of awake segments. So that part seems pretty accurate.

True. I think most people care more about trend lines than anything else. If it isn't accurate, at least let it be consistently inaccurate.