Related
This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

Fitness comparison with population probably suffers from sampling bias?!

9314056
over 4 years ago

Hi, I'm honestly not sure where to ask this, this forum does not seem to have space for general questions like this. 

With that said, I'm currently using my forerunner to track my progress as many of you do and I noted that my fitness level is constantly below average (45% percentile). I understand I'm not a particularly sports focused person, but I try to take care of myself. I feel it can't be possible that a 30 year old that can run a 10k is below average fitness. I understand my times are not particularly impressive (about 1 hour 5 min for a 10k), but I know only a handful of people in my circle that can even run a 10k to begin with.

As a scientist and data analyst, I think these results presented to the end user must be heavily biased towards a population of people with an already above-average level of fitness, since those are more likely to spend their money in devices like this to be part of the database. I mean, in the U.S. the obesity rate is 40%, how can it be possible that people just in the threshold of obesity are capable of running 10k's?! (for context, my BMI is 22). I would really like to see a more detailed discussion on what Firstbeat is using as a benchmark for the population.

There's no discussion in Firstbeat's papers of this caveat in the data to my knowledge, which means they're either unaware of it or I'm completely unable to evaluate the population based on my personal experience, which is also possible. I feel this lack of transparency is quite concerning and that something should be done about this by Garmin.

I'm sorry if this is as ramble but I didn't see this being asked and I think a customer deserves either better data or better information on its caveats. Otherwise the users are more likely to simply disregard it, which is seen over and over in discussions in other platforms.

Thanks.

  • 0 over 4 years ago
    9314056 said:
    these results presented to the end user must be heavily biased towards a population of people with an already above-average level of fitness, since those are more likely to spend their money in devices like this to be part of the database

    www.firstbeat.com/.../

    I can't find anything in writing, but I believe your fitness level is compared against other 30 yr old runners and not random 30 yr olds.  This doesn't bother me as I know these type of watches are targeted towards athletes and I prefer to be compared against them instead of the general population.  Especially against the population here in the US where the general population has a fitness level that's so far behind the rest of the world that it's an embarrassment.  Anyway, FirstBeat got their start 20 years ago studying athletes in Finland. 

  • 0 over 4 years ago in reply to tess

    Thanks Tess, I think the issue I have with the percentiles is that the Garmin Connect software calculates your "fitness age" (I forgot to mention that) by comparing your estimated VO2max with the "population", which I also agree must be biased towards runners or at least people capable of taking VO2max tests.

    This skews the information and makes you believe that you're unfit even when you work hard. In all honesty, it has been a little demotivational to me, since I work my ass out and still see something like this "fitness age" being thrown at me with 7 years above my actual age. I think this metric should be re-evaluated to at least be more representative of the population, if you're going to present it as a metric of the body aging.

    I understand perhaps comparing with the US is not a great idea, given the US is known to be one of the most obese countries in the world. I wish I was doing research on this field and was able to contribute to this, because the bias is too obvious not to notice. It is amazing they dare to even build a product based on such skewed datasets.

  • 0 over 4 years ago

    If you mean with Fitness Level your VO2Max, the biggest caveat is your maximum heartrate. If that value is underestimated, your VO2max will be not correct. A bad running technique will additionally have a negative impact on the VO2max value. Furthermore I have experienced that VO2Max calculated with heartrate reserve method works better for me than with maximum heartrate method. Maybe you give that a try.

  • 0 over 4 years ago in reply to 9314056
    9314056 said:
    It is amazing they dare to even build a product based on such skewed datasets

    Well, this is just speculation at this point. I thought the metrics were coming from FirstBeat.  This might not be the case.  If I look at my Vo2Max tab in GCM and tap "help" it says fitness age is based on your VO2Max and goes into a little more detail of how it's calculated.  What I found interesting (and I don't have time right now to further investigate) is there's a statement saying VO2Max data is used with permission from The Cooper Institute out of Dallas, TX. 

  • 0 over 4 years ago

    Hello! 

    As a almost 40 year old man who have spent his life sitting on a couch watching tv and eating crap food... The only kind of excersise I have practiced daily is Yin yoga and meditation... I never exhaust my muscles, just simply allow the body to relax.

    Taking care of your body and feeding it nutrious fresh produce and drinking fresh water will help your heomoglobin carry oxygen more effeciently through your body... 

    Edit; My VO2Max is 51 (after a brisk walk)

  • 0 over 4 years ago

    My fitness age is 20 and I am 55 years old so I cannot believe that this is comparing me with 20 year old athletes, therefore I take it with the same pinch of salt as with my Tanita body composition analyser that gives me a metabolic age of 12. I think that once you are far from the mean, the assumptions that the software works on become invalid.

  • 0 over 4 years ago

    My fitness age is 20 and I am 55 years old so I cannot believe that this is comparing me with 20 year old athletes, therefore I take it with the same pinch of salt as with my Tanita body composition analyser that gives me a metabolic age of 12. I think that once you are far from the mean, the assumptions that the software works on become invalid.

  • 0 over 4 years ago in reply to Garmin-Brooks

    Thanks, Brooks, for some starting point. I think reference [33] is what I'm looking for, I've already read Firstbeat's publications. I don't think that reference exists, though. Searching the Cooper Institute's website doesn't give any trace of its existence, which is surprising given its supposed to be recent and one of their major publications. My university library also does not have it in its catalog (it's a major US university). In order to arrive at my own conclusions about the table presented, I need to see how the data was acquired and what are the potential biases in the sample. That's impossible if the reference doesn't exist or is inaccessible to academics like me, right? 

    Given the title of the publication, it looks like it is supposed to be setting standards for law enforcement, which implies some degree of bias towards more fit individuals. But without the publication in hands, it is hard to say anything. Looks like research transparency is not the strength of Firstbeat, right? Or perhaps you can prove me wrong and link me to where I can buy this book referenced in [33]??