This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

Scientific Review - Many Garmin Watches Extremely Inaccurate

I had watched this video before buying my Epix, but it sure took the shine off the purchase. I can only hope that these issues have been or will be addressed. I wanted to share this because I think we deserve better for the prices we pay.

https://youtu.be/YJRUW0azIqk

  • Bias - yes he does - watch his reviews - he never justifies his conclusions for any of his data- all he will say is excellent, good, bad; so you get watches with very similar results in one area, but one is excellent and another that is similar is bad - as he never justifies his stance I can only conclude its bias as the data often doesn't suggest a specific decision.

    Within means yes - but then don't tell everyone how scientific it all is when it isn't (false advertising) and if using a base standard device like EEG device and it isn't particularly accurate to start off with then you need to justify it. There are lots of people out there that due to his him calling himself quantified scientist, mentioning his his, explaining some of the basic flaws of his test methods, and then using a device like EEG and telling everyone its the only way to baseline think that his reviews are accurate - when in reality they are seriously flawed and currently other than comparing a watch with a person in a sleep study there is no accurate way to test it. I may have the stats wrong, but if you test with a baseline device that is 75% accurate and your device is only 75% accurate doesn't it mean that your device is less than 60% accurate? Based on his own correlation tables that is very bad - yet he is telling everyoen that its doing an excellent job....

    I never said he shouldn't compare. I think what he needs to do is improve his testing by a) much larger data pool (yes does mean being able to test fewer devices) - or at provide initial thoughts based on small pool and then provide final review say 60 days later with 60 days of data. b) qualify true estimate of each stage by taking into account the limitations of the current EEG devices (but then does sort of make his testing pointless beyond a general indication). c) Actually justify his conculsions i.e. this is best device for sleep tracking as deep sleep is x percent compared to others as an example

    personally I think he should move away from sleep stages and focus on the reactiveness of sleep tracking ie.. how it reacts to lifestyle changes, injury and illness; as far more useful than sleep stages. I understand injury/illness not really feasible, but lifestyle is very easy as do x days baseline, then throw in x days with y alcohol, then x days with 2y alcohol - and easy enough to find some other items that impact. 

  • I think you need to look up the word bias. "prejudice in favor of or against one thing, person, or group compared with another, usually in a way considered to be unfair."

    That he doesn't explain precisely how he used his data to arrive at "excellent", "good", etc, doesn't mean biased, it means he failed to explain properly how his data arrived at those labels. 

    If he liked Apple better, and always gave Apple a better score, that would be biased in favor of Apple. If you notice on his HR accuracy charts, some Apple Watches are more accurate than others, same goes for Garmin. If he had a bias in favor of Apple, all of the Apple devices would be on the top.

    You keep slinging words like "bias" around when you have no idea what the word means and you stir up confusion and nonsense ideas in people's heads.

    If you think I'm "biased" against Garmin, you are wrong. I've weighed the benefits versus the drawbacks of both Apple and Garmin and landed on Garmin, the Epix, for now. I happen to have a *** load of money to constantly buy and try things for myself. I've tried every Apple Watch including the Ultra, the Fitbits, etc. Apple has the accuracy, but Garmin interprets the data better. I prefer Garmin's data analysis. However, Garmin could do better with accuracy. It's bemusing why anyone wouldn't want better accuracy. I LIKE when someone shines a light on the flaws of a device, even when the device might be the one that I own and am fond of. It means there's an opportunity for improvement. 

    Comparing the devices to the most accurate device he can get his hands on doesn't mean unscientific. He's still using proper scientific methods to do the comparisons, even if the device is not the same as going in for a sleep study. BTW, the Dreem IS accurate based on studies: https://academic.oup.com/sleep/article/43/11/zsaa097/5841249

    Because he doesn't collect the data you want doesn't invalidate what he says about Garmin not accurately identifying the sleep stages. Garmin needs to improve. 

    It would absolutely be great if he expanded the things he looked at in these devices as well as stayed on top of firmware updates, etc. He could also study more people than just himself which would be a HUGE improvement. Even he recognizes that this is a huge weakness. I agree with you there.

  • Hi guys. I run speed wingfoiling.

    And we sport of global competing through asynchrinus GPS races.

    I bought myself an epix gen 2 sapph 47.

    I eas surprisef tonfind the Watch "banned' from official use in competition on One of the major British Speed GPS sites...

    Then i experiencef the frustration of the publically exposed incoherent data of my Watch.

    Basically, all other foilers have more consistente data...meaning at least One hit every second.

    Mine scores as it's best 50% of 1 point x sec, and then anything else .

    50% of the points are anything berween 1 point every 2,3,4,5,6,10 secs.

    Of course batteri saver functionneere disactivated.

    Watch wssvexposed ti the constellation (not covered) and open line of light.

    Using all constellation + multiband.

    Should i try GPS only, nullifing the reason of the buy (i could have gone with the cheapest  Watch in catalog...

  • Mine scores as it's best 50% of 1 point x sec, and then anything else .

    50% of the points are anything berween 1 point every 2,3,4,5,6,10 secs.

    Sounds like you need to change your Data Recording Rate to Every Second - here's how:

    MENU > System > Data Recording > Frequency > Every Second

    HTH