Garmin Fenix 6 heart rate measurement accuracy tested in new study

The Garmin Fenix 6 smartwatch has been included in a recent study to determine the relative accuracy of heart rate measurements across various devices. Also included in the experiment were the Polar H10 chest strap and a new military-grade sensor called OMNI. The research flagged problems with data collected by the Garmin wearable.

https://www.notebookcheck.net/Garmin-Fenix-6-heart-rate-measurement-accuracy-tested-in-new-study.775907.0.html

Top Replies

All Replies

  • Strange they decided to test the Fenix 6, which was released 4 years ago (2019) and not the fenix 7 Pro, which was released in May of this year and has the Garmin ELEVATE V5 optical HR sensor.

  • The search may have started with the Fenix 6 and in the meantime the 7 appeared.

  • Maybe, but the publication date was 5 days ago and the Fenix 7 Pro had already been released 6 months earlier. Seems they should at least put a disclaimer in there somewhere. But that wouldn't increase the clickbait ads being presented.Rolling eyes 

  • When you start a study, you collect a lot of data and you cannot afford to change your reference and lose all the data and have to start again.
    They used a Fenix 6 as a tool to connect the other modules to be tested and took the opportunity to test the one in the watch at the same time.
    I don't know the basic reasons for doing the research, but if it's for development then they may have also made a lot of change during the study. And between the date of the start of the study, data collection, analysis and diffusion(publication), then this explains the reason for the delay.
    I don't see any caveats to point out. This only involves the Fenix 6 and not all Garmins or versions of the Fenix 7.
    I have just received the study. I will read it first to try to understand the research question.

  • Strange they decided to test the Fenix 6, which was released 4 years ago (2019) and not the fenix 7 Pro, which was released in May of this year

    Publishing research findings in journals can take a long while. It is not beyond the realms of possibly that the data gathering for the research started 3 or 4 years ago. Then took a year to write up the results and even longer to get published. Publication date does not mean the research occurred last week, or even last year.

    I've always said research is the science of the bleeding obvious as shown by the conclusions drawn from this study:

    Conclusions Chest-based ECG devices are preferred to wrist-based PPG devices due to superior HR accuracy over a range of exercise intensities, with the OMNI device demonstrating equal, if not superior, performance to other commercial ECG monitors. Additionally, wrist-based PPG devices are significantly affected by exercise intensity as they underestimate HR at low intensities and overestimate HR at high intensities.

    My highlighting. 

    While the watch tested might have been an F6 I'd be surprised if other watches with WHR from around the same time produced any different outcomes. The newer watchhes, particularly the new Gen 5 sensors show much better results overall but as some users will attest, still do not produce acceptable results for everyone.

    Nothing's really changed. If you want consistency and accuracy during active e activities use a strap.

  • The link that is indicated in the article for the search is not the correct one.
    But here is the summary of the research:

    ''Accuracy of heart rate measured by military-grade wearable ECG monitor compared with reference and commercial monitors

    Abstract

    Introduction Physiological monitoring of soldiers can indicate combat readiness and performance. Despite demonstrated use of wearable devices for HR monitoring, commercial options lack desired military features. A newly developed OMNI monitor includes desired features such as long-range secure data transmission. This study investigated the accuracy of the OMNI to measure HR via accuracy of R-R interval duration relative to research-grade ECG and commercial products.

    Methods 54 healthy individuals (male/female=37/17, age=22.2±3.6 years, height=173.0±9.1 cm, weight=70.1±11.2 kg) completed a submaximal exercise test while wearing a reference ECG (Biopac) and a randomly assigned chest-based monitor (OMNI, Polar H10, Equivital EQ-02, Zephyr Bioharness 3). All participants also wore two wrist-based photoplethysmography (PPG) devices, Garmin fēnix 6 and Empatica E4. Bland-Altman analyses of agreement, concordance correlation coefficient (CCC) and root-mean-squared error (RMSE) were used to determine accuracy of the OMNI and commercial devices relative to Biopac. Additionally, a linear mixed-effects model evaluated the effects of device and exercise intensity on agreement.

    Results Chest-based devices showed superior agreement with Biopac for measuring R-R interval compared with wrist-based ones in terms of mean bias, CCC and RMSE, with OMNI demonstrating the best scores on all metrics. Linear mixed-effects model showed no significant main or interaction effects for the chest-based devices. However, significant effects were found for Garmin and Empatica devices (p<0.001) as well as the interaction effects between both Garmin and Empatica and exercise intensity (p<0.001).

    Conclusions Chest-based ECG devices are preferred to wrist-based PPG devices due to superior HR accuracy over a range of exercise intensities, with the OMNI device demonstrating equal, if not superior, performance to other commercial ECG monitors. Additionally, wrist-based PPG devices are significantly affected by exercise intensity as they underestimate HR at low intensities and overestimate HR at high intensities.''

  • In fact, the research was submitted to BMJ on July 26, 2023 and first published on November 24, 2023.

  • See here also the analysis:

    ''Resting heart rate is reported differently by most wearables, some thoughts below.

    Oura: your lowest heart rate is called resting heart rate.

    Garmin: your lowest 30-minute heart rate in the last 24 hours is called resting heart rate.

    Whoop: your night average, weighted more strongly during deep sleep, is called resting heart rate.

    Oura also reports the average of the night, and that is what is highly recommended to use as resting heart rate if you prefer not to measure in the morning for some reason.

    Using the lowest heart rate is artifact-prone (any min or max could be very far from the truth in case of inaccuracies in the measurement, which are frequent for optical sensors) and also compares data collected at different times every day, hence I think it was a poor choice.

    Whoop's approach is in fact extremely similar to Oura's night average, the data gets a bit more noisy with their weighting (because of course, deep sleep cannot be detected accurately), especially for HRV, but for resting heart rate, it is basically equivalent to using just the average of the night.

    Garmin's approach is quite problematic as sometimes those 30 minutes are even during the day or at dramatically different times, making the data of no use. Using 30 minutes is better than using a single minimum, but the issue here is that 24 hours of data are used.

    A final note on Fitbit, which is probably the worst of all, using the average heart rate of "resting" periods, defined as non-movement periods, even during the day. Seems like the engineers who came up with that algorithm many years ago forgot to talk to any physiologists. This data reflects your behavior (e.g. you are more active, your resting heart rate is higher) more than your physiological response (which needs to be measured at rest, i.e. far from stressors).

    A morning measurement allows you to capture data as far as possible from the previous day's stressors, as the morning comes after the night.

    This is what I would call your resting heart rate: it reflects your response to stressors, and your "recovery" (for lack of a better word), as opposed to just your behavior (which is what happens when you measure too close to stressors).

    The value itself will depend also on the protocol (e.g. if you are lying down or sitting, it will be different), but as you know already, what matters are relative changes over time, not absolute values.

    This being said, a night average is my second-best option.''

    https://medium.com/@altini_marco/wearables-for-heart-rate-variability-hrv-measurement-analysis-of-data-quality-and-issues-with-a50ae8127a8b

  • the research was submitted to BMJ on July 26, 2023 and first published on November 24, 2023.

    The paper was accepted for publication on July 26 2023. Before that date there would have been a peer review process resulting in the need for corrections and amendments to be made that likely would have taken several months, even up to a year. Very, very few papers are accepted for publication on first presentation. And then there's preparing the paper for submission, and collating and analysing the data, and collecting the data, and getting participants, and writing the study process, and finding somebody to undertake the research. 

    Quite often, the journal of publication would have not been the first choice or even the second. I had one paper of my PhD returned from two journals before finding a third. The whole process from first submission to final acceptance took nearly two years. And that did not take into account the data collection, analysis, and writing up beforehand.

    As already noted, publication date has little to no bearing on the actual date the study was undertaken.