Related
Options

Sleep tracking issues and impact on other metrics/features

Chris
9 months ago

While I know that sleep tracking (especially detecting the correct sleep phases) is far from being perfect, I'm getting more and more fed up with how badly it actually works and how it impacts other metrics on the watch, in particular overnight HRV readings, the training readiness and quite possibly also the suggested workouts. 

Three examples:

Today I got a really bad sleep score reading of 51. The watch detected quite accurately my bedtime around 9 pm yesterday. But around 1 am the watch stopped recording any metrics for a while (maybe it applied an update?) and consequentially detected this as my wake-up time as soon as it stopped recording. I actually slept normally until 5 am and woke up very well rested.

A few weeks ago I had quite a bad night, when I was woken up around 2 am and couldn't get back to sleep due to external circumstances. Nevertheless, my watch recorded sleep until my normal wake up time at 5 am, even resulting in a "Good" sleep quality, although I was really tired. It even detected REM phases while I was awake.

I encountered another issue on a recent weekend, when I was watching TV on the couch with my girlfriend starting around 8 pm. I was very relaxed and my body was not particularly stressed from training that day. We watched until midnight and then went to sleep. The next day I was surprised that the watch measured my sleep times starting around 9 pm. I'm surprised how this could be possible, since I was also constantly moving slightly, was on the phone and drinking water. I can just assume that it started to detect sleep because of low stress and my heartrate went quite low (below 50 bpm) at some point for an extended periode of time. The wierdest thing was that it even detected REM and deep sleep phases while I was actually awake. The worst thing again, because it thought I slept for a really long time that night (about 11 hours) it again gave me a poor sleep score. In this case I was lucky though, because after adjusting the bedtime hour to cut away sleep time, the remaining deep and REM phases were sufficient for an excellent sleep score... on bar with how I actually felt that morning.

So, I know that I can adjust the sleep time within the Garmin Connect app, but this doesn't change the sleep score significantly (today only from 51 -> 61) because it automatically categorizes the added sleep time as light sleep. But for a good sleep score you need a sufficient amount of deep and REM sleep.

It also doesn't help with overnight HRV readings, because the added time is not taken into consideration. Since I get usually higher HRV readings in the second half of the night (especially after a workout in the evening) due to getting more rested, my overnight HRV average is cut short if wake-up times are detected too early. That suggests that my body is in a more stressed condition than it actually is. 

If it would only be a metric on its own, I would just shrugg it off. What concerns me the most about these issues is, that they also influence other measurements like the HRV readings (as mentioned above), training readiness and I assume the suggested workouts, too. When I get a bad sleep score like today it increases my recovery time on the watch and consequently my training rediness. Furthermore it seems to impact the suggested workout feature. It got rid of a an anaerobic session it had planned previously in order for a base session. Also, I noticed that it adjusts the Watt values for base sessions based on my recovery status. I have an cycling FTP of about 225 (manually set to my watch). Other platforms like Xert even suggest a higher FTP of 235, Strava suggests on the power curve 226, Intervals.icu gives 223. Based on my training based on FTP, I always feel appropriately challenged, therefore I think that the value set on my watch is currently ok.  When I'm well rested according to my watch, it suggests Base training with 150  Watts which is well at the center of zone 2 at about 55% – 75% FTP. When it thinks I need more recovery time it sets the suggested Watts significantly lower to 120-130 Watts for base(!) training, which is already within or very close to the border of active recovery. This doesn't make any sense... if it thinks that I should only do active recovery, it should suggest active recovery since it is a possible and available workout option. But ignoring this issue, it should not adjust the training so significantly based on bad sleep score readings. In general I'm quite happy how the watch naturally suggests a lot of base training and 1-2 threshold and VO2max sessions per week, quite similar to a polarized training plan that I try to apply.

In my opinion there are very simple fixes to this problem, so that the user can intervene if the detected sleep metrics go wrong.

My suggestions how to improve these problems:

  • Most importantly: Let only sleep metrics impact other functionality of the watch that the user can influence, i.e. in the current state only the length of sleep not the quality.
  • If you want to keep a metric describing the sleep quality, give the user the option to at least overwrite the sleep quality based on your default categories (Poor, Fair, Good, Excellent). Then set the sleep score to an average value in the respective range. Basing the sleep quality on the sleep phase detection which you obviously can't do properly and probably never will, because it is even difficult in a lab, is just ridiculous. Why don't you just ask during the morning report how the user's sleep felt and how well rested they feel? Merge this to the actually sleep length and you will end up with a more reliable sleep score. 
  • You probably already track HRV constantly as a metric to determine stress throughout the day. If someone changes the start/end of their sleep time fill the overnight HRV readings with those values. 
  • If an update is applied during the night (sleep times), just set a flag on the watch so that it doesn't forget after booting up again, that the wearer was at sleep at the time of the update and didn't wake up. 
  • If it was an update tonight that messed up my sleep score, don't apply updates during sleep times automatically and let me manually apply them during the day as I do so often already.

As further information, I wear my watch daily, tightly fit on my left arm, slightly above the wrist and clean the sensors on a daily basis with water especially after training.

I have really big issues with how strongly these sleep readings affect other metrics and features on an already quite expensive watch, rendering many of said metrics/features completely unreliable and useless. Currently it just seems like a feature implemented by people, who don't know what they are doing, overengineering the feature to make it appear well thought through but eventually making the metric significantly worse than it could be, while also negatively impacting other features.

All I can say as a conclusion is that based on these issues that render so many features of the watch useless, I currently can not recommend a Garmin watch to anyone, which is sad, because in general I like the watch if there weren't the sleep scores which affect so many other features negatively.

  • 0 9 months ago

    I'm in the same boat at the moment. Sleep has not been tracked properly for the last 4 weeks (Coinciding with a software update), which means no HRV status, training status etc. I like the suggestion of being able to add in manually, it would be great if you could add in manually and then it could "re-analyze" the manual set time for sleep stages etc, but even if it just allowed the manual duration to flow through to other metrics then that would be an improvement too 

  • 0 9 months ago

    The thing is, it's not Garmin, it's all fitness trackers.

    Watching the Quantified Scientist, he shows that none of them accurately track sleep stages which as you say renders all of the above metrics advisory at best 

    The watch shines, I feel, in display, navigation, VO2 and GPS (for my needs at least). For what it's worth I always wear a strap as I zone train not for recovery advice.

    For training, once you have a solid grasp of the relevant concepts , it is probably best to create your own plan and follow that whilst guaging how you feel from a recovery perspective and adjusting accordingly.

    I really wanted to buy into suggested workouts (with the race calender for an ultra) but as you mention above these are not always accurate if sleep hasn't been adequately tracked (and to be frank sleep stages are only adequately tracked 50% of the time) so what's the point in following them at all.

  • 0 9 months ago in reply to 8924630

    I totally agree with you, but I think you slightly missed my point Slight smile

    I absolutely think that many metrics/features of the watch should only be seen as advisory, but what is the point of having them, if they are based on fundamentally wrong data... causing consequently fundamentally wrong advice? And one major factor, why many of the metrics/features of this watch are not even usable as an advise, is the bad sleep tracking, which you can barely fix  manually and which influences other features. 

    Some metrics are in my opinion very valuable for all athletes, i.e. RHR and HRV are very valuable to monitor how well the body is rested or even detecting sickness early. Monitoring those in combination with an assessment of how my body feels works for me very well to make consistent progress and knowing when to slow down a bit with my training. But again: Those are also connected on the watch to some extent with your sleep and getting the sleep times wrong will consequently give you false readings.

    You have to keep in mind that so many metrics and features of this watch are also used by people with limited knowledge about the science of physiology and exercising, who may think that it is ideal to follow the watch's suggestions. 

    I'm simply advocating for an option to manually intervene and correct the data, if you know that something didn't work right, which is currently not sufficiently possible with regard to the sleep data.

  • 0 9 months ago in reply to Chris

    I completely got your point and I understand your suggestion for manual tweaking ( although even this would not be particularly accurate as you would be unable to alter your sleep stages which would mean the extrapolated metrics would still be a guesstimate).

    As to your question why have them in the first place?

    Because as you rightfully state, many users will not be well versed in exercise physiology and that is a demographic which Garmin wants to incorporate into its client base.

    Plus they are competing with other providers on functionality.

    I don't think there is a fool proof solution although using the metrics derived from sleep data will still provide a good spring board to improved performance just not optimal performance perhaps.

  • 0 9 months ago in reply to 8924630

    You are fixating too much on the fact that sleep measurements are not accurate to begin with and should just be taken as an advice (which is true).

    The issue, as I wrote multipe times, is not that it is not very accurate. It is that the readings are quite frequently horribly off/wrong (not even slightly in the ballpark) and then the inferred metrics and advice that the watch gives is not useful to anyone at all. In that case it also doesn't help Garmin competing with other brands. With readings so inaccurate you can also base your training on how you think your pet seems to feel each day... that wouldn't be worse.

    And the fix would be rather simple, because you wouldn't even need to fix the sleep stages themselves. I'm quite convinced that Garmin doesn't even use the sleep stages as input for other metrics but only the sleep score that they calculate from it. And you could easily overwrite the score with a generic value (i.e., Excellent Sleep with a range of about 90-100 would be set to 95) by simply selecting how you felt after sleeping, if you see something went wrong (similar to how you use to set your perceived exertion after a workout). That would still yield much better results (even though not perfectly accurate) than having to life with completely wrong sleep readings as I described in my original post here.

  • 0 9 months ago in reply to Chris

    I understand.

    The watch is horribly inaccurate for you. I don't find it wildly inaccurate at all.

    From time to time the sleep may not tally up if I am watching a movie in bed but on the whole it is more than in the ballpark.

    I would suggest either trying another brand or perhaps posting your idea to Garmin and see if they incorporate it in to the next update.

    Good luck!

  • 0 4 months ago

    I have a Fenix 8 and my experience has been similar to yours. Garmin should allow for manual override when the sleep metrics don’t align with how you feel the next day. I have a feeling my marathon block is being sabotaged by this issue, but we’ll see if I hit my goal time in May… this may be the last time I train using DSW.

  • 0 4 months ago

    Congratulations! It looks like your watch does a better-than-average job of tracking sleep.

    Actually, while the sleep tracking isn't terribly bad, that's only the case when your sleep is good and regular. Anything out of the ordinary, such as sleeplessness, it falls apart. I am regularly told I'm dreaming while I'm just pondering whether to get out of bed. And as a poor sleeper (old age), it's encouraging to see I'm being credited for all those hours spent wide awake.

  • 0 4 months ago

    Very well said, and seconded. I have an almost identical suggestion in over at Amazfit (T-Rex 3, Cheetah Round) as I try and navigate this upgrade process. Garmin used to be a lock, but this latest upgrade for me is proving more difficult for these very reasons.

  • 0 4 months ago

    I completely agree with you and I am extremely annoyed by it as well.

    There are companies such as Oura, Apple and Google/Fitbit that offer a reasonably accurate sleep tracking including sleep phases. They show that it is technologically possible nowadays. Of course, developing a robust and reliable sleep tracking algorithm requires effort, time and money. What I find appalling is, that Garmin asks the highest prices for their fitness watches but refuses to invest the necessary money.

    I completely agree that Garmin shouldn‘t base their sleep score on the amount of deep and REM sleep knowing how bad their algorithm is. It just doesn‘t make sense and gives the false impression to us customers that they can accurately track our sleep, which they can‘t.

    One solution I‘d like to see is, that I could import my very reliable sleep matrics of my Oura ring. But since my experience is, that Garmin refuses to open the Connect app for the import of data from other apps and that Garmin most of the time refuses to listen to its customers, I predict that this will never happen.

    The other thing that really annoys me, are the number of bugs that are introduced with every firmware update. The worst has been the last version 20.16, on my Epix Gen2. It broke the body battery, displaying wrong factors from the day before in the morning and deleting the stress bars in the middle of the day.

    Within the last two years, the wind compensation in power measurements suddenly didn‘t work, the help sceens for running dynamics graphs crashed Garmin Connect. I did a run where my location was shown completety wrong kilometers away from my actual positon for the entire run, I had my watch crash so badly one day before a race. At first, it couldn‘t be rebooted at all and I had to try several times and several button combinations to revive the watch. 

    Therefore I am with the OP that I cannot recommend a Garmin watch to anybody at the price point they‘re asking. I am so disappointed with Garmin at the moment that I consider switching to an Apple watch instead. I don‘t see me buying another Garmin watch any time soon again.