Sleep tracking - REM classified as awake?

Sleep analysis in labs is tricky; by consumer devices, it is approximative, and self-analysis is extremely unreliable (because, well, we are sleeping).

There is a low probability (around 2%) that  watch was confusing your awake periods with actual REM periods. See below. More likely they were confused with light sleep.

The phase classification is not very precise and you cannot do anything about it except wearing the watch properly (not too tight, not too loose) to help with HR/HRV data collection.

More details:

When sleep experts are asked to identify sleep patterns, they agree 67%-82% of the time according to FirstBeat Research. Firstbeat is the technology used in the Garmin watch. 

https://assets.firstbeat.com/firstbeat/uploads/2019/11/Firstbeat-Sleep-Solution_white-paper_short.pdf

Firstbeat designed an algorithm that has about "reasonably good" (sic) accuracy when using an ECG device to measure HR and HRV and various accelerometers.

What the table shows is only 1% of the 5s-30s periods ot sleep identified as "awake" by the watch were in fact "deep" periods, 9% were in fact "light" sleep, and only 2% of them were in fact "REM" periods (4th column).

Overall, the agreement between the watch and experts is about 66%. This means that 1 out of 3 period of 5s-30s will be misclassified.

A wrist-based watch however will fare worse because of the challenges of accuracy for HR and HRV measurements.

Very helpful.  Thanks for posting!

damn, thats really interesting and thats a super helpful graphic too. Thanks for the helpful response!!! :)

I ended up resetting the watch anyway and it seems like it's reduced the amount of "awake" spikes and replaced them with light sleep - which lines up with what you are saying. Not sure why resetting it changed it but I'll keep collecting sleep data and see if it's consistent.

It's super annoying because it affects all my training readiness/recovery etcetera. Oh well, hopefully the algorithm improves over time.

Thanks again :)

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8120339/

I did not have time to read the whole thing, but the conclusion seems to be that Garmin devices are the worst sleep trackers among consumer devices  (but they only included older generations like Fenix 5)

"Results

Most devices (Fatigue Science Readiband, Fitbit Alta HR, EarlySense Live, ResMed S+, SleepScore Max) performed as well as or better than actigraphy on sleep/wake performance measures, while the Garmin devices performed worse. [...]"

ooh, i love a bit of research. i find it ironic that the devices perform worse when you have a worse sleep hahahah. hopefully with the discontinuation of fitbit garmin up their game for sleep tracking through software somehow. the sample size is also not huge but it wouldnt surprise me if the results were easily reproduced. maybe i should pick that to study if i get the opportunity at uni

The article you cite is from 2020 and used Garmin Fenix 5S. Since then, Garmin has started using FirstBeat's sleep tracking algorithm instead of their own attempt, so any conclusions in that article don't apply to current Garmin models.

Bitti said:

The article you cite is from 2020 and used Garmin Fenix 5S. Since then, Garmin has started using FirstBeat's sleep tracking algorithm instead of their own attempt

Garmin started using Firstbeat's algorithms for the sleep analysis long before they acquired Firstbeat Analytics in 2020. Minimally since June 2018, when not even earlier, on some models. The first sleep analysis using Firstbeat's technology was done on the server (you had to upload the sleep data to the GC account first). Newer models already have the analysis implemented directly on the watch, so the uploading to the server is not necessary. However, both methods use algorithms from Firstbeat.

That told, Fenix 5 indeed did not use the Advanced Sleep analysis by Firstbeat. It was released in 2017

Ok, I didn't know that the on-server sleep analysis was also from Firstbeat. I was under the impression it came from somewhere else. Thanks for the info!

This is great content. Thank you for sharing.

Interesting to see that the Garmin devices in this study show a bias towards detecting too much sleep time, and too little wake time during sleep. Unsurprisingly, Garmin devices have a better matching of sleep epochs to actual sleep, but the specificity is low, meaning they have a lower ability to correctly detect actual wake epochs than sleep epochs.

Wrt to sleep stages, the devices do not perform very well compared with polysomnography (PSG) analysis, and when they misclassify a PSDG REM, Deep or Wake period, they all tend to put it in the light sleep class.

As the study suggest, Garmin's sleep phase performance is aggravated by the poorer performance in sleep/wake state detection.

All in all, the results of the study feel right in my experience: watch detecting sleep when I am not sleep (nap, evening, or wake periods during the night).

I think you are right thats rem not awake. Mine is doing the same thing.  Right where it should be rem it shows awake.  I just woke up and checked mine. Been awake half sleeping for past hour and that reads as REM!  Not perfect, no big deal except as you said all the other great readiness features rely on sleep analysis.