New VA4 - Live review

You did good. Thanks for your time.

I upgraded from cheap mi band4 to garmin thinking: more money must be better. But owning the va4 I am amazed how accurate the mi band4 actually was at sleep. Garmin can't seem to differentiate "just lying in bed browsing the net" and actual sleep while the mi band did, mi band also did naps accurate.

Yes I have to admit. The garmin app is beautiful and just works every time, including notifications. But with data gathered every second, it disappoints you can only deep dive in 2min interval in the app on daily monitoring. Zooming function on the graphs would be welcome, if only to satisfy my 1sec curiosity.

Hi, where do I find the 'heart report' exactly?

Thx

Had a ***ty sleep so it might not be the best example, but nevertheless results are interesting:

- VA4 completely misinterpreted the start time (should have been after the first awake period, just sat and watched TV before that) but detected the short "getting out of bed, then coming back for a doze" around 8AM
- AW3 kind of got the right times but decided i was already deep sleeping when i got into bed, lol? Also stopped tracking after the 8AM move
- GT2 well....measured everything (at least what i consciously remember, like sleep start & end times and waking up for couple of minutes around 8AM, then going back trying to sleep some more) with perfect accuracy. See for yourselves.

Did also a battery drain experiment overnight:

AW3 (Cinema/DND mode ON): start 68% end 57% loss 11%
VA4 (PulseOX sleep only, DND mode): start 20% end 10% loss 10%
GT2 (DND mode, everything else ON): start 67% end 65% loss 2% !!

From left to right: AW3, VA4, GT2

Later today stay tuned for an outdoor walk and outdoor cycle experiment... 
I kind of feel (and probably look) strange with 3 watches on my hands, so i will make sure i wear long sleeves LOL

Allright, cycling (in the city) comparison test done. I didn't do the walking because the results would be the same.
I don't run, maybe i should do it once tomorrow to see how VO2max values compare..and then i'm done.

Upfront conclusion for the cycling test: results for all 3 watches are pretty much the same, with some notes:

- GT2 had a hard time (for the first time, never happened before) to lock on GPS. I even restarted it once...1-2 minutes later still no GPS lock. Then i moved few steps away from the bike and voila, GPS lock straight away...strange.
- AW3 seems to LOVE cutting corners, i think the developers don't like straight lines, pretty much everything is rounded to some more or less extent (i think AW4 has the best GPS mapping, even better than AW5 apparently)
- AW3 is the only one who shows speed on the GPS map by color-coding the trajectory: green fast - yellow medium - red slow speeds
- GT2 and VA4 indicate the colored speed bar but the trajectories are plain.
- VA4 had the most accurate GPS route of all 3 watches. However both AW3 and GT2 were very close, with the GT2 slightly better.
- Some very strange altimeter/climb/descent values for all 3 watches, here were the most differences found.
- Some differences in calories burnt between VA4/GT2 (which were close) and the AW3.
- No clear winner here as in the other tests, they all do good.

APPLE WATCH 3
The worst GPS representation of the route but nicely speed-sensitive coloured 
OK not THAT bad, but compared to VA4 and GT2 it is...cutting and rounding corners and trajectories too much.
Detected 4.33km, average HR 130, max HR 156, average speed 12.3 km/h, elevation 80min/91max (gain 15m)

   

VIVOACTIVE 4
The best GPS representation of the route, best accuracy.
Detected 4.42km, average HR 131, max HR 168, average speed 12.5 km/h, elevation 129min/138max (gain 16m)

HUAWEI WATCH GT2 (just one photo since the Huawei Health app "export to file" bundles everything)
Detected 4.39km, average HR 130, max HR 167, average speed 12.7 km/h, elevation 45min/54max (gain 11.6m)

Allright, stay tuned tomorrow for the 3rd night sleep analysis and maaaaaaybe a small run to test VO2 values.
 This time i'll make sure i go to bed earlier 

As the weekend draws to a close (and also my patience running low with 3 devices strapped on 24/7 for 3 days), what is your recommendation on which watch to keep? What would you choose and why?

So, dear Garmin Support,

Besides the swimming detection issue i've been spending some time demonstrating, can you please explain why the PulseOX keeps functioning during the day even if I set it for sleep only? It's been like this for 2 days, see below.

Thanks!

  

I'll be following this topic wonderingly. No doubt AW series 3, 4 and 5 has the most accurate wrist HR sensor in the world. In the past, Mio Global was famous for that but they stopped manufacturing and i dunno the reason.

As you know, wrist HR sensors are not as good as chest straps, especially for the activities involving lots of intervals. BUT EXCEPT AW, especially  AW 4 ! If you watch below video between min. 2:55 - 4:17, you will see how Garmin Instinct is a shame versus AW 4. After a set of pull up, AW 4 updates HR from 121 to 149 with 5-10 seconds delay, on the other hand Instinct wanders around 104 bpm for centuries !

https://www.youtube.com/watch?v=7cy6YRcmwBE

It's really a big shame for a SPORTSwatch to be defeated by a SMARTwatch on a sport&health focused test !

However, I'm not very sure about the accuracy of sleep analysis by AW 3. Because it doesn't have a native sleep tracker. And also AW series don't have continuos HR recording. And HR sampling frequency is insufficient. You'd better add a fitbit device to the comparison.

Additionally, due to lack of continous HR sampling, these health metrics will be affected negatively in tandem;
Resting HR >> Overall Burned Calories >> Progress/Regress in Performance/ or Getting Fitter/Unfitter.

For instance, when my resting HR becomes higher than usual, i know there is something wrong with my body. Generally it becomes a sign of my body before an ilness.

Yes, i agree that no wrist watch is as accurate as a chest strap, but i'm comparing apples to apples here (no pun intended) 

For most normal, healthy users, a wrist device is sufficient as we're not that into exact measurements that can lead to other more advanced and accurate statistics, such as effort estimation, recovery rates, oxygenation, etc. So these 3 devices are pretty much in the same ballpark, feature and price-wise...yeah, could throw a Fitbit in there, but from my experience they also have same weird swimming detection peculiarities. Plus, Fitbit's gone subscription based now for slightly more advanced stats & insights, which i totally do not agree with. 

AW3 is usually pretty good in sleep detection, even though it's not native, but because it's very accurate and has good 3rd party API support that enables others to make sense out of the data...just another approach vs. others who do it "in-house".
Its algorithms are quite good at sampling at right times given its battery life estimates, so yes, they are very balanced and optimised devices, compared to VA4 or GT2 or even Fitbits which can afford 24/7 HR sampling at 1sec intervals without having to worry about battery impact that much. My guess obviously is that Apple don't bundle yet native sleep tracking due to battery constraints...maybe next-gen watch will do this, because it's simply amazing how GT2 lasts for 10-14 days with 24/7 sampling and a 1.3" AMOLED always-on display.

Well, the GT2 is the elephant in the room here, there's nothing compared to it yet that even comes close battery life and efficiency-wise. VA4 lasts 3-5 days, AW3 1-4 days...all depending on what other functions are enabled or not.

RHR is an important indicator as you say, correct. But from what i've researched, Garmin has a different way to calculating this than most others, that's why it may have other values. So in the end, it's how smart you sample and interpret the data based on how often you gather it.

They are not medical devices, not even the latest & greatest Fenix 6 is, nor latest AW5, etc. They are prosumer devices, some more or less fashionable  that "democratise" fitness & basic health statistics. A very productive industry indeed for an increasing anxious, stress & hypochondria driven population  The politically correct term would be however "health conscious"

But again, here i'm just comparing 3 devices in the things that interest me: battery, swimming, cycling and sleep scenarios.

BTW, last night messed all 3 devices sleep statistics, none handled the switch to DST winter time properly 

So, final sleep analysis comparison conclusions. DST switch to winter time messed up all 3 watches one way or another.

- AW3 (AutoSleep app) shows start/end times right but the total calculation is 1h too long
- VA4 shows both start/end times and total calculation wrong (1h too long)
- GT2 same as VA4

BATTERY LOSS OVERNIGHT

- AW3 11%
- VA4 10%
- GT2 2%

SLEEP ANALYSIS

The GT2 missed the awake minutes around 6 AM (i remember looking at the watch, don't remember which one lol). Probably because i didn't get out of bed this time, just opened my eyes, saw 6 AM, cussed few times cos it's too early and went back to sleep :)
The VA4 seems to make a habit out of not detecting deep sleep in the latter part of the night, it's not the first time it did this. I see deep sleep patterns on both AW3 & GT2 after 6 AM

STEPS COUNT ANALYSIS

Had the chance today to wear all 3 without charging. No idea how many steps i did in reality (lol) but here are the results now:

- VA4 8748
- GT2 8685
- AW3 9498 (activity app on the watch, rotate crown to see total steps)

RUNNING ANALYSIS

Haven't run in a looong time and it showed  But that's a small step in the name of science and comparison tests, right....
Actually it was a stop & go affair, each time the GT2 vibrated to alert me of HR above 180 i walked. And so on...
VO2MAX was recorded only by the AW3 and GT2
For some reason i have in Garmin Connect an old VO2MAX value of 44 from a walking workout (0.9km) i did with the Venu couple of weeks back. But that's it. I understand this is a bug on the VA4 (separate thread in here about it) which is going to be fixed in the next FW release.
* the GPS lock issue i had yesterday with the GT2 is no more. It locked today for both running & walking within 10 seconds.

AW3

VA4

GT2

FINAL CONCLUSIONS

What can i say, it was a fun experiment, i learned a lot about each of these watches with good & bad things. They are sensor-wise not so far apart. What makes the difference is the "presentation layer": data analytics, 3rd party ecosystem, social integration, etc.

DESIGN

Here i must say the GT2 looks and feels the best. The strap is really comfortable and it doesn't feel big at all at 46mm. The VA4 comes second, strap and feel also pretty good. Slightly lighter than GT2 i feel. The AW3 comes last because i'm fed up with the same design for years now  Its strap does feel really nice though, given its different way of locking it compared to the other two.

HARDWARE

This is a mixed bag because each has its strenghts: VA4 the very precise GPS & SpO2 sensor, GT2 the incredible battery, display & sensors, while AW3 has the smoothest / fastest operation and compatibility with my iphone (duhh)

SOFTWARE

Here it gets interesting. AW3 we all know pretty much what it's up to. Between VA4 and GT2 there's quite a few bugs that need to be squished asap. GT2 is simple & straightforward, no complications (pun intended). VA4 feels sliiiightly slower than GT2 and definitely slower than AW3. VA4 wins big time over GT2 in 3rd party apps & integration w/fitness platforms, watch face market. GT2 also needs an Android phone to make the most out of its functions vs. GT1 (music player, make/receive calls, watch faces, etc.)

BUGS

AW3 (WatchOS 6.0.1)..hmm, none that i found?

VA4 (FW 3.50) : PulseOX measures 24/7 even if set to sleep only. HR sensor remains on even if it's set to off (separate thread in here). VO2MAX not recorded. 2.4GHz Wifi errors (separate thread in here). Larger battery drain than expected (>20% day). Swimming detection errors.

GT2 (FW 1.0.1.16): Resting HR not shown/detected. SpO2 sensor not active (apparently China has it enabled).

GRAND CONCLUSION

AW3: Oldie but goldie. Dismal battery life. Dated design. What it does, it does very good with the help of some friends (3rd parties). Health apps (yea, Apple, why have 3 apps - Health, Activity and Watch??) confusing.
VA4: Excellent GPS accuracy. Accurate sensors. Comfortable to wear. Thank God for Garmin Connect. 2 apps (Connect & IQ) needed.
GT2: Pragmatic. Killer looks, battery and display. Locked OS & platform. Stupid bugs. Good price. Awesome tracker. Best sleep stats. Huawei Health pretty clean and good (on Android). Single app that handles it all.


In 5 days i already had to fully charge both AW3 and VA4, while the GT2 had 75% out of the box and now it's at 43%.
Used all 3 in exactly the same way.

I have kindly @mentioned some Garmin mods (even PM-ed one) few posts back, haven't received anything back.
I really hope someone takes a good look at this thread and tries to figure out the swimming, pulse OX, HR monitor and DST issues with sleep tracking. My VA4 is being returned tomorrow, i don't think it brings any extra value vs. the GT2 that is justified by the price difference and my "sporting profile & ambitions" 

Hope you enjoyed those last couple of days as much as i did. Thanks @Garmin for allowing this comparison on your support forums. If it's an issue, pls. let me know in advance and i'll move it somewhere on Reddit.

Night, over & out.