Race predictions way off

I also switched from a FR 245 to 965 at Christmas.  For me, the race predictions and VO2Max all stayed the same.  So I don't think it was changed for everybody across the board.

Yeah- so Garmin had my predicted 10K time at nearly 44 minutes.  So then I ran a 10K race in about 41 minutes.  I had the race on my schedule, in my watch and on garmin connect and it knew it was a race.  And even after I ran much faster than the predictor… it didn’t update my prediction.  Oh well.

Etupes25 said:

peterdb said:

On the day that I switched, the race predictions dropped enormously

The first suspect is your HR Max value. Your new watch can estimate your HR Max based on your training history. The accuracy is good if you curate your HR data. If you have been using the wrist sensor or had issues with your chest strap, it will take some time for the bad data to fall off.

Assuming VO2 Max didn't change, I would guess the reason for the immediate drop is that the new algorithm takes mileage and training load into account. HR Max is already used to estimate VO2 Max, so my educated guess is that it wouldn't be used a 2nd time for race predictions.

Also, I don't think changing HR Max would have any immediate effect on your current VO2 Max, but only on VO2 Max estimated from future runs with the new HR Max.

My totally unscientific experiment: currently, my max HR is manually set on my 955. I changed the manual value to be 50 beats lower and saw no immediate change in my VO2 Max or race predictions.

As far as Stryd goes, its estimates have always been minutes slower than what I can actually run (and far outside of the range shown in the app.)

I do agree that it's very important for max HR to be set correctly for VO2 Max calculations though. Based on everything Garmin / Firstbeat has said about VO2 Max estimations, max HR is the cornerstone of the algorithm. Basically a linear relationship is assumed between pace and HR, and your maximal effort for a given race distance (based on max HR) is extrapolated from your sub-maximal efforts recorded by the watch. For example, if you can run at a certain speed at a certain HR, then it's assumed you can run 10% faster (for example) at an effort where your HR is 10% higher (all other things being equal). The HR you're assumed to be able to sustain for a given race distance (e.g. 10k) is based on your max HR. As a concrete example, if you can run 10k in 50:00 at 140 bpm (submaximal effort), then if you can sustain an average of 185 bpm for a 10k race (which is 32.14% higher than 140), you should be able to run 32.14% faster (which would be 37:50). But your ability to average 185 bpm for 10k would of course depend on your max HR being higher than 185. That's my layman's understanding, anyway.

My understanding is that runalyze uses the same (or a similar) algorithm for VO2 Max, but the cool thing is it does adjust race predictions on your actual results. (Let's say you run a race 10% slower than it predicted - all future predictions would be adjusted to be 10% slower than normal.)

FlowState said:

I changed the manual value to be 50 beats lower and saw no immediate change in my VO2 Max or race predictions.

Obviously, you have to get a new VO2 Max value: just changing the HR Max value doesn't trigger a new estimate. Since Garmin has dampened the fluctuations of VO2 Max estimates, you won't see an abrupt change, and it may take a couple of weeks for the adjustements to be reflected. Of course, in the interim, other factors will also influence the new values. I recently took down my HR Max by 5 bpm. The cycling VO2 Max went down by a point after the following ride, and then another point. The running VO2 Max started decling slowly and lost about a point as well. I am confident that the adjustment was beneficial and I am happy that both VO2 Max are within a couple of points of each other, while the cycling VO2 Max matches other platforms estimates (these other platform use a formula based on the power duration curve).

FlowState said:

so my educated guess is that it wouldn't be used a 2nd time for race predictions.

As described above, Garmin uses your VO2 Max and your training history (which means the relationships between ventilation rates, paces and durations). The power of the Firstbeat algorithm is in the cleaning, filtering and processing of the raw sensor data embedded in the device. Races are just data points among others in the process. They are particularly useful because they represent "best effort".

Using a different perspective,  consider that research has established that threshold performance is a better predictor for race performance than VO2 max. I wouldn't be surprised if this was taken into account in Garmin's approach. But then we know that HR Max values also influences LTHR detection. So there are several ways HR Max value and training focus can influence the predictions. IMO, it is better to expose the watch to a variety of actual best efforts across durations than to let it make predictions. Practically, it means keeping the training focus balanced.

FlowState said:

Basically a linear relationship is assumed between pace and HR, and your maximal effort for a given race distance (based on max HR) is extrapolated from your sub-maximal efforts recorded by the watch.

In fact, there is an expected linear relationship between HR and speed *where* the lactate threshold plays a big role because it is a point where the correlation changes significantly.

As you can see in the chart, there are 2 key necessary points: (HR Rest, null pace), (LTHR, LT pace) and (HR Max, VO2 Max pace). That brings to life the critical role of HR Max which influences 2 of these.

ttps://www.ncbi.nlm.nih.gov/pmc/articles/PMC3438148/

FlowState said:

My understanding is that runalyze uses the same (or a similar) algorithm for VO2 Max

I don't think so.

Runalize explains they use the Daniels/Gilbert formula for their estimates and race predictions.

https://runalyze.com/help/article/vo2max?_locale=en

Garmin/Firstbeat doesn't mention that formula, but instead mentions machine learning using neural networks for the processing of HR sensore data, VO2 Estimates, EPOC and respiration models. Very interesting white paper here, but not for the faint of heart.

https://jyx.jyu.fi/bitstream/handle/123456789/13267/951391707X.pdf

TL;DR I don’t think the poster you were replying to actually recorded any new activities (based on the fact they said their race predictions immediately went down — on the day they switched), and even if they did, i don’t think the resulting drop in the activity VO2 Max (i.e. due to incorrect HR) would be enough to move the needle significant on either average VO2 Max or race predictions. But I could be wrong.

Etupes25 said:

Obviously, you have to get a new VO2 Max value: just changing the HR Max value doesn't trigger a new estimate.

Right, I was referring to the poster you were replying to who said that their race predictions went down on the day they changed watches and your response that it’s probably because of incorrect max HR.

1) We don’t know if they recorded any new activities on the new watch before the race predictions went down. My guess is they didn’t, but I could be wrong.

2) We don’t know if the VO2 Max on the new watch went down as the race predictions went down (of course that would mean they must’ve recorded new activities). My interpretation is it didn’t, or the poster would’ve mentioned it. Further complicating this question is the fact that Garmin stores a VO2 Max with at least 2 decimal places but only displays 0 decimals of precision (everything is rounded to the nearest integer), so you wouldn’t necessarily know if your vo2max has changed (unless, ironically, you record another activity and look at the vo2max in runalyze with 2 decimal places)

I think we both agree that the only way changing your max HR would affect race predictions would be indirectly - you’d have to record a new activity or activities which would produce an incorrectly low VO2 Max which would affect your race prediction. However, there’s also the fact that Garmin VO2 Max appears to be some sort of rolling average (obviously no one knows the exact period of time or weighting here, although some have mentioned that the average seems to change much more slowly in newer watches than in the past). So my question is even if the poster had recorded 1 or 2 new activities with an incorrect max HR, would that be enough to cause a dramatic change in average VO2 Max and therefore race predictions? I know that in the past, one bad run or one good race would be enough to move the needle on my average VO2 Max by a noticeable amount, but again, I think the newer algorithm has the average VO2 Max moving a lot more slowly in response to changes from individual activities.

Thanks for all the other information, I’ll have to digest it.

I’ve never had a solid grip on all the details of the VO2 Max calculations, although I have read the firstbeat whitepaper (not the one you linked, the one on firstbeat’s site.) My takeaway from the firstbeat white paper was:

A. a linear relationship is assumed between pace and hr
B. a maximal race effort is extrapolated from submaximal efforts using max hr
C. the “special firstbeat sauce” is the part of the algorithm which discards irrelevant data such as when you’re standing still at a red light

https://www.firstbeat.com/en/science-and-physiology/fitness-level/

[https://www.firstbeat.com/wp-content/uploads/2017/06/white_paper_VO2max_30.6.2017.pdf]

Calculation steps
The following calculation steps are used for VO2max estimation:
1) The personal background info (at least age) is logged
2) The person starts to exercise with a device that measures
heart rate and speed
3) The collected data is segmented to different heart rate
ranges and the reliability of different data segments is
calculated
4) The most reliable data segments are used for estimating the
person's aerobic fitness level (VO2max) by utilizing either
linear or nonlinear dependency between the person's heart
rate and speed data.

Obviously where I’ve already made a mistake is that firstbeat utilizes both linear and non-linear relationships between HR and speed.

https://runalyze.com/tools/effective-vo2max?vo2max=50.00&units=km&paces=1

Runalyze estimates for you the "Effective VO2max" also from non-maximal training performances based on the ratio of heart rate and speed

From my extremely simple surface-level understanding of the topic, the description of the runalyze algorithm sounds similar to A and B above, but I’ll defer to your knowledge. I freely admit that I have a very limited understanding of this stuff.

Etupes25 said:

In fact, there is an expected linear relationship between HR and speed *where* the lactate threshold plays a big role because it is a point where the correlation changes significantly.

As you can see in the chart, there are 2 key necessary points: (HR Rest, null pace), (LTHR, LT pace) and (HR Max, VO2 Max pace). That brings to life the critical role of HR Max which influences 2 of these.

That’s pretty interesting, thanks! I wonder if the Garmin/Firstbeat algorithm actually takes lactate threshold into account, since we know that Garmin won’t estimate LT unless you perform the guided test with a chest strap. Clearly the guided test isn’t necessary for VO2 Max estimation or race predictions. (Nor is a chest strap required for VO2 Max estimation, although it may

I realize it’s possible that Garmin has some hidden LT estimate based on max HR which is only used for VO2 Max estimation purposes.

Since you seem to have a very good understanding of this topic, do you think weight has any direct bearing on Garmin’s VO2 Max estimation (meaning that the weight you enter on your Garmin would be a direct input to the VO2 Max estimation algorithm)? I’ve seen it suggested many times in these forums and elsewhere (like reddit) but I don’t buy it. People will say “make sure your weight and max HR are entered correctly”, but my (layman’s) understanding is that the only physiological parameter that Garmin actually uses for VO2 Max estimation is max HR.

I think lowering your weight (by losing non-functional mass) will allow you to run faster for a given heart rate, thus resulting in a higher VO2 Max estimate, but I always figured that weight itself is not used in the calculation. Same goes for age and gender. e.g. People slow down after a certain age which results in a lower estimated VO2 Max, but I think age itself isn’t used in the calculation. 

Similarly, the Daniels formula for estimating VDOT from a race result only cares how far and fast you ran, not any of the secondary factors which may have influenced how fast you were. (Yeah, I realize it’s not the same as what Garmin does, although the part where Garmin estimates your maximal effort based on actual sub-maximal efforts, and uses the maximal effort to estimate VO2 Max seems to be similar to taking a race result — i.e. an actual maximal effort — and using that to estimate VDOT.)

Again I could be wrong.

To clarify, I have entered the exact same max HR and HR zones as in my previous watch. So all the input is exactly the same.

I  have recorded 7 run activities since I switched the watches two weeks ago, and the estimates are still way off.

I did a run on the day that I got the watch, I do not remember if it already showed estimates before the run.

To give an example, I have never run a half marathon race, but I ran a marathon two months ago and recorded a half marathon best time of 1:47 during that marathon. My 245 showed 1:38 as half marathon race prediction, which seems quite possible and consistent. My 965 now shows 1:55, which is much slower than the time I ran during that recent marathon!

Maybe the 965 just needs much more runs to give better predictions.

FlowState said:

I think lowering your weight (by losing non-functional mass) will allow you to run faster for a given heart rate, thus resulting in a higher VO2 Max estimate

Starting with physiology, your weight has a direct linear influence on your ventilation. A study of treadmill exercise at 75% of VO2 Max, varying additonal weight of 10,20 30kg has shown that each additional kilogram increases oxygen uptake by 33.5 ml/min and heart rate by 1.1 beats/min. Interestingly, this relationship doesn't vary much with the intensity of the exercise.

https://pubmed.ncbi.nlm.nih.gov/648506/#:~:text=In%20dynamic%20exercise%20oxygen%20uptake,of%20the%20dynamic%20work%20load.

Garmin's uses HR and HRV data to estimate ventilation (VO), and uses machine learning to predict your VO2 Max from sub-maximal efforts given altitude and temperature data after filtering and qualifying the data by taking into account movement vs rest. Weight is not needed here.

However, when the watch gives you a VO2 max number, it has been normalized by your weight. While absolute ventilation is measured in liters of oxygen per minute (L/mn), the number that the watch gives is the relative ventilation, which you can use to compare with others. The relative ventilation is typically expressed in 1/1000 liters per minute and per kg (mL/mn/kg), so this where your weight input is important. If your weight input is wrong, your relative VO2 Max will be wrong mathematically. You don't need to update your weight for every couple of lb of variation though. If an athlete weighing 200lb enters 190lbs instead, his VO2 Max of 50 will change to 52. These 2 points are within the average 5% error of the absolute VO2 Max estimate, so the weight change might not impact the estimates that much.

FlowState said:

Same goes for age and gender. e.g. People slow down after a certain age which results in a lower estimated VO2 Max, but I think age itself isn’t used in the calculation

Age and gender is certainly used for the relative VO2 max benchmarking (the color coding of the VO2 Max scale on the watch). Just like weight, they are not needed for the absolute evaluation of ventilation.

FlowState said:

although the part where Garmin estimates your maximal effort based on actual sub-maximal efforts, and uses the maximal effort to estimate VO2 Max seems to be similar to taking a race result — i.e. an actual maximal effort — and using that to estimate VDOT

It is similar yes, but the key difference is that in Garmin/Firstbeat world we are talking about an individualized model whereas in the fixed regression model world a la Daniels/Gilbert we are applying a standard formula. Same thing/problem with HR Max detection vs formula based HR Max.

peterdb said:

My 965 now shows 1:55, which is much slower than the time I ran during that recent marathon!

Take these predictions for what they are. Garmin has implemented an algorithm that predicts your performance based on your training history and your input. If we forget for a moment that the algo comes with an error rate, the fact that your HM prediction is way off can only stem from a wrong HR Max or your training history that doesn't reflect your capabilities during a race. Aside from the possibility of wrong data capture, I don't see another explanation.

I am not commenting on the difference between the 245 and the 965. I had a vivoactive 3 before the Epix Gen 2, and all my performance metrics have changed. I trust that more recent watches are updated with better sensors and algorithms from an accuracy perspective and I don't think the Garmin's UI software has much risk to impact that accuracy.