Related
This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

[PacePro] How broken is pace pro with elevation (e.g. Trailrunning)?

Fredderic
over 1 year ago

Today I did a evenly paced "easy" trail run, and I wanted to show how Pace-Pro would recommend me to pace this effort, and contrast it with reality (how does a real evenly paced effort look like).

The run:

17km, 1360m elevation gain, 650m elevation loss. Mix of road, forest roads and trails. Total time 2h46min.

Here are the kilometer splits as pace-pro suggests, as happened in reality, and also as a different tool called plot-a-route suggests. Each pacing strategy was given the "correct" goal time of 2h46min.

As you can see, plot-a-route is on average 14% wrong on the kilometer splits, while pace-pro is 25% wrong on average. This is a huge difference, and we can see if we look at the details, that the problem is especially bad when the hills are steep, as for kilometer 7, where the Pace-Pro suggests a 13min/km, reality was a 20.5min/km, and plot-a-route gave 19min/km. This is 56% error from pace pro!

Plot-a-route is not using rocket-science for this much better prediction, they simply calculate a "flat equivalent" distance using:

Every year millions of people use a Garmin watch to pace their trail races - could we put in the minimal effort to make Pace-Pro remotely possible when it goes up-hill?

See also this post, now 1 year old, showing the discrepancies when using pace-pro to predict uphill performance:

forums.garmin.com/.../1526426

  • 0 over 1 year ago

    PacePro first flat kilometer: 8:47 min/km => Thinks route is roughly equivalent to 19.5 km flat.

    Plot-a-route first flat kilometer: 6:24 min/km => Thinks route is roughly equivalent to 26 km flat.

    Anybody who has ever done anything similar to 17 km with 1360m gain will attest to you that it is significantly more than a 19.5 km flat :D.

  • 0 over 1 year ago in reply to Fredderic

    Hi Fredderic!

    Even though PacePro is unrealistic with its paces in mountainous terrain, I'm not sure if that's its biggest issue. I'm also not sure if PacePro's programming is associated with our position on the track rather than distance covered; I believe it's more likely the latter.

    In any case, I think pace and split time planning in trail running makes sense between aid stations or significant points, not kilometer by kilometer, or altitude changes. 

    Based on the above, I suppose PacePro can be useful in a marathon, and it was probably created with that purpose in mind. 

    I would be satisfied if Garmin's AHD/BHD field didn't reset every time we slightly deviate from the track...

    On the other hand, I'm a bit surprised by the pace differences between PlotaRoute and your actual pace. In my case, I have it fairly well calibrated. If you could share the track of your route with me (for example, using the link to PlotaRoute), I can play around with the paces for a while.

  • 0 over 1 year ago in reply to carglp

    Strong agree on not resetting the ahead/behind field for any "off-course" warning, it is almost impossible to never receive this off-course warning on a more complex trail run, and it negates the use of the virtual pacer.

    Pace-Pro actually has a setting to go by climbs instead of every kilometer, which you point out correctly is probably more sensible for pacing in a trail race, but for simplicity I just posted the kilometer splits, as I want to highlight how badly wrong the assessment is for the very steep uphills and also for the flats in comparison.

    I agree that plot-a-route could definitively be fine-tuned to show much better results for me for this course - much of the "too slow" kilometers on the end are because of the "steep downhill penalty" that plot-a-route has, but the ski-slope is simply too runnable even at 20% down-hill grade ;). I wanted to show a route where plot-a-route is not having a easy time either, to give the gamin predictions any benefit I can think of, and to show that even then they are wildly worse.

  • 0 over 1 year ago in reply to Fredderic

    did you see that now It's possible to edit manualy the pace in every segment?

    do you know what happen if you lost the track for a few meters with the programed pace? Are the programed pace change points geolocated on the track? or linked to the distance traveled?

  • 0 over 1 year ago

    What did you choose for Splits? If you chose Every Mile, can you change it to Elevation Based and test it out?

  • 0 over 1 year ago in reply to Garmin-Sierra

    How are you doing, mates?

    I've been doing some tests with PacePro, and here are my conclusions:

    Field Test: I didn't encounter any issues with PacePro

    • When you stop and the autopause kicks in, both the pace in the segment and the AHD/BHD stop as well. When you resume running, both fields start working again.
    • If you ever lose the signal or the track, both fields will stop working. When you return to the track, PacePro updates AHD/BHD, remaining distance of the segment and programmed pace, based on your position relative to the original plan.
    • So, at least in this test, it has worked correctly.

    Design Test: At least for me, in my trail runs, it's not useful yet.

    • I'm not sure why, but when converting an activity into a route, approximately 15% of the elevation gain is lost. The quality of the altitude profile is reduced, so the distribution of intervals based on changes in slope is not faithful to reality.
    • The pace calculation for uphill sections is overly optimistic, even when selecting the minimum effort value. Conversely, the pace for downhill sections is too conservative.
    • Being able to manually edit the pace of each interval is a great improvement, but it would be even more useful if we could also manually modify the length of each interval.

    Here are some improvement suggestions:

    • Personally, I prefer to plan my runs based on aid stations or specific points such as high points and low points. So it would be very helpful to be able to manually select the start and end points for each interval.
    • For determining the pace of each interval, I find the method used in PlotaRoute more useful. It involves starting with the flat pace and penalizing it based on the incline for uphill sections and downhill sections separately.
    • It would also be useful to integrate a waypoints view on the map and profile, as well as being able to dynamically see the time of passage for each waypoint as the parameters of PacePro are modified.
    • I'm not entirely sure how the ETA at Next or Time to Next fields work when navigating a track with PacePro, but they should be consistent (fix: I have checked that the ETA at Next, Time to Next do not match the expected pace from PacePro)
    • Regarding the effect of stops on AHD/BHD, I don't feel prepared at the moment to define a specific behavior.

    And that's all for today! Hopefully, you will keep improving.

  • 0 over 1 year ago in reply to Garmin-Sierra

    Yes, I chose distance based splits (every kilometer), here is the result with "Elevation Based" splits:

    (1) When creating the pace pro plan the default finish time (before I edit it to 2:46:18, the real time) is 1:35:04, which is only a little slower than my half-marathon time, I can definitively not run 17km with 1360m gain anywhere close to that time.

    (2) Also when I select the route via the watch the predicted time shows this crazy values that are almost (but not quite!) ignoring elevation. See this screenshot for my watch suggestion for a "vertical kilometer" route, which just goes up 1000m very steeply over 3.76km: https://postimg.cc/pp1LSfCD 18:48 would beat the world record by 10 minutes https://en.wikipedia.org/wiki/Vertical_kilometer - Note that Garmin suggests my 5km time would be 19:15, so it is taking elevation into account (only not enough).

    Pace Pro elevation based

    Settings: https://postimg.cc/zH7Vv4TM - I put uphill effort all the way to "easy" to try to improve the predictions as they are always too fast on uphill. Set the target time to what I ran in reality as training run.

    Plot-a-route settings: 6:04 min/km flat equivalent pace (to reach 2:46:18 total time), default hill settings with no penalty for steep downhill: 1m elevation equivalent to 7.92m extra. 1m elevation down equivalent to -2.72m flat. Steep downhill (>12%) 1m elevation down equivalent to 0.00m flat.

    Result: Even with pace-pro set to the maximum "easy on uphill" settings and with elevation based splits, the steep uphill segment is still 30% too fast with pace-pro prediction. Looking at the mean squared error on the pace-pro selected segments against the plot-a-route basic flat-equivalent prediction, the error of plot-a-route is 6.5%, the error of pace pro is 16.5%, 2.5x worse.

    Table with results:

    https://postimg.cc/XG154QC4

    If you want you can use your Garmin super-power Sierra, and view the PacePro plan I created at https://connect.garmin.com/modern/pacepro/edit/3202810 (it is not public so you do need those super-powers)

  • 0 over 1 year ago in reply to carglp

    Yes, I did see this. The points in the thread opening post are every 1km of the loaded track, but see below my response to Sierra for a re-analysis using the elevation based splits. Even with "Uphill Effort" slider set all the way to easy (as far as it goes!) the uphill still gets *severely* underestimated and suggests a time that is almost 30% faster than reality.

    That difference is like suggesting to me in the flat to go 4min/km (my 10km race pace) instead of 6min/km (easy jog pace)

  • 0 over 1 year ago in reply to carglp

    Hey carglp,

    I filed a bug-report asking to not reset "time ahead/behind" every time there is a "off-course" warning: https://forums.garmin.com/beta-program/forerunner-965/forerunner-965---public-beta-bug-reports/i/public-beta-version-4-12/do-not-reset-virtual-pacer-on-off-course-time-ahead-behind-gray-arrow-on-map Maybe if it gets some upvotes it will at least be discussed :)