This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

Extracting "precise" Vo2Max from fit file

[This is a repost of something I posted on Reddit a few minutes ago but I think some of my fellow 945 owners might be interested in]

Hello,

I'm a software developer and stat geek. One of my frustrations when it comes to GarminConnect UI is that you cannot get a precise value of your estimated Vo2Max - it's always a round number. I wanted to have precise values to see trends better (especially since I've been stuck in the 54-56 range for months...) so I wrote a little program that extracts data from the Fit files and renders them in a webpage.

I though I would share it with you guys so I published the source code : https://github.com/bleenhou/fitparser/releases

To use it, just copy the fit files of your activities in a folder, run the program and select the folder in which you just copied the files. (You cannot point the program directly to the watch storage...)

Quick notes : 1/ It's written in Java so you need Java installed on your system to run it. 2/ It can be downloaded at the link above, but if you want to recompile it, you will need the thisisant java fit SDK jar. 3/ It's only parsing Running Fit files, not the rest (Strengh, Bike, etc...)

I tested it with chrome but I think the code is simple enough that it will work with any browser.

Top Replies

All Replies

  • many thanks, 0.4 now opens any large/old directory with a large mixture of good/bad files, it's perfect

    only bonus features I could think of for now is some kind of small dialog that shows it is actually processing files and doing something because some of these directories have 1000 files and it takes it a real full minute to digest it - either a count or if you want to get fancy it could be a scrolling list of accepted vs rejected files but that's not important at all

    also not sure if possible with java/jar file but if I add fitparse to my windows "send to" menu. it could be neat if it knew to accept directories sent to it from explorer instead of the open dialog but this is just a super bonus and not critical

    the only bug to report is it does not open all the generated reports by default, only the last one like "weight" and then I manually have to go into the temp folder it uses and manually open the vo2max html but this is also not a showstopper

  • I found some more cool fields you could extract by taking a peek at how GoldenCheetah does it:

    github.com/.../FitRideFile.cpp

    they all seem to be in that "140" message id

    Some are very exciting because I've been looking to log them for years like "recovery time" and aerobic/anaerobic TE

    There is also the ending "Performance Condition" !

    case 7:   // METmax: 1 METmax = VO2max * 3.5, scale 65536
                        active_session_["VO2max detected"] = QString::number(round(value / 65536.0 * 3.5 * 10.0) / 10.0);
    
    case 4:   // Aerobic Training Effect, scale 10
                        active_session_["Aerobic Training Effect"] = QString::number(value/10.0);
    
    case 20:   // Anaerobic Training Effect, scale 10
                        active_session_["Anaerobic Training Effect"] = QString::number(value/10.0);
    
    case 9:   // Recovery Time, minutes
                        active_session_["Recovery Time"] = QString::number(round(value/60.0));
    
    case 17:   // Performance Condition
                        active_session_["Performance Condition"] = QString::number(value);
    
    case 14:   // If watch detected Running Lactate Threshold Heart Rate, bpm
                            active_session_["LTHR detected"] = QString::number(value);
    
    case 15:   // If watch detected Running Lactate Threshold Speed, m/s
                            active_session_["LTS detected"] = QString::number(value/100.0);
    

    Does anyone see where the "recovery heart rate" is stored? Is it stored? Have to go through my numbers to search.

    aha! it's in there

    event (21, type: 5, length: 14 bytes):
      timestamp (253-1-UINT32): 2020-11-21T00:0:00 (00000000)
      data (3-1-UINT32): 106
      event (0-1-ENUM): recovery_hr (21)
      event_type (1-1-ENUM): marker (3)
      event_group (4-1-UINT8, INVALID): 255
    

  • Somewhat interesting discovery now that I know to look for message id 140 field 7 for vo2max in activity file

    Seems like my fenix5 estimates/records a vo2max for cycling activities even though I don't have a power meter (but hrm-tri and cadence/speed sensor so maybe that makes a difference)

    Just not displayed in garmin-connect for cycling.

    Not that I think it should be included in your charts, but maybe someday if there is a cycling vs running vo2max chart.

    This is all pretty neat. I am working on a native PHP version that doesn't need the SDK

  • I'm still looking to figure out where garmin/firstbeat's version of TRIMP is hidden "EPOC"

    Has to be in global 140 with all the others.

    I doubt it stores the weekly average, probably only for that session in the FIT file and then the watch must keep a flying total average somewhere else.

    Since it based on field 4 and 20 and it existed before they made a model that tracked anaerobic 20, it's probably under 20

    So we know it's not 4,7,9,14,15,16,17,20,27,28,29

    Knocking out the zero values in multiple files, so it's not 1,11,12,13, 10 is the same in multiple files

    0, 8 and 18 are all only 8bit values, 8 is signed which is odd and I doubt load can be negative

    18 is too low some days so it can't be load.

    That leaves fields 2 3 5 and 6 as 32bit signed, those number are very big and no clue what the multiplier is, maybe some variant of 32768 or 65536

    I think it might just be field 0, it's a small unsigned 8bit integer and I think that's how much my weekly load goes up every day, I have to double-check my screenshots.

    That field might be the rounded reduced version of one of the 32bit fields holding the unrefined raw load just like metmax is raw vo2max

    Oh wait no, field 0 is the recovery heart rate, so it's the undocumented original of event21 recoveryhr

    field 8 is also too low, not sure what it is though, curious

    that leaves 2,3,5,6

    still can't figure it out but wanna see something cool, apparently it's possible to flip settings on the watch via FIT files places into the NEWFILES directory

    support.firstbeat.com/.../360015729193-How-to-Set-on-the-RR-recording-for-Older-Garmin-Devices

    enable_hrv_settings_file.fit

    file_id (0, type: 0, length: 2 bytes):
      type (0-1-ENUM): settings (2)
    hrm_profile (4, type: 0, length: 2 bytes):
      log_hrv (2-1-ENUM): 1

    Maybe that means it's possible to save/restore settings on garmin watches that are lost during hard resets.

  • Just to add my two cents to this, I think the "precise" VO2Max value (as seen in the FIT file and runalyze) is a moving average (based on the activities from some time period up to and including the current activity) and not an absolute value. I base this on the fact that no matter how slow / bad a given run is, the value of the "FIT from file" in runalyze never deviates much from the previous activity's value. OTOH, the estimated VO2Max that runalyze calculates can swing wildly from activity to activity depending on the actual performance.

  • more EPOC / training load detective work

    looking at the numbers from day to day was too confusing, wasted too much time on that

    here are three 5K repeats in a row, less than 30 minutes apart from each other, so load would only be increasing, I've left field 9 in there for recovery minutes as a gauge but removed the other fields we know about, actually I guess we know field 4 is aerobic effect / 10

    field 2 and field 5 actually drop inbetween, so that's not load, can't be, though I ran that repeat slower due to bad wind

    so field 3 and 6 are suspect

    field3   5425236   6039806   6782002

    field6  17466005  19198070  19908795

    field 3 goes up by almost same amount 614570 and then 742196

    field 6 goes up by 1732065 and then  710725, that's too big of a jump

    so field 3 is very suspect

    it's likely the raw load for the activity, I just cannot figure out the multiplier

    oh the training load on the watch after each one read: 

    752   835   927

    so what is the relationship to

    752 = 5425236  |  835 = 6039806  |  927 = 6782002

    remember that's over 7 days and the watch drops the realtime load daily on the hour the week-ago activity expires, but the current FIT never changes and the value is stored on connect.garmin

    it's interesting to note EPOC is measured in ml/kg by firstbeat

    they also track TRIMP in their own systems, I am not sure if on the watch

    but that means the runner's weight is used from the profile elsewhere in the FIT ?

    adding: look at firstbeat's patents on "readiness index" https://patents.justia.com/patent/20160184637

    https://patentimages.storage.googleapis.com/2a/e0/67/f359686c677dec/US10238915.pdf

    now that is facinating and a good lead

      unknown2 (2-1-SINT32): 4142250
      unknown3 (3-1-SINT32): 5425236
      unknown5 (5-1-SINT32): 157813
      unknown6 (6-1-SINT32): 17466005
      unknown9 (9-1-UINT16): 1106
      unknown4 (4-1-UINT8): 29
      unknown8 (8-1-SINT8): 17
      unknown18 (18-1-UINT8): 21
      unknown2 (2-1-SINT32): 4021790
      unknown3 (3-1-SINT32): 6039806
      unknown5 (5-1-SINT32): 94401
      unknown6 (6-1-SINT32): 19198070
      unknown9 (9-1-UINT16): 1164
      unknown4 (4-1-UINT8): 30
      unknown8 (8-1-SINT8): 22
      unknown18 (18-1-UINT8): 97

      unknown2 (2-1-SINT32): 4615547
      unknown3 (3-1-SINT32): 6782002
      unknown5 (5-1-SINT32): 254004
      unknown6 (6-1-SINT32): 19908795
      unknown9 (9-1-UINT16): 1222
      unknown4 (4-1-UINT8): 31
      unknown8 (8-1-SINT8): 22  
      unknown18 (18-1-UINT8): 100

  • okay I am onto something tracking down EPOC on the watch

    I caught my watch sync-ing with garmin connect

    it made a special FIT file to transfer data, they are stored in \GARMIN\METRICS

    it uses global 232 to track changes in firstbeat metric, but it sends the calculations and not the original numbers from the watch

    it sends the "before" and the "after" for the activity!

    field number 5 is the load before the activity, and then the load after, it matches my screenshots exactly

    the other metrics stay the same, which is odd but they are not the loads from previous day so no idea

    unknown (232, type: 4, length: 18 bytes):
      xxx253 (253-1-UINT32): (timestamp before)
      xxx5 (5-1-UINT16): 985   <<<<<<<<<<<< load before
      xxx6 (6-1-UINT16): 503
      xxx7 (7-1-UINT16): 1088
      xxx8 (8-1-UINT16): 1312
      xxx0 (0-1-ENUM): 1
      xxx1 (1-1-ENUM): 0
      xxx2 (2-1-ENUM): 7
      xxx3 (3-1-ENUM): 2
      xxx4 (4-1-ENUM): 2

    unknown (232, type: 4, length: 18 bytes):
      xxx253 (253-1-UINT32): (timestamp after)
      xxx5 (5-1-UINT16): 1082   <<<<<<<<<<<< load after
      xxx6 (6-1-UINT16): 503
      xxx7 (7-1-UINT16): 1088
      xxx8 (8-1-UINT16): 1312
      xxx0 (0-1-ENUM): 1
      xxx1 (1-1-ENUM): 0
      xxx2 (2-1-ENUM): 7
      xxx3 (3-1-ENUM): 2
      xxx4 (4-1-ENUM): 2

    Based on firstbeat whitepapers, this is how they calculate EPOC

    EPOC in ml/kg = (average vo2 at recovery X seconds of run) - (vo2 baseline X seconds of run)

    We have vo2max baseline in field 7, we know the length of the run in seconds, one of those numbers in 140 is the recovery vo2

    The thing is the number would have to be bigger than the vo2max, assuming its stored the same way with the 65536/3.5 multiplier

    EPOC

  • I think I am onto something with figuring out EPOC 7-day training load on the watch from the FIT data global 140

    140,6 looks to be some kind of workload for the activity

    if you take 140,6 and divide it by 140,5 and then add up those numbers for 7 days and then divide by 2 you get a number really close to what the watch shows for 7-day load, but it's not exact - however the daily and weekly number seems to change directly related

    you can also see the weekly load here in garmin-connect though garmin refuses to share the daily breakdown

    https://connect.garmin.com/modern/report/activity_training_status/all/last_four_weeks

    some watch models apparently display the number as a daily breakdown but not mine so it's hard to compare but the number is darn close

    it may be that the final divisor is not exactly "2" but I just can't figure out what it is based on