Where can I find definitions for the large proportion of messages types generated by my Garmin devices which are not defined in Profile.xlxs?

In the FIT files which I download from Garmin Connect which are generated by either my Epix gen2 or my Fenix 6 watch, there are 11 of 30 message types which do not have a mesg_num defined in Profile.xlxs. Does anyone know where these are defined and how you could generate an SDK which actually works without a full definition?

For my purposes, I don't actually need to decode and encode the unknown fields because I only want to fix some mesg_num = record = 20 messages.

However, fields that will not decode cannot be re encoded into an output file and Garmin import throws an error message if you try to upload a FIT file that is missing these undefined fields.

On the other hand, Strava doesn't care that these fields are missing.

I noticed that the Muktihari's Golang FIT SDK provides a raw decode encode facility which would allow passthrough of unknown fields.

However, I'd like to use the Garmin C++ FIT SDK if it were possible.

Also, the Goland SDK notes in its profile doc that:

"supplementary items related to "Global Profile" that are not
declared in the Profile.xlsx itself but are provided by Garmin
or its affiliates website, such as common file types,
are added manually."

I couldn't find these manual additions.

As an example of the astounding number of undefined messages in a file generated by my Garmin watch see below:

- Undocumented FIT_MESG_NUM according to FIT File Viewer
- In a run activity file from 2/28/2024
- Out of 30 message types 11 are undocumented
- mesg 288 3 rows of timestamps + 3 integer fields
- mesg 327 4 rows of timestamps + 6 integer fields
- "GPS Events" 34 rows of timestamps + event type + mode
- mesg 147 3 rows of 11 fields
- mesg 79 1 row of timestamp + 41 fields
- "Other Settings" 1 row of 50 fields
- mesg 325 104 rows of timestamps + 6 integer fields
- mesg 104 5 rows of timestamps + 4 integer fields
- mesg 233 196 rows of 1 field
- mesg 140 1 row of timestamp + 65 fields
- mesg 133 3 rows of timestamps + 6 fields

  • Check Integrity looks at the header and the CRC, it does not look at the data in the file. 

  • Running Decode::CheckIntegrity on my re encoded file did show me a problem. The decoder doesn't just discard unknown messages but rather returns a message with a null profile. When this is passed to the encoder to write, the encoder has a null validator and so skips validation and writes gibberish. All I had to do to fix it is to return from OnMesg without re encoding unknown messages by checking  mesg.GetNum() == FIT_MESG_NUM_INVALID).

    After the fix, I no longer see gibberish in place of unknown messages if I fitprint or fitdump and CheckIntegrity and passes.

    Also, FitTestTool still passes.

    Sadly, Garmin still rejects the file. 

  • Yes, CheckIntegrity doesn't catch the problem but it does call ReadByte on the entire file and ReadByte throws an exception when it tries to decode data for which it doesn't have a matching message definition.

    Perhaps, if I run a full decode on my current re encoded file, it will throw an exception and lead me to the remaining problem.

    Thanks a lot for getting me into a more productive line of thinking.

    I know I would have had fewer problems if I constructed the file by repackaging the records like the other Fix utility does. But, it just seems like it should work this way as well and ultimately require a lot less code.

  • Oh well, I had no luck finding any more errors with the decoder. Apparently, the decoder used by the Garmin Connect service is quite different from the one provided by the C++ SDK. Too bad. Explicitly repackaging all the known messages that you would like to keep in a file when you only need to alter one kind seems like a huge amount of unnecessary code.

  • FIT is a binary file format and it does not matter which SDK, provided by Garmin or a third party, is used to encode the file or which SDK, provided by Garmin or a third party, is used to decode the file. All of the SDKs are interchangeable when it comes to encoding and decoding files.

    After decoding a file, a platform may apply its own business logic to the data decoded from the file. Platform specific business logic is independent of the FIT SDKs.

  • I suppose it depends on whether you consider the "business logic" which categorizes the FIT file as invalid when uploaded to Garmin Connect and valid when uploaded to Strava as part of the decode process. I agree that the invalidating logic is not part of the publicly available decoders because I can successfully decode the file myself using the C++ SDK. But, unless I'm missing something, I have no access to these additional "business logic" validity checks. So, it is quite difficult to make the file conform to them. At this point, it seems that my only option is to work from the other direction by exactly following the example of the ActivityRepairTool.java to hopefully encode a minimal FIT file which Garmin will accept and then start adding additional messages types to preserve as much information from the original FIT file as possible.

    Anyway, I hope to get back to building a robust model for predicting heart rate at some point :->

  • This is great work. I was building an app to fix incorrect HR measurements in my fit files when I ran into these undocumented fields. Ultimately, I wrote an app that discarded all the unknown fields which was very unsatisfying. The fact that Garmin doesn't bother to keep dev resources up-to-date convinced me not to spend time on FIT file app development. I'm glad that you persevered.

  • There has always been unknown messages and fields in the FIT SDK, and the expectation is that there always will be unknown messages and fields. 

    With the newer SDKs, the Swift SDK that was just released and the encoder that was just added to the JavaScript SDK, you can decode a file and re-encode it without losing any of the unknown values. At the moment there are no plans to updated the other SDKs to do the same. We may add an encoder to the Python SDK that would mimic what the JavaScript encoder. There is no ETA for that right now.