First list of FIT decoding libraries and basic benchmarking

Hi,

Just wanted to share, I started to put together a list of libraries that decode fit files and a simple benchmarking of the performance of each as for my own application performance is an issue.

Hopefully can be useful to others as reference, feel free to add other libraries or suggest other to look at.

https://github.com/roznet/fit-benchmarks

  • The device appends the data from the HRM to the end of the original Activity file created by the device. The device does not merge the data. This is know as Chained FIT Files. In this case the second FIT file, which is from the HRM, contains a FileId message and one or more HR messages. Each HR message containing 8 HR values. The merging of the data happens in post processing. The SDK contains the HrToRecordMesgBroadcast plugin that handles the merging of the HR values into the Record messages.

    If you open your test file in a hex/text editor you should see a second file header with the ".FIT" signature in it.

    More info on Chained FIT files and the Plugin Framework can be found here: https://developer.garmin.com/fit/protocol/


    An example of how to use the HrToRecordMesgBroadcast can be found here (at the bottom of the page). Plugins can be expensive at runtime, especially if you do not encounter a lot of files that require the logic in the plugin, and at the very end there is a discussion about using the logic from the plugin without using the plugin itself. 
    developer.garmin.com/.../

  • Found some more interesting public FIT files for testing on another review site like Rainmaker's

    https://the5krunner.com/2016/11/05/test-route-for-gps-devices/#Course_Description

    the dropbox link is right there

  • Oh wow "chained FIT" is fascinating, just wish it didn't violate the header filesize which violates backwards compatibility?

    I guess since it came out in 2015 that should be enough time for most decoders to handle it, I'll have to add it to my decoder, if there is a filesize vs data declaration mismatch, I'll check if that extra "garbage" is actually another FIT

    In theory I could use it to append shoe and gear data to FIT files, extracted from Strava, etc.without tampering with the original data.

  • The size given in the header is the data size not the file size. It does not include the size of the header or crc, so it is always less than the file size. In the parser, if you are done reading the header + data size + crc and there are still more bytes to read then you can treat that as the start of of the next FIT file and read the FIT header. Chained FIT files allow you to concatenate FIT files. They are not meant to append other binary data.

  • So chained files are literally FIT-files concatenated as-is? This means all one would have to do is to compare file size with total fit size of the first FIT-file (and so on), get the new offset and just continue parsing as normal? Sounds good to me. I assume the chained data is meant to be read in parallel (i.e. time will restart for each chained file)?

  • Great find! That's a lot of FIT-files AND a wide device selection.

  • That is correct. They are literally just FIT files concatenated, or chained, together. Currently the only use in the Garmin ecosystem for chained fit files is to chain the FIT file from an HRM. In this case the data in the two files is recorded in parallel and then chained into a single file by the wearable. 

  • In that FIT-pack, does the file `2019-05-07\ FR945\ Galileo/CV\ Run20190506184615.fit` parse ok for you? I had to add an alignment check (= performance loss) because of a 32-bit value (according to definition) that's defined as 1 byte in length. Error seems to be in a data message starting at offset 65203/65708 of the data part so presumably starting at byte offset 65217 for the file.

    Below a "debug" print I added where "DEF" is the definition (parsed underneath at "PARSED"), "DAT" is the data message. The message type is "event"/21 and the problematic field is "data"/3 which should be a uint32, i.e. field length dividible by 4. Wonder if I missed something.

    [65182/65708] HEADER: 0b01000000
      DEF U8 [64, 0, 0, 21, 0, 5, 253, 4, 134, 0, 1, 0, 1, 1, 0, 4, 1, 2, 3, 1, 134]
      DEV U8 None
      PARSED DefinitionMessage {
        header: 64,
        reserved: 0,
        architecture: 0,
        global: 21,
        definition_fields: [
            DefinitionField {
                field_definition_number: 253,
                size: 4,
                base_type: 134,
                field_name: "timestamp",
                units: None,
                scale: None,
                offset: None,
            },
            DefinitionField {
                field_definition_number: 0,
                size: 1,
                base_type: 0,
                field_name: "event",
                units: None,
                scale: None,
                offset: None,
            },
            DefinitionField {
                field_definition_number: 1,
                size: 1,
                base_type: 0,
                field_name: "event_type",
                units: None,
                scale: None,
                offset: None,
            },
            DefinitionField {
                field_definition_number: 4,
                size: 1,
                base_type: 2,
                field_name: "event_group",
                units: None,
                scale: None,
                offset: None,
            },
            DefinitionField {
                field_definition_number: 3,
                size: 1, <-- THIS MUST BE WRONG
                base_type: 134,
                field_name: "data",
                units: None,
                scale: None,
                offset: None,
            },
        ],
        developer_fields: [],
        data_message_length: 9,
    }
    [65203/65708] HEADER: 0b00000000
      DAT U8 [0, 212, 59, 51, 55, 0, 0, 0, 0]

  • With the speed of modern browser javascript compilers these days (and the performance "war" between Firefox and Chrome) I am really curious to see how that JS decoder someone posted in another thread compares to the other languages.

    180K is not a trivial amount of javascript before all the memory requirements but it's impressive that it is done completely in the browser.

    https://forums.garmin.com/developer/fit-sdk/f/discussion/254794/javascript-parser

    https://github.com/backfit/backfit/issues/6

  • I added this javascript library to the benchmarks https://github.com/roznet/fit-benchmarks

    it's similar speed to the python library.