First list of FIT decoding libraries and basic benchmarking

It's currently a cli tool where one of the functionalities is to inspect a fit-file. The fit parsing code is isolated enough to become a library, but I'm quite new at this. Tests, docs, sorting out some spaghetti-parts is required for it to become a lib. It's currently not intentionally optimised, save for a re-structure of how I store extracted basetypes, so I should probably check that it's idiomatic Rust or if there are further optimisations or simplifications to be made as well. As a simple tool it can be released as is. Will happen sometime after New Years. 

The project I'm in have an underused gitlab repo it will be released on. It'll be Open source, but I'm sure anyone looking a my code will have to use eye bleach afterwards - does work so far though!

I think your reported speed are quite good. nicely done. definitely you should release it.

On the M1, after checking it and realizing it beats my MacBook Pro, I decided to check on an iPhone 12, as the main use case of my library is on iPhone... And the iPhone 12 is on par with the m1! and hands down beats the MacBook Pro intel i9 in parsing fit files.. quite amazing...

Detailed discussion of this may be outside this thread or even garmin forums but I just wanted to share that I think I figured out how websites do segment identification quickly, or could be done even more quickly

The trick is don't try to identify the segment start or end or waypoints, it takes too long to scan all records vs all segments

Instead do lat/lon bounding boxes

I got the idea while studying FIT global 18 sessions after being annoyed that Garmin doesn't save the lat/lon endpoint of the session in the data and instead you have to look at the last record (heck some other brands don't even bother storing the start lat/lon in the session and you have to also check the records)

Instead Garmin stores the upper NE corner and lower SW corner lat/lon of the session, it's a bounding box for map drawing, etc.

  nec_lat (29-1-SINT32)
  nec_long (30-1-SINT32)
  swc_lat (31-1-SINT32)
  swc_long (32-1-SINT32)

The same trick could be done for segments

It's relatively easy and fast to check if a smaller box is completely inside a larger box and then you do a more detailed scan comparison

Pretty exciting to figure this out, now the trick is to code it which is not so easy or exciting lol

Wait, "timestamp ms" - is that fractional timestamps? what watches do that because that is exciting.

Is that in conventional devices like Fenix5 and I am just missing it?

Oh is that just more Virb data? That's a great device if so, I've got to look into getting one. It probably has much more processing power than a watch so they store a lot more data like accelerometer and fractional seconds. Would be neat if next watch series also did that, cpu and storage is plentiful now. I'd also love to see L5 GPS ability for 1 meter accuracy.

By the way, here's another fun FIT file to analyze from DC Rainmaker when he reviewed the Fenix 6X Pro Solar

analyze.dcrainmaker.com/

https://dcranalyzer-qxyvqnghga-uc.a.run.app/api/v1/set/d807bb6a-ee1c-41b4-676d-7192ed9edfc0/files/set.zip

I like it because it's one of the few files that takes more than a second to process because it has 16,000+ records, he was climbing a glacier very slowly at altitude!

He's a good source for other public FIT files from other manufacturers so you can see variances in how they store data.

timestamp_ms is the millisecond part of the [relative] timestamp. Virb stores timestamps in seconds counted from when the camera was turned on (meaning you have telemetry outside of recording video, which a great feature imo). So as mentioned above, you need to augment timestamps with a correlation value to get absolute timestamps.

For the Ultra 30 gps_metadata (the good, full gps data, as opposed to "record") is logged at 10Hz, whereas the sensors, such as accelerometer, log up to 30 data points in bursts, each 300ms (the sensor data contains ms offsets for each sample).

Ok that's nice to hear, thanks! I'll try my hands at a library but need to find ways of macroing away some verbose code (if possible) for turning "raw" fit base type values into e.g. decimal degrees.

The power of small devices todays is mind boggling to me. Just a bit sad that web tech (and locked up mobile tech) is seeing more progress than open, native ditos. Computers are fast, software not so much. :)

This is valuable. I also want a solar watch. :) Don't know whether this is ok or not, considering most files contain location + date time, but I'd like to make a large library of test files. Broken/corrputed, large, small etc.

I get (as usual for extracting base types, but converting to more usable form is very quick so far):

SUMMARY
---------------------------------------------------
Header

      size: 14
  protocol: 16
   profile: 2096
  datasize: 275423
    dotfit: ['.', 'F', 'I', 'T']
       crc: Some(8523)
---------------------------------------------------
Data

 Global ID | Message type                 | Count
...................................................
         7 | zones_target                 |      1
        19 | lap                          |     10
        13 | UNDEFINED_MESSAGE_TYPE_13    |      1
       141 | UNDEFINED_MESSAGE_TYPE_141   |      1
         0 | file_id                      |      1
        18 | session                      |      1
       140 | UNDEFINED_MESSAGE_TYPE_140   |      1
        23 | device_info                  |     12
       288 | UNDEFINED_MESSAGE_TYPE_288   |      8
        22 | UNDEFINED_MESSAGE_TYPE_22    |      3
         2 | device_settings              |      1
        72 | training_file                |      1
         3 | user_profile                 |      1
        34 | activity                     |      1
       104 | UNDEFINED_MESSAGE_TYPE_104   |     54
        12 | sport                        |      1
       233 | UNDEFINED_MESSAGE_TYPE_233   |  16108
        49 | file_creator                 |      1
       113 | UNDEFINED_MESSAGE_TYPE_113   |      4
        79 | UNDEFINED_MESSAGE_TYPE_79    |      1
        21 | event                        |    108
        20 | record                       |   5474
       216 | UNDEFINED_MESSAGE_TYPE_216   |     11
...................................................
                                    Total:   21805 
---------------------------------------------------
UUIDs in file

  None
---------------------------------------------------
Result

  Message types 160, 161, 162 (*) present: NO
  Parsed in full, no errors
---------------------------------------------------
Done (0.050s)

SUMMARY
---------------------------------------------------
Header

      size: 14
  protocol: 16
   profile: 2096
  datasize: 633325
    dotfit: ['.', 'F', 'I', 'T']
       crc: Some(63730)
---------------------------------------------------
Data

 Global ID | Message type                 | Count
...................................................
        23 | device_info                  |     12
        21 | event                        |    111
       233 | UNDEFINED_MESSAGE_TYPE_233   |  16056
        22 | UNDEFINED_MESSAGE_TYPE_22    |      6
       113 | UNDEFINED_MESSAGE_TYPE_113   |      4
        19 | lap                          |     10
        34 | activity                     |      1
        49 | file_creator                 |      1
       104 | UNDEFINED_MESSAGE_TYPE_104   |     53
       141 | UNDEFINED_MESSAGE_TYPE_141   |      1
       140 | UNDEFINED_MESSAGE_TYPE_140   |      1
        18 | session                      |      1
        13 | UNDEFINED_MESSAGE_TYPE_13    |      1
        72 | training_file                |      1
         0 | file_id                      |      1
         3 | user_profile                 |      1
        20 | record                       |  16024
       216 | UNDEFINED_MESSAGE_TYPE_216   |     11
         7 | zones_target                 |      1
         2 | device_settings              |      1
        79 | UNDEFINED_MESSAGE_TYPE_79    |      1
        12 | sport                        |      1
       288 | UNDEFINED_MESSAGE_TYPE_288   |      9
...................................................
                                    Total:   32309 
---------------------------------------------------
UUIDs in file

  None
---------------------------------------------------
Result

  Message types 160, 161, 162 (*) present: NO
  Parsed in full, no errors
---------------------------------------------------
Done (0.094s)

By the way, the "record"/20 message contains temperature (field 13) for both watches! Just a nod to your earlier comments above. E.g.:

[16024] Global ID: 20 | Message type: record | Header: 2/0b00000010
    id: 253 timestamp             : UINT32([935330443]) 
    id:   0 position_lat          : SINT32([547987933]) 
    id:   1 position_long         : SINT32([82530551]) 
    id:   5 distance              : UINT32([1519922]) 
    id:  73 enhanced_speed        : UINT32([1101]) 
    id:  78 enhanced_altitude     : UINT32([12101]) 
    id:  87 UNDEFINED_FIELD_87    : UINT16([0]) 
    id:  88 UNDEFINED_FIELD_88    : UINT16([300]) 
    id:   3 heart_rate            : UINT8([131]) 
    id:   4 cadence               : UINT8([0]) 
    id:  13 temperature           : SINT8([26]) 
    id:  53 fractional_cadence    : UINT8([0]) 
    id:  90 UNDEFINED_FIELD_90    : SINT8([-20])