Why is this project so awful?

I am trying to pull data from an Instinct Solar for a little side project. Nothing fancy, just want some basic data so I can correlate metrics like heart rate with another data stream.

As a software engineer, every step of this experience has been horrifying.

1. I couldn't just download CSVs of individual data timeseries. Even though Garmin Connect does the conversion, and shows me a graph, I can only download the original FIT binary data. What end user could have any conceivable use for this?

2. Converting these FIT files requires use of a poorly documented SDK and command line tool.

3. These files have evolved over time, without documentation or logs, making utilities created by individuals as recently as 2018 completely useless (see https://maxcandocia.com/article/2017/Sep/22/converting-garmin-fit-to-csv/). See other threads in this forum with people making comments to the effect of "oh I guess they changed it so I'm looking through previous versions of the SDK to do so-and-so..."

4. The protocol data is inexplicably confusing. Many data streams don't even use the correct field names, structure is malformed, conventions not followed, timestamps are unix epochs shifted by 20 years, except for timestamp_16s which are relative to... Something? I still can't tell what they are relative to. There's no excuse for any of this, as these bluetooth messages have no real data limitation, and the difference between a normal unix epoch at a timestamp_16 is all of 32 bytes (tops).

So what gives? How are people dealing with this? Has Garmin not learned their lesson? Security by obscurity is not an option, and these problems are not just surface-level frustrations. They're indicative of a deeply flawed engineering organization. Garmin needs to pivot immediately to leverage their community and start pushing updates that make these devices useful and secure.

0 dpawlyk over 3 years ago

8213573 said:
See other threads in this forum with people making comments to the effect of "oh I guess they changed it so I'm looking through previous versions of the SDK to do so-and-so..."

There was one such thread and that person was wrong. The thing he thought was removed was never present in the SDK.

I wonder if something like Golden Cheetah would do what you need.

0 Ben FIT over 3 years ago

If your goal is to get the data from a FIT file in to a spreadsheet application, then using the FIT CSV Tool is a good place to start. The tool is meant for debugging, but in the tool’s documentation https://developer.garmin.com/fit/fitcsvtool/commandline/ there is an example showing the use of the “—data” option that will output a file with just the record messages, which is probably what you are looking for. If you are using Windows, there is a bat file “FitToCSV-record.bat” that runs the FIT CSV Tool with the “–data record” option. If you are on OSX, there is documentation on how to create an Automator app for running the FIT CSV Tool and shows the use of the “—data” option.

In the FIT SDK Cookbook there is a recipe for working with FIT Epoch values, which coincidentally is the date that Garmin was founded. Each SDK provides a Date Time class that handles the conversion from the FIT Epoch to a language specific date time object, and in many cases the developer does not need to work directly with the FIT Epoch values. But that depends on how the data is being used. For example, if you want to get a relative time offset from the beginning of the activity, then subtracting epoch values is more efficient than subtracting date time objects.

https://developer.garmin.com/fit/cookbook/datetime/

If your goal is not to get the data in to a spreadsheet application, but rather write your own application to perform the analysis, there is a recipe in the Cookbook for decoding FIT Activity files. The recipes are written in C# but the concepts apply to each SDK. This project also writes out a CSV file, so if you want to customize the CSV file for your own needs this would be a good place to start.

https://developer.garmin.com/fit/cookbook/decoding-activity-files/

If you do not want to use C#, each SDK has an example project for decoding FIT files. Those example projects are generic and work with all FIT files, not just Activity files. There are many open-source projects that support additional languages as well.

https://developer.garmin.com/fit/example-projects/

The binary format itself has only had on breaking change, and that was back in mid-2017. SDKs released prior to the change cannot read files created after the change, but the SDKs released after the change can read both versions of the file. At this point there are not that many maintained projects that have not been updated to handle change.

The majority of updates to the SDKs involve adding new values to existing enums, exposing new fields in existing messages, and on occasion adding new messages. As a rule, once something has been added to the SDK it is not removed.