Reading developer fields using the C SDK

Decoding developer data is not supported by the FIT C SDK. If switching to the C++, or other, SDK is not an option then the following information may help you implement this on your own.

First look at the Activity Encode examples in the C# or Java SDKs to see how developer data is written to the file. This will help understand how to decode it.

When adding developer data to a file, there are one or more Developer Data Id messages written to the file that identifies the app or data field that is contributing the developer data. There may be more than one. These will have a zero-based index that is unique within the file. Then there is one or more Field Description messages that describe the data. These messages store the id of the Developer Data Id message that they are related to, and a zero-based index that is unique per Developer Data Id. Both ids show up later in the message definition.

These messages are written to the file before adding developer data to a message. This is so that the information exists ahead of time on how to decipher the developer data.

Next review the Definition Message section in the protocol doc, and the sub-section on the Developer Data Field Description.

developer.garmin.com/.../

The message definition refers to both a Developer Data Id message and Field Description message that should be used interpret the developer data in each message. These are the messages that I mentioned earlier.

In the C decoder, these three cases are handling decoding of the developer data field definitions from the message definition

case FIT_CONVERT_DECODE_DEV_FIELD_DEF: // Currently ignores the value

case FIT_CONVERT_DECODE_DEV_FIELD_SIZE: // Keeps track of the total size, so it can read past the developer fields later

case FIT_CONVERT_DECODE_DEV_FIELD_INDEX: // Currently ignores the value

You will need to store these three values somewhere along with the local message id so they can be related back to the message definition that they are part of. There may be more than one developer data field present in a message, and the order of the definitions needs to be maintained. To do this, you will need to figure out the data structures that work best in your app. You should be able to save off this information without affecting the rest of the decoder.

Then back in the C decoder, this is where the developer data is read-past when reading in a message. This data needs to be captured and then interpreted based on the Field Description messages.

case FIT_CONVERT_DECODE_DEV_FIELD_DATA:

This data will need to be captured and then interpreted based on the message definition, Developer Data Id messages, and Field Description messages. This is another place where you will need to figure out the data structures that work best in your app. You should be able to read this data and store it without affecting the rest of the decoder. Assuming you read the correct number of bytes.

In the Protocol doc look at the FIT File Example section. It should help visualize what all of this looks like.

Many thanks for the detailed response!  I already had been processing the  FIELD_DEV FIELD_SIZE and FIELD_INDEX cases, but it wasn't quite clear to me as to how the field data was actually going to get delivered. Thanks to your response I got a first version working.

I added a few additional fields to the FIT_CONVERT_STATE structure which allow me to track the individual field sizes (as opposed to just the sum of the sizes that is already stored in dev_data_sizes) and allocate sufficient space to store the field data. Now when my application processes the FIT_MESG_NUM_RECORD message it goes ahead and reconstructs the developer fields from the stored field data in the convert state in conjunction with the DEV_FIELD data type information.

I have one question regarding string and byte data types. The documentation states that the string type developer field is null terminated. However, reading the decode source, a flexible data size isn't possible. If the string would somehow be of non-fixed length, how does the decoder know when it is finished with the developer fields?

Below is the modified FIT_CONVERT_DECODE_DEV_FIELD_DATA case in decode.c where the field data gets stored. Flexible string lengths don't appear to be possible because dev_data_sizes is fixed (set in the FIT_CONVERT_DECODE_DEV_FIELD_SIZE case of decode.c).

         case FIT_CONVERT_DECODE_DEV_FIELD_DATA:
            state->dev_data[state->dev_data_index] = datum;
            state->dev_data_index++;
              
            state->field_offset++;
            if (state->field_offset >= state->dev_data_sizes[state->mesg_index])
            {
               // Done Parsing Dev Field Data
               state->decode_state = FIT_CONVERT_DECODE_RECORD;

               // We have successfully decoded a mesg and there is no dev data to read.
               return FIT_CONVERT_MESSAGE_AVAILABLE;
            }
            break;

The same applies to byte arrays. I don't have FIT files available for testing that contain those two types, maybe you could point me into the right direction how it is supposed to work?

Thanks again!

Best Regards

Herb

Correction: "decode.c" should of course be "fit_convert.c"

Regards

Herb

Make sure to read all three values from the definition. If there are multiple CIQ data fields contributing data to the file, DEV_FIELD_DEF will not be unique. The DEV_FIELD_INDEX identifies the app, and DEV_FIELD_DEF identifies a field written by the app. More than likely each app will start numbering its fields at zero.

The field size is fixed for all string and array fields, so the decoder must read that number of bytes. For strings, calling strlen() on that block of memory may return something less than the field size. It is possible, although I am not sure if it exists in the wild or not, to have an array of strings. The corresponding field_description message will tell you if it is an array or single value.

To test, in the app store look for data fields asking for permission to “Record additional information into activity files”. That will let you know if the data field is contributing data to the file. Then add a few of those to a device and record an activity. There should be multiple DEV_FIELD_INDEX values and more than likely multiple DEV_FIELD_DEF values that are the same.

I am pretty sure that this app uses arrays for at least one field.

apps.garmin.com/.../c5d949c3-9acb-4e00-bb2d-c3b871e9e733

Ben FIT said:

Make sure to read all three values from the definition. If there are multiple CIQ data fields contributing data to the file, DEV_FIELD_DEF will not be unique. The DEV_FIELD_INDEX identifies the app, and DEV_FIELD_DEF identifies a field written by the app. More than likely each app will start numbering its fields at zero.

So just to be clear I understand correctly, you mean I should make sure that developer_data_index of the respective field in FIT_FIELD_DESCRIPTION_MESG  matches the developer_data_index I'm getting throught FIT_DEVELOPER_DATA_ID_MESG so I can positively identify the field and know it comes from the vendor I'm interested in. Is that it?

The state->field_index in FIT_CONVERT_STATE is numbered sequentially over all fields that are reported, regardless of the developer. Also, the developer data arrives as one long stream of bytes, the fields are only distinguishable by using the field definitions. That's my understanding anyway. I'm quoting the definitions of the structs below for clarity.

Regards

Herb

typedef struct
{
   FIT_STRING field_name[FIT_FIELD_DESCRIPTION_MESG_FIELD_NAME_COUNT]; //
   FIT_STRING units[FIT_FIELD_DESCRIPTION_MESG_UNITS_COUNT]; //
   FIT_FIT_BASE_UNIT fit_base_unit_id; //
   FIT_MESG_NUM native_mesg_num; //
   FIT_UINT8 developer_data_index; //
   FIT_UINT8 field_definition_number; //
   FIT_FIT_BASE_TYPE fit_base_type_id; //
   FIT_UINT8 scale; //
   FIT_SINT8 offset; //
   FIT_UINT8 native_field_num; //
} FIT_FIELD_DESCRIPTION_MESG;


typedef struct
{
   FIT_BYTE developer_id[FIT_DEVELOPER_DATA_ID_MESG_DEVELOPER_ID_COUNT]; //
   FIT_BYTE application_id[FIT_DEVELOPER_DATA_ID_MESG_APPLICATION_ID_COUNT]; //
   FIT_UINT32 application_version; //
   FIT_MANUFACTURER manufacturer_id; //
   FIT_UINT8 developer_data_index; //
} FIT_DEVELOPER_DATA_ID_MESG;

What is a bit strange is that in the .FIT files I'm working with the manufacturer_id in the FIT_DEVELOPER_DATA_ID_MESG is always FIT_MANUFACTURER_INVALID. The file was recorded with a FR745 HRM PRO and external Stryd sensor (whose data I'm interested in). I also have older test files recorded with an FR735XT + Stryd sensor that has developer fields that exhibit the same problem. Not sure if I'm looking at the wrong thing here.

Regards

Herb

The CIQ sub-system on the devices does not populate the manufacturer field. Instead, look at the developer_id field to identify either the CIQ app or CIQ data-field that wrote the data. 

You mentioned Stryd, so I will use them as an example. This is the URL to the Stryd Zones data-field

apps.garmin.com/.../18fb2cf0-1a4b-430d-ad66-988c847421f4

This is unique id that identifies the app

18fb2cf0-1a4b-430d-ad66-988c847421f4

This is the binary version. Note that the binary guid format is one uint32 value, three uint16 values, and then six 1 byte values. The uint32 and uint16 values are all little-endian. 

f0 2c fb 18 4b 1a 0d 43 ad 66 98 8c 84 74 21 f4

That is what the developer_id field should be equivalent to in the FIT_DEVELOPER_DATA_ID_MESG. That is not going to change, so for an efficient comparison you could treat that as four uint32 values.

Depending on what the requirements are, you might be able to check that the sport type is run and that there is power data in the record messages, and not specifically look for a list of matching developer_id values.

For your earlier question, that is correct. You need to look at both the field_definition_number and the developer_data_index in the FIT_FIELD_DESCRIPTION_MESG to identify the data. If there are multiple CIQ data-fields contributing data to the file, then there will probably be multiple field_definition_number with the value zero, but each will have a different developer_data_index value.

Showing how the applID is formed from a GUID has been exceptionally instructive for what I was hunting for, but Ben FIT I'm suspecting (but not sure) that the bytes of the 3rd uint16 are in the wrong order.  Shouldn't the binary bytes for the specified unique ID be:

f0 2c fb 18 4b 1a 0d 43 66 ad 98 8c 84 74 21 f4   (?)

A GUID is a data structure that resembles:

{

  long

  short

  short

  char[8]

}

FIT does not have a GUID data type, so in a FIT file GUIDs are stored as byte arrays. I suggest using the available GUID class or library in the language you are using to convert GUIDs to/from byte arrays. That way you do not need to worry about the endianness of the long and short values. The class/library will take care of it for you.