FIT WRITE Solution: No Gaps for Invalid Data

Discussed in another thread is the unfortunate issue in which we can't generate a GAP in line graphs for our FIT User Data... for data that is invalid or unavailable. I have a solution that works for me. I have a metric to display Heart Rate Stress. This is relative to nominal HR values for a given sustained and steady power (watt) level. I won't go into the math and research. The point is, the metric can be from about -10 to +20 or so depending on your stress, temp, hydration level, etc. A value of ZERO means your HR matches the model or expected.

But if your power isn't steady or one of several other criteria, I can't generate a meaningful stress value. I wait until conditions are right to generate a data point. Setting it to ZERO is misleading...

So what I now do it oscillate from -0.5 to +0.5 when I'm in an invalid condition state. The graph produces a thicker bar around ZERO which means: "no valid data" during this timeframe. In my case this was a MTB ride and there were times my power level was surging and therefore my HR wasn't steady, and any stress value would be unreliable, so those are times I dropped into the oscilation mode. Also my HR Stress was in the + range because it was hot out today (over 80F) and I wasn't drinking enough. Often my HR Stress starts out in the negative range.

  • I've just wanted to ask about it. So can you confirm that it's impossible to write in fit file NULL value?

    I need some time to count metric so from beginning of activity I don't have anything to save so thought I save null.

    But what does do system's data field if watch can't read e.g. HR during activity? What to do of in compute info.currentHeartRate is null?  

    maybe system simple connects two valid points with line, no gap but line

  • Yes I can confirm. We've tried NULLS and we've tried the official INVALID values as defined by the FIT standard. Nothing works.

  • But what does do system's data field if watch can't read e.g. HR during activity? What to do of in compute info.currentHeartRate is null?  

    maybe system simple connects two valid points with line, no gap but line

    I think I've seen gaps from native data fields in the past, but it was a long time ago. I also never had a chance to look at what the data actually was in the FIT file. (i.e. Was it "no data" / gaps, or was it invalid values, or maybe something else?)

  • I need some time to count metric so from beginning of activity I don't have anything to save so thought I save null.

    If your gap is at the start of the activity, you can simply avoid calling FitContributor.setData() until you have data to write. This is an exception to the problem because of course if you never called setData() even once, the system has no idea what you want to write, so it won't write anything (afaik).

    If I recall correctly, I tried this in the past, and it works, although the specifics of how the "gap" is rendered may vary depending on whether you look at the activity in the Connect app or on the website.

    This thread is more than 4 years old >_<

    https://forums.garmin.com/developer/connect-iq/f/discussion/7509/fitcontributor-field-setdata-should-accept-null-as-a-value/1435857

    Now consider the case of a Connect IQ field recording the exact same data.

    If there's no HR data available at beginning of the run and the field hasn't called setData yet, no problem. The records for the 3rd-party field won't be present for that time period. But the first time the app calls setData, it's obligated to keep doing so every second, even if it has no data. If you don't call setData, the device just records the last value you set. And if you call setData with null, it crashes.

  • So I resign, I don't want to explain users why data in graph are bad especially the answer can be only one (again) bug in ciq/hw, platform limit etc.

  • I tested this now: from the average user's point of view using setData(INVALID_VALUE) - where INVALID_VALUE depends on the field's type, but it's something like 0xFF or 0xFFFF, etc) does the following:

    - visually: line that connects the last valid datapoint with the next valid datapoint after the invalid values.

    - when advanced user looks at the graph in Garmin Connect and moves the cursor over the graph then they can see that there's no data every second [opening parenthesis] even though in other places of the graph there is, even when the value doesn't change for multiple seconds [closing parenthesis]

    In my app/activity I use every second recording, and in my code I call setData every second. So where the value is valid I see a data point on the graph. Even when I set it to the same value for 60 seconds, you can see (but only when moving the cursor) that there IS a value for every second.

    However when I call setData(0xFFFF) (got a DATA_TYPE_UINT16 field) every seconds for 60 seconds, then there's a (non visual) 60 second gap. The line looks like it has a value for every second, but when moving left-right it jumps between the two valid datapoints.

    So I'm thinking that maybe I'll do something like: detect when the data becomes invalid, set a valid (from the fit data type's point of view) value that makes no sense for the graph (i.e negative HR) for the 1st second, and then the INVALID_VALUE for the rest of the time it keeps being invalid. Of course the same thing will have to be done every time the data switches from valid to invalid.

  •  Since there's no more forum about the forum I'll spam here. In the above comment of mine I struggled posting the text. At the end I edited and added it sentence by sentence, until it turned out that I can't have parenthesises in the 3rd sentence, where I indicated with: [opening parenthesis] and [closing parenthesis]. In later sentences it allowed me to use them.

    I have no idea what causes this, but this is so frustrating. Maybe I'll start to invest my time in making a new developer forum (maybe together with a few other developers) instead of wasting my time in refactoring comments or posting code as images (unless the forum blocks the image as well of course) or just posting code as unreadable text...

  • that makes no sense for the graph (i.e negative HR)

    I see a few practical problems with writing a literal negative HR value as some sort of "marker" to indicate when bad data starts:

    - it could mess up the graph scale

    - it might confuse users

    - it might change the FIT data type that you use and waste space. e.g. To record HR, you could normally use a UINT8, I think, but if negative values are required, then you'd have to use INT16.

    - it could cause issues if people want to do their own analysis of the data that is written to the FIT file (e.g. take the average of the data)

    Ofc all of these things could apply (and even more so) if you wrote 0 for all invalid data (which I think some 3rd party apps do).

    But maybe the compromise is to write 0 for the first invalid data point and the invalid value for the rest. I think everyone knows that it's impossible for their heart rate to be literally 0.

    [opening parenthesis] even though in other places of the graph there is, even when the value doesn't change for multiple seconds [closing parenthesis]

    I know you think this passive-aggressive way of circumventing forum bugs is meant to annoy Garmin employees and prove a point, but it really only makes it harder for the actual audience of your content.

    You realize you could have easily typed something like this and nobody would've cared:

    "[even though in other places of the graph there is, even when the value doesn't change for multiple seconds]"

    Garmin isn't gonna be motivated to fix forum issues because you made it harder for other people to read your posts.

  • I have no idea what causes this

    I think it's pretty obvious that it's some kind of anti-malicious code injection protection, which is why things like parens and numerals randomly cause posts to be rejected.

    I think you also realize this, since you are the first one to bring up cloudflare whenever this happens.

    In the old forums, we had similar issues, except even the names of built-in javascript functions like "alert" (or something like that) would trigger this apparent protection. Ironically, I think one of the reasons that Garmin switched to this platform is that the old platform was so bad and had so many issues preventing normal people from posting.

    What I've never understood is why some sort of general mechanism for avoiding code injection isn't used (like parametrized SQL queries), instead of this apparent whack-a-mole approach of randomly blocking certain combinations of input.

    What I mean is that we should be able to post literally anything at all without being a security risk, given correct data handling on the back end. But maybe I'm just being naive.

  • Also, for context, the invalid values for various FIT data types are documented here:

    https://developer.garmin.com/fit/protocol/ 

    Search "invalid values".

    (click to enlarge)