• Replies 26 replies
  • Subscribers 19 subscribers
  • Views 4409 views
  • Users 0 members are here
Options
Related

FitContributor::Field::SetData should accept null as a value

FlowState
over 6 years ago
And gaps in both native and CIQ activity FIT records are not displayed correctly by Garmin Connect.

Hopefully I'm not missing something, but I think there's a major gap in the functionality for FitContributor::Field::SetData. Native fields can have "gaps" (null/no data) in their data (in the middle of an activity) but 3rd-party fields can't -- once they write their first value, they have to keep writing for the rest of the activity. So if there's an actual gap in the real data, a 3rd-party app has no choice but to write zeros.

Consider the case of going for a run recording HR (from wrist) to the FIT file. Let's say the runner takes off their watch in the middle run and the wrist HR turns off. The native graph in Garmin Connect will show a nice gap where that happened. It won't show zeros, it will show nulls (no values) -- if you roll over the graph it will skip that portion completely and if you look at the FIT file contents, there are no HR records for that time period.

Now consider the case of a Connect IQ field recording the exact same data.

If there's no HR data available at beginning of the run and the field hasn't called setData yet, no problem. The records for the 3rd-party field won't be present for that time period. But the first time the app calls setData, it's obligated to keep doing so every second, even if it has no data. If you don't call setData, the device just records the last value you set. And if you call setData with null, it crashes.

So in the example of runner who takes off his wrist HR in the middle of a run:
- The native field is able to write no data
- The 3rd party field has two choices:
-- Don't call setData, in which case the graph will have a nice horizontal line where the same old data was written over and over again
-- Call setData with 0, which doesn't match the actual situation. HR data was unavailable -- the runner's heart rate wasn't zero

Oh, and I tested the scenario where a CIQ app doesn't write data at the beginning of the activity (e.g. recording HR while not wearing watch)-- this is displayed properly in the Garmin Connect mobile app but not the website. The website acts as though the first available value existed all the way to the beginning (even if you waited for a few minutes to start recording.) The native field has the opposite problem: Garmin Connect mobile shows a bad graph for the native HR, but the website shows the correct graph.

It would be nice to have some kind of parity with native fields, especially if the data is to be analyzed automatically later.
  • over 2 years ago in reply to jim_m_58
    jim_m_58 said:
    The fit contrib data will be written every second, even if setData isn't called for something lie HR.  That's why setData must be called before the data is written.  setData does not cause the data to be written

    I understand that. Please explain how:

    1) If you delay calling setData() *for the first time* (by say 5 minutes), then the first 5 minutes of your graph will have "no data". Obviously since there's no previously written/set value, the firmware chooses to write nothing. (As opposing to writing 0 or a random number)

    2) Native fields are able to have "no data" at any point in the graph.

    Obviously it's possible for native or CIQ fields to have "no data" (missing FIT records for a given timestamp), it's just not part of the setData() design to be able to do that once you've called setData() for the first time, as I've tried to point out over and over again.

    You are describing things as they are, in response to my over-4-year-old request to change that status quo.

    My request was for the ability to call setData(null), which would mean "please write 'no data' at the next possible opportunity". I also would have been satisfied with a new function, say "clearData()", which does the same thing.

    I think we all know this will never happen, so the discussion is academic.

  • over 2 years ago in reply to FlowState

    1), that's likely a FW thing and not a CIQ thing.  I don't know.  But I do know the reason that devices default to smart recording to reduce the size of the fit files Could be related, as fit contrib causes things to go to 1 sec recording, and no reason to do that until there is data to save

    2) Native code works differently than what can be done in CIQ

  • over 2 years ago in reply to jim_m_58
    jim_m_58 said:
    1), that's likely a FW thing and not a CIQ thing.  I don't know.  But I do know the reason that devices default to smart recording to reduce the size of the fit files Could be related, as fit contrib causes things to go to 1 sec recording, and no reason to do that until there is data to save

    A) I always have 1-second recording enabled

    B) There's other stuff being written to the FIT file other than developer fields, so I'm not sure if this argument holds water. If I delay calling setData() for five minutes, the watch will happily write other fields to the FIT file regardless of that fact.

    C) When 1 native field has a gap, it doesn't mean the other native fields also have gaps. (e.g. if I take my watch off my wrist to induce a gap in the HR data, GPS data and cadence data is still recorded.)

    It feels like you just don't want to admit that it is totally valid for there to be gaps in FIT data for FIT fields (due to no data being available, not due to smart recording or anything like that), and that this even naturally and intentionally happens for CIQ FIT fields when you delay calling setData() for the first time.

    I mean clearly it's a design oversight that this should be possible before you call setData() for the first time but not after.

    Obviously Garmin didn't forsee (or care) that any developer would want to write "no data".

    But clearly "no data" is not equivalent to 0 or "previous value", either from a mathematical or physical/real-world standpoint.

    e.g. Consider an app that connects to a power meter and writes the data to a FIT field:

    - 0 means you were standing still

    - "no data" would mean that the sensor wasn't connected, and/or there was a drop-out. I mean, ideally it would, except that we can't write "no data" (unless setData() hasn't been called yet.) So we can properly handle the case where the sensor doesn't connect until 5 minutes into the run, but we can't handle the case where sensor disconnects in the middle of the run.

    jim_m_58 said:
    2) Native code works differently than what can be done in CIQ

    Thanks I realize that. Hence:

    FlowState said:
    It would be nice to have some kind of parity with native fields, especially if the data is to be analyzed automatically later.

    I will explicitly state what perhaps I didn't previously make clear: this is a *feature request* asking Garmin to bring parity to CIQ FIT fields and native FIT fields, with regards to gaps in the data.

    Also, I feel like you moved the goalposts. First it was "this probably is due to the FIT specification". Then "probably something to do with smart recording". Finally "native code can do stuff CIQ can't". Yeah, I realize that, which is why I made this request.

    Also, as I said before, I don't care about this anymore.

  • over 2 years ago in reply to FlowState

    Maybe I moved the goal posts, because as I have said, I don't know for sure.  Just stating ideas on what may be the cause.  And to be honest, it's never bothered me.

    As far as the fit spec, I was saying that could have to do with writing a null.  As far as smart recording vs 1sec, that has to do with writing anything at all

  • over 2 years ago in reply to jim_m_58
    jim_m_58 said:
    Just stating ideas on what may be the cause. 

    Not sure why you can't even entertain the idea that the cause might just be that Garmin designed something in a less-than-optimal way. The response to a large number of complaints is "this isn't really a bug" or "I don't use that feature". That sounds a lot like "I don't care" or "Garmin doesn't care", to me. (Which is fine.)

    If by your smart recording argument, you're implying that my watch was previously in smart recording mode (allowing gaps to be written in native fields), but the use of a CIQ data field kicked it into 1-second (at which point gaps were no longer allowed in either native fields or CIQ fields), that simply isn't possible, because:

    1) I'm always in 1 second mode (although you had no way of knowing that)

    2) I already said that data for a CIQ field is allowed to be absent at the beginning of the activity, regardless (simply delay called setData() for the first time, for any length of time you want.) The first data point for the CIQ field will only be written when you call setData() the first time -- it could be 1 hour into a 2 hour activity, for example.

    jim_m_58 said:
    As far as smart recording vs 1sec, that has to do with writing anything at all

    But "recording no data to save space" (when the value hasn't changed) is clearly a different concept than "recording no data when the source data is absent." I haven't looked at the FIT spec in a long time, but I would even guess it would look different in the file itself (otherwise how would Garmin Connect know to "fill in the gaps" in the graph, which it def doesn't do in the case where the source data is absent)

    My point was that you said you think the FIT specification doesn't allow for recording no data for given a field, during a certain period of time, when it clearly does, even by your own argument.

    To recap - the different known ways that "no data" can be recorded for given field at a given time (even when a watch is set to 1-second mode):

    1) When data for a native field is absent (e.g. HR sensor disconnection / dropout)

    2) When data for a CIQ field is absent at the beginning of the activity

    I don't see why it should be controversial that CIQ can't do what native fields can clearly do, in this regard. The OP was simply a request for parity in that sense. I realize it's not important, and it's never gonna happen, it's just one of those little things that adds up to frustration for me personally, although I don't really care about this problem anymore.