(moved edits to the end as I'm starting to ramble down there...)
Do newer devices still use Compressed Timestamp Headers? I'm trying to determine whether this is at all worth to implement or not for my parser.
It seems I'd have to get the actual value out of the last, previously logged specific timestamp field (253 presumably) and temporarily store that for every logged data message, just incase a compressed timestamp header shows up...? Seems it would bring down performance quite a bit...
Also, the last logged "proper" timestamp could very well exist in a message type that is not even officially documented. What then? While most of us on this forum are familiar with field def no 253 as *the* timestamp field in seconds I don't like presuming/hardcoding things.
The reason performance is important is partly because I'm automatically matching VIRB video clips with FIT-files as part of a tool, and we have *a lot* of those at work. On an M1 Mac I'm currently down to roughly 200ms for parsing + looking up scale, offset and field names incl developer data for the absolute largest and/or complex (?) FIT-file I can get my hands on. But if it gets much higher than that, going through multi-TB of data would take ages since it quickly adds up. (any good performance should probably be linked to the M1, rather than my crappy parsing method)
Maybe I'll have to do a custom parse algorithm just to deal with compressed timestamps, then keep my current faster one that returns an error (together with a partial read) if encountered...?
EDIT1: Perhaps implementing "halfway" support would be ok: check if Compressed Timestamp Header -> Get the local id (0-3), but ignore timestamp part of header -> continue parsing with a (documented) caveat that those timestamps have not been derived. That way I could at least continue parsing the file.
EDIT2a: Does anyone have any FIT-files with compressed timestamp headers that they are willing to share? I only have a broken one from the old ant+ forums.
EDIT2b: As for the broken FIT-file, I've changed my parser a bit, and it now returns the same error as FitCSVTool (I'm returning the partially read data up until that point, and FitCSVTool returns a partial CSV). The file is this one: https://www.thisisant.com/forum/viewthread/7019/. Architecture/endian error: "Endian 35 not supported. Error at byte: 298252". This is "architecture" which can only be either 1 or 0 depending on if it's Little or Big Endian. Just in case anyone is curious.
EDIT2c: So in the broken file above there are data messages containing both a compressed time stamp header and an explicit timestamp (field def no 253), which can't be right. I'm guessing header bit 7 was incorrectly set as part of the corruption. To the hex editor mobile...?
EDIT3: Since I'm storing the message headers as part of the extracted data anyway, one could derive compressed timestamp header as a post-process method. I don't have a non-corrupt FIT-file to try this on, however.
