Stack trace addresses in FW.

I have been down this path before, but I don't think I ever made sense of the response.

Here's an error log report from my watch yesterday. (Nothing in ERA - it's still reporting "A server error has occurred...")

Error: Out Of Memory Error
Details: 'Failed invoking <symbol>'
Time: 2021-11-18T07:58:17Z
Part-Number: 006-B2697-00
Firmware-Version: '24.10'
Language-Code: eng
ConnectIQ-Version: 3.2.5
Store-Id: 74c474f8-4140-4794-afb7-f56817083bd5
Store-Version: 43
Filename: 4C065B22
Appname: raceQs
Stack: 
  - pc: 0x300035c0
  - pc: 0x300033d0
  - pc: 0x10002946
  - pc: 0x10001256

The first two addresses of the stack trace (- pc: 0x300035c0 - pc: 0x300033d0) do not lie in my code, so why are they being provided to me as the developer?

I suspect a memory leak in the FW VM as I am, (and have been for the past four years!) meticulously managing the memory in the app. The crash occurred after 70 minutes of faultless running.

Can anything be deduced from the fact that the first two addresses of the stack trace are in the FW VM?

jim_m_58 over 3 years ago

What about the other two addresses? Do they show what your code was doing that resulted in a call to the system where it ran out of memory?

Alan.raceQs over 3 years ago in reply to jim_m_58

Yes, 0x10002946 is:

dc.setColor(Graphics.COLOR_BLACK, 0xFFFF00);

But my question is: Can anything be deduced from the fact that the first two addresses of the stack trace are in the FW?

jim_m_58 over 3 years ago in reply to Alan.raceQs

What that means is that something in the VM (not the base FW) ran out of memory when handling your code. What are the lines around the setColor?

Alan.raceQs over 3 years ago in reply to jim_m_58

Can we stick with the issue of the VM addresses in the error report please?

Is it normal for an Out Of Memory Error report to reference VM addresses?

jim_m_58 over 3 years ago in reply to Alan.raceQs

I've seen it before, so the answer in Yes. You need to look at what's happening before that.

jim_m_58 over 3 years ago in reply to jim_m_58

Bugs like this can be hard to track down as they can be the result of something else in the code.

Alan.raceQs over 3 years ago in reply to jim_m_58

Yes, I understand the debugging process. However, there are 7,000 lines of code around this one, and it occurred after 70 mins of processing (that's ~ 4,200 main code loops).

I have very little to go on, so am trying to extract as much information as I can from the limited diagnostics available, and so am attempting to determine what, if any, information is encoded in the stack trace.

Does an out of memory error always produce a stack trace with VM addresses? If not, what does it mean?

jim_m_58 over 3 years ago in reply to Alan.raceQs

"Writing code is easy. Debugging it is hard" - My mantra for over 40 years!

In this case I'd probably monitor available memory and the start to log things when memory gets tight..

Real example is if you have a large array, and do an add(), doing that add() is done in the VM (it can crash there) and an cause a spike in memory usage

Alan.raceQs over 3 years ago in reply to jim_m_58

Thanks, Jim. All sound advice and already implemented.

However, I am hoping someone in the Garmin VM development team can throw some light on the appearance of VM addresses in the error report.

Alan.raceQs over 3 years ago

Further interesting data on the crash.

The CIQ_LOG (above) reported the app crash at 07:58:17Z, but the Activity file continued to record for a further 10 minutes until 7:08 pm (local UTC+11 = 08:08 UTC), which possibly coincided with my input to clear the IQ! crash notice on the watch. (ACTIVITY BBIH5000.FIT below):

So although the app had crashed, the ActivityRecording session remained active while the IQ! symbol was displayed.