• State Verified Answer
  • Replies 35 replies
  • Subscribers 23 subscribers
  • Views 5076 views
  • Users 0 members are here
Options
Related

Multiple watch faces crashing simultaneously with "out of memory" error (minimal repro app included)

myaro
over 2 years ago

For the last 1-2 weeks, a few watch faces have been crashing on my Epix 2 after it's been running for more than one day. They all crash at the same time, and the only way to get them to load again is to restart the device.  I've generally ruled out a memory leak.

The source of the problem seems to be use of Graphics.createBufferedBitmap.  This call is "randomly" working fine and randomly causing an out-of-memory issue.  ("Randomly" is in quotes because it works fine when the device is freshly started, but after some unknown time, all watch faces making this call fail - including ones I made and one from the store).

Here's what I know:

  • The 3 watch faces I made (all still in private beta):
    • Generally use 70% - 80% memory (on device) - I added a gauge to see it realtime on all my faces
    • Memory usage remains constant according to this gauge - there's no memory leak / change in memory over multiple days of consistent use of the same watch face
    • 1 of the watch faces worked consistently well with no crashes (memory-related or otherwise) from Dec 1, 2022 through the end of December. The other 2 faces are brand new as of the last 1-2 weeks since the new year.
    • All my faces use BufferedBitmap to enable 1hz updating (with a second indicator).  For each minute, I draw to the BufferedBitmap and then draw that to the screen.  Then, between minutes, if the device is awake, I'll draw the background minute via the BufferedBitmap.  Then I'll also draw the second indicator directly to the DC over the top.  This allows me to keep the onUpdate call for each second to about 6 milliseconds of execution time.
    • All my watch faces use exactly 1 BufferedBitmap set to the full dimensions of the DC.  I use anti-aliasing, so I cannot use a pallet.  I also do not specify a color depth.  This BufferedBitmap is stored in a variable inside the class that extends WatchUi.WatchFace.
  • All watch faces (ones I made + one I didn't):
    • Work fine for the first day or so
    • They all crash together at the same time / they all work fine when the device is restarted
    • The crash generally happens when the watch face goes from not being displayed to being displayed again (e.g. after the Morning Report or after exiting an activity)

The watch face that's crashing that I didn't make is "Analog Switch"

About a week ago, I began some research into this issue by looking at the error logs on the device and tracking it back to the function where the Graphics.createBufferedBitmap call happens.

To test my hypothesis that this is a system-level issue (and not something related to the design of the watch faces), I created a minimum reproducible watch face (I call it MemoryX for "Memory Explorer"). To create it, I took the new Watch Face project and made very slight modifications to make it use a BufferedBitmap to draw the time to the screen (based on a setting).  Once the watch goes into the mode where all the watch faces crash, MemoryX loads fine if I turn off the use of the BufferedBitmap.  But if I toggle the setting to make it use the BufferedBitmap, it crashes, too. When using the BufferedBitmap, MemoryX only used 7.3/123.9kB on the simulator. But this crashes on the device.

Questions:

  • Is anyone else experiencing this or something related?  I noticed Gavriel reported another memory issue recently.
  • Why would they all crash at the same time?  Even if one of the faces had an out-of-memory issue, shouldn't that be isolated?
  • It's odd that this happens "randomly" after some time -- can anyone think of something I need to be cleaning up in onHide/onShow that I'm not?

I can't seem to post formatted code (it's telling me I'm blocked).  And I know I can't post images, either.  But here's the code (unformatted) inside the onUpdate function.  I'll try to post it more clearly in a comment below:

function onUpdate(dc as Dc) as Void {
var timeFormat = "$1$:$2$";
var clockTime = System.getClockTime();
var hours = clockTime.hour;
var timeString = Lang.format(timeFormat, [hours, clockTime.min.format("%02d")]);

if (getApp().getProperty("UseBB")){
if(bbDisplay == null){
//We need a BB but it's null - init it
bbDisplay = Graphics.createBufferedBitmap({
:width=>dc.getWidth(),
:height=>dc.getHeight(),
}).get();
}
}
else if(bbDisplay != null){
bbDisplay = null;
}

//Will be either main DC or BufferedBitmaps's DC
var drawContext;

//Will print whether we're using BB or not
var textPrint;

if(bbDisplay != null){
drawContext = bbDisplay.getDc();
textPrint = "Using BB";
}
else{
drawContext = dc;
textPrint = "NOT Using BB";
}

drawContext.setColor(Graphics.COLOR_WHITE, Graphics.COLOR_BLACK);
drawContext.clear();

var font = Graphics.FONT_SYSTEM_SMALL;

//Draw time
drawContext.drawText(
drawContext.getWidth() / 2.0,
drawContext.getHeight() / 2.0 - Graphics.getFontHeight(font),
font,
timeString,
Graphics.TEXT_JUSTIFY_CENTER
);
//Draw whether using BB
drawContext.drawText(
drawContext.getWidth() / 2.0,
drawContext.getHeight() / 2.0,
font,
textPrint,
Graphics.TEXT_JUSTIFY_CENTER
);

if(bbDisplay != null){
dc.drawBitmap(0, 0, bbDisplay);
}
}
  • 0 over 2 years ago in reply to flocsy🤠

    If which WF is starting the problem is IDd, that would be very useful for Garmin to know so they can use it to narrow down the issue.  As I've said, this has happened in the past, and once Garmin had a way to reproduce it, the fix was relatively fast

  • 0 over 2 years ago in reply to flocsy🤠

    Thanks, jim_m_58 for the link to that 2021 announcement.  I wish all that great content had been linked to (or included directly) in the main docs for new developers like me who weren't in the community when this happened.  I'm glad you surfaced it for me.

    I'm not sure that trying the watch faces one after another will help, since for the first day after a reboot, they all work fine.  I can't really trigger this crash in a repeatable way right now

    I just had an idea about what might be special about the 2 new watch faces I made that could be a corner case Garmin didn't anticipate:

    Both newer watch faces use a BufferedBitmap that is created from inside a Barrel and is contained as an instance variable inside that Barrel's class.  My watch face view classes extend this and then access that variable.

    When I moved on from making Watch Face 1 to starting Watch Face 2, I realized that I had developed a pretty common pattern of how I like to use the Buffered Bitmap and some other logic to help me know what to paint. For example, I like to track when the last full paint was so that I can decide whether we should repaint the whole thing or just paint the 1hz update experience.  

    I created a Barrel with a class that extends the WatchUi.WatchFace class and held most of this common logic.  My new class handles most of my usual logic and flow, and it contains the buffered bitmap both watch faces use.  

    Then, in the new watch face apps, I extend my own class that extends WatchUi.WatchFace, and I get to access the Buffered Bitmap provided by the parent class.  It cuts down on a lot of the boilerplate code I found myself about to set up for Watch Face 2, and it made developing Watch Face 3 a lot faster.

    But I wonder if Garmin's OS is somehow not cleaning up the memory associated with the BufferedBitmap that's defined in a Barrel?

    Either way, when I get back to coding, I'll explore trying to clean up resources in onHide() and see if that resolves things for 2-3 days.  Then, if not, I'll proceed to see if the BufferedBitmap inside the Barrel is the cause.  I'll move the variable & definition of that up into the main View.mc file instead of inside the barrel.

    I'm also going to try some Logging to see what I can find about when the various lifecycle hooks are called.  

    By the way, here's the relevant part of the class in that barrel:

  • 0 over 2 years ago

    As a quick update, I discovered the page in the docs that outline the lifecycle of WatchUi.WatchFace.
    https://developer.garmin.com/connect-iq/api-docs/Toybox/WatchUi/View.html

    onHide is never called for Watch Faces. 

    So I'm going to try instead tapping into Application.AppBase.onStop().  I came up with this solution to pass that information down: I hold a reference to the WatchFaceView, which I can then use to call custom methods.  Here's the approach:

    I'll report back in a few days if this resolves / isolates the issue.

  • 0 over 2 years ago
    myaro said:
    About a week ago, I began some research into this issue by looking at the error logs on the device and tracking it back to the function where the Graphics.createBufferedBitmap call happens.

    the graphics pool memory is not contained within your app but it is also not endless, have you considered using a limited color palette for your buffered bitmap?

  • 0 over 2 years ago in reply to peterdedecker

    But it works. It works for him for more than 86400 seconds :) So it can't be the problem. Somewhere the memory is leaking. Maybe not even the whole buffer, just a pointer somewhere.. but it adds up after a while.

    maybe at the beginning just try to log and count how many times onLayout is being called up to the crash.

  • 0 over 2 years ago in reply to flocsy🤠

    Update from this morning:

    After nulling the BufferedBitmap in onHide and onStop, I loaded the watch face up and went to sleep.  Woke up this morning and it crashed right after the morning report.  Now I have 5 different watch faces crashed - 1 downloaded from another developer, my 3 real WFs, and my 1 tester "MemoryX."  They all were crashed this morning.

    Moving onto next step:
    I moved the BufferedBitmap variable outside of the Barrel and directly into the WF View.mc file.  I also removed all other installed watch faces except the newest one which uses that Barrel for the View class.  I'll load this up, hard reset the device (by holding the Light button for 20 seconds) to clear out any possible issues from this morning's crash, and see what the status is tomorrow morning.

    If that seems to resolve the matter, I'll consider making a minimum test case for others to try involving a BufferedBitmap inside a Barrel.

  • 0 over 2 years ago in reply to myaro

    What happens if you include the mc from the barrel in your project instead of using it as a barrel?

  • 0 over 2 years ago in reply to jim_m_58

    Good thinking.  I'll try that next after completing the exploration about just moving the BB variable from the Barrel class directly to the view class.  

  • 0 over 2 years ago in reply to jim_m_58

    Hard to believe it matters. At the end the generated binary code is probably the same.

  • 0 over 2 years ago in reply to flocsy🤠

    Barrels do things differently and induce about 1k of overhead.  Rez is different for example.