Simulator Randomizer

The Simulator is really useful. However as I'm developing some useful cycling apps, my logic adapts to sensor input that changes values faster than expected, and reacts differently to different levels vs, say, FTP or Lactate Threshold HR, etc.

Anyway, a REALLY cool and I believe useful feature to the simulator in general, would be to allow for controls over the simulator randomizer.

1. One aspect would be the mean value - say I want POWER to be about 220W on average. Right now it picks a value at random.

2. Another would be the variance - say I want POWER to spread from +/- 20% from mean. Right now it generally stays nearly constant with slight variances and drift.

3. Finally, drift of the mean over time. Say, I want the POWER mean to slowly drift -10W per min. Or maybe + 5W per min.

That would allow us to model real life and exercise dynamic logic a lot better. Yes, I can build a simulator debug module to do this. But this would be generally useful I think.

Maybe a pulldown in which we can pick a particular metric and set those 3 controls. POWER, CADENCE, HR, ALTITUDE... others?

jim_m_58 over 6 years ago

I'm thinking you might have an issue with how you handle settings if they keep getting reset on a device. Make sure you look at the New Developer FAQ, point 10. Also (though it might be fixed on some devices now), if the user did an "update" of the app from a phone, that was actually an uninstall/reinstall in the background, which causes app settings to be reset to the default. Suggest users only use Garmin Express to get an app update if that's the case.

Having onSettingsChanged or not won't impact crashing due to memory. The onSettingsChanged() callback is to just let you know they've changed, and then it's up to the app to use the new values or not. (some apps will use them the next time onUpdate() is called for example) A user just trying to change settings from their phone while the app is running could cause it with or without oonSettingsChanged. I allow many things to be changed while a DF is running, and the DF knows there was a change with onSettingsChanged().

If an app crashes when a user tries to change settings from a phone while it's running and the app crashes as a result (something that can be done with most apps of any type), I'd look ar changing things - simplify settings or maybe even reducing functionality - so it doesn't happen, as that's just something that can trigger bad reviews/contact developer messages.

flowstate over 6 years ago

jim_m_58 to be clear, in my experience, settings only get reset in the simulator not the device. That's the crux of my complaint. I load settings in the sim, send them back, and not only are the new settings ignored, but the existing settings are reset to defaults. This doesn't happen for every app, but when it does, it happens fairly consistently. I have also had the situations where I simply cannot apply settings at all.

I can't imagine what I could be doing "wrong" to make the simulator spontaneously delete my settings, unless it's the fact that I have extensive "conditional compilation" of settings based on different devices.

Furthermore I have had crashes in my data fields on changing settings (either the first time or second time), but only in the simulator. I've never seen it in real life (as far as I know, based on the device I have to test, and on lack of user reports).

---

As far as the impact of onSettingsChanged() goes, I am only speaking of the memory impact for the code to read and apply settings while the datafield is running, which may be slightly higher than just doing it on init. In some cases, it may be a lot higher, if the running state of the app takes more memory than the init state. i.e. Certain persistent data structures are now initialized. Yes, they could be deinitialized, but again that's slightly more code. In some cases, as with loading resources and caching the resource table, CIQ itself has permanently allocated memory which can never be freed by the app. What would've been an innocuous memory hit during init could be fatal after init.

My apps typically don't crash in real life when users send settings, afaik. I have had data fields that crash in the simulator in onSettingsChanged(), especially with some of the earlier 3.0.x SDK updates. In the sim only, I have seen some of my data fields crash on either the first or second onSettingsChanged(). This is because my data fields are typically packed with features, especially on CIQ1.

I used to worry about this, but:
- Nobody has complained about this once. To be fair, most of my apps have tiny userbases. However, I usually hear about it if I have any kind of crash.
- I can't get it to happen in real life
- Even when I cut away code code to make the app survive first onSettingsChanged() in the sim, sometime it will still die on the second onSettingsChanged() in the sim. At what point do I stop trying to make the simulator happy? Especially when the first onSettingsChanged() is fine but the second is not?

So at what point is worth sacrificing features handle a use case which won't or can't happen for practical purposes? I have never once changed CIQ app settings during a run, especially because I don't bring my phone with me. I know very few runners who bring their phone during a run, although I realize that not every runner is like me.

I can see how watchface settings would be completely different, since the watchface is always running.

Thanks for the advice. I do agree that:
- In a perfect world I would handle any changed settings that I could while it's running. However, I have seen very popular data fields (200k+ downloads) which don't handle changed settings on the fly, and as I said, some of my apps have mostly settings which can't really be properly changed after the activity starts anyway.
- I would never want my app to crash on changed settings, but to this date, I have not had a single report of this happening, even though it does happen in the sim. So either people aren't changing settings on the fly for my data fields, or my data fields don't crash in real life when you change settings.

Thanks again for your advice.

flowstate over 6 years ago

jim_m_58 case in point, I have a datafield which is right against the memory limit in CIQ2. In the sim, if I max out memory usage by enabling certain features, then send new settings twice, it will crash for sure. Only in the simulator.

On the real device, I can send settings as many times as I want and nothing happens. Not one single crash. I am 100% sure that settings were sent, because as soon as I stop and discard the activity, the new settings take effect. Because, at least on a 935, discarding the activity terminates and restarts all CIQ data field apps.

So I've stopped worrying about it, but I def used to. I admit that the UX of forcing the user to restart the activity to apply new settings (for cosmetic settings which could be applied) is not nice, but it's a compromise as with all things. Again, I assume that most people don't change settings in mid-run, as many don't even run with a phone.

But of course, since the simulator is not an emulator, I can never know for sure if my app would really crash on settings change for every platform unless I owned a device from every family and tested it in real life. This is where hobbyist devs are at a disadvantage to orgs that can test on all the real hardware.

Thanks again.

jim_m_58 over 6 years ago

I simply don't publish if something crashes the sim. The sim might be 100 bytes high for peak memory causing the crash, but adding/changing a bit of code could cause it to happen on devices. Or possibly new FW.

I think there are a number of people that run with their phone - in cases where they run to music and don't have a watch with on board music. And for me, when I install a new DF, the first few times I use it I'll often tweak the settings while it's running, as that's when I remember I wanted to change something.

flowstate over 6 years ago

jim_m_58, yes I agree that would be the smartest solution which covers all the bases and is the most considerate of users. Yes, I know a few runners who like to run with music, too.
.
However, I decided to make one exception for onSettings() crashes that only happen in the simulator (especially if it only crashes the second time), because I think it's unreasonable to keep 100s or even 1000s of bytes available just to handle a crash scenario that I can't even observe in real life. I guess I am a little stubborn, and also a little greedy when it comes to including features.

As you pointed out, new firmware could also cause crashes. So even if you avoid crashes with the previous firmware, there is literally nothing you can do to prevent crashes in the future, except retest. I think we've both seen the other thread where people have complained about increased memory usage with the newer CIQ versions.

I am sure it will come back to bite me if I ever publish a data field that actually becomes widely used....

But then again, if that happens, I'll just tell users not to change settings during the activity. I can already see some watchface devs cautioning users to only change a few settings at a time, or to switch watchfaces before changing settings. It isn't ideal, but I don't see a perfect solution, except just needlessly limiting some of my apps.

As I said, it would be ideal if apps could opt out of settings changes in the manifest. Sort of the perfect compromise.

Thanks again.

jim_m_58 over 6 years ago

So even if you avoid crashes with the previous firmware, there is literally nothing you can do to prevent crashes in the future, except retest.

Actually, what you can do is make sure your apps aren't that close to the edge. That might mean reducing functionality or even removing targets.

Keep the future in mind - code knowing this could happen. Even in my DFs for devices with a 16k limit, I have 1-2k free for this.
That's what 40 years of coding and 4 years with CIQ has taught me! :)