How to unit test code marked as (:background) in a Datafield project?

Is there any way to unit test a class marked with (:background) in a Datafield project without receiving permissions errors.

This class that I want to test has references to constants from modules that are not allowed to be accessed in a datafield app, but can be accessed in a background service delegate.

  • My challenge is that I am really on the edge of memory. And that can happen any time to any device. 

    I show user's calendar events and it is quite difference if you can load 5 or 6 events, because you might not show him events of his full day. 

    Testing it manually is desperately time-consuming russian roulette I was hoping I can just get rid of.

    a) Do you claim there is no way to give permissions to makeWebrequest to the test mocks

    b) or are do you claim you don't know if there is a way? 

  • a) I think you won't see what you want to see if it works as memory limits are not checked for unit tests, as Peter said.

    b) what are you hoping to find that you can see based on sim runs with various devices or a sideload on the device you have?

    Understand that Peter has a WF with 1m+ downloads, and mine combined are in the 500k range.  You may want to understand the advice that's been posted.

  • I don't challenge your knowledge guys. I appreciate your answers and respect your work. 

    I'm hoping to find a way how I can save time of 1 h+ testing before every release so apps of my users won't get broken. 

    There are bugs everywhere in Garmin (including makeWebrequest, Position, Weather API), simulator on Mac OS crashes with every Background.exit and tough memory limits on Background varying across devices not predictable by getSystemStats().freeMemory (some crashes with 2k, some with 6k left in the Background process). 

    Memory-wise the makeWebrequest works the same on device and simulator for me: when it exceeds the limit in the simulator, it exceeds the limit in the watch as well.

    So please do not interpret my questions as a challenge to your knowledge. I'm just searching for a way to guard my users from Garmin's bugs including the problematic makeWebrequest. 

  • Understand that with unit tests, you probably won't see issues with memory limits as they aren't checked.  You really want to do an integration test where you see what happens when the app runs for real.

    If you use app settings, be sure to check what happens when you change those while you app is running, as the extra memory needed then can be a real impact and cause crashes..

  • With makeWebRequest and 2k vs 6k available in a background service, understand there can be a dependence there based on the data you are receiving.  If you are doing JSON data, and let's say it's X bytes long, the impact on your code is greater than that, as the data gets converted into a dictionary, and the complexity of the data will also have an impact on the real impact to your code. But the impact won't be "X" bytes. And running out of memory during the conversion is different than running out in your code itself.

  • If you really want to do this via unit testing, the only thing I can think of would be to use a different manifest to change the app type. You should be able to build and run via the command line using the different manifest, and then you'd have access to the functionality you mention.

    We might be able to do something where we fake the app type to watch-app when building and running unit tests. Maybe it doesn't make sense to use the app type for unit test builds at all.

  • , the problem is that unit test framework does not respect the manifest nor the features available for the device it was compiled for: 

    a) even when the Communications permission is requested in the manifest, the unit test of ServiceDelegate mock raises:

    Error: Permission Required

    Details: Module 'Toybox.Communications' not available to 'Watch Face'

    b) even when the code is compiled for a device without FloorsClimbed, the unit test of mock view of WatchFace returns a value Toybox.ActivityMonitor.getInfo().floorsClimbed correctly when both simulator and device crash. 

    Or how to make it work? How to test any device is not trying to ask for feature not available on it?

  • Finally I implemented integration tests by: 

    1. Mac OS shell to test for all selected devices and screenshot results
    2. AppleScript to restart that bloody crashing simulator, and start Background events
    3. (:debug) overloaded resource from testing-specific.jungle to set variables for each testing scenario
    4. following testing in (:release) keeps those variables (from properties/storage) and runs as a production code with right memory limits and device features

    It took me a whole day, but it works as a charm and I'll save weeks in the future:) 

    Now I just run the one test, and after couple of minutes (after a coffee break:) I see screenshots of all significant devices; if they display everything properly, do not crash and load all background data in full limits and even check the layout looks beautiful across resolutions.

    If someone is interested in devices variations/fragmentations: I initially defined the next sets, that seem to cover all variations relevant to my app: 

    • Background data loading memory limits and display: (fenix6xpro venusq fr245 fr945 fenix5s) # data 280 240 218 OLED rectangle nofloors weakest-with-data no-storage-from-background
    • Just display the data above on resolutions not in above: (wearable2021 venu smallwearable2021 vivoactive4) # 416 390 360 260 
    • Just display on no-data devices, including CIQ1: (fenix3 fr230 fr45 vivoactive_hr fr735xt) # no-data 218 65k 3CIQ1 180 semi-round weakest old disabled-data
    • Devices without FloorsClimbed: (venusq)
    • Alternative design variant on all resolutions: (wearable2021 venu smallwearable2021 fenix6xpro venusq fr945 vivoactive4 fr745 fr735xt garminswim2 vivoactive_hr)  # 416 390 360 280 260 240 rectangle 218 16c 180 semiround 208 CIQ1   rectangle