• State Suggested Answer
  • Replies 26 replies
  • Answers 1 answer
  • Subscribers 23 subscribers
  • Views 5282 views
  • Users 0 members are here
Options
Related

Automating the Connect IQ Simulator for Testing

9356949
over 3 years ago

Hello,

I am trying to setup an automation framework around a Garmin watch app that we have developed. I am NOT concerned with unit tests. I am further down the line from there and focused on integration tests and UI tests. 

The question:

is there a way to control the connect IQ simulator? 

can you attach a debugger to a currently running app?

can you send actions (press a button, change gps location, etc) to the simulator once its created?

can you read the state of the app while its running?

Basically want to do this:

compile prg file

start simulator with that prg file

run automated tests on a simulator running the prg file.

The problem I have is that no window inspection (windows inspector, appium, various github inspector tools) can detect anything inside the panel window to "click" on the buttons or read the text on the screen. It is just a "blank" panel object.

Anyone done this?

Any resources you can point me to?

Thanks for reading and I appreciate any help you can give me.

  • 0 8 months ago in reply to bombsimon

    This is great!  I actually put together a similar script for MacOS and then got stuck because the emulator listens for keystrokes at a layer lower than the Cocoa window manager, so there isn't a way to feed keystrokes into it.  I'll try modifying your script for Windows to see if I can make it work on that platform.

    There is an interesting tool in the ConnectIQ SDK called "shell" that looks like it provides some ways to control the simulator, but it's not documented from what I can tell.  I was trying to figure out if I could use that to simulate keypresses to the simulator.

  • 0 2 months ago in reply to bombsimon

    Hi Simon, great idea, out of interest do you find that when you switch devices on the simulator it messes up some resources? I find that SVG's and images change colour/size the first few times you load them, then after a few restarts it sorts itself out, like there is a cache that needs clearing when you change device...maybe you already found a solution for this?

    I found there is Github Actions workflow (https://github.com/matco/connectiq-tester) for automating the testing, this currently only supports a single device so I might look at using your code to extend it for testing all devices in the manifest + running tests + taking a screenshot all in a Github Actions runner....If that works then all we need is a way to control the simulator inputs to have a real CI test suite.

  • 0 2 months ago in reply to DaveB

    I haven't used it in a while but lat time I used it I didn't have much problems. The biggest one was that it's so slow to build and load a new device so it took some time. Other than that, no. Do you have any repo I can test on? And what version of the SDK are you using?

    Yeah I've seen that, I don't think they're much related, that one is mostly about containerize the testing with Run No Evil. But if you think you can combine them somehow just give it a go!

    I never ended up doing much more work on it and trying to get it working on macOS was a hassle since it requires so much more external dependencies to do all the manipulation needed such as finding processes and switching windows etc. But any change or improvement is of course much welcome!

  • 0 2 months ago in reply to bombsimon

    Cool, it must have been an issue with my setup, I'm using SDK 8.1.1 but noticed the same issue on a previous version, but I haven't noticed it yet when running your script.

    I managed to get your script working on my Mac, well mostly, it just required a manual click on the simulator window each time, I cant seem to fully automate it.

    In any case I will try and go down the containerisation route, so it will be running on linux and not taking over my laptop, I'll post back if I get it working

  • 0 1 month ago

    If anyone is interested I managed to automate running the simulator and unit tests in a docker container, this is a combination of bombsimon's and matco's projects.

    https://github.com/DavidBaddeley/connectiq-tester

    The docker run command will fire up a linux instance, load all your supported devices from your manifest file, build the debug release for each device then for each device, fire up the simulator, take a screenshot then run the unit test's. At the end you will have a montage of all the screenshots and an HTML test report showing each device.

    I have found it really useful for debugging UI differences between devices.

    It would be great if the Garmin devs gave us some official key bindings for controlling the simulator as the way I have been doing it is a bit hit and miss, but I have managed to get the simulator to change the UI by pressing ENTER / SPACE / UP / DOWN keys then get corresponding screenshots