Automatic UI tests with SIM

There's something for watch faces where you can run time simulation to go though hours in minutes and the heat map for devices that can burn in, but that's really it.

In the case of apps in general, you can cut down the number of devices to test if you know what's similar to what.  Like the legacy series and the va4/va4s, the venu and the D2 Air, 935 and fenix 5, etc,

I really wish this were possible, too (as a built-in Monkey C feature), because I hate manually testing all the different families of devices after updating an app (e.g. for cosmetics or to make sure that it doesn't crash due to running out of memory.)

I suppose you could use a third-party scripting tool, like Autohotkey on Windows, or Automator or Hammerspoon on Mac.

If it helps, the sim has some undocumented keyboard shortcuts for the various watch buttons and gestures, which would save you from scripting the mouse movements to click on the buttons:

START: Enter

LAP/BACK: Escape

UP: Up

DOWN: Down

I think you have to use the mouse for touchscreen gestures, although depending on how your app is written, using the keyboard shortcuts (to activate non-existent buttons like UP and DOWN) may work for your user-defined views (but not for native views). (This seems like a bug to me, but oh well.)

Thanks for the ideas.

jim_m_58, I know what you say and I only test one device per "group" anyways. Still, supporting only CIQ3.2 devices, I already have to test ~10 devices. I have a lot of settings in my app, I can "group" them as well (if I did not touch the data layer) to ~15 types. This means I need to check 150 screens for a full test (in my optimised setup, meaning I already skip some possible settings). If I find a visual bug (that's what I mostly want to do in my case) I may need run all tests again which is a lot of effort.

FlowState, I already though about using AutoIT or a simple Powershell Scripts but I thought there may be a better approach like e.g. following simple process that would fit my needs (although it's slow, but I would not mind)

• start the sim
• "upload" a custom settings file to the sim
• wait for the UI to update (or I just wait a few seconds)
• make a screenshot (with a custom name if possible)
• repeat this with all desired devices and settings

Starting the sim is possible, but updating the settings is not possible (as far as I know) - the eclipse plugin does do it though, so maybe it's somehow possible?

Simply being able to change settings via a script would already give me a lot of possibilites to make some automatic tests very fast - just something like push a single setting to the SIM or pushing a whole setting file to the SIM would make it possible to write UI tests within minutes... (simple loop + change a setting + screenshot and repeat)

If you generate canned settings files, you should be able to simply copy them to the simulator folder in your temp directory -- it looks just like the folder structure on the real Garmin device.

Windows: %TEMP%\GARMIN

MacOS/Linux: %TMPDIR%/GARMIN

It works for me. I tested it in Eclipse as follows:

1. Run app

2. Edit settings and send them to simulator (ensure changes are applied as this part can be finicky)

3. Copy "custom" settings file from GARMIN\APPS\SETTINGS to safe place

4. Reset all app data in simulator (ensure settings return to default)

5. Close simulator

6. Copy "custom" settings file back to GARMIN\APPS\SETTINGS

7. Run app in simulator (with the same device as before). Note that custom settings are applied

You mean the Temp\GARMIN\Settings\Settings.fit file? So you suggest to change the settings via eclipse and then copy away the settings.fit file and reuse this file? That's a smart idea, but needs a lot of preparation time (and you can't reuse those files if you make any changes to the settings.xml file)

If I could somehow create the fit file from the settings.xml file your idea would fully fulfill my needs (closing and starting the sim is something I can live with).

EDIT

No, I just tested this - you meant the %Temp%\GARMIN\APPS\SETTINGS\<APP_NAME>.SET file...

No, I mean GARMIN\APPS\SETTINGS\{YOURAPPNAME}.SET.

It's not a FIT file. You would have to reverse-engineer the process of creating the SET file, for a given settings XML definition, and settings values.

Probably wouldn't be impossible.

SETTINGS.FIT is device settings, not CIQ app settings.

No, under garmin/apps there is a settings folder.  The structure in the sim's temp directory mimics the watch's

For sure. And I'm betting any dev who wanted to release such a tool to CIQ dev community would become an instant hero .

Given that the SET files aren't encrypted (you can see the property text in cleartext), it should be fairly straightforward (if not a bit tedious), to reverse-engineer the format through trial and error.

(They don't seem to be crytographically signed, either, although that should also be very easy to verify)

Actually that's nothing I'm familiar with... But do you know how to build a project for one device on windows via command line? And also where the corresponding files are located at?

It seems like you can run everything on the command line in linux but not in windows...

Seems like following is possible on linux, is something similar possible on windows?

// Launch the simulator:
> connectiq

// Compile the executable:
> monkeyc -d fenix5plus -f /path/to/monkey.jungle -o project_name.prg -y /path/to/Dev_Key

// Run in the simulator
> monkeydo myApp.prg fenix5plus

EDIT

I found it and it works on windows as well. The plugin bin folder is here on my machine, e.g.:

C:\Users\<USERNAME>\AppData\Roaming\Garmin\ConnectIQ\Sdks\connectiq-sdk-win-3.2.5-2021-02-12-6d31f4357\bin

With the code above it's possible to run the emulator with the given device + a custom settings file, make screenshots and repeat this with all desired settings and devices