Automatic UI tests with SIM

This worked for me (Windows batch file, just like yours). (I run it in the root of my project folder)

set SDK=%APPDATA%\Garmin\ConnectIQ\Sdks\connectiq-sdk-win-3.2.5-2021-02-12-6d31f4357\bin
call %SDK%\connectiq
call %SDK%\monkeyc -d fr245m_sim -f monkey.jungle -o tmp.prg -y PATH_TO_MY_GARMIN_DEV_KEY
call %SDK%\monkeydo tmp.prg fr245m

Thanks for trying but please make sure to also completely reset the sim state before trying this via File > Delete all apps. I thought it is working as well in the beginning, but after I wrote my UI test script I saw, that the settings did not change so I went back to this simple test and found out, it does not work correctly.

Resetting the sim state and starting an app from eclipse works but not from the command line...

It still works after clicking "Delete all apps". Just for good measure, I tried deleting the whole sim folder and it still works.

Not sure what's happening differently here. I did the simplest test possible....

Just for good measure I tried "fenix6xpro" and it works too.

flowstate said:

That was already suggested, but they specified that they wanted to create the settings *programmatically*.

I suppose another solution could be to script the app settings editor... Not as elegant as creating your own tool to do it directly tho.

The other problem is that the app settings editor can fail to push settings due to the way it pushes them while an app is running. If the resulting memory spike crashes the app, then default settings will be applied.

Have you tried this with the simplest possible app? e.g. Nothing esoteric like multiple jungles, or shared code using symlinks or eclipse links, etc.

That helps me, I will try a simply example as well. It does not work with my setup, but I do use a more "complex" monkey.jungle with relative paths.

I tried the exact same code as you tried and I also put the batch file directly into my project folder just as you did and tried with a resetted sim and it still fails.

My monkey.jungle does look like following though (excerpt), I will make some more tests now:

# ---------
# Allgemein
# ---------

project.manifest = manifest.xml

# SOURCE
srcBase=source;../shared-src-core;../shared-src-data/source

# RESOURCE
resBase=resources;../shared-res-strings/resources;../shared-res-core/resources

# RESOURCE - LOCALISED STRINGS
base.lang.ces=$(base.lang.ces);../shared-res-strings/resources-ces
base.lang.dan=$(base.lang.dan);../shared-res-strings/resources-dan
base.lang.deu=$(base.lang.deu);../shared-res-strings/resources-deu
base.lang.dut=$(base.lang.dut);../shared-res-strings/resources-dut
base.lang.fin=$(base.lang.fin);../shared-res-strings/resources-fin
base.lang.fre=$(base.lang.fre);../shared-res-strings/resources-fre
base.lang.hrv=$(base.lang.hrv);../shared-res-strings/resources-hrv
base.lang.hun=$(base.lang.hun);../shared-res-strings/resources-hun
base.lang.ita=$(base.lang.ita);../shared-res-strings/resources-ita
base.lang.nob=$(base.lang.nob);../shared-res-strings/resources-nob
base.lang.pol=$(base.lang.pol);../shared-res-strings/resources-pol
base.lang.por=$(base.lang.por);../shared-res-strings/resources-por
base.lang.rus=$(base.lang.rus);../shared-res-strings/resources-rus
base.lang.slo=$(base.lang.slo);../shared-res-strings/resources-slo
base.lang.slv=$(base.lang.slv);../shared-res-strings/resources-slv
base.lang.spa=$(base.lang.spa);../shared-res-strings/resources-spa
base.lang.swe=$(base.lang.swe);../shared-res-strings/resources-swe
base.lang.zhs=$(base.lang.zhs);../shared-res-strings/resources-zhs
base.lang.zht=$(base.lang.zht);../shared-res-strings/resources-zht

# default setting für build (Fenix 6X)
base.sourcePath=$(srcBase);../shared-src-devices/source-fenix6x
base.resourcePath=$(resBase)

# --------------------
# GROUP 280 - Fenix 6X / Enduro
# --------------------

fenix6xpro.sourcePath = $(srcBase);../shared-src-devices/source-fenix6x
fenix6xpro.resourcePath=$(resBase)

I may need to switch to start the sim from within eclipse to make sure that everything works - that's possible as well quite simply... That way my test script works but it looks so "dirty"...

Well I *think* the connectiq and monkeydo commands are fairly straightforward, so the only thing that eclipse can be doing differently is the equivalent of the monkeyc command (which can be different then what we have in our scripts if you have multiple jungles, for example).

In Eclipse if you watch the console closely during a build (Run As > Connect IQ app), you will see a build command flash quickly by, soon to replaced by the run console itself.

Click on the console selection icon to the far right of the console tab (looks like a console with a down arrow beside it -- hovertext is "Display Selected Console") and select the "Connect IQ" console.

You will see a build command line such as:

BUILD: C:\Program Files\AdoptOpenJDK\jdk-11.0.9.101-hotspot\bin\java -Dfile.encoding=UTF-8 -Dapple.awt.UIElement=true -jar C:\Users\USERNAME\AppData\Roaming\Garmin\ConnectIQ\Sdks\connectiq-sdk-win-3.2.5-2021-02-12-6d31f4357\bin\monkeybrains.jar -o PROJECTPATH\bin\PROJNAME.prg -w -y "DEVKEYPATH" -d fenix6xpro_sim -s 3.1.0 -f PROJECTPATH\monkey.jungle

Everything after monkeybrains.jar is the arguments to monkeyc. Try comparing that to what you have in your script and see if it's different. If it's exactly the same (or equivalent), then something else is wrong here.

Unfortunately the other commands (connectiq and monkeydo equivalents) aren't in the console log. (I haven't dug deeply enough to see if they're logged somewhere else.)

Thanks for the detailed response. I'm aware of all that and I even tried calling java -jar ...\monkeybrains.jar <ARGS> and  am using the exact same args in my script but I really do think that there is some special handling inside eclipse in the monkeydo part... Adding the -w + -s 3.2.0 to my script does not help even though they are the exact same commands as eclipse is logging in my case...

Cool, sorry for telling you stuff you already knew haha.

In that case you can use a tool like Process Lasso:

https://bitsum.com/

Set it to log process launches and run your CIQ app from Eclipse.

I see a cmd line that looks like this:

java -classpath c:\Users\USERNAME\AppData\Roaming\Garmin\ConnectIQ\Sdks\connectiq-sdk-win-3.2.5-2021-02-12-6d31f4357\bin\monkeybrains.jar com.garmin.monkeybrains.monkeydodeux.MonkeyDoDeux -f PROJPATH\bin\PROJNAME.prg -d fenix6spro -s SDKPATH\connectiq-sdk-win-3.2.5-2021-02-12-6d31f4357\bin\shell.exe

Of course, monkeydo.bat looks like this:

java -classpath "%home%monkeybrains.jar" com.garmin.monkeybrains.monkeydodeux.MonkeyDoDeux -f %prg_path% -d %device_id% -s "%home%shell.exe" %test_flag% %test_names%

So I don't really see where there's room for it to be anything other than what we expected, except for any possible tests (which you are not running.)

Maybe try adjusting the current working directory that you launch your script from.

I assume you're not doing anything unusual like launching Eclipse as an administrator or something.