unit test results with monkeydo

I am successfully executing unit tests from the command line using the following. The logger output is displayed

monkeydo.bat bin\test_edge520plus_myApp.prg edge520plus /t

The tests execute, but the documentation @ https://developer.garmin.com/connect-iq/api-docs/Toybox/Test.html implies there should be output summarising the results, similar to the below:

RESULTS
Test:                                Status:
aDebugTest                           PASS
aTestOfAssert                        ERROR
Ran 2 tests

FAILED (passed=1, failed=0, errors=1)

What am I missing to get this results summary?

0 flowstate over 5 years ago

As far as I can tell, you can currently only get this summary (in graphical form) if you run tests through Eclipse, not through VS Code or the command line.

It's worth noting the Eclipse plugin runs tests for all supported devices (it launches the test runner once for each device), and summarizes *all* results. So it's clearly functionality on top of the test runner (monkeydo).

If you need a summary when running tests from the command line, the only suggestion I can make would be for you to create your own script on top of monkeydo which parses the output. (I see you're on windows, so I would highly suggest using WSL and tools like bash, awk and grep for this.)

0 gary_l over 5 years ago in reply to flowstate

Thanks, I figured as much. I'm not sure I *need* a summary but it will save reviewing the output to see if any tests failed.... I have something working with powershell but agree that VSL would probably be easier/better. I'll just need to install the dev environment there first...

monkeydo.bat bin\test_edge520plus_myApp.prg edge520plus /t > testResults.log

results.ps1

Write-Output (Get-Content testResults.log)
$passCount = (Select-String -Path "testResults.log" -Pattern "PASS" -AllMatches).Matches.Count
$failCount = (Select-String -Path "testResults.log" -Pattern "FAIL" -AllMatches).Matches.Count
Write-Output "TOTAL: " ($passCount + $failCount)
Write-Output "PASS: " $passCount
Write-Output "FAIL: " $failCoun

It's not very elegant but it will do the job for now!

0 peterdedecker over 5 years ago in reply to gary_l

you could also execute test by test by specifying the test name (in case that would make your automation any easier)

0 flowstate over 5 years ago in reply to gary_l

Personally I wouldn't install the whole dev env in WSL, but just use the tools from WSL that I need.

If you use wslpath to translate paths from unix to windows (and vice versa), and (possibly) wsl-proxy to launch WSL binaries, you could write a bash script that runs with your existing CIQ installation.

If you can do everything you want from powershell, that's even better. I just prefer cross-platform tools whenever possible.