Running tests from visual studio code not working

Robbin 6 days ago

I made a very simple testcase to make sure everything is working:

import Toybox.Lang;
using Toybox.Test;

(:test)
function testTrueEqualsTrue(logger as Test.Logger) as Boolean {
    logger.warning("warning");
    logger.debug("debug");
    logger.error("error");
    return 42 == 42;
}

When trying to run this from Visual Studio Code command pallet (ctrl + shift + p => Monkey C: Run Tests) the program compiles, starts a simulator, but then crashes the simulator. It will just say that the simulator is 'not responding', while not every finishing this simple test. What do I do wrong here?

The program does actually compile and run normally, just not the tests. I use Ubuntu and the 8.4.0 connect iq sdk. I tries a few different devices for the tests, but all of them fail.

flocsy🤠 5 days ago

The 1st thing I do in every new project is to add this file to .vscode/launch.json:

{
  // Use IntelliSense to learn about possible attributes.
  // Hover to view descriptions of existing attributes.
  // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
  "version": "0.2.0",
  "configurations": [
    {
      "type": "monkeyc",
      "request": "launch",
      "name": "Run App",
      "stopAtLaunch": false,
      "device": "${command:GetTargetDevice}"
    },
    {
      "type": "monkeyc",
      "request": "launch",
      "name": "Run Tests",
      "runTests": true,
      "device": "${command:GetTargetDevice}"
    },
    {
      "type": "omonkeyc",
      "request": "launch",
      "name": "Build and Run Optimized - Release",
      "stopAtLaunch": false,
      "device": "${command:GetTargetDevice}",
      "typeCheckLevel": "Strict",
      "releaseBuild": true
  },
  {
      "type": "omonkeyc",
      "request": "launch",
      "name": "Build and Run Optimized - Debug",
      "stopAtLaunch": false,
      "device": "${command:GetTargetDevice}",
      "typeCheckLevel": "Strict",
      "releaseBuild": false
  },
  {
      "type": "omonkeyc",
      "request": "launch",
      "name": "Run Optimized Tests",
      "stopAtLaunch": false,
      "runTests": true,
      "device": "${command:GetTargetDevice}",
      "typeCheckLevel": "Strict",
      "releaseBuild": false
  }
]
}

If you don't use Prettier Monkey C, then you only need the first two sections.

After you added this file, go to the Run & Debug tab in the left side of VSCode, and there choose Run Tests. You can now run or debug your tests and it'll also ask you which device to run on.

flowstate 4 days ago in reply to flocsy🤠

1) I don't think it should be necessary to create launch.json to use tests in Monkey C / CIQ

2) If you do opt to create launch.json, I think it's better to have the contents generated automatically as opposed to simply copying what you posted. For example, for a project that does not already have launch.json, if I select the Run and Debug tab > create a launch.json file, this is what I get (with SDK 8.4.0):

{
    // Use IntelliSense to learn about possible attributes.
    // Hover to view descriptions of existing attributes.
    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
    "version": "0.2.0",
    "configurations": [
        
        {
            "type": "monkeyc",
            "request": "launch",
            "name": "Run App",
            "stopAtLaunch": false,
            "device": "${command:GetTargetDevice}"
        },
        {
            "type": "monkeyc",
            "request": "launch",
            "name": "Run Tests",
            "runTests": true,
            "device": "${command:GetTargetDevice}"
        },
        {
            "type": "monkeyc",
            "request": "launch",
            "name": "Run Complication Apps",
            "stopAtLaunch": false,
            "complicationSubscriberFolder": "${command:GetComplicationSubscriberFolder}",
            "complicationPublisherFolder": "${command:GetComplicationPublisherFolder}",
            "device": "${command:GetTargetDevice}"
        },
        {
            "type": "monkeyc",
            "request": "launch",
            "name": "Run Native Pairing",
            "runNativePairing": true,
            "stopAtLaunch": false,
            "device": "${command:GetTargetDevice}"
        }
    ]
}

There's 2 "new" entries there which aren't present in your boilerplate (ofc they are also available via the command palette).

flocsy🤠 4 days ago in reply to flowstate

It amazes me (albeit in a bad way) how it is possible to learn new things from the forum (and that some things are ONLY possible to learn from the forum, because the information is not present in any documentation, sample, announcement, etc)

I'll try what this Run native pairing does. So far I thought it is supposed to be triggered from Simulator > Settings > Sensors manager

Robbin 4 days ago in reply to flocsy🤠

Thanks for the help so far. Unfortunately, it has not been fruitful, since the simulator still seems to hang while running the tests with either configuration.

My end goal is to run the tests in CI by the way, but I first tried to get it working locally, but so far have not been successful.

The app compiles correctly, and runs correctly, but trying to run the example test, crashes the simulator. It puzzles me, also because i am not sure where to start debugging in the first place, else I would have provided more information, but it so consistent across multiple devices and sdk versions (tried 8.2.1, 8.3 and 8.4), that I suspect it is something I do wrong.

flocsy🤠 4 days ago in reply to Robbin

try this: in the simulator's File menu:

- Reset all app data
- Delete all apps
- Reset simulator

Then close the simulator window, and try to run the tests again

gasteropod 4 days ago in reply to flocsy🤠

If you trigger native pairing from the simulator menu, any print output goes to the simulator output rather than the vscode debug console, so for most is lost. Furthermore, you cannot debug the code via that method. If you start it from vscode using "Launch Native Pairing", you see the output in the debug console and you can debug. I pulled my hair out over this too when I first started using native pairing and couldn't debug what I consider is poorly documented functionality. I should have logged a bug, although I do believe that the documentation is going to be updated to mention this. I only recently noticed there is a vscode "Launch Complication" too when I looked at a launch.json.

zmullett 4 days ago in reply to Robbin

I'm unsure if this will be of any help, but I wrote some Rust-based tooling that better manages the simulator lifecycle for CI-esque unit test execution. See https://github.com/semolina-solutions/rules_ciq/blob/main/simulator/test.rs - you can build this and run it from the command line, although it's intended to be run via the build tooling (but it only supports macOS currently and also it's a bit of a paradigm shift).

flocsy🤠 3 days ago in reply to gasteropod

There's already a bug report: bug: logs from the app are not displayed in debug console when running in "Sensor Pairing mode" and I doublt anything will be added to the documentation. Usually I'd say "let's hope", but I don't really hope any more. But adding it to the forum might help someone in the future.

gasteropod 3 days ago in reply to flocsy🤠

flocsy🤠 said:
Usually I'd say "let's hope", but I don't really hope any more.

I know. It's a real shame that we can report problems and never hear back again.