ConnectIQ Docker image and GitHub Action

With the latest releases of the ConnectIQ SDK, it is finally easy to run the SDK on Linux. Thanks to this update, I've been able to create a Docker image that tests a ConnectIQ application (if it uses "Run No Evil"). The command to test an app is:

docker run -v /path/to/your/app:/app -w /app ghcr.io/matco/connectiq-tester:latest fenix7

The code source of the image is available here: https://github.com/matco/connectiq-tester. It's currently based on ConnectIQ 4.1.4 and the device bits retrieved on 29 June 2022.

In addition to the Docker image, I created a GitHub Actions that can test your application using GitHub CI. Just add the following step to your GitHub workflow:

- name: Test application
  id: run_tests
  uses: matco/action-connectiq-tester@v1
  with:
    device: fenix7

The web page of the action is here: https://github.com/marketplace/actions/connectiq-tester and its source code is here: https://github.com/matco/action-connectiq-tester. You can have a look at the CI workflow of my own ConnectIQ application here: https://github.com/matco/badminton/blob/dev/.github/workflows/test.yml.

Feel free to use these tools and do not hesitate to provide your feedback. I already have ideas to improve them. My goal was to have the simplest GitHub Action possible, but I'm not sure if this approach is the good one. It may be better to have an action that setups ConnectIQ directly in the GitHub Action runner. This would allow choosing the version of the SDK to install and give more flexibility to the Action (for example, choosing custom flags for the compiler).
I found that someone built a Docker image that setups the full ConnectIQ SDK here: https://github.com/cedric-dufour/connectiq-sdk-docker. It's funny because the Docker files are very similar, even if the goal is slightly different. This image could be a good base for another kind of GitHub Action.

I also have a message for the Garmin ConnectIQ team: it's still far too complicated to retrieve and set up the device files that are required to launch the simulator. A direct download link to these files would be good. An option to configure the path of the devices bits would also be appreciated (instead of hard-coding a path inside the user's home folder).
I also think it would be nice if Garmin provides and supports these kind of tools themselves. By the way, both the Docker image and the GitHub Action are licensed with the "unlicense" (public domain) so you can do whatever you want with them. I would be happy if you decide to copy and support them as your own.

jim_m_58 over 2 years ago

How does this help with testing the Ui of things, button interaction, recording/playing back a fit, etc?

You can't automate the Ui for example.

Even for a watch face, low/hi power. You can run "Run No Evil" but does that really test your app? There are cases where you want to test a sideload, as the sim has some issues. (It's a simulator, not an emulator - to quote Garmin!)

waterkip over 2 years ago in reply to jim_m_58

There is a whole lot of business logic you can test without an UI.

But UI components need more ways to inspect them. You cant query the amount of items an Menu2 object has for example. With views you cant inspect the amount of drawables a Dc has.

waterkip over 2 years ago

You can't download devices automaticly. Or... you can. But the problem is you either need to expose your credentials to the CI. Than you need to do some magic (I've documented the steps, but I'll call it magic) and you'll be able to download the devices and fonts.

Ps. I also have an image for Docker and CIQ: github.com/.../connectiq-sdk-docker

jim_m_58 over 2 years ago in reply to waterkip

Huh? when you can't test the UI, record or play back a fit file or switch from low to high power in a Watch face?

maybe to see if it compiles and you can charge for it? Is that the busines purpose?

waterkip over 2 years ago in reply to jim_m_58

Well, you can't test those things. But if you model your code correctly, you can exactly do any of those things in a test, eg..

Testing getting coordinates in a circle or a box:

https://gitlab.com/waterkip/opn-monkeyc/-/blob/master/source/t/TestMathCoordinates.mc

Test functions:

https://gitlab.com/waterkip/opn-monkeyc/-/blob/master/source/t/TestMore.mc

Conversion to various bits and pieces:

https://gitlab.com/waterkip/opn-monkeyc/-/blob/master/source/t/TestConversionWeight.mc

You can test all sorts of stuff you call from your views in various modes and you can all test that without the UI. I'm not familiar with all the bits and pieces from MonkeyC, but depending on how you order your code you can test a whole lot of it.

For my (unpublished) app, I'm doing most of the model work which I test and than hook it into the UI components. I can test the models and I just need to plug the correct method in the delegate and views.

jim_m_58 over 2 years ago in reply to waterkip

You are clearly VERY new to CIQ. How many apps have you actually published? I mean in the CIQ app store.

The total downloads of all your apps in the store?

Unless you've done testing on at least one real device, I wouldn't publish anything.

waterkip over 2 years ago in reply to jim_m_58

Doesnt invalidate a testsuite in a CI setup. Plus I don't participate in **** measuring contests.

jim_m_58 over 2 years ago in reply to waterkip

But you don't understand that's just a small part of testing.

So you have not published even one app and dealt with users on a bunch of devices...

waterkip over 2 years ago in reply to jim_m_58

Jim, you probably are very knowledgable but the fact that you just want to shoot down testing in a CI is just insane. While I acknowledge the fact that you can't test everything you don't seem to understand the fact that when you model your code accordingly that you can test behaviour of your models. Those behaviours can be triggered in tests.

matcora over 2 years ago

@jim_m_58 I really don't understand why you are spending time denying the usefulness of CI. Anyway, I will rephrase @waterkip who already explained it to you. It is true that for now we may not be able to test every part of the app, but we can still test a lot. If your application is well organized, you should have a model layer that is not linked to the UI. You can easily test this part automatically. I will give you a perfect example (like @waterkip already did, actually):

In my badminton app, I have a class that handles the state of a match as you can see here: github.com/.../Match.mc. I wrote some tests for this class here github.com/.../MatchTest.mc. What I do here is called unit testing. Being able to unit test your model automatically for free helps a lot. I don't know why you would deny this benefit.

@waterkip Thanks for the link to your repository, I'll have a look. The fact that you need credentials to download the device files is a real pity. Instead of searching for a convoluted way to download them, I would prefer that Garmin offers a direct download link, as they do for the SDK. Why would they make things so complicated?!