LLMs for Monkey C - What's Your Experience?

Hi folks,

I've been experimenting with using Large Language Models (LLMs like GPT, Gemini, etc.) for Monkey C, and I'm getting mixed results. 

I'm curious about the community's experience. Have you successfully used LLMs?? And which model seem to do better??

Specifically, I'm wondering about:

  • Generating reliable code (both whole/new project from scratch and adding to existing)

  • Debugging tricky issues

  • Upgrading to new SDK version. (e.g. the whole changing of variable typing from a while back)

  • Any prompting tricks you've discovered?

  • Late to this but I've a fair bit of recent experience. Since August I've put four things on the store (StairSprint, ClimbPower, Gaelic Ref and Weekend Pint) and most of the Monkey C was written with Claude. I come from Python and computer vision, had never touched Monkey C before.

    flocsy is right, it doesn't really know the language. It knows Java-ish syntax and guesses at Toybox. The things that made it workable for me:

    Keep the actual logic in plain functions and unit test them. The climb power maths and the watch face week/fill maths are just functions with tests. That part the model is fine at, it's the Toybox glue where it makes things up.

    Keep a project notes file it reads every session. Target devices, memory limits, SDK version, and every gotcha I've hit. Half of that file is forum knowledge, like the simulator's monkeydo not reconnecting after you've driven its menus so you have to kill simulator.exe and java.exe first. Once something like that is written down it stops repeating the mistake.

    Tight loop. Build, push to the sim, paste the error back. It reads compiler and runtime errors well. Writing correct code blind, not so much.

    A couple of scripts around the simulator helped more than anything, one that builds and pushes to the running sim and a PowerShell one that clicks through it and screenshots the window, so the model can actually see the result on AMOLED, MIP and the 1-bit Instinct layout.

    Where it still falls over is memory on the small watches and anything the docs don't cover. For that it's still forum search first.

    So, a very fast pair of hands if you give it a good build loop. Not a Monkey C expert and won't become one from arguing with it.

  • So AI is not yet good for writing Monkey C code, but maybe it could "summarize" the whole forum in topics with an index and this could be added to the above GitHub repo and then another AI session could be better after reading the GitHub repo?

  • Well, IMO Grok did an acceptable job coding. Not always FTR, but overall good enough.

  • Thanks! Very interesting. Would you mind sharing those 2 scripts??? But sounds like you're on windows. I'm Mac.

    Did you by chance try that "skill" I posted in this thread last week? As I understand, that should teach it all/most of what it misses about the Monkey part.... I'm neck deep into a iOS project, so no more bandwidth right now.

    I also wonder if we could give it the dev forum as a resource. It should be able to read that and memorize it all. That's the strength of a LLM, right?

  • Since you're also on mac and interested in using LLMs to write Monkey C projects, it'd be interesting to see if my build system works for you - https://github.com/semolina-solutions/rules_ciq is a Garmin CIQ SDK extension for Bazel (the open-source version of the build system Google uses). Nowadays I use this in conjunction with Gemini to (almost) one-shot fairly elaborate Garmin apps, and it's quite LLM friendly since it includes samples and is otherwise reasonably well documented, plus it can do everything (building, simulator automation, etc) via scripting. Note that I think it should also work for non-mac platforms now, but I haven't personally tested it.

  • For mac users (or maybe for any users; it's built but not tested), I have a very LLM-friendly build system based on the Bazel build system that Google open-sourced. I've been (almost) one-shotting quite complex Garmin apps with it, since the system is well-documented with working samples, and can manage the SDK files and binaries itself through scripting. Search for semolina-solutions/rules_ciq on github (anti-spam rules won't let me post a link, guh). Lately I've made it capable of spinning up a simulator in a container and perform UI automation in order to do more elaborate testing and take screenshots.

  • Thanks for sharing Zac. I look forward to try it out.

    PS: I see you got flagged too. There is something weird with this thread. All but my post earlier today have been flagged. Just plane text....

  • It's not just you and not just this thread. In the past there was another sub forum under Connect IQ a forum about the forum. But unfortunately it was merged into this forum. That was full of reports like this, people were unable to post code, pictures, or some characters. For me it got better since, but apparently there is still place for improvements.