Big update to prettier-extension-monkeyc

I've posted about prettier-extension-monkeyc before, but I've added a bunch of new features that developers will probably like (well, I've been missing them, so maybe you have too).

The new features it implements for VSCode include:

  • Goto Definition. Point at a symbol, Ctrl/Cmd click, and it will take you to the definition. Or F12
  • Goto References. Right click on a symbol and select "Goto References". It will show you all the references. Or Shift-F12
  • Peek Definition/Peek References. Same as above, but in a popup window so you don't lose your place in the original document.
  • Rename Symbol. Right click on a local, function, class or module name, and select "Rename Symbol". It will rename all the references. It doesn't yet work for class members/methods.
  • Goto Symbol. Type Ctrl/Cmd-Shift-O and pick a symbol from the drop down (which has a hierarchical view of all symbols in the current file). This also appears as an outline across the top of the file.
  • Open Symbol By Name. Type Ctrl/Cmd-T, then start typing letters from a symbol name. A drop down will be populated with all matching symbols from anywhere in your project.

Older features include a prettier based formatter for monkeyc, and a monkeyc optimizer that will build/run/export an optimized version of your project.

[edit: My last couple of replies seem to have just disappeared, and the whole conversation seems to be in a jumbled order, so tldr: there's a new test-release at https://github.com/markw65/prettier-extension-monkeyc/releases/tag/v2.0.9 which seems to work for me on linux. I'll do more verification tomorrow, and push a proper update to the vscode store once I'm sure everything is working]

  • So this looks like some serious bug in the compiler, but still what I don't understand is why this happens only when I compile with the prettier optimizer

    My guess is that that MyApp/bin/optimized/group005-debug/source/source/test/Test.mc is a stale version of the file? Maybe my extension is finding an old copy via some arcane monkey.jungle rule that only applies to fenix6? if thats the case its probably a bug in how I interpret the jungle file...

  • No, the "generated" Test.mc is as it should be, exact copy of the original (as it only has empty lines and comment lines).

    Also, the problem only happens when I add a Dictionary variable or const to global scope.

  • Another interesting curiosity: "recursive usage of prettier optimizer": I copied the generated output to the source folder of a new project to test how that compiles, and I accidentally compiled it with prettier which generated different code and bigger data size than without prettier.

    And I managed to reproduce the problem:

    garbagestr.zip

    You'll need to edit run.sh, but the main point is this:

    1. you comment out dfminApp.mc#6: 

    var TEST_DICT_ARR as Dictionary<Number, Array> = {} as Dictionary<Number, Array>;
     
    2. you compile with -g and save stderr to dump.txt
    3. grep -E '^( STRING "/Users/|str_Users_)' dump.txt
    You'll see 4 lines, that are basically 2 strings for dfminApp.mc and dfminView.mc
     
    So far we can say that this is because of the debug build.
    But then:
    4. comment in dfminApp.ms#6, and do step #2, #3 again. You 'll see now that 2 more strings appear: Empty.mc and CommentOnly.mc
    So I think there are at least 1 bugs in the compiler:
    adding a global Dictionary causes all files that don't have even 1 line of code to be compiled in with "debug" info.
    So I think one thing that the code generator can do until Garmin fix this: remove files that don't have meaningful code (only comments, empty lines, and if it's possible then maybe even files that end up only having imports (because all the constants and functions in them were inlined)
    This is still not a full fix, that probably can only be made by Garmin.
    Ah and the other interesting thing that in my real project there's no difference when I compile without prettier but all these strings are added when I compile with prettier. I don't think though that you'll be able to see that with this project.
    Let me know if you find other interesting things.
  • and bigger data size than without prettier

    But this is what I've said before - if you compare debug builds, then the optimized data size will look bigger, because the paths to bin/optimized/source/foo/bar/baz.mc are longer than the paths to the originals.

    So yes, if you run again on the output, the paths are just going to get longer, and the data will get bigger. You generally don't notice this effect because the optimizer reduces the data size enough that it still gets a win, even with debug builds...

    If you set the output path to /tmp/x instead, you'll (probably) find that the data size is smaller (assuming the path to the root of your project is longer than /tmp/x/group_000/source).

    So I think there are at least 1 bugs in the compiler

    I guess what's happening is that if the global module requires an <init> method, then it generates debug strings for every file (since they all "contribute" to the global module). But I agree that its silly to do so for files that produce no code at all (although again, this only affects debug builds, so not a big deal).

    So presumably in your original project, fenix6 didn't require an init method until you added the dictionary, while the other devices *did* require one.

    FYI, an init method is needed when you initialize a heap based type (so, Array, Dictionary, Long or Double), or the initializer requires code - ie "var foo = bar();" as opposed to "var foo = 42;"

  • Opps, I did it again... I'm not used to VSC, and I don't really understand also why are there so many ways to build/run. I am OK with having different shortcuts to go the same, but in VSC I know at least these ways to build, all have slightly different meaning:

    Command+Shift+P Prettier Monkey C: Build and Run Optimized project (this is what I usually do)

    But there are also the different options from launch.json:

    Run App, Build and Run Optimized - Debug, Build and Run Optimized - Release

    Can you rename "Prettier Monkey C: Build and Run Optimized project" to "Prettier Monkey C: Build and Run Optimized project - Debug" and add "Prettier Monkey C: Build and Run Optimized project - Release" ?

  • I noticed that constants that are Array or Dictionary are kept in the optimized code

    So I just took a look at this - and I'm not seeing that. In fact there's a bug - it removes them even if they're initialized by a function.

    eg "const foo as Array = [bar(), baz()];" will be removed, even though bar() and baz() may have important side effects.

    Did you mean that *variables* don't get removed? Because thats true. For no good reason, I only remove unused constants, not unused variables. But it has nothing to do with arrays and dictionaries...

    Also note that "unused" is pretty conservative. If you have a constant named FOO, and somewhere in your program you mention :FOO, it won't remove the constant (because you can access the global FOO as $[:FOO]). Similarly if you have a variable x, and say x.FOO, it won't remove it, because you might have said "x = $;" somewhere. This is all old code that predates my type checker, so I could do much better now; but as I've said a couple of times recently, Im going to rewrite the dead code removal anway...

    So - if you have a specific example that fails can you post it?

  • ok the :FOO thing probably explains what you did not drop it at some phase, though it doesn't explain why it's kept at the end because every other mention of "FOO" is gone because of the constant inlining:

    const TEST_FALLBACK = 12;
    const TEST_DICT =
    {
    26008707 => [null],
    26008205 => [null],
    26008713 => [null],
    } as Dictionary<Number, Array>;

    (:inline, :use_or_not)
    function test_me(h as Number) as Number? {
    var arr = TEST_DICT[h];
    return arr == null ? TEST_FALLBACK : arr[0] as Number?;
    }
    So when both constants are only mentioned in test_me, and test_me isn't used at all (depending on :use_or_not) then test_me is correctly removed, and also the Number const, but the Dictionary const is kept without any mention of it in the generated code

  • But there are also the different options from launch.json

    The options in launch.json are whatever you put there - including whatever their names are. There are no default launch configurations. I posted mine a while back, and I think you may have copied them.

    Also, you can add build configs to tasks.json - but those are less useful except for quick rebuilds to check on sizes, and for customized export settings.

    Can you rename "Prettier Monkey C: Build and Run Optimized project"

    I could; but basically I was trying to keep things simple. If you want more complex options, you can create them using launch.json and tasks.json.

    Also, it mirrors Garmin's "MonkeyC - Build Current Project" command, which takes the debug/release from your current settings.

  • So when both constants are only mentioned in test_me, and test_me isn't used at all (depending on :use_or_not) 

    It's not quite clear what you mean by that.

    If I try your example, and set use_or_not as an excludeAnnotation, then test_me, TEST_FALLBACK, and TEST_DICT are all gone. If thats not the case for you, its a bug.

    On the other hand, if I don't set use_or_not as an excludeAnnotation, and call test_me from somewhere, then test_me and TEST_FALLBACK are indeed gone, while TEST_DICT is still there - but it needs to be, because its used at the point where test_me was inlined.

    Finally, if I don't set use_or_not as an excludeAnnotation, and don't call test_me from anywhere, then again, test_me and TEST_FALLBACK are gone, while TEST_DICT is still there. And this is the same problem I've mentioned a few times recently. At the end of the optimization phase, test_me looks like

    const TEST_FALLBACK = 12;
    const TEST_DICT = {
      26008707 => [null],
      26008205 => [null],
      26008713 => [null],
    } as Dictionary<Number, Array>;
    
    (:inline, :use_or_not)
    function test_me(h as Number) as Number? {
      var arr = TEST_DICT[h];
      return arr == null ? 12 : arr[0] as Number?;
    }
    So test_me isn't referenced from anywhere, TEST_FALLBACK isn't referenced from anywhere (because its value was substituted into test_me), but TEST_DICT is still referenced from test_me (and you don't want to substitute its value into test_me, because there's a *lot* of code to initialize that dictionary).
    So now the unused code pass runs, and removes TEST_FALLBACK and test_me, but leaves TEST_DICT, because it's "used". If I ran the optimization phase again, and then ran the unused code phase again, it would indeed remove TEST_DICT. But thats an extremely inefficient way to do it, and could still miss things - so I'd need yet another pass.
    Worse, if we had foo calls bar, and bar calls foo, but both are otherwise unused, I could keep going forever, and never remove them.
    The solution (as Ive said before) is to start from all the entry points, and work outwards to all the reachable code. And Im going to do that.
    But for now, if you have literally unused code/data, the best thing is to exclude it via exclude annotations.
  • v2.0.51 is out.

    The big change in this version is that I've added a post-build optimizer. That takes the .prg file, disassembles the code section, optimizes it, and writes out a new .prg file.

    At -O0 or -O1 this saves a lot (Garmin's compiler leaves a lot of dead code at these optimization levels). For me, around 500 bytes on an 11k executable.

    At O2 the savings are much less - only 108 bytes for my project. But it depends what you do. If you use try/catch much, it will save you a lot. If you have a lot of global or module scope functions, it will probably save quite a lot. And I have a number of planned optimizations that should make things much better.

    For now this is "experimental" so it's off by default. You can turn it on in settings via the "Post Build Optimizer" setting, or you can turn it on in specific tasks or launch configs via the "postBuildOptimizer" field.