Big update to prettier-extension-monkeyc

I haven't try mcreduce yet, but looking at the example test function in github and remembering how log sometimes it takes for me to find a particular part of the generated code (because it becomes quiet a bit different from the original), it looks like the hard part will be to write the correct test function. Fortunately the comments with the original variable names can help.

In case of looking for the if not being removed maybe I would try the following trick and add some comment:

if (gaugeOnTop) {
  // LOOK FOR THIS IN THE OUTPUT
  ly += gaugeHeight1;
}

I figured out the bug, and will probably push a fix later today.

I also experimented a bit more with mcreduce, and got it to come up with:

import Toybox;
class a extends WatchUi {
    function b(c, e) {
        var d;
        var dy;
        var ly = d;
        if (gaugeOnTop) {
            ly += gaugeHeight1;
        }
        do {
            c.f(ly);
            ly = dy;
        } while (e);
    }
}

I've pushed the changes. So now there's a real world example as a template. The main changes I made were:

• turn on invalid symbol diagnostics, and bail if the optimizer produces any diagnostics
• tweak the regexes a little. In particular check that after optimization the if is gone, but "var lv" is still there AND there's a reference to lv later on. Otherwise creduce could simply remove everything after the if, except for a solitary "lv;". That would pass the regex on the input, but the optimizer could still optimize everything away. 

v2.0.56 is out.

This fixes the bug flocsy reported earlier, where an if statement went missing, and also makes some small improvements to dce in the post build optimizer.

v2.0.57 is out.

This adds some new optimizations. While working on the infrastructure for local coalescing in the post build optimizer, I noticed there were cases where sizeBasedPRE had pulled out a constant with the value zero (for example), but the function still had lots of literal zeros - as if PRE had missed some. When I looked closer, I realized that there are several monkey c constructs that can result in literals that don't appear in the source code.

For example "x = [1, 2, 3]" obviously involves the literals 1, 2 and 3. But in addition, the compiler also generates 0, 1, 2 and 3 as part of the array generation (it generates something very similar to "x = new [3]; x[0]=1;x[1]=2;x[2]=3;". To fix this, I need to port the PRE pass into the post build optimizer; but meanwhile, it was quite effective to simply check whether a literal was already in a local (or on the stack), and re-use it if so.

Other non-obvious cases are "x++", which is literally implemented as "x = x + 1" in the bytecode; and "-x" which is implemented as "0 - x" in the bytecode. I was able to rewrite those in the source-to-source optimizer, so that the PRE pass can see the literals, and optimize where appropriate.

Works well, both the original and the post optimized shrank.

I have a question: until now when I noticed something odd (i.e: size increased, or running code didn't behave as it was supposed to) I could look at the optimized code and understand what happened.

Also when I looked at ERA reports I could find the device in release.jungle and then open the relevant file and see in which line (in the optimized code) the error happened.

How are we going to do these things now, with more and more post-build optimizations? 

flocsy said:

How are we going to do these things now, with more and more post-build optimizations

You can always turn the post build optimizer off :-)

I've not actually tried ERA, but the post build optimizer updates all the internal tables, and everything in debug.xml, so in theory it should map back to the source just as well as the original (it certainly seems to work in the debugger). But obviously, if I have bugs, the source could look fine, while the byte code is broken.

Of course, thats true of Garmin's compiler too...

One thing I can do to help is to add an option to dump the byte code before and after (and write some documentation about exactly what it means - since it took me a while to figure it all out). But clearly, debugging that is going to be much harder than debugging broken source code.

But the reason for writing the post build optimizer is that there are lots of things that simply can't be done at the source code level.

This last release I found a couple of new things that I *can* do at the source level - and got some nice wins in the source level optimizer. There are still a couple more things like that (that I'm currently aware of), and there's always room for improvement in the existing optimizations. But ultimately, there's a lot that can only be done in bytecode.

How do you know if some opcode you'll use is really available on the real device? Can there be things that were added at some version of the VM and if you use that then it won't run on older devices or on devices who haven't upgrade the FW yet?

flocsy said:

How do you know if some opcode you'll use is really available on the real device

Good question!

I hadn't really thought about it - although I'm testing on an fr235, a fenix5xplus and an edge1030plus; so I've got a reasonably wide range of devices, and for the most part I see the same byte codes on all three.

The only exception (I've noticed) is "newba" - which allocates a new ByteArray (as opposed to "newa" which allocates a new array). That isn't supported on older devices. But while I do recognize, and optimize the initialization sequences for newba, the optimizer is never going to introduce a newba where there wasn't one already.

And apart from that, there really aren't any "exotic" byte codes. Its almost the minimal possible set of byte codes to implement monkeyc (I suppose things like newa/newba could have been implemented as system calls, and then use some magic in the system library - but that would have made arrays much more unwieldy).

Finally, most of the optimizations are just removing dead code, reordering code to remove branches, and sharing duplicate code blocks; where I do introduce byte codes that weren't there before its all very simple - eg push a local onto the stack (which has to be supported everywhere) instead of pushing a constant onto the stack (because I know the local has the constant in it already, and pushing a local takes 2 bytes, while pushing a Number takes 5).

v2.0.58 is out

Fixes this bug, improves some existing byte code optimizations, and adds a remove argc optimization.

Every function begins with a 2-byte argc bytecode, which says how many arguments the function expects. The only purpose I can find for the opcode is to check that it was called with the correct number of arguments, and raise a fatal error if not. Since Garmin's compiler checks the number of arguments that are passed at compile time (and my extension checks the number of args passed as you type), this seems like its not strictly needed (in the same sense that asserts aren't needed in release builds); so I've added an option (off by default) to remove them, saving 2 bytes per function. I've not seen a problem with any of my apps, or tests.

nice!

2.0.57:
> Sizes for optimized-X.original-fenix6: code: 10074 data: 2293 <
> Sizes for optimized-X-fenix6: code: 9816 data: 2293 <
2.0.58 default:
> Sizes for optimized-X.original-fenix6: code: 10074 data: 2293 <
> Sizes for optimized-X-fenix6: code: 9805 data: 2293 <
2.0.58 remove argc:
> Sizes for optimized-X.original-fenix6: code: 10074 data: 2293 <
> Sizes for optimized-X-fenix6: code: 9697 data: 2293 <