A Whole New World of Graphics with Connect IQ 4

The smallwearable2021 and wearable2021 in the Connect IQ SDK manager represent the first products that support API level 4.0.0. We're releasing these device configurations in advance of launch so you can begin porting your apps to these new products and also try out the new capabilities of API level 4.0.0. This post covers some of the new graphics features available.

Bitmap Packing Formats

Images can grow your executable size, which can add extra wait when users install or update your app. To help reduce executable bloat, Connect IQ 4.0 introduces bitmap attribute for packing an image into your executable.

ATTRIBUTE DEFINITION VALID VALUES DEFAULT VALUE NOTES
packingFormat The format with which the image will be encoded into the PRG default, png, jpg, yuv. default Only available on certain devices

Each of the formats can have their advantages and disadvantages:

FORMAT ADVANTAGE DISADVANTAGE USE CASE
default Available on all products, fastest to load, supports alpha channel No compression App runs on pre-Connect IQ 4.0 devices. Low palette images can have very small runtime costs
png Lossless, compressed and supports alpha channel Slowest to load, which can add runtime cost if purged and reloaded frequently from the graphics pool Importing non-photo images with or without alpha channel
jpg Compresses very well, fast to load Lossy format and does not support alpha channel Importing photo imagery without alpha channel
yuv Compresses well, supports alpha channel, fast to load Lossy format Importing photo imagery with alpha channel

Compile Time Image Scaling

The resource compiler can scale your image resources at build time. Images can be scaled to a pixel size, to a percentage of their size, or to a percentage of the screen’s size.

ATTRIBUTE DEFINITION VALID VALUES DEFAULT VALUE NOTES
scaleX How should the image be scaled in the x dimension? Pixel size or percentage If scaleY is set, will default to scaleY’s value. Otherwise will default to 100% of image width See scaleRelativeTo
scaleY How should the image be scaled in the x dimension? Pixel size or percentage If scaleX is set, will default to scaleX’s value. Otherwise will default to 100% of image height See scaleRelativeTo
scaleRelativeTo What should the scale factor be based on? Screen or image Screen Sets what to base relative scaling on. If set to screen, image will be re-scaled based on product it is being built for at compile time

Alpha Blending

Connect IQ 4.0 adds some powerful new tools to the Dc:

FUNCTION PURPOSE ACCEPTS API LEVEL
Dc.setFill() Set fill tool for drawing primitives. Graphics.ColorType, Graphics.BitmapTexture 4.0.0
Dc.setStroke() Set pen tool for drawing primitives Graphics.ColorType, Graphics.BitmapTexture 4.0.0
Dc.setBlendMode() Set blend mode for drawing Graphics.BlendMode 4.0.0

Previously, the setColor() API allowed the setting of a foreground or background color based on a 24-bit RRGGBB value. setFill() and setStroke() both accept 32-bit AARRGGBB values, allowing you to provide an alpha channel value with the RGB value. The setStroke() API allows setting the pen tool for the Dc, while setFill() sets the fill tool.

You can also set the blend mode with setBlendMode(). By default, the system will blend your color with whatever is being drawn over. However, you can use BLEND_MODE_NO_BLEND to set the color and alpha of a BufferedBitmap directly. 

In addition to colors, you can now also provide a BitmapTexture. This allows a primitive to be filled by a bitmap and opens up many new drawing possibilities.

Graphics Pool

In the past, all resources loaded at runtime into the application heap. This heap is used to hold your code, data, stack and runtime objects, so loading images could quickly limit the runtime functionality of your app.

Connect IQ 4.0 introduces a new graphics pool that is separate from your application heap. When you load a bitmap or font at runtime, the resource will load into the graphics pool, and you will be returned a Graphics.ResourceReference.

The graphics pool will dynamically cache, unload and reload your resources behind the scenes based on available memory. All the drawing primitives that accept resource objects also accept references, as well, so your app should not have to be reworked to take advantage of the new system.

Calling ResourceReference.get() on a reference will return a resource object. As long as the object returned is in scope, the resource will be locked in the graphics pool.

Buffered Bitmaps and the Graphics Pool

BufferedBitmap objects, like other graphics resources, now take advantage of the graphics pool, as well. The advantage of this scenario is that you can now liberally use temporary graphics buffers without running out of application heap.

As noted earlier, the graphics pool will intelligently purge and restore resources from the pool if the loaded resources exceed the available pool space. Unlike static resources that are reloaded from your executable, BufferedBitmap are not restored if they have been purged. This works fine if you are using a short-lived, temporary buffer, but if your bitmap is purged after allocation, you need to re-render its contents. Alternatively, you can call the get() method on the reference to get a locked version of the bitmap. This will prevent the BufferedBitmap object from being purged from the pool, but it can also lead to the graphics pool running out of available space if more resources are loaded.

To create a BufferedBitmap, use the Graphics.createBufferedBitmap() API. If your application runs on pre-Connect IQ 4.0 devices, use a has check for allocating your BufferedBitmap:

import Toybox.Graphics;

//! Factory function to create buffered bitmap
function bufferedBitmapFactory(options as {
            :width as Number,
            :height as Number,
            :palette as Array<ColorType>,
            :colorDepth as Number,
            :bitmapResource as WatchUi.BitmapResource
        }) as BufferedBitmapReference or BufferedBitmap {
    if (Graphics has :createBufferedBitmap) {
        return Graphics.createBufferedBitmap(options);
    } else {
        return new Graphics.BufferedBitmap(options);
    }
}

Putting it into Action

To wrap our heads around these new APIs, let's create an animated watch face for the upcoming wearables. With our new alpha blending capabilities, we can have semi-transparent overlays drawn in real time. We are going to use this to create a fade to white effect to fade between multiple images.

To start, we will add our images to resources:

<drawables>
    <bitmap id="image1" filename="pretty_picture1.png" scaleX="100%" scaleRelativeTo="screen" packingFormat="jpg"/>
    <bitmap id="image2" filename="pretty_picture2.jpg" scaleX="100%" scaleRelativeTo="screen" packingFormat="jpg"/>
    <bitmap id="image3" filename="pretty_picture3.jpg" scaleX="100%" scaleRelativeTo="screen" packingFormat="jpg"/>
    <bitmap id="image4" filename="pretty_picture4.jpg" scaleX="100%" scaleRelativeTo="screen" packingFormat="jpg"/>

    <bitmap id="LauncherIcon" filename="launcher_icon.png" />
</drawables>

You'll need to pick four images (preferably with a square aspect ratio) to use. This takes advantage of both compile time image scaling to resize the image to the screen size as well as JPG image packing to reduce the resource size of the PRG. Next, we will create a standard layout:

<layout id="WatchFace">
    <drawable id="Blinds" class="FadeDrawable"/>

    <label id="DateLabel" x="45%" y="5%" font="Graphics.FONT_SMALL" justification="Graphics.TEXT_JUSTIFY_RIGHT" color="Graphics.COLOR_BLACK" />
    <label id="TimeLabel" x="center" y="78%" font="Graphics.FONT_NUMBER_MEDIUM" justification="Graphics.TEXT_JUSTIFY_CENTER" color="Graphics.COLOR_BLACK" />
    <label id="BatteryLabel" x="55%" y="5%" font="Graphics.FONT_SMALL" justification="Graphics.TEXT_JUSTIFY_LEFT" color="Graphics.COLOR_BLACK" />
</layout>

I'm going to assume you can handle wiring up the date and battery fields. The FadeDrawable is where we will be doing all our work. This will be a drawable that uses WatchUi.animate()  to create the fade effect:

import Toybox.Application;
import Toybox.Application.Storage;
import Toybox.Lang;
import Toybox.Graphics;
import Toybox.Timer;
import Toybox.WatchUi;

//! Class to execute a fade effect across multiple bitmaps
class FadeDrawable extends WatchUi.Drawable {

    //! What part of the fade are we executing?
    enum State {
        STATE_FADE_IN,
        STATE_HOLD,
        STATE_FADE_OUT
    }

    //! Key for storing where we left off
    private const IMAGE_INDEX_KEY = "imageIndex";

    private var _imageArray as Array<Symbol>;
    private var _curImage as BitmapType?;
    private var _curImageIndex as Number;
    private var _state as State;
    private var _nextState as State;
    private var _timer as Timer.Timer;

    //! This variable is public so it can be modified by animate
    public var fade;

    //! Constructor
    //! @param options Layout options
    public function initialize(options as Dictionary) {
        Drawable.initialize({:identifier => options[:identifier]});
        // Array of images from resources
        _imageArray = [
            Rez.Drawables.image1,
            Rez.Drawables.image2,
            Rez.Drawables.image3,
            Rez.Drawables.image4
        ] as Array<Symbol>;

        // Restore the image index if we persisted it
        var index = Storage.getValue(IMAGE_INDEX_KEY);
        if (index instanceof Number) {
            _curImageIndex = index;
        } else {
            _curImageIndex = 0;
        }

        // Initialize values
        fade = 0;
        _state = STATE_FADE_IN;
        _nextState = STATE_FADE_IN;
        _timer = new Timer.Timer();
    }

    //! Handler to switch between states
    public function handleStateChange() as Void {
        _state = _nextState;
        switch(_nextState) {
            case STATE_FADE_IN:
                // Clear the current image
                _curImage = null;
                // Load the next one
                _curImage = Application.loadResource(_imageArray[_curImageIndex]);
                // Advance the image count
                _curImageIndex ++;
                _curImageIndex = _curImageIndex % _imageArray.size();
                // Persist where we are
                Storage.setValue(IMAGE_INDEX_KEY, _curImageIndex);
                // Kick off the animation
                WatchUi.animate(self, :fade, WatchUi.ANIM_TYPE_EASE_OUT, 254, 0, 1, method(:handleStateChange));
                _nextState = STATE_HOLD;
                break;
            case STATE_HOLD:
                // Wait three seconds
                _timer.start(method(:handleStateChange), 3000, false);
                _nextState = STATE_FADE_OUT;
                fade = 0;
                break;
            case STATE_FADE_OUT:
                // Animate the fade back to white
                WatchUi.animate(self, :fade, WatchUi.ANIM_TYPE_EASE_OUT, 0, 254, 1, method(:handleStateChange));
                _nextState = STATE_FADE_IN;
                break;
        }
    }

    //! Kick off the animation
    public function start() as Void {
        _state = STATE_FADE_IN;
        _nextState = STATE_FADE_IN;
        handleStateChange();
    }

    //! Terminate the animation
    public function stop() as Void {
        WatchUi.cancelAllAnimations();
        _timer.stop();
    }

    //! Draw the images
    public function draw(dc as Dc) as Void {
        var bmp = _curImage;
        // Make the fade color (white + alpha)
        var color = Graphics.createColor(fade.toNumber(), 255, 255, 255);

        if (bmp != null) {
            // Draw the bitmap
            dc.drawBitmap(0, 0, bmp);
            // Draw the fade on top
            dc.setFill(color);
            dc.fillRectangle(0, 0, dc.getWidth(), dc.getHeight());

            // Draw the semi-transparent circles above and below
            // to offset the time text
            dc.setFill(0x80FFFFFF);
            dc.setStroke(0x80FFFFFF);
            dc.fillCircle(dc.getWidth() / 2, dc.getHeight() * 1.5, dc.getWidth() * .7);
            dc.fillCircle(dc.getWidth() / 2, -(dc.getHeight()/2), dc.getWidth() * .7);
        }
    }
}
 

Here is the watch face in action, featuring appearances by Commodore the dog:


A Whole New World

As you can see the new APIs open a lot of possibilities to add graphical "oomph" to your apps. The APIs are available and the products are coming, so now is the time to think about how you can get your apps ready.