any way to create an efficient byte array?

There have been a couple of threads that touch on the subject but none have specifically addressed this to the best of my knowledge. But is there any way to create a byte array that isn't wasting a lot of memory, where you really are only using 1 byte per element, plus a small overhead for the entire array? With the limited resources on these devices (especially some), having an array of numbers where the values always fits in a byte could be really useful. One very cumbersome, and still inefficient way would be to have an array of long's that you break apart to access individual bytes with masks but I wouldn't consider that a great solution.
  • No, there is really no great way to do this. You could definitely make an array of Long and then use bit manipulations as you suggest. It wouldn't be difficult, but it wouldn't be pretty either. Of course it could potentially save you 8x the memory, so it may be worthwhile.

    Do you really need to store that many values?

    Travis
  • I have a few things that save a fair amount of data (lat/lon pairs for example), and a couple of things to consider are how you store the data, and how own you save the data. Also, no matter how much data storage you have, at some point you could run out and need to handle that. Here's a few things I'll do

    In the case of lat/lon, I store them as floats, so no easy way to squeeze them into less space, but for example, I don't store them in a 2 dimensional array, as that's really "an array of arrays", and can chew through objects fast. Instead, I use either (2) one dimensional arrays, or a single 1 dimensional array, with the first half for lat, and the second half for lon.

    When it comes to the size, I'll also figure that out at runtime, using "freeMemory"/"totalMemory", so it can vary based on code you add/remove, and 1.x (64k limit for apps) vs 2.x (122k or more limit for apps). (so you don't get into a case where adding a small function pushes you past the memory limit crashing the app)

    As far as the frequency, using lat/lon again, I'll also do something like initially save at say every 30 seconds. When the space is filled, I'll reduce it by removing every other value, and then double the time between samples, so 1 hr of 30 second samples, becomes 2 hr's of 60 second samples, and when that fills, the same space will hold 4hrs of 2 minute samples, etc, so I can run for a long time, and never run out of space, just resolution of the data.

    As far as using a big chuck of data, you'll also want to be careful, and not do things like walk the array all the time doing something like looking max/min/avg, but instead figure that out as you add to the array, as when you're dealing with a large number of data points, you'll not only impact performance of your app, but also get into the "watchdog danger zone", where the watchdog timer fires and kills your app for executing too long. Again, using lat/lon, with say 1000 points, you don't want to display all 1000 points, as that could (likely will) trigger the watchdog. but instead, only display every 5th point, and never try to use more than say 200 points, for example. (it's not just displaying the point, but calculating the screen position to display it, as that could change with each new sample)
  • Do you really need to store that many values?

    Travis


    Right now this was as much an "exercise" as a "need". With such limited memory, hard to feel good about 13x overhead for a byte. Handling the payload data from the Tracker Profile, which can have up to 20 dogs, each dog comes across as 4 separate 8 byte packets. The code could have been simpler and quicker just to save the packets in a single byte array that grows as the number of reported dogs grows. But then I realized even the payload array isn't really even a byte array. Reality is that I'll never have more than 2 to 4 dogs. But the code should handle more.

    I already am pushing up against the 64k (really 52k) limit of my 630 by my "watch" app. This is the only app I run 95% of the time as it lets me see the current time, get data from my personal weather station, control the lights in my house, and record an activity with a couple of 10 fld data pages. I will be adding a basic dog tracking page to this. So I'm always trying to think about how to reduce the memory foot print. I still have 1 custom font taking up 3k so I might have to ditch that until they come out with a 640 with more memory / app.

    I could break this up into multiple apps (I do have more sophisticated apps for golfing and full activity app including mapping when I need more functionality) but it is really handy still having access to my home automation while I'm recording an activity or being able to track my dog while I'm recording a run.

    I already do all of Jim's suggestions with arrays. I never use dictionaries. Jim's response did make me think I might want to look into storing my lat/lon pairs for my mapping page as long's using the semi-circle format where each lat and each lon is in 32 bit format. Currently I'm limited to about 200 data points.
  • If you're tight on memory, dropping the custom font will help, along with things like bitmaps (every few hundred bytes help! :) ), and if you use layouts, look at just using dc calls instead.

    Like I said, I use floats for lat/lon pairs, and have a data field that saves over 200 in a 16k DF on 1.x devices. I don't recall the specifics, but was cheaper memory wise than other types (the only comment in my code is "convert to floats to save memory" :) You might lose a bit of precision, but GPS itself is only correct within a certain range ("accuracy is 20 feet" or something, on devices that show you that kind of info)
  • I want to use a multidimensional array to test whether lines intersect. I was trying to implement some functions and I quickly ran out of memory, so it looks like I'm gonna have to conserve. I'm having a hard time understanding what the exact procedure would be to create arrays to work with.. like what code to type...

    (1) in particular, how would I create an array of type "floats"?

    (2) and could I save some memory by creating a const array at the start of my project to handle the "boundaries" which will not change throughout my program, i.e.,
    static const points = [
    2.74744, 9.37372,
    8.36362, 8.22383,
    7.32951, 3.38384,
    2.02910, 4.29191
    ];

    Can you post a link to an example or throw some code up for me to look at? There has to be some code in some of these forums pertaining to working with arrays...

    (3) and, in terms of memory usage, is there a quick and dirty way to understand how much memory I have to work with, during run-time, for debug purposes? Something involving a GetSystemStats function?
  • An array isn't type dependant. You can store floats, strings, booleans, etc, in any array

    Ok, to create a multidimensional array, here's the basic for something like a 20x2:
    var dim1=20;
    var dim2=2;
    var array=new[dim1];
    for(var i=0;i<dim1;i++) {
    array=new[dim2];
    }
    [/code]
    So you actually create 21 arrays in this case. It's not cheap object wise (you just used 21 for just the array), and if you wanted to make it 100x2, you'd likely run out of objects.

    So instead, what you can do is use 2 arrays:

    var dim1=20;
    var index0=new[dim1];
    var index1=new[dim1];

    In this case you only create 2 arrays, and if you change dim1 to 100, you still only have 2 arrays. [100x2] would use more memory, but not more objects.

    Or, use just 1 array:
    var dim1=20;
    var array=new[dim1*2];

    You have an array that's double the length needed (but only 1 array), and when you want to access values, (say at "idx"), they are in array at array[idx] and array[dim1+idx]. So from the first case that uses 21 arrays for a [20x2], you can use 1 array for a [20x2], and this may be something you'd really want to consider for something like a [20x20] array.

    BTW: Sys.getSystemStats() - freeMemory, totalMemory,usedMemory in your code.

    In the sim, the bottom line shows current usage, max available, and peak memory.
  • THANKS!!!, I will examine this... and btw, in terms of my (3) third question, I found this code helps:

    using Toybox.System as Sys;
    ...

    var mys;

    mys = Sys.getSystemStats();
    Sys.println("total [" + mys.totalMemory + "] - used [" + mys.usedMemory + "] = free [" + mys.freeMemory + "]");

    Happy holidays
  • I want to use a multidimensional array to test whether lines intersect. I was trying to implement some functions and I quickly ran out of memory, so it looks like I'm gonna have to conserve. I'm having a hard time understanding what the exact procedure would be to create arrays to work with.. like what code to type...

    (1) in particular, how would I create an array of type "floats"?

    (2) and could I save some memory by creating a const array at the start of my project to handle the "boundaries" which will not change throughout my program, i.e.,
    static const points = [
    2.74744, 9.37372,
    8.36362, 8.22383,
    7.32951, 3.38384,
    2.02910, 4.29191
    ];

    Can you post a link to an example or throw some code up for me to look at? There has to be some code in some of these forums pertaining to working with arrays...

    (3) and, in terms of memory usage, is there a quick and dirty way to understand how much memory I have to work with, during run-time, for debug purposes? Something involving a GetSystemStats function?


    There isn't really such a thing as an array of floats. It's an array of objects and every element in this case would be a float. So there is a basic overhead for the array of objects. Then you add in the memory for each object, which in this case is a float. In a language like C, think of it as an array of pointers, each pointer pointing to a float. As long as each object is just a base type (ie float, int, long, bool), the memory is pretty constant. It is when the objects themselves are more complex such as an array or dictionary, the memory usage quickly goes up.

    I don't believe creating the array as a const has much impact on memory usage.

    When running in the simulator, at the bottom, there is a line telling you how much you are currently using and the maximum you have used during the session.

    Looks like you are already doing one of the most important things and that is just using a single dimensional array.

    const D2_COUNT = 2; //size of second dimension
    var array2D = new [cElements * D2_COUNT ];

    for (var i = 0; i < cElements; ++i)
    {
    arrray2D[i*D2_COUNT] = x; // array2D[i,0]
    arrray2D[i*D2_COUNT+1] = y; // array2D[i,1]
    }

    If I have a large array of objects, I'll do the following:
    enum {OFLD1, FLD2, FLD3, FLD4, FLDCOUNT}
    var arrayObj = new [cElements * FLDCOUNT];
    arrayObj[iObj*FLDCOUNT+FLD2] = n; //setting FLD2 of element[iObj];
  • I don't believe creating the array as a const has much impact on memory usage.


    Same here - the same space and just saying to the compiler, "don't let anyone change me".

    But one of the Garmin compiler folks would know for sure, and maybe it won't. Maybe Coleman can ask when he gets back.