Big data

I can confirm similar error when I tried to use iterators and copy values to internal array (2.1.0 beta SDK used). Around 250 items were enough to crash app due to memory limit. I stopped investigations, as I wait for release. But now priorities changed and I realized heart rate iterators do not exist on many watches (without internal HR sensor, i.e.).

It was not possible to know size of the array, so I made such code: first pass is to know how many items is in collection, second pass is to copy items to internal array.
if (Toybox has :SensorHistory) {
mSensorIter = getIterator();
sample = mSensorIter.next();
while ( sample != null ) {
if (sample.data != null) { cnt+=1; }
sample = mSensorIter.next();
}

Writelog("Total cnt: "+ cnt.toString());

SensorValues = new [cnt];
SensorValuesW = new [cnt];

mSensorIter = getIterator(); // once more time - to get values

cnt = 0;
sample = mSensorIter.next();
while ( (sample != null) && (cnt < SensorValues.size()) ) {
if (sample.data != null) {
SensorValues[cnt]= sample.data;
Writelog(sample.data);
SensorValuesW[cnt]= sample.when;
Writelog(sample.when);

cnt+=1;
}
sample = mSensorIter.next();
}

Writelog("Total cnt: "+ cnt.toString());
Writelog(SensorValues.size());

}


Can anyone tell me when exactly a 'too many objects' exception is thrown? I have an array of about 150 objects.

I'm not sure exactly what the limits are, but I'd venture to guess that they limit varies by device firmware. Given your code, it looks like you will have at least one instance of number2, 150 instances of myclass, and then potentially two objects per myclass instance (assuming a and b refer to unique objects). So we're talking about ~450 objects.

If I increase it further I get 'too many objects' errors. I also get other random errors, such as 'unexpected type' errors.

This error is similar to an 'out of memory' error in that the virtual machine is only capable of tracking some number of objects and the system itself will have allocated some objects that count against the limit. You get to use what remains.

Has anyone else had this problem or found a way around it?

Use fewer objects would be my first suggestion. You may need to apply a different limit for each device, but you should be able to do this via device-specific resources. I'm not sure if it possible, but you might be able to catch the exception and deal with it, but it is probably easier to avoid the condition entirely by using fewer allocations.

Another possible solution is to reorganize your code. From what I can see, it looks like myclass just provides access to a and b via the methods x() and y(). You can reduce the number of allocations by 30% by simply re-organizing...

var a;
var b;

function myclass_x(index) {
// behave as if calling arr[index].x() in your old code
}

function myclass_y(index) {
// behave as if calling arr[index].y() in your old code
}

class MyApp extends App.AppBase
{
function initialize() {
AppBase.initialize();
}

function onStart(state) {
var n = 150; // figure out how many

a = new [n];
b = new [n];

// initialize `a' and `b'
}

function onStop(state) {
a = null;
b = null;
}
}


I seen to recall that the max number of objects is something like 256, but there may be more based on the device/vm.

I can confirm similar error when I tried to use iterators and copy values to internal array.

Is there some reason you'd want to make a copy of this data as opposed to just using the data directly?

Of course yes. Main reason - how many items are in collection? Then I want to get average value for selected period, min, max and access to the element by index. Don't forget time values can be non-linear and some values can be null. Need to analyze all these stuff. All of these is possible in array. Have no idea why it was made so complex and of course memory greedy.
For example you have different scales and decide to output graph. You cannot do this not iterating all the elements. However in array you need i.e. to select each 2nd or 3rd simply using array[idx*multiplier].

I'm guessing that the iterator is a facade over an implementation that pages in the data as needed. This is likely how the system is able to give you so many data points without running out of memory. Without it you'd never have access to 150 data points in the first place.

You should be able to do all of those things you want except access by index pretty easily in a single loop. If you are using the data to draw a graph, you just iterate over all of the samples, filtering samples as needed. It doesn't seem like it is that big of a problem.

Travis

Of course yes. Main reason - how many items are in collection? Then I want to get average value for selected period, min, max and access to the element by index. Don't forget time values can be non-linear and some values can be null. Need to analyze all these stuff. All of these is possible in array. Have no idea why it was made so complex and of course memory greedy.
For example you have different scales and decide to output graph. You cannot do this not iterating all the elements. However in array you need i.e. to select each 2nd or 3rd simply using array[idx*multiplier].

When you mention using an iterator, all I can think of the getHeartRateHistory in actmon, or the data in SensorHistory, and as Travis said, you get to in a cheap way already The readings are quite regular, and you'll see the bad readings in time, so why copy it? By the time you use the copy, the data itself could have changed anyway....

This version gives the average of n samples.

// given an iterator `it', advance through `n' valid samples, returning
// the average of the samples encountered. if less than `n' samples are
// found, null will be returned.
function advance(it, n)
{
assert (n != 0);

var first_data = null;
var first_when = null;

var data_off = 0;
var when_off = 0;
var count = 0;

var sample = it.next();
while (sample != null) {

if (sample.data != null) {

if (first_data == null) {
first_data = sample.data;
first_when = sample.when.value();
}

data_off += (sample.data - first_data);
when_off += (sample.when.now() - first_when);

count += 1;

if (count == n) {
break;
}
}

sample = it.next();
}

var result = null;
if (count == n) {
result = new SensorHistory.SensorSample();

// average of the data values
result.data = first_data + data_off / count;

// average of the times
result.when = first_when + when_off / count;
result.when = new Time.Moment(result.when);
}

return result;
}


Thanks all for the advice, and the code optimisation tips Travis!

I was a little surprised to find that despite having plenty of memory still available, I was getting errors due to the number of objects that I was creating. And that the simulator wasn't really representative of the device. The maximum number of objects available creates another constraint (on top of memory, screen size, input methods, etc.) that I'll need to work within.