FlowState

do you have any experience about amount of data in makeWebRequest?

 It's a shame the data doesn't have unix time, as now, "dateCreated" need to be parsed to understand what it is.  With unix time, it's a simple comparison of two numbers.

jim_m_58 said:

If you save the newest 10 (and sort them while saving) you get just what you need.  You can't trust the order in a dictionary, but if you can see all the values, no problem!

I still have no idea how you will determine the "newest 10" entries *before* you sort (as you seem to be implying). You have to sort all the values to determine the newest 10, since the dictionary you end up with is unordered, as you said.

EDIT: now I see that this is possible if you keep a running tally of the 10 newest entries, although that seems like more work (code-wise) than just sorting the whole list, especially if 10 entries that you return have to be sorted. (You might gain some efficiency if the complete list is huge, although ofc Monkey C can't handle huge amounts of data in a web request.)

jim_m_58 said:

It's a shame the data doesn't have unix time, as now, "dateCreated" need to be parsed to understand what it is.  With unix time, it's a simple comparison of two numbers.

dateCreated is in the very common standard ISO 8601 format. e.g.

"dateCreated": "2022-07-01T21:10:37.972Z"

You can sort data of this type with simple string comparisons (*) -- no need to parse the data. (* Assuming it's all in the same time zone -- since this data is in UTC as per the Z suffix, it's reasonable to assume that it's always in UTC and not local time.)

Only problem is that Monkey C doesn't support comparing strings with ">" and "<", so you'd have to write the string comparison function yourself (which would compare two strings lexicographically). You'd also have to write the sort function yourself.

e.g.

Something like this:

    // Compare two strings
    // Returns:
    // -1 if a < b
    // 0 if a == b
    // 1 if a > b
    function compareStrings(a, b) {
        var a_array = a.toCharArray();
        var b_array = b.toCharArray();

        var minLength = a_array.size() < b_array.size() ? a_array.size() : b_array.size();

        for (var i = 0; i < minLength; i++) {
            if (a_array[i] < b_array[i]) {
                return -1;
            } else if (a_array[i] > b_array[i]) {
                return 1;
            }
        }

        if (a_array.size() == b_array.size()) {
            return 0;
        } else if (a_array.size() < b_array.size()) {
            return -1;
        } else {
            return 1;
        }
    }

FlowState said:

I still have no idea how you will determine the "newest 10" entries *before* you sort (as you seem to be implying). You have to sort all the values to determine the newest 10, since the dictionary you end up with is unordered, as you said.

There is a time stamp in each record (though it is ascii and you need to parse it).  That gives you the order.

jim_m_58 said:

There is a time stamp in each record (though it is ascii and you need to parse it).  That gives you the order.

As I just posted above, you *don't* have to parse it, you just use string comparison, which is much easier.

For example in javascript: "2022-07-01T21:10:37.972Z" < "2022-08-01T12:10:37.972Z" == true


In C: strcmp("2022-07-01T21:10:37.972Z", "2022-08-01T12:10:37.972Z") == -1

Using the Monkey C compareStrings() function I posted above: compareStrings("2022-07-01T21:10:37.972Z", "2022-08-01T12:10:37.972Z") == -1

Note that the ISO 8601 date/time format was obviously designed with this use case in mind:
https://en.wikipedia.org/wiki/ISO_8601#General_principles

• Date and time values are ordered from the largest to smallest unit of time: year, month (or week), day, hour, minute, second, and fraction of second. The lexicographical order of the representation thus corresponds to chronological order, except for date representations involving negative years or time offset. This allows dates to be naturally sorted by, for example, file systems.

• Each date and time value has a fixed number of digits that must be padded with leading zeros.

Also not to state the obvious, a great advantage of this format over "seconds since the Unix epoch" is that it's human readable.

FlowState said:

still have no idea how you will determine the "newest 10" entries *before* you sort (as you seem to be implying).

EDIT: Ok I see what you mean. You would keep track of the latest 10 timestamps as you iterate through the unsorted list of values, rather than sorting first. Sorry for being super obtuse there.

Yes, your way of tracking the latest 10 as you go would work, but you *would* have to parse the data to be able to keep track of the latest 10 values as you iterate through an unsorted array of values.

My way of sorting the whole list first (by simply comparing the timestamps as strings) would be simpler (but perhaps a tad less efficient), because then you don't have to parse the timestamps.

Sorry for misunderstanding.

EDIT: actually, either approach works regardless of whether you parse the data or not. The timestamps can be still compared as strings (without parsing the data). You don't need to know the meaning of the timestamps, just their relative order, which can be accomplished via simple string compare

I still think it's simpler in terms of code to sort the entire list first.

_psx_ said:

do you have any experience about amount of data in makeWebRequest?

I don't use it personally but there's this old thread:

forums.garmin.com/.../max-length-of-json-response

Max length of JSON response?

Coleman.Apps said:

I was able to find out that it should be 16k max.

I assume it could be lower depending on device and app type. jim_m_58 might be able to shed some more light on the practical limits.

Thanks @all for your help and suggestions. And thanks for the compare function.
I think I will sort all entries into a array using the compare function from FlowState. This way I can reduce the data by only taking over the needes JSON elements into the array.

Perhaps I add a setting where the user can define the number of notifications he want to see. This way I'm flexible for changes. I will implement it later - first I have to work befor going to freetime development :-)

If you are doing the makeWebRequest from a background service it can be less than 16k, based on the max size for a background service. A 16k response can cause an out of memory condition in a device that only has 32k for a background service

The request comes back as JSON, but then needs to be converted to a dictionary, which can require something like double the size of the response when it's converted, so in the case of a 8k response, the available memory  needs to be on the order of 16k,  For a full 16k response, 32k, so there needs to be enough memory available in the app for that.

It's a widget where I making these large web requests (not background). And I had to sort out all devices with <=64kB available widget memory. The request are ~10kB plain text, and ~20kB or morre as dictionary. So with the widget itself it's more than 64kB peak memory usage.