Memory management, Unable to serialize app data

I have a problem of memory management on my app : https://apps.garmin.com/fr-FR/apps/95dd5586-4a26-401f-83b5-6e408d4dbd0e

Common Resources file is 3Ko. Data is about 600 octets
French Resources file is 3Ko. Data is about 600 octets
English Resources file is 3Ko. Data is about 600 octets
Properties.xml file is 5Ko. Data is about 1700 octets (52 string, 2 boolean, 1 number).
I have to load all properties at the same time.

In the device.xml, I have for the fenix3 memory_limit="65536" and fit_session_memory="7168"
PRG file is 86Ko
SIM is showing peak at 58,2Ko with my properties (52 string, 2 boolean, 1 number) loaded

In the documentation for AppBase.setProperty(), there is a limit on the size of the Properties of 8KB. If you reach this limit, the properties will not be saved or reloaded.

I woud like to have more proprerty, about 100 strings instead of 52, then about 3400 octets (much less than 8KB).
But when I'm trying with 55 (and more) strings, and when I quit the app, I have a message on Eclipse "Unable to serialize app data", and my properties are destroyed on the SIM. And I think it's a problem of memory, the properties are not saved and cannot be reloaded.

If someone could tell me more ? Thanks

travis.vitek over 8 years ago

Given the error message, it sounds like you've exceeded the 8K limit on data in the object store. Given the numbers you're throwing out there, it seems that you should also be running into out-of-memory issues (65536 available, take 7168b for activity recording leaves 58368b = 58K) since you're using about 58K.

It seems that it would be a good idea to figure out how to reduce both memory usage and the object store size. What, specifically, do you need to know?

Travis

jim_m_58 over 8 years ago

Actually, if fit recording has been started, "peak" should reflect that in the sim. (it jumps 7k when recording starts).

It does look like memory is tight at 58k and a max of 64k. That would leave at most 6k for the data (as well as the keys and overhead needed to manage the data), the way I understand it - the 8k is a max, and can be smaller based on the overall size of the app.

Travis is correct. You probably need to reduce the size of the app and maybe look into a better way to store the data

dududu13 over 8 years ago

OK, it's seem to be a memory error, but the app is running well, without the error "out of memory". I had this one before, and I reduced my code
And, I'me using activity recording, then with 7K less memory avalaible
Really, I can't see what's wrong
I also have a resource file, settings.xml, wich is 6Ko. It permit me to modify all the strings in my properties.

Here are my properties :
<properties>

<property id="appVersion" type="string">0.3</property>
<property id="isWhiteBackground" type="boolean">false</property>
<property id="IsAutoRecording" type="boolean">false</property>
<property id="AlertWP" type="number">1</property>

<property id="1" type="string">41.332353 ;9.255809;;Iles Lavezzi</property>
<property id="2" type="string">41.383892 ;9.145946;;Bonifacio</property>
<property id="3" type="string">41.593878 ;9.297180;;Porto Vecchio</property>
<property id="4" type="string">41.682143 ;8.897552;;Propriano</property>
<property id="5" type="string">41.870966 ;8.580322;;Iles Sanguinaires Ajaccio</property>
<property id="6" type="string">41.924759 ;8.751983;;Ajaccio</property>
<property id="7" type="string">42.564965 ;8.767776;;Calvi</property>
<property id="8" type="string">42.639452 ;8.942227;;L'Ile Rousse</property>
<property id="9" type="string">42.693854 ;9.455022;;Bastia</property>
<property id="10" type="string">42.873856 ;3.055272;;Port Leucate</property>
<property id="11" type="string">42.961102 ;9.457726;;Saint Florent</property>
<property id="12" type="string">43.007910 ;6.200065;;Porquerolles</property>
<property id="13" type="string">43.015191 ;9.406700;;Macinaggio</property>
<property id="14" type="string">43.038532 ;9.405326;;Cap Corse</property>
<property id="15" type="string">43.044303 ;5.824127;;Cap Cepet St Mandrier</property>
<property id="16" type="string">43.080173 ;6.165390;;Hyeres</property>
<property id="17" type="string">43.080549 ;6.375503;;Cap Benat</property>
<property id="18" type="string">43.080800 ;5.786447;;Les Embiez</property>
<property id="19" type="string">43.132685 ;5.759668;;Bandol</property>
<property id="20" type="string">43.134188 ;6.370868;;Le Lavandou</property>
<property id="21" type="string">43.141328 ;6.439361;;Cap Negre</property>
<property id="22" type="string">43.158361 ;5.591697;;Bec de l'Aigle</property>
<property id="23" type="string">43.165373 ;6.650161;;Cap Taillat</property>
<property id="24" type="string">43.173385 ;6.542015;;Cavalaire</property>
<property id="25" type="string">43.176765 ;5.615558;;La Ciotat</property>
<property id="26" type="string">43.196791 ;5.506896;;En Vau</property>
<property id="27" type="string">43.200546 ;5.232667;;Planier</property>
<property id="28" type="string">43.207302 ;5.427246;;Sormiou</property>
<property id="29" type="string">43.208929 ;5.449218;;Sugiton</property>
<property id="30" type="string">43.209742 ;5.536079;;Cassis</property>
<property id="31" type="string">43.213214 ;5.336866;;Cap Croisette Les Goudes</property>
<property id="32" type="string">43.265768 ;3.507041;;Cap d'Adge</property>
<property id="33" type="string">43.270831 ;5.302019;;Frioul la ferme</property>
<property id="34" type="string">43.295699 ;6.675910;;Golfe St Tropez</property>
<property id="35" type="string">43.296198 ;5.357122;;Vieux Port</property>
<property id="36" type="string">43.322242 ;5.047187;;Carro</property>
<property id="37" type="string">43.324365 ;5.108985;;Sausset</property>
<property id="38" type="string">43.326301 ;5.155420;;Carry</property>
<property id="39" type="string">43.330141 ;5.184731;;Carry Calanque eaux salees</property>
<property id="40" type="string">43.338070 ;5.261120;;Niolon</property>
<property id="41" type="string">43.391763 ;3.702220;;Sete</property>
<property id="42" type="string">43.409911 ;6.774616;;St Raphael Frejus</property>
<property id="43" type="string">43.445877 ;4.424657;;Stes Marie de la mer</property>
<property id="44" type="string">43.513109 ;7.048373;;Iles de Lerins</property>
<property id="45" type="string">43.522165 ;4.122276;;Le Grau du roi</property>
<property id="46" type="string">43.523285 ;3.938426;;Palavas</property>
<property id="47" type="string">43.532122 ;7.018203;;Cannes</property>
<property id="48" type="string">43.548050 ;7.149353;;Cap d'Antibes</property>
<property id="49" type="string">43.551191 ;4.081892;;La Grande Motte</property>
<property id="50" type="string">43.282769 ;5.300526;;Frioul Morgiret</property>
<property id="51" type="string"></property>
</properties>

Perhpas I coud reduce with float instead of string, but I don't think to win much memoy, and it will ask me more code in my app

And I will see to reduce my memory program too, but don't know how to. I'm trying to have variable only in functions, but cannot do that everywhere, and my app is not very big (I think)

dududu13 over 8 years ago

lincoln street

Hi
I've tried to reduce my memory, but nothing to do.
with 50 strings, it's ok, with that (I've put some println(...) at the end :

maj distances, i=48 freeMemory =3224
maj distances, i=49 freeMemory =3200
Menu principal Exit, Before save settings : freeMemory =15024
Menu principal Exit, After save settings : freeMemory =15136
Menu principal Exit, After .SaveRecord() : freeMemory =22320
onStopApp before Position.LOCATION_DISABLE : freeMemory =22320
onStopApp after everythings : freeMemory =22448
Complete
Connection Finished
Closing shell and port

When I try with a little more properties, here's the result :
maj distances, i=53 freeMemory =2160
maj distances, i=54 freeMemory =2136
Menu principal Exit, Before save settings : freeMemory =15288
Menu principal Exit, After save settings : freeMemory =15400
Menu principal Exit, After .SaveRecord() : freeMemory =22584
onStopApp before Position.LOCATION_DISABLE : freeMemory =22584
onStopApp after everythings : freeMemory =22712
Unable to serialize app data
Complete
Connection Finished
Closing shell and port

In the app, I'm just, with 2136 Byte free at the max.
But when I quit the app, I have 22712 Byte, but always this message
I'm still searching ...
An idea ?
Thanks

travis.vitek over 8 years ago

Given the error message and your findings, it is most likely the object store. You need to reduce size of the object store either by reducing the lengths of the strings, the length of the keys (the property id string), or the number of properties.

One thing that you could do would be to eliminate the space and the extra semicolon in your property strings. That would save ~100 bytes. Unfortunately there is no way to prevent the user from entering data that takes up too much space, unless you want to put a limit on the length of the property strings.

You could also consider removing the last digit or digits of the latitude/longitude. I believe dropping down to 5 digits after the decimal point would introduce a little bit more than 1m of error in your position data. That would save another ~100 bytes.

Another possible option would be to forget using the object store/application settings entirely. You could expose the coordinate data via a web service, and then download it with makeWebRequest(). Once you have the data locally, you could save it in a more compact format. For example, you could use one key in the app settings, and the value would be an array of locations (latitude, longitude, and name).

I'm not sure any of these ideas will work, but they might be worth investigating.

Travis

dududu13 over 8 years ago

OK, thanks for this usefull response
But, after my firts post and before my last post, I already have reduced the digit to 3 after the decimal point (about 100m, just sufficient for sailing), and removed the space, like this : <property id="0" type="string">43.296;5.357;;Vieux Port</property>

But it does not change much. And I would like to have about 100 waypoints.
The property file in SIM, Voilewaypoint.SET is about 3 or 4K
Then, I think the only possible option would be to use a web service like you says. My waypoints in the app, are already stored in an array, with a string (name) and 2 floats (coordinates).
But I don't know anything in web, I will learn

I think there is a unknown limit or bug with the object store, cause I am much less than 8K

Thanks for the help

riedid74 over 8 years ago

Salut Dudu

Et si tu décomposais tes données

L'objectif est de gagner sur la taille des properties et ne pas avoir trop de strings

<property id="1" type="string">Iles Lavezzi</property>
<property id="2" type="string">Bonifacio</property>
<property id="3" type="string">Porto Vecchio</property>
<property id="4" type="string">Propriano</property>
<property id="5" type="string">Iles Sanguinaires Ajaccio</property>

Tu mets toutes les coordonnées GPS dans un seul string

<string id="CoordGps">41.332353|9.255809|41.383892|9.145946|41.593878|9.297180|41.682143|8.897552|41.870966|8.580322| .....</string>

Tu peux limiter aussi les chiffres derrière la virgule qui ne sert pas vraiment (4 est suffisant je pense)

Tu crées ensuite une fonction d'extraction de CoordGps avec le séparateur "|" qui mets dans un tableau

Tu peux donc pour le property 1 récupérer les coordonnées par arrayCoordGps[0] et arrayCoordGps[1]

arrayCoordGps = split(Ui.loadResource(Rez.Strings.CoordGps), "|",200);

property 2 tu sais que c'est arrayCoordGps[2] et arrayCoordGps[3]

Avec cette méthode tu diminues ton code

Didier

jim_m_58 over 8 years ago

Unless you want the user to be able to change the stings and lat/lon, I'd just do away with storing this data in the .set at all. I have an app that stores up to 50 waypoints, but it's based on a user action to mark one. I do store them for reuse, but do it in the Object Store (the .str file) like this. A single array:
[lat1,lat2,lat3,...lon1,lon2,lon3...] and do it as floats. I don't use name, but I number them so I don't have the name bit to deal with.

But, if you just want a list used by your app, you could do something without using the .set or .str like:

var lats=[lat1,lat2,lat3,...];
var lons=[lon1,lon2,lon3,...];
var names=["name1","name2","name3",...];

With this you're limited to available memory and objects, and not by what the .str/.set allows.

dududu13 over 8 years ago

Bonjour Didier,
Oui, j'ai déjà limité les coordonnées GPS à 4 chiffres après la virgule, et aussi limité un peu mes noms de waypoints, mais ça ne change pas grand chose
En fait, j'ai mis 50 waypoints en dur dans mon appli, mais je voudrais que ça reste personnalisable. Dons j'ai aussi mis les waypoints, sous la forme "nom;latitude;longitude" dans les properties, et modifiables avec un settings.xml
Si je fais comme tu le dis, ça va devenir compliqué à modifier (faire le lien entre le nom et les coordonnées)
J'ai à peu près 20K de libre lorsque je quitte l'appli, et c'est après que le message "unable to serialize app data" arrive
Je pense donc qu'il y a un souci dans le SDK avec un nombre de strings trop important

Pour le moment, je vais me contenter des 55 waypoints possibles, ça ira bien,

merci

Yes, I have already limited the GPS coordinates to 4 digits after the comma, and also limited my waypoint names a little, but it does not change much
In fact, I put 50 waypoints in my app, but I want it to be customizable. I also put the waypoints, in the form "name, latitude, longitude" in the properties, and modifiable with a settings.xml
If I do as you say, it will become complicated to modify (to make the link between the name and the coordinates)
I have about 20K free when I leave the app, and after the "unable to serialize app data" message arrives
So I think there is a concern in the SDK with too many strings

For now, I will content myself with the 55 possible waypoints, it will be fine,

thank you