-403 Error in makeWebRequest callback / BG / WF

Hi,

fighting with memory and other tricks.

I have a strange error that come not at anytime. I guess it could be from the way i use the simu, App Settings Editor and OS but not sure of anything now.

in a watch face, background :

I am doing 3 makeWebrequest, 1st callback calling 2nd makeWebRequest, 2nd callback calling 3rd makeWebRequest.

1st is to get a Stock AMrket value, 2nd for weather and 3rd for 2nd stock market.

below is all println i made, from init to error :

****getInitialView =****
in userSettings : modification true
in userSettings Application.getApp().getProperty StockCode: GRMN
in userSettings StocksArray[0][2]: GRMN
in userSettings FULL StocksArray[0]: [true, 12, GRMN, null, 404, null, false, 5]
in userSettings changed: true - StocksArray[0][true, 12, GRMN, null, 404, null, false, 5]

in userSettings Application.getApp().getProperty StockCode1: MSFT
in userSettings StocksArray[1][2]: MSFT
in userSettings Changed : true - StocksArray[1][false, 13, MSFT, null, 404, null, false, 5]

in userSettings Changed : true - Updating Weather API Key to demo

****background init false intervalWebRequest ****5
****background register **** interval =>5
Background: ****makeRequestFirstStock **** RequestStocksArray 0= [true, 11, GRMN, null, 404, null, false, 3]
Background: in FIRST receivestock responseCode = -403 -- dataFirstStock = null
Background: in parseresponse receive StockResponse=> null
Background: in parseStockResponse exit with returnStocksArray [null, 400, false]
Background: ****makeRequestWeather ****
Background: ****receiveWeather **** responseCodeWeather = 401 -- data={message=>Invalid API key. Please see openweathermap.org/faq for more info., cod=>401}
Background: ****makeRequestNextStock **** RequestStocksArray 1= [false, 10, MSFT, null, 404, null, false, 5]
Background: in NEXT receivestock responseCode = 200 -- data = {Information=>The **demo** API key is for demo purposes only. Please claim your free API key at (www.alphavantage.co/.../ to explore our full API offerings. It takes fewer than 20 seconds, and we are committed to making it free forever.}
Background: in parseresponse receive StockResponse=> {Information=>The **demo** API key is for demo purposes only. Please claim your free API key at (www.alphavantage.co/.../ to explore our full API offerings. It takes fewer than 20 seconds, and we are committed to making it free forever.}
Background: in parseStockResponse exit with returnStocksArray [null, 408, false]
Back in onBackgroundData ----------- dataFromWebRequest = [[[null, 400, false], [null, 408, false]], [false, 401, KEY, A, RED, 1.100000, 90.000000, 24.500000]]

in userSettings : modification true
in userSettings Application.getApp().getProperty StockCode: GOOGL
in userSettings StocksArray[0][2]: GRMN
in userSettings FULL StocksArray[0]: [true, 11, GRMN, null, 400, null, false, 3]
in userSettings changed : modification
in userSettings changed: true - StocksArray[0][true, 11, GOOGL, null, 404, null, false, 5]

in userSettings Application.getApp().getProperty StockCode1: MSFT
in userSettings StocksArray[1][2]: MSFT
in userSettings Changed : true - StocksArray[1][true, 10, MSFT, null, 408, null, false, 6]

in userSettings Changed : true - Updating Weather API Key to xxxxxx

Background: ****makeRequestFirstStock **** RequestStocksArray 0= [true, 11, GOOGL, null, 404, null, false, 3]
Background: in FIRST receivestock responseCode = -403 -- dataFirstStock = null
Background: in parseresponse receive StockResponse=> null
Background: in parseStockResponse exit with returnStocksArray [null, 400, false]
Background: ****makeRequestWeather ****
Background: ****receiveWeather **** responseCodeWeather = -403 -- data=null
Background: ****makeRequestNextStock **** RequestStocksArray 1= [true, 10, MSFT, null, 408, null, false, 3]
Background: in NEXT receivestock responseCode = -403 -- data = null
Background: in parseresponse receive StockResponse=> null
Background: in parseStockResponse exit with returnStocksArray [null, 400, false]
Back in onBackgroundData ----------- dataFromWebRequest = [[[null, 400, false], [null, 400, false]], [false, -403, mem, A, RED, 1.100000, 90.000000, 24.500000]]

sometimes i will get the correct data in my call back, sometimes no.

Like for weather, where the error key return the correct error msg, and with a correct key return data is null.

I am absolutly sure it is not a return data memory error as it works.

for example, here are the data in HTTP Traffic in simu, for weather when i get the 403 error :

--------------------------- HTTP TRAFFIC LOGS ---------------------

GET /data/2.5/weather?units=metric&lon=6.83002&lat=43.4283&appid=xxxHTTP/1.1\r\nHost: api.openweathermap.org\r\nUser-Agent: Mozilla/5.0\r\nAccept: */*\r\nContent-Type: application/x-www-form-urlencoded\r\n\r\n

HTTP/1.1 200 OK\r\n

Server: openresty\r\n

Date: Tue, 03 Dec 2019 18:03:54 GMT\r\n

Content-Type: application/json; charset=utf-8\r\n

Content-Length: 444\r\n

Connection: keep-alive\r\n

X-Cache-Key: /data/2.5/weather?lat=43.43&lon=6.83&units=metric\r\n

Access-Control-Allow-Origin: *\r\n

Access-Control-Allow-Credentials: true\r\n

Access-Control-Allow-Methods: GET, POST\r\n

\r\n

{"coord":{"lon":6.83,"lat":43.43},"weather":[{"id":800,"main":"Clear","description":"clear sky","icon":"01n"}],"base":"stations","main":{"temp":10.11,"pressure":1019,"humidity":87,"temp_min":5.56,"temp_max":13.33},"visibility":10000,"wind":{"speed":1.5,"deg":360},"clouds":{"all":0},"dt":1575396174,"sys":{"type":1,"id":6507,"country":"FR","sunrise":1575355644,"sunset":1575388658},"timezone":3600,"id":2977246,"name":"Saint-Raphael","cod":200}

------------------------------

and for my Stock security with 403 error :

--------------------------- HTTP TRAFFIC LOGS ---------------------

GET /query?function=GLOBAL_QUOTE&symbol=MSFT&outputsize=compact&interval=60min&apikey=xxxxxxHTTP/1.1\r\nHost: www.alphavantage.co\r\nUser-Agent: Mozilla/5.0\r\nAccept: application/json\r\n\r\n

HTTP/1.1 200 OK\r\n

Connection: keep-alive\r\n

Server: gunicorn/19.7.0\r\n

Date: Tue, 03 Dec 2019 18:03:55 GMT\r\n

Transfer-Encoding: chunked\r\n

Content-Type: application/json\r\n

Allow: GET, HEAD, OPTIONS\r\n

Vary: Cookie\r\n

X-Frame-Options: SAMEORIGIN\r\n

Via: 1.1 vegur\r\n

\r\n

183\r\n{\n "Global Quote": {\n "01. symbol": "MSFT",\n "02. open": "147.4900",\n "03. high": "148.4100",\n "04. low": "146.6500",\n "05. price": "148.3179",\n "06. volume": "12740675",\n "07. latest trading day": "2019-12-03",\n "08. previous close": "149.5500",\n "09. change": "-1.2321",\n "10. change percent": "-0.8239%"\n }\n}\r\n

0\r\n\r\n

------------------------------

Memory usage is 62.3/92.0kb

Peak mem : 66.7kb

Object Mem : 93

Peak Objects : 135

What can it be from ?

i'm lost...

thanks.

Marco over 4 years ago

here are data i just received, after closing all and re open them (Simu, App Settings Editor, run my WF and send my settings, laucnhing in Simu : BG Events -> Temporal Event

Background: ****makeRequestFirstStock **** RequestStocksArray 0= [true, 11, GOOGL, null, 404, null, false, 3]
Background: in FIRST receivestock responseCode = 200 -- dataFirstStock = {Global Quote=>{02. open=>1278.6600, 05. price=>1292.6100, 10. change percent=>0.2910%, 04. low=>1277.0500, 01. symbol=>GOOGL, 09. change=>3.7500, 03. high=>1292.9907, 06. volume=>633260, 07. latest trading day=>2019-12-03, 08. previous close=>1288.8600}}
Background: in parseresponse receive StockResponse=> {Global Quote=>{02. open=>1278.6600, 05. price=>1292.6100, 10. change percent=>0.2910%, 04. low=>1277.0500, 01. symbol=>GOOGL, 09. change=>3.7500, 03. high=>1292.9907, 06. volume=>633260, 07. latest trading day=>2019-12-03, 08. previous close=>1288.8600}}
Background: in parseStockResponse exit with returnStocksArray [1292.6100, 200, true]
Background: ****makeRequestWeather ****
Background: ****receiveWeather **** responseCodeWeather = -403 -- data=null
Background: ****makeRequestNextStock **** RequestStocksArray 1= [true, 7, GRMN, null, 404, null, false, 3]
Background: in NEXT receivestock responseCode = -403 -- data = null
Background: in parseresponse receive StockResponse=> null
Background: in parseStockResponse exit with returnStocksArray [null, 400, false]
Back in onBackgroundData ----------- dataFromWebRequest = [[[1292.6100, 200, true], [null, 400, false], [null, 404, false], [null, 404, false], [null, 404, false], [null, 404, false], [null, 404, false], [null, 404, false]], [false, -403, mem, A, RED, 1.100000, 90.000000, 24.500000]]

Travis.ConnectIQ over 4 years ago

According to the docs, error code -403 is NETWORK_RESPONSE_OUT_OF_MEMORY. When running in a background process on most devices, you only have 32KB of memory. I'm willing to bet that you're just running out.

You'd have to use System.getSystemStats() to get information about free memory in the background process (you can't see that in the debugging tools right now).

jim_m_58 over 4 years ago in reply to Marco

While your watch face may have 92kb of space, your background process likely only has 28kb (I forget what devices you're using.)

So you're seeing -403 in the background, which has far less memory than your watch face.

Looks's like Travis and I had a posting race! When I mention 28K, that's the 32k he mentioned minus the memory used by the VM.

Marco over 4 years ago in reply to Travis.ConnectIQ

thanks for answering.

here are getSystemStats() :

***background init false intervalWebRequest ****5
****background register **** interval =>5
Background: ****background Init ****true
Background: **** initialize**** TOTAL MEM28688
Background: **** initialize**** USED MEM25504
Background: **** initialize**** FREE MEM3184
Background: ****makeRequestFirstStock **** RequestStocksArray 0= [true, 11, GRMN, null, 404, null, false, 3]
Background: **** makeRequestFirstStock**** TOTAL MEM28688
Background: **** makeRequestFirstStock**** USED MEM25480
Background: **** makeRequestFirstStock**** FREE MEM3208
Background: in FIRST receivestock responseCode = -403 -- dataFirstStock = null
Background: **** receiveFirstStockPrice**** TOTAL MEM28688
Background: **** receiveFirstStockPrice**** USED MEM25608
Background: **** receiveFirstStockPrice**** FREE MEM3080
Background: in parseresponse receive StockResponse=> null
Background: in parseStockResponse exit with returnStocksArray [null, 400, false]
Background: ****makeRequestWeather ****
Background: **** makeRequestWeather**** TOTAL MEM28688
Background: **** makeRequestWeather**** USED MEM25752
Background: **** makeRequestWeather**** FREE MEM2936
Background: ****receiveWeather ****
Background: **** receiveWeather**** TOTAL MEM28688
Background: **** receiveWeather**** USED MEM25664
Background: **** receiveWeather**** FREE MEM3024
Background: ****receiveWeather **** responseCodeWeather = -403 -- data=null
Background: ****makeRequestNextStock ****
Background: **** makeRequestNextStock**** TOTAL MEM28688
Background: **** makeRequestNextStock**** USED MEM25960
Background: **** makeRequestNextStock**** FREE MEM2728
Background: ****makeRequestNextStock **** RequestStocksArray 1= [false, 10, MSFT, null, 404, null, false, 5]

i probably should work on freeing mem, but getting stocks data seems to cost 404octets, and weather 416octets ?

wouldn't i be able to get them ? even with only 3000 in mem ?

sure, I am going to work on improving memory. But for this particular point, is -403 the ideal response code ?

I thought -403 was for 8K http size ! (not sure but i saw this somewhere...)

jim_m_58 over 4 years ago in reply to Marco

it means memory in general. the 8k is "up to 8k", based on available memory. There's also more memory used that it seems as the json data gets converted into a dictonary and both use memory at the time. How complex the json data is impacts the amount used for the dictionary, so the more complex, the more memory, even for the same number of bytes of data.

Marco over 4 years ago in reply to jim_m_58

Thanks for all theses answers.

That explains my random errors 403, depending on stuff in on BackgroundData that affected free mem for the WebRequests.

I now have to think in another way to get data back, store them, use them. all this between BG and View,

I understand, now, why there is so much threads about cost effective memory data and optimization.

I should just have chose to make a simple watch face that give hour...

thanks for your help, Travis and Jim.

(developping mainly for my MK1, but design for 240x240 rounded)

Marco over 4 years ago in reply to Marco

Sorry, just one thing :

Does the Sys.println after compliation took some mem ?

jim_m_58 over 4 years ago in reply to Marco

Yes. You are adding code.

And if you do something like

Sys.println("value="+value)

you are also adding a string.

But I don't think you need to look at a few bytes. You need to look for something bigger, like cutting the size of the responses to the makeWebRequest or optimize your code, as it looks like you;''re using 25k before you even do the comm.

Marco over 4 years ago in reply to jim_m_58

thanks.

I'm now down to 19816 used, 8872 free, just optimizing onBGdata().

I was doing lot of stuff in App vs View since i wanted to put in viex only display stuff. But that's not optimal for WF BG !

I will work now on parsing data, from request to View functions (i did it in callback to minimize data to on BGdata, as i thought i was limited here too)!

lot of job to do... But it works !

thanks again.