Is there a way to debug a makeWebRequest call?

could it be a similar issue like this thread: https://forums.garmin.com/forum/developers/connect-iq/106905-?p=757201#post757201

Android or iOS? I recall a case a while back where Android would work, but iOS would return a -400, and it had to do what was in the url vs what was stated as a param, if I recall (something need to be moved from the URL to the param).

What's your url look like? It's not in the code you posted.

Do you see anything with something like curl?

Hi Peter and Jim,

Thanks for your replies. I'm going to try to address them below.

Before I start I must clarify that I have 2 sets of JSON. One that works perfectly (returning a code 200, the parser is very happy about it) and one that never works (code -400, the parser will never want to go further).

They both come from the exact same web server, I just change the URL constant in my program to point to a different resource. Then I run my program twice to make sure I exercise exactly the same code.

To extract the JSON set I simply replaced the makeWebRequest call by an openWebPage call which allows me to save the output as a file directly as received from the simulator.

One important point as hinted above is that I am using the simulator only at the moment. I want to rule out any potential weird effects of the watch, or Android and/or iOS app.

Here is the URL I use for the one that works: https://api.test.com/v1/advertisements
The JSON returned to the sim that always works
{"collection":{"version":"1.234.5","href":"api.test.com/.../search","data":[{"name":"team_id","value":null},{"name":"member_id","value":null},{"name":"medium","value":null},{"name":"page","value":null},{"name":"ad_spot","value":null},{"name":"page_size","value":null,"prompt":"The number of items to return for each page."},{"name":"page_number","value":null,"prompt":"The number of the page to be returned."}]}]}}

Here is the URL I use for the one that never works: https://api.test.com/v1/time_zones
And the JSON returned to the sim that never works
{"collection":{"version":"1.234.5","href":"api.test.com/.../search","data":[{"name":"id","value":null},{"name":"team_id","value":null}]}]}}

Jim, I looked at your other post. I have tried few of the tricks recommended in that post on the one that fails.

• I have tried a JSON validator online tool (JSONLint). Both sets are said to be Valid.
  
• I have tried to open the files in a Windows Notepad and save them back, it didn't complained about any encoding
  
• Notepad++ shows the files as UTF-8. There is no BOM mark or any special characters in the files from what I can tell. I looked at the hex/ASCII correspondance and everything is OK from that standpoint
  

Peter, let me address your comments here:

• To your comment about the URL used, I have pasted them above. As you can see they are quite similar in nature, just the last part describing the resource I'm pointing to that defers.
  
• I can be wrong, but I believe params are used when you have slightly more complex URL to craft with parameters after the resource that you are trying to address. The Garmin API doc gives the following example:
  

Communications.openWebPage(
"www.bing.com/.../search",
{"q" => "cute kitten"},
null
); // passes the url: bing.com/.../search kitten to the browser on the phone

You do have a very good point with regard to CURL though. When making a similar request to the one that fails but directly from CURL, I'm actually getting a different output than the one I receive through the Garmin call. Basically I get a quite longer and more complete JSON from the server, whereas for the one that works, I get exactly the same output through CURL. However, I have checked the dump from CURL (response headers, etc) they are exactly the same for both dumps, so I don't quite understand why one request would still fail compared to the other.

It would really help if there was a mode in the simulator that would dump the request and response from the makeWebRequest call into a temp file that could be observed for debugging purpose!

I appreciate any other tricks you guys may have that could help me debug what comes in and out of that blackhole call!

When making a similar request to the one that fails but directly from CURL, I'm actually getting a different output than the one I receive through the Garmin call. Basically I get a quite longer and more complete JSON from the server, whereas for the one that works, I get exactly the same output through CURL.

When you say it's much longer, how long is it?

This is where I'd look - why are they different?

Well, when I do the request through CURL, the JSON has all the definitions of the time_zones. This is what I would expect with the makeWebRequest call as well. It does works on other resources (the ones that do not fail) but somehow, the server does not respond the same for this one (and some others as well). If you look at the JSON I provided, you'll see there isn't much meaningful data returned in the JSON. Yet the JSON is syntaxly correct so I would have expected the parser to be able to take it still.

The difference returned from the server is probably because it does not fully like the HTTP GET sent by the makeWebRequest call. The answer probably lies in the difference between what CURL sends as an HTTP GET request and what the makeWebRequest call sends as well. Although, I did checked that the exact same headers, target URL and parameters are passed to the call and the CURL request.

Unfortunately, I do not have the control on the server, so I cannot debug the HTTP GET request from it and clearly see the difference. Because it is HTTPS, I don't think I can sniff the request either from my PC. If only I could print out or log or trace what goes out of the makeWebRequest call (like CURL is able to do with the --trace option), that would help me tremendously. At least to understand the differences between the two requests and maybe understand why the server is answering differently.

First off, we have a feature that I believe made it into the 3.0.0 beta3 plugin that will help with debugging web traffic. If you start the simulator and click File > View HTTP Traffic..., you should get a window that will display all incoming and outgoing data. It could use some improvement, but it is functional.

Second, I tried to reproduce the behavior you're seeing, but failed because the URL you've given above just redirects to www.test.com, which results in the accept header being discarded and HTML being given in the response. I believe the curl request I'm using below is equivalent to what you were using above...

vitek$ curl -H "Content-Type: application/json" -H "Accept: application/json" -Lv api.test.com/.../time_zones
* Trying 69.172.200.109...
* TCP_NODELAY set
* Connected to api.test.com (69.172.200.109) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
* CAfile: /etc/ssl/cert.pem
CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server accepted to use http/1.1
* Server certificate:
* subject: CN=test.com; O=Test.com, Inc.; L=Cleveland; ST=Ohio; C=US
* start date: Dec 21 16:03:41 2017 GMT
* expire date: Dec 17 22:03:41 2020 GMT
* subjectAltName: host "api.test.com" matched cert's "*.test.com"
* issuer: C=US; ST=Illinois; L=Chicago; O=Trustwave Holdings, Inc.; CN=Trustwave Organization Validation SHA256 CA, Level 1; [email protected]
* SSL certificate verify ok.
> GET /v1/time_zones HTTP/1.1
> Host: api.test.com
> User-Agent: curl/7.54.0
> Content-Type: application/json
> Accept: application/json
>
< HTTP/1.1 302 Moved Temporarily
< Server: nginx/1.13.12
< Date: Sat, 28 Jul 2018 20:54:03 GMT
< Content-Type: text/html
< Content-Length: 162
< Connection: keep-alive
< Keep-Alive: timeout=20
< Location: https://www.test.com/
< X-DIS-Request-ID: 28e1de629e198867250c955d638f985c
<
* Ignoring the response-body
* Connection #0 to host api.test.com left intact
* Issue another request to this URL: 'https://www.test.com/'
* Trying 69.172.200.235...
* TCP_NODELAY set
* Connected to www.test.com (69.172.200.235) port 443 (#1)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
* CAfile: /etc/ssl/cert.pem
CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server accepted to use http/1.1
* Server certificate:
* subject: OU=Domain Control Validated; OU=nsProtect Secure Xpress; CN=www.test.com
* start date: Jan 15 00:00:00 2017 GMT
* expire date: Jan 24 23:59:59 2020 GMT
* subjectAltName: host "www.test.com" matched cert's "www.test.com"
* issuer: C=US; ST=VA; L=Herndon; O=Network Solutions L.L.C.; CN=Network Solutions DV Server CA 2
* SSL certificate verify ok.
> GET / HTTP/1.1
> Host: www.test.com
> User-Agent: curl/7.54.0
> Content-Type: application/json
> Accept: application/json
>
< HTTP/1.1 200 OK
< Server: nginx/1.13.12
< Date: Sat, 28 Jul 2018 20:54:04 GMT
< Content-Type: text/html
< Transfer-Encoding: chunked
< Connection: keep-alive
< Keep-Alive: timeout=20
< X-DIS-Request-ID: 96e1f82618276131436f76a2fb4d5eff
< P3P: CP="NON DSP COR ADMa OUR IND UNI COM NAV INT"
< Cache-Control: no-cache
<
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "www.w3.org/.../strict.dtd">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<meta http-equiv="Content-Script-Type" content="text/javascript">
<script type="text/javascript">
function getCookie(c_name) { // Local function for getting a cookie value
if (document.cookie.length > 0) {
c_start = document.cookie.indexOf(c_name + "=");
if (c_start!=-1) {
c_start=c_start + c_name.length + 1;
c_end=docuent.cookie.indexOf(";", c_start);


if (c_end==-1)
c_end = document.cookie.length


return unescape(document.cookie.substring(c_start,c_end));
}
}
return "";
}
function setCookie(c_name, value, expiredays) { // Local function for setting a value of a cookie
var exdate = new Date();
exdate.setDate(exdate.getDate()+expiredays);
document.cookie = c_name + "=" + escape(value) + ((expiredays==null) ? "" : ";expires=" + exdate.toGMTString()) + ";path=/";
}
function getHostUri() {
var loc = document.location;
return loc.toString();
}
setCookie('YPF8827340282Jdskjhfiw_928937459182JAX666', '73.240.97.121', 10);
try {
location.reload(true);
} catch (err1) {
try {
location.reload();
} catch (err2) {
location.href = getHostUri();
}
}
</script>
</head>
<body>
<noscript>This site requires JavaScript and Cookies to be enabled. Please change your browser settings or upgrade your browser.</noscript>
</body>
</html>
* Connection #1 to host www.test.com left intact
vitek$


Hi Travis,

Thanks for your reply, it would be awesome to have such feature in the simulator to allow debugging of all requests and responses going through the API. Unfortunately, I have installed the 3.0.0 beta3 SDK and I'm afraid "View Memory" and "View Watchface Diagnostics" are the only two view menus provided in that SDK. But if this is a feature coming soon, I'll be sure to watch out for it.

To investigate further, what I did is to install my own local web server in hope to see if there are differences between a simple cURL request and the equivalent using the API call. I'll skip the details working around self-signed certificates and unchecking the "Use Device HTTPS Requirements" option to make sure I can talk to my local/private/dummy server.

I must say that I'm really puzzled because there are none. Tracing the request at the server, the request, the headers, the number of bytes, everything is identical beside the User-Agent, which is set to Mozilla/5.0 using the API call whereas cURL puts its name and version in that field.

I even went on and overrided the User-Agent in cURL to be the same as the API call and ran again the query to the real server to see what it would do. Same results. cURL always returns the full time zone data I'm looking for and is a syntaxly correct JSON, whereas through the makeWebRequest API I systematically get a -400 and when I use the openWebPage call to try to troubleshoot, I get a JSON result that is syntaxly correct but does not contain the info I need.

Now, I'm not sure if there is any difference in the HTTP request made through makeWebRequest and openWebPage. I'm starting to suspect that there are differences and that it may not be a good way to debug. Could it be that makeWebRequest makes an identical request to cURL and gets an identical response, but somehow fails the parsing and that openWebPage does not pass the same headers ending in having the server answering with a different JSON. As you said having a view of all the HTTP traffic would really help!

Something you need to know is that the "test" URL I provided is not the real one. I'm using a service on the web and it was a little bit complicated to explain everything in the forum as you need to create an account, have some data in there, use the authentication APIs to login and get an authentication token, use it in the header, etc. I didn't want to put all that in order to focus the discussion on debugging tools/methods. That said, the beside the base host everything else in the JSON is truly what I get from the server.

Thanks again for your help!

Hi Travis,

Sorry I wasn’t too activate lately, vacation took precedence over work :) Now that I’m back, I have tried a number of things, let me recap what I found:

1) Thanks for pointing out that The code path for makeWebRequest() is not the same as the one for openWebPage(). I can confirm that I see differences between the two based on what I found below.

2) I have installed the latest 3.0 SDK to benefit from the (awesome) HTTP Traffic logger tool in the simulator (File > View HTTP Traffic). I strongly recommend to use this tool as it gives the full request and full response from the server in great details. Thanks for adding this extremely useful tool! (If I may suggest an improvement: having a button to save the dump to a text file - similar to the trace option in cURL – could be a nice addition)

3) I have tried HTTP_RESPONSE_CONTENT_TYPE_TEXT_PLAIN as the :responseType option with no luck. The parser still returns a -400 code, which is surprising as it means the parser still tries to do something with the body of the request. Which leads me to my next point.

4) From the HTTP traffic trace, I could observe that requests to any resources that I’m making to the server is actually successful. Basically the response from the server is a success (HTTP 200 OK), with always the same header structure and expectedly the body that differs.

Which to me means that the makeWebRequest() call sends the right request, with the right headers and parameters similar (even identical) to cURL (beside the agent name, as expected). It also means that the response returned by the server is the same as a cURL call. Headers returned by the server seems to be understood (or skipped) without any problems by the parser as some requests are going through.

Which lead to the conclusion that there is definitely something in the body of some responses that the parser does not like (hence the -400 code) whereas some others are perfectly fine. I'm trying to attach a file that is a text output of a cURL to a resource that fails on the server, but the forum does not seem to allow me to upload a text file for some reason (seems uploaded attachments can only be images as it complains about it). I can paste it as code in the thread but it is quite long. Let me know what would be the most efficient way.

Thank you again for your support!