How does Communications.makeWebRequest() works if responseType does not match the Content-Type which server returns?

I am trying to use Communications.makeWebRequest() to get data from Google Spreadsheet like this

var url = "">docs.google.com/.../pub
var params = {
};
var options = {
:method => Communications.HTTP_REQUEST_METHOD_GET,
:headers => {
"Content-Type" => Communications.REQUEST_CONTENT_TYPE_URL_ENCODED
},
:responseType => Communications.HTTP_RESPONSE_CONTENT_TYPE_TEXT_PLAIN
};
Communications.makeWebRequest(url, params, options, method(:onReceive));

There are only a few responseTypes, which I can use in Garmin API and it is mentioned here https://developer.garmin.com/connect-iq/api-docs/Toybox/Communications.html

Basically, only these 2 for text content.

HTTP_RESPONSE_CONTENT_TYPE_TEXT_PLAIN (Content type "text/plain")

HTTP_RESPONSE_CONTENT_TYPE_JSON (Content type "application/json")

It works in simulator, but on Fenix 7 I received "-401" error. I read a lot of forum discussions about the same error. As I understand from all these discussions, Garmin API just return "-401" if content type returned by the server does not match responseType I set like this?

:responseType => Communications.HTTP_RESPONSE_CONTENT_TYPE_*

so, for me to read Google spreadsheet in CSV format, the only thing I can do is to contact Google and ask them to change Content-Type from what they return now ("text/csv") to "text/plain".

If I correctly understand this, then is there any workaround for this? Maybe setting something in monkey.jungle to make it less restrictive?

  • Unfortunately you can't make the API less restrictive. The only workaround is to set up your own proxy server to serve the data in a way that makeWebRequest() will accept.

  • Why does it work in Simulator?

  • That's a good question haha. Not sure.

  • This is a bug that the simulator allows this while the actual device does not. I'll file a ticket to get this addressed in the simulator.

  • I think it will be better if watch will be less restrictive. Even google spreadsheet returns CSV, and there is no way to get CSV. Google Spreadsheet can return JSON, but Google puts Content type as "application/javascript", but it is still not accepted by Garmin devices because it is not "application/json".

    In most of the cases we as developer are using third party providers of data and we do not own/control the server. I cannot say to google "Please change Content-type header for me".

    Ideally, it would be good to add one new constant HTTP_RESPONSE_CONTENT_TYPE_RAW, which will accept any content type. Maybe you can limit the amount of data and generate REQUEST_TOO_LARGE error. It is ok for small devices such as watch, but why to limit the content type it is unclear for me.

    Even for JSON there are 5 content types widely used

    application/json
    application/ld+json
    application/geo+json
    application/manifest+json
    application/x-web-app-manifest+json

    and plus this one

    application/javascript

    I guess only first one will work in Garmin watches.

    What if I want to display news from rss feed, which is xml? or any other format? There are hundreds of formats. It is not practical to only support a few of them.

    At least, could you please explain why Garmin has this limitation?

  • I second this ^.

    Much of the underlying technology of the internet was designed with Postel's Law in mind - "be conservative in what you do, be liberal in what you accept from others".

    I don't know of any web clients in wide use which have the kind of super-restrictive behavior we see in makeWebRequest(). I get that CIQ apps don't have access to a whole lot of resources (memory and CPU), and they necessarily can't handle every single internet content type, but at the very least, it would nice CIQ apps had a way to tell the API "I know for a fact the content is JSON or plain text (for example), regardless of the content-type in the headers", and for the API to respect that decision.

  • At least, could you please explain why Garmin has this limitation?

    Probably bc the API automatically handles most content types (like JSON), and they don't want to try to decode the wrong kind of content.

    Speaking of the way the API handles JSON, as a matter of fact, not all valid JSON is accepted by CIQ. CIQ only accepts JSON where the top-level value is an object (not an array, number, string, or null), because makeWebRequest() wants to be able to return a Dictionary in every case that JSON is received.

    I think this is another example of overly-restrictive design that could've been avoided.

    For example, to preserve the rule that makeWebRequest() always returns the same Monkey C type for JSON, it could wrap the top-level response in a Dictionary. e.g.

    { :response => /* actual response */ }

    (I realize this only moves the "problem" of multiple possible return types down one level. But doesn't the same "problem" exist even when the top-level JSON value is an object? That object's values can be any valid JSON types, which means the Monkey C Dictionary that's returned will have values with multiple possible types.)

  • I fully agree with you. If Garmin API cannot parse the content, just send it to my program as a string, regardless of the content type. I can parse it myself. It will be my problem. Now I have 20 bytes CSV to parse from Google Spreadsheet. I just need to extract one value and I cannot do this because CSV content-type is not supported.

  • Yeah ppl had similar problems when an API simply returns a number (valid JSON for everyone except Garmin).

    Another related problem is with APIs that return one type of content when successful (e.g. a JSON object), but a different type of content on error (e.g. an unquoted string, which is not valid JSON for anyone.) Obviously it is impossible for the app to know which will be returned in advance, so it has to assume the success case when specifying the expected content type. (Or it could leave the expected response content-type unspecified, but that would still necessitate that the server returns the "right" content type in the error case, which is not a given.) In the error case, Garmin's overly-restrictive error handling might prevent the app developer from actually seeing the error text response (and HTTP response code), and possibly doing something with it (passing it along to the end user, displaying a friendly error message, retrying, etc.)

    Unfortunately I think it would be very hard to change this stuff without breaking backwards compatibility. I guess the API could always give devs a way to opt in to the new behavior.

  • I have had similar issues with this in my apps. The server I connect to sends content type = json but the response it gives is not a valid json string, according to CIQ. It worked well in the sim but not on target. Took me a while to figure out what the problem was. Now I have to run a web proxy serber in between that converts the response to something that CIQ accepts. It sounds like a brilliant idea to just have the app figure out how to parse the content by itself, if CIQ cannot. That would allow me to remove the proxy server and help many others I assume.