Web request PUT method does not follow 307 redirect like GET method does

verified
#1

I have a piston that sends both GET and PUT web requests to the same URL (of the form https://address/…, using a basic authentication type authorization header like “Bearer xxx”.) The GET request works fine, but the PUT request results in an $httpStatusCode of 307.

I tried both of these same requests using Postman (with “Automatically follow redirects” turned off), and I see that the server responds to both with a status of 307 and provides the redirection URL in the Location header. (To be clear, it provides the same redirection URL to both requests.) If I reconfigure Postman to automatically follow redirects, then both requests succeed with a status of 200.

As a temporary workaround in my WC piston I changed the web requests to use the redirection URL I obtained using Postman instead of the original URL, and in both cases (GET & PUT) they now work. Of course, if the server provides a different redirection URL in the future, my piston will stop working until I go through the same process again to get the new redirection URL via Postman.

Note that the API docs of the server I’m interacting with specifically mention redirects and suggests caching the redirect URL and using that instead of the original URL unless the redirect URL eventually no longer works, in which case, the original URL should be used again to obtain (& cache) a new redirect URL.

So the questions are:

  1. Why doesn’t a PUT web request in WC automatically follow redirects when a GET web request does?
  2. Is there anything that can be done to get PUT web requests to automatically follow redirects?
  3. This is more of a related feature request, but can the redirect URL be made accessible (e.g., via a new system variable such as $httpRedirectURL) if redirection happens so that the piston that invokes the web request action can cache the URL and use it in the future? This will reduce network traffic and the number of HTTP requests seen by the server.
#2

I should clarify that the URL corresponds to an external address (i.e., not something on the LAN that the Hub is connected to.) In my particular case it is https://developer-api.nest.com.

#3

Hi webCoRE developers!

Just wondering if any of you have seen this and what you think about it. I do have a workaround, but I think this is a bug that needs addressing.

I’d like to either have WC/ST deal with the redirect automatically for the PUT just like what seems to happen on the GET, or barring that, at least be able to get the Location header from the response with the error 307 so I can redo the request with the new URL myself. Ultimately, though, I’d like to get the redirect URL, whether or not the redirect is followed automatically, so that I can cache it and use it for future web requests to minimize traffic for all concerned (WC/ST, network, and target web server.)

Thanks!

#7

Thank you for reporting the issue here, but I think that this problem may be better addressed by SmartThings than by us. It’s probably possible to handle redirects in webCoRE but that is something that should be done at the network rather than smart app level. If SmartThings can support a 307 redirect it would improve any smart app trying to connect to the Nest API.

I double-checked to make sure there is no precedent of handling redirects manually in the webCoRE code, certainly seems to be handled higher up the chain. Specifically, we are using the httpPut function provided by SmartThings.

#8

Hi!

Thanks for responding. FWIW, I did browse the webCoRE SmartApp code and I know what you’re referring to. I’ve also browsed the Nest Manager SmartApp code a bit in the past. It uses similar ST functions, of course. (Although in some cases it makes asynchronous http requests.)

Anyway, I partially agree with you. Certainly, if httpGet is able to follow redirects, one would think httpPut should as well. My problem here is, I’m not a SmartApp developer. I don’t know that I could get anywhere if I tried to bring this up with ST Support. Is it possible that someone on the WC team could bring this up with them? After all, it is indirectly affecting the functionality of WC as the higher layer application.

Regarding Nest’s API, they definitely want users to cache redirects and use them until such time as they become invalid (i.e., the Nest server either provides a different redirect URL, or simply returns an error.) By doing so it cuts down the number of requests to the server in half (which would be the case even if the redirect was handled inside httpGet/httpPut and WC and the WC piston never knew.) But, I know exposing the redirect URL to the WC piston is really a feature request, so please consider it in that light.

In the interim, the WC code could definitely handle the 307 redirect error, and simply retry the original request, substituting in the redirect URL provided in the response Location header. If you look at the Nest Manager SmartApp code, that is what it’s doing. (FWIW, they’re not trying to cache redirect URLs either.)

But as things stands, I’m kind of between a rock and a hard place: ST httpPut is not following the redirect automatically, and WC is not passing up the information I need at the piston level to do it myself. Even if in the case of a 307 error if WC could somehow make the contents of the response Location header available to the piston, that would work.

Lastly, FWIW, I am, and have been, using the Nest Manager SmartApp (and its DTH’s) for a while, first with CoRE, and now with webCoRE, but I’m in the process of implementing the two or three simple Nest interactions directly in webCoRE, since I don’t use like 95% of the Nest Manager’s functionality. Hence the need to deal with this myself via WC. I suppose I could try resolving this myself by hacking up my copy of the webCoRE Groovy code, but I’m nowhere near comfortable enough with doing that. :slight_smile:

#9

@ipaterson, I figured out how to pass the redirection URL back to the web request caller in the WC piston. It’s only two lines of code. The first added line is in vcmd_httpRequest():

1
1.JPG1099×65 20.1 KB

For this test/workaround, I only did this in the case of an external address, since that’s where I need it. Obviously, if something like this was eventually added I’m sure it would need to go in other places, too.

The second added line is in getSystemVariables():

2

Now in my WC piston I can check $httpRedirectURL to see if a redirect URL was provided, and if so, cache it, and rerun the current request.

I’m not sure what should eventually be done in WC and/or ST, but IMHO something needs to be, because the current situation is not fully functional. And even if all redirects are fully handled (so the piston doesn’t have to), it would still be nice to expose the redirect URL to the calling WC piston no matter where in the stack it’s processed, as I said, to allow caching which minimizes network traffic and server load.

But for now I at least have a workaround. :slight_smile:

#10

This is challenging since the redirect url will not be available in this way once the status code is handled properly. What do you think about something like $httpResolvedURL that is populated with a) the Location header if redirect is not followed or b) the final URL after any redirects.

#11

Yeah, that sounds good.

In my temporary hack, I’m just returning the Location header if $response.status is 307 (i.e., redirect not handled.) The piston will then cache the URL for future requests, and use it to rerun the current request. So it doesn’t really matter if it’s available after that. But, yes, in the long run it would be nice to return the ultimate redirect URL (if there is one, or I suppose the last of multiple redirects if that’s the case), whether or not it was handled for the caller, and of course, return 307 if it wasn’t.

Thanks!

#12

Sounds good. There will be a release today but including this fix so you will need to reapply your patch but I’ll get the resolved url into the next release.

#13

Hello. I was wondering if this was ever implemented? My Nest setup is exactly the same as in I am using the redirect instead of original URL. I had a look through the code and cannot see the two lines that the OP added or any mention of $httpResolvedURL.

Thanks!

#14

It was not added, so SmartThings still doesn’t support a 307 redirect as far as you can see?