-
Notifications
You must be signed in to change notification settings - Fork 0
Posting data
I spent some time working on the code.
- Tested whether POST method would work - no repsonse from wunderground.com when I did
- Upgraded to use HTTPS
I'm using the Arduino IDE programming environment for the ESP8266 and this is a new platform for me (only a couple of months of experience). I ran into the problem of no support for floats in sprintf, which leads to a lot of somewhat ugly use of string += operators. I'm not worried about running out of space on the ESP8266 for this project, but I can see where things could get really tight on some of the smaller Arduino MCUs.
So, one of the aspects of getting HTTPS transactions to work is that they don't like piecemeal prints to the client, since there's an encryption function involved that would prefer to have the whole transaction to work on. This means space in memory has to be used to hold the transaction before transmission.
The drawback, as I see it, of the GET method is that the data payload is exposed for logging at more points than when the POST method is used. For most web frameworks, there's a straightforward translation between POST method requests and GET method requests, but the evidence so far is that wunderground.com does not make any use of POST requests and simply ignores them. Malformed GET requests usually get a response indicating what went wrong, so getting no response from an attempted POST request may mean that these are disabled in their web server.
As you may see from the code, the data in the body of the POST request is exactly what is appended as a "query string" to the web page name in the GET request.
The documentation for wunderground.com did not explicitly say that it accepted HTTP requests; the examples use HTTPS. But, since the project code I was starting from used HTTP, that's how I started sending data and found that it worked.
The longer-run problem is that there's not enough room on these tiny microcontrollers to support a full cert validation chain for HTTPS. The workaround is to compare a "fingerprint" of the cert with one that was recorded on a more fully functional computer that could do the full validation. But certs have to be renewed periodically, And the renewed cert will have a different fingerprint.
This fragility problem leaves a dilemma. One solution would be to simply relay the data posting through a more capable computer, using HTTP on the local network. A more complicated solution would be to communicate via HTTPS with both the wunderground.com site directly and with a local server that could supply an updated fingerprint when the wunderground.com cert changes. But you'd need two local servers as a sort of tag team to cover the situation where the fingerprint on one of the local servers changes.... This starts to seem too byzantine to work well.
In viewing the history on wunderground.com, I see that the pressure readings are consistently there, every 10 minutes, but the temperature and humidity (dew point) measurements have holes. It looks like I'll need to implement some local monitoring to figure out why the DHT sensor is not reporting. I don't think it was an environmental problem, but having the sensors covered with snow does not seem like a good thing; I've put them in a plastic box into which I've slotted some breathing holes (I'm rather far from a precision Dremel tool operator). It's too early to tell if this box is going to collect condensation, or otherwise distort the readings.
Noticed that station had stopped reporting on 6 May. I thought perhaps the cert for wunderground.com had changed, so the signature might need to be updated. I should have just tried hitting the reset button.
When I brought the device inside and hooked it up to the Arduino IDE, I could not initially get it to accept downloaded code or display output on the serial monitor. Unplugging and plugging in again, hitting reset button, etc. After a few tries, and I can't be certain which things actually fixed the problem--but with no new code loaded--the serial monitor showed normal operation, no changed signature needed. So I'll put it back in place as-is (with the D0->reset jumper restored). It has made a few reports on schedule since I put it back on duty, so we'll just watch it from here.
After much struggle with other options, I settled on a very modest upgrade from my former Phant database. Instead of modified Phant codeto get https service, I now run completely stock Phant code behind HAProxy. This is all running on a VM built from the Turnkey Linux node.js appliance. HAProxy provides some glue that allows my Electric-Imp-based garage door monitor to continue to report as it always has (the switch of hosts was transparent to it) and translates https requests from both internal and external connections into local http requests on port 8080 to get to Phant.
Oh, yes, the current code for posting to Phant uses POST method. Waiting to see how long the battery lasts, among other things. So far, it's still going strong.