Updated SPC decoder to handle SPC Mesoanalysis text files

[keltonhalbert]

Updated SPC decoder to handle SPC Mesoanalysis text files

What Changed

SPC Mesoanalysis text files from their archives often include the Omega field, and have a slightly diferent metadata header for the date. This would cause the SPC Decoder to not recognize these files (I've attached one for reference).

20130531_0025_EF1_36.12_-95.23_93_455311 (copy).txt

This change includes some try/except blocks to test whether this is an observed SPC file, or one from the Mesoanalysis and handle the appropriate situation. As far as I can tell, this doesn't break functionality for the observed sounding text as I've been able to read the old format just fine.

Additionally, in sharptab/profiles.py, the line that specifies the location and date of the file saved was commented out. This prevented users from being able to open a file saved by SHARPpy itself, which I doubt was intended. I've uncommented that line and left it as-is.

Why It Changed

The reason for these changes is that we're using SHARPpy to look at the SPC Mesoanalysis files provided to us to look into initializing CM1 simulations. Sometimes that requires modifying the sounding in some way, which the GUI is very useful for. We can then save out the modified sounding to a text file, and send it to our preprocessor that converts it to the format desired by CM1 and interpolates it to 100 meter height intervals.

In the near future, I will also be making a pull request for a new decoder that reads in the CM1 sounding format so that the resulting output can be checked. I haven't written it yet though because this format only contains the surface pressure, which means programming in some hypsometric equation stuff I really don't feel like doing right now.

[wblumberg]

It lives!

[keltonhalbert]

Now I'm not sure what all of these fancy build checks do but if there's anything broken that's my fault, let me know :)

[wblumberg]

So it looks like most things I'd expect to pass are passing. The Azure pipelines (the rocket ship icon) is failing on the Windows build, which I haven't been able to try to fix yet. I turned off the testing on the Azure CI machines to try to better understand why it's unable to build the binary but never turned it back on. But the Travis-CI still runs the tests and they're all passing, so that's a good sign for Kelton's code. Not sure if Appveyor does tests. I'll have to check later.

That's the major issue holding up release right now...the binaries can't be built cleanly.

[keltonhalbert]

Right on.

Well if there's anything I need to do before getting this merged into the working Andover branch, let me know. I feel like at minimum, being able to open the text files that SHARPpy saves would benefit those brave enough to work with Andover.

[wblumberg]

Hey @keltonhalbert ...everything looks good, but I was thinking of two things you could do that would help others know about the new file format. These changes shouldn't take long:

1.) Could you update the Sphinx documentation text talking about the file format to add in information about the omega field, units, that it's optional, etc.? Here's the link to the file you'll need to modify:
https://github.com/sharppy/SHARPpy/blob/andover/docs/source/datasource_guide.rst

you may also want to update the interacting with the GUI page, as this discusses saving the data:

https://github.com/sharppy/SHARPpy/blob/andover/docs/source/interacting_gui.rst

The environment.yml file I think has all of the correct packages you need to remake the documentation to see how it looks. The command is "make html" and should be run within the docs/ folder.

2.) Write a quick test to read a file with the new file format. You'll have to upload a new file in the examples/data/ folder, and I'd keep the file name shorter than the one you have provided. Right now the tests that run read in the older file format - these are passing, so it might be useful to include a failsafe within the tests where it tries to read in a file with the new data format too. Here's the test script you should modify:

https://github.com/sharppy/SHARPpy/blob/andover/sharppy/tests/test_decoders.py

It's not very well commented or laid out. I kinda just threw it together so I apologize for that, but I think you should be able to figure it out easily.

@tsupinie, do you have any idea why this was commented in the code? We don't have a test written for saving the data and reloading it, and I have no idea how this happened. I can't go hunting for why right now:

Additionally, in sharptab/profiles.py, the line that specifies the location and date of the file saved was commented out. This prevented users from being able to open a file saved by SHARPpy itself, which I doubt was intended. I've uncommented that line and left it as-is.