-
Notifications
You must be signed in to change notification settings - Fork 27
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Cannot read LAC files from NOAA CLASS #130
Comments
@vysitu thanks for reaching out regarding this issue! I recommend you download the TLE for the full life of the satellites, they are available here: https://github.com/pytroll/pygac/wiki/Gapfilled-TLE-files-for-(A)VHRR-satellites Tell us how it goes! |
@mraspaud Hi, thank you for the information! I downloaded the TLE_noaa11.txt in the link you provided. The first part of the error message became:
The rest is the same. I don't think the correct TLE file helped much ... or it wasn't loaded correctly when I set reader_kwargs = {'tle_dir': '.', 'tle_name': 'TLE_noaa11.txt'} and put the txt file with the notebook file. I forgot to mention that the error message popped up when I used
after
and I was using jupyter notebook. It is worth noting that I also tried TLE_noaa7.txt on a file from 1983 (filename: NSS.HRPT.NC.D83178.S1954.E2006.B1036868.WI), and the error message is as the following:
|
@vysitu thanks for reporting back. I looked at this a bit and I think I tracked this down to a problem in pygac, that the dataset name in the tbm header could be ebcdic-encoded. Pygac wasn't able to decode it correctly, hence the header was wrongly discarded. The PR here should fix this. Do you have the possibility to test this? |
@mraspaud Thank you very much! I didn't know how to get the test branch, so I downloaded the files you modified (luckily there were only 2..). After replacing the local pygac files with the modified ones, I was able to load datasets from a Scene. There were some warning messages, but I don't think it's pygac's issue. Not sure if it's the best practice, but I was able to export each band (dataset) to a tiff file, so I could use longitude, latitude and corresponding pixel values. The warning messages are as follows:
There was a file from NOAA-10 ("NOAA-G") it could not read. The error message was "array of sample points is empty". Other files from NOAA-10 seems readable. I will do more tests before I can say what the problem is. |
Nice that it's working! I'm interested in the faulty noaa 10 file. It's probably a file with some missing data or something like that, but it would be great to improve pygac to even support these. So if you can share it, it would be great. |
@mraspaud Thanks again! I have uploaded two of the NOAA-10 files I noticed to have the issues to the "with_issues" folder: https://drive.google.com/drive/folders/1ZfAwveWBawVQW5t9dZ7KUbZtqb8akLj4?usp=drive_link I am requesting some files from earlier NOAA satellites with AVHRR onboard. Is it ok to notify you if I have issues reading them? |
@vysitu thanks for linking to the broken files. I found the problem with them (some of the first scan lines were missing, messing with the PRT gap detection), and address it in a larger refactoring pull request I have open. But if you want to try it, here is the relevant code change that should make it work. |
@mraspaud Thank you very much! I tried adding noaa.py and replacing test_reader.py, as mentioned in the commit, but it didn't work... I also tried replacing the pygac folder in site-packages with the one in git repo, which only made things worse. The error message was still "array of sample points is empty". Maybe replacing the file was not enough.. Will this fix be available in conda or pypi some time? I tried it on some files from 1983 (NOAA-C) and the issue with "spacecraft_id" showed up again. I uploaded them to the same GoogleDrive folder as above. Would you mind taking a look? Files I have that were from NOAA-E and NOAA-F were all readable. |
@vysitu ok, i think installing pygac properly will help. |
Describe the bug
I requested some AVHRR L1B data from NOAA CLASS. The data were NOAA10 and NOAA11, in August 1989.
This might look like a previous issue in the satpy github repo: pytroll/satpy#2494
But I made sure it's 10bit/pixel when I requested the data.
Could you please take a look at the following information? Thank you!
To Reproduce
Not sure if it's the best way to do, but one of the files have been uploaded to Google Drive: https://drive.google.com/file/d/1JM0OCf9L7-Ps3z-iMzfB_1qLvt5Sm6Or/view?usp=drive_link
I did the following:
I copied the whole thing into a .txt file called "NOAA.txt".
Expected behavior
Tried the files available in the satpy github repo: pytroll/satpy#2494
saw some warnings but the scene was load-able. I was able to export the loaded scene to a Tiff file.
My thoughts on the cause of the error
The error message is a bit long, so I'm putting my thoughts here.
Looking into the files mentioned in the error message, I think it is caused by the pod-reader.py not setting the position of the file pointer right, therefore getting the "89" as spacecraft ID. I requested data from 1983-1989, none worked. 16bit/pixel data didn't work, either. I also tried selecting the "scanline" box and the "signature" box, neither seemed to make a difference. I haven't tried data after 1992 since I don't really need them at this point.
I first thought it was caused by the get_start_date function, but in the second run of the reading part of the script, the first error line about the number of scanlines went away. So the "auto" mode for header date seemed working properly...
Actual results
got the following error:
Environment Info:
The text was updated successfully, but these errors were encountered: