save_kmz.py for HyP3

[SMJMirza]

It seems that for the HyP3 option of the MintPy, the velocity map in KMZ format, generated by "save_kmz.py", can not be opened in Google Earth properly.
It is due to the UTM coordinates of processing that is done for the HyP3 service products by MintPy. You can find a sample of this KMZ file in the attachment.

velocity.zip

[welcome]

👋 Thanks for opening your first issue here! Please make sure you filled out the template with as much detail as possible. We appreciate that you took the time to contribute!
Make sure you read our contributing guidelines

[geofysue]

The same happens for the outputs of save_kmz_timeseries.py. Is there any way to solve this?

[yunjunz]

KML takes south, north, east, west in degrees, so you could use ut.to_latlon() to do the conversion at save_kmz.py#L338.

A similar approach can be done for save_kmz_timeseries.py as well.

[NKakar]

Dear @yunjunz,
I was not able to fully understand your last comment,
Do you mean to change the west, east, south, north = ut.four_corners(meta) to west, east, south, north = ut.to_latlon? in the save_kmz.py file? or something else.

[geofysue]

Hi @NKakar, it has to be something like this:

west, east, south, north = ut.four_corners(meta)

############# new lines added by me
north, east = utm_to_latlon(east, north, 29, 'T') # you need to specify zone_number and northern=True or False, or letter.
south, west = utm_to_latlon(west, south, 29, 'T') # idem
#############

where utm_to_latlon() is the function that converts UTM coordinates to Latitude and Longitude.

[NKakar]

Thank you @geofysue,
some minor changes worked for me as followings:
############# new lines added by me
import utm # Please install utm as pip install utm first and the following code must work as described by @geofysue
north, east = utm.to_latlon(east, north, 42, 'T') # you need to specify zone_number and northern=True or False, or letter.
south, west = utm.to_latlon(west, south, 42, 'T') # idem
#############

There is a problem in this method, the KML file don't correctly overlay on the google imagery, I think there is about 3-5km offset, any suggestions about fixing that? look at the following screenshot.

[geofysue]

@NKakar, I do have an offset too and I don't know how to correct it, but mine is about 100 metres. Hopefully someone can help with this.

[NKakar]

@geofysue, I hope there is a solution for it. I wanted to ask what changes are required in save_kmz_timeseries.py did you fix this one too? if yes, can you share the changes too.

[geofysue]

@NKakar, yes, it is similar. I import my function from save_kmz, but you could use utm package.

Line 131:

def flatten_lat_lon(box, ts_obj, coords=None):
    if coords is None:
        lats, lons = ut.get_lat_lon(ts_obj.metadata, box=box)
        lats, lons = save_kmz.utm_to_latlon(lons, lats, 29, 'T') #####added by me

Line 383:

# 1.2 Parse Spatial coordinates
lats, lons = ut.get_lat_lon(ts_obj.metadata, box=box)
lats, lons = save_kmz.utm_to_latlon(lons, lats, 29, 'T') #####added by me

I think this works fine.

[yunjunz]

Note that the UTM_ZONE and EPSG code are saved in the metadata as well, for hyp3 products. So one could extract them to make this conversion generic.

[NKakar]

The same approach introduced by @geofysue works fine for the save_kmz_timeseries.py. There is no such offset in the kmz file and google image. If the data is same, why the program is introducing an offset.

[yunjunz]

One possible issue could be that: in save_kmz_timseries.py, the coordinate of every pixel is converted, while in save_kmz.py, only the 4 corners are used in the conversion to define the Google Earth overlay object, then the image is stretched, the location accuracy for the rest of pixels could be due to the distortion between lat/long and UTM coordinate system. How much are these potential distortions, I don't know.

Update: Google Earth could show UTM coordinates directly, in "preferences -> 3D View -> Show Lat/Long -> University Tranverse Mercator". We may want to check if we could write KML overlay objects with UTM coordinates directly.

[SMJMirza]

Sorry, Yunjun, I followed your updated information about opening the UTM-based KMZ files from MintPy into Google Earth, but it did not work. So, do you think that another way (changing the codes to get the suitable format of KMZ files) can be better?

[mosyhey]

Thanks to all,
Some more changes in save_kmz.py to make it more generic (for hyp3 products):

    ########## New lines for converting utm to lat/lon (https://github.com/insarlab/MintPy/issues/686)
    strProcessor = readfile.read_attribute(inps.file)['PROCESSOR']
    if strProcessor == 'hyp3':
        import utm #Install utm as "pip install utm" if not installed
        strUtmZone = readfile.read_attribute(inps.file)['UTM_ZONE'] #Read from metadate, for example: "11N"
        intUtmZone = int(strUtmZone[0:-1]) #In this example: 11
        strHemisphere = strUtmZone[-1] #In this example: "N"
        north, east = utm.to_latlon(east, north, intUtmZone, strHemisphere)
        south, west = utm.to_latlon(west, south, intUtmZone, strHemisphere)
    ##########

*Update: It is better to check the PROCESSOR first.

[SMJMirza]

@mosyhey Thank you for your changes. Did you see any shift in your generated KMZ files by these changes in the "save_kmz.py" file? I can see that others reported this shift in the files.

[mosyhey]

@SMJMirza I'm still working on the example dataset RidgecrestSenDT71. There is a small negligible distortion here, but I think it's ok. As you can see, corner points DL and UR have located nearly exactly, and DR and UL have an offset of 650m and 1300m respectively. The Reference Point at the bottom of the map has a 450m offset. The lake in the middle also has a slight shift.

velocity.zip

[SMJMirza]

@mosyhey Thank you so much for your reply and description. I can see the KMZ file format in the HyP3 downloaded folders. I am trying to ask the HyP3 support team to find a way to fix this issue.

[mosyhey]

Thanks to all.
Some attempts to make save_kmz_timeseries.py more generic (for hyp3 products):
save_kmz_timeseries_new.zip

We need an import and a user-defined function at top of the file, in line 28:

########## New lines for converting utm to lat/lon (https://github.com/insarlab/MintPy/issues/686)
import utm
def fncUtmToLatLon(lats, lons, ts_obj):
    strProcessor = readfile.read_attribute(ts_obj.file)['PROCESSOR']
    if strProcessor == 'hyp3':
        strUtmZone = readfile.read_attribute(ts_obj.file)['UTM_ZONE'] #Read from metadate, for example: "11N"
        intUtmZone = int(strUtmZone[0:-1]) #In example: 11
        strHemisphere = strUtmZone[-1] #In example: "N"
        lats, lons = utm.to_latlon(lons, lats, intUtmZone, strHemisphere)
    return lats, lons
##########

Also adding this piece of code in lines 131, 236 and 380:

        ########## New lines
        lats, lons = fncUtmToLatLon(lats, lons, ts_obj)
        ##########

[mosyhey]

In this case, it seems the points are located exactly, something like persistent scatterer (PS) points.
Here, we set the --step of points to 1 i.e. every pixel (80m):

[Arctic-Ambrun]

Share something I found recently. these images should be transfer from WGS84 UTM to WGS84 lat/lon first. but when a Image cross the Equator, the transform process could be wrong. Now I`m still working on it.

[Arctic-Ambrun]

@yunjunz Mr.YunJun, I have figured out the reason. Here are two method, which do you want me to supply? One method is reprojection from WGS84 UTM to WGS84 lat/lon and clip all the AOI.(finished scripts, testing, especially when crossing the Equator) One method is change the geo coordinates of Geometory.h5 (Have not tried yet).

[mosyhey]

Thank you @Arctic-Ambrun. We are interested to see the result.

[Arctic-Ambrun]

@mosyhey Mr.mosyhey, it goes like this

[yunjunz]

Share something I found recently. these images should be transfer from WGS84 UTM to WGS84 lat/lon first. but when a Image cross the Equator, the transform process could be wrong. Now I`m still working on it.

@yunjunz Mr.YunJun, I have figured out the reason. Here are two method, which do you want me to supply? One method is reprojection from WGS84 UTM to WGS84 lat/lon and clip all the AOI.(finished scripts, testing, especially when crossing the Equator) One method is change the geo coordinates of Geometory.h5 (Have not tried yet).

Hi @Arctic-Ambrun, sorry for the late response. I think method one is great if I understand you right: reprojecting a 2D image/matrix from UTM to lat/long. This can be a very useful function, and a PR is more than welcomed!

[Arctic-Ambrun]

sure, I will upload a script next week 获取 Outlook for Android<https://aka.ms/ghei36>

…

________________________________ From: Zhang Yunjun ***@***.***> Sent: Friday, July 1, 2022 8:07:59 AM To: insarlab/MintPy ***@***.***> Cc: WANG, JiaGeng [Alumni] ***@***.***>; Mention ***@***.***> Subject: Re: [insarlab/MintPy] save_kmz.py for HyP3 (Issue #686) Share something I found recently. these images should be transfer from WGS84 UTM to WGS84 lat/lon first. but when a Image cross the Equator, the transform process could be wrong. Now I`m still working on it. @yunjunz<https://github.com/yunjunz> Mr.YunJun, I have figured out the reason. Here are two method, which do you want me to supply? One method is reprojection from WGS84 UTM to WGS84 lat/lon and clip all the AOI.(finished scripts, testing, especially when crossing the Equator) One method is change the geo coordinates of Geometory.h5 (Have not tried yet). Hi @Arctic-Ambrun<https://github.com/Arctic-Ambrun>, sorry for the late response. I think method one is great if I understand you right: reprojecting a 2D image/matrix from UTM to lat/long. This can be a very useful function, and a PR is more than welcomed! ― Reply to this email directly, view it on GitHub<#686 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ATPQPAHUZV5DEYQAKABONB3VRYZF7ANCNFSM5HFNQENQ>. You are receiving this because you were mentioned.Message ID: ***@***.***> [https://www.polyu.edu.hk/emaildisclaimer/85A-PolyU_Email_Signature.jpg] Disclaimer: This message (including any attachments) contains confidential information intended for a specific individual and purpose. If you are not the intended recipient, you should delete this message and notify the sender and The Hong Kong Polytechnic University (the University) immediately. Any disclosure, copying, or distribution of this message, or the taking of any action based on it, is strictly prohibited and may be unlawful. The University specifically denies any responsibility for the accuracy or quality of information obtained through University E-mail Facilities. Any views and opinions expressed are only those of the author(s) and do not necessarily represent those of the University and the University accepts no liability whatsoever for any losses or damages incurred or caused to any party as a result of the use of such information.

[geofysue]

Hi, I'm wondering if the code presented at Arctic-Ambrun:prep_hyp3_crop (pull request) is working and if it will be added to main. What are the steps to use it correctly? Thanks

[Arctic-Ambrun]

Hi, I'm wondering if the code presented at Arctic-Ambrun:prep_hyp3_crop (pull request) is working and if it will be added to main. What are the steps to use it correctly? Thanks

Yeah I have uploaded the script but not approved yet, It works fine for me, so far. unzip all ASF zipped files in a INPUT folder and make a OUTPUT. USE -h to see the usage. the crop option is a optical parameter, you can put up to four parameters of NESW

[geofysue]

Therefore, if we use prep_hyp3_crop, there is no need to modify save_kmz.py and save_kmz_timeseries.py?

[Arctic-Ambrun]

Hi @geofysue , Yes, Sorry to reply you so late. the script has changed the coordinate already. And I realize so far the little script is not qualified enough and needs further maintenance. But It can deal the coordinate now. Maybe we can do it together? I think

[geofysue]

Hi @geofysue , Yes, Sorry to reply you so late. the script has changed the coordinate already. And I realize so far the little script is not qualified enough and needs further maintenance. But It can deal the coordinate now. Maybe we can do it together? I think

Hi @Arctic-Ambrun , thanks. Alright, I can help you with this, but I'm not an expert in programming.