geoid issue-19 #20
geoid issue-19 #20
Conversation
|
Looking at the initial issue and the implementation of this PR, I am going back and forth on the best approach. On the one side it is nice to have some sort of automation for converting between CRS and the other is that we are assuming that the point cloud is the preferred/correct one. You mentioned pulling this logic out of the pipeline and have it as a separate tool/script, which I am currently leaning towards. For this PR, I would suggest that we compare the reference DEM and the point cloud and raise an error in the workflow when there is a difference. Then we could give the user the exact conversion call for GDAL if they opt to convert the reference DEM. Basically print the call on line 74 Making the user aware of this detail is important and then have ice-roads as the tool to assist the user in getting everything done. Thoughts? |
|
I really support some sort of error print and engage the user to make a
decision. Especially in the case of geoid- ellipsoid issues. I havent
gotten into software development at this scale for a long time, but
ideally, i think there would be a configuration gui that could be used at
start up to check what you what done with the run, then let it loose. It
would still be basically command line driven, but the modules would be user
specified (if desired) to allow for individulalized workflows.
I dont know how hard this would be in python, or who would be available to
do the programming. Maybe just an option to do CRS or run separate would
be good fior now?
…On Tue, Jan 21, 2025 at 4:32 PM Joachim Meyer ***@***.***> wrote:
Looking at the initial issue and the implementation of this PR, I am going
back and forth on the best approach. On the one side it is nice to have
some sort of automation for converting between CRS and the other is that we
are assuming that the point cloud is the preferred/correct one.
You mentioned pulling this logic out of the pipeline and have it as a
separate tool/script, which I am currently leaning towards.
For this PR, I would suggest that we compare the reference DEM and the
point cloud and raise an error in the workflow when there is a difference.
Then we could give the user the exact conversion call for GDAL if they opt
to convert the reference DEM. Basically print the call on line 74
<https://github.com/cryogars/ice-road-copters/pull/20/files#diff-68f7c9293bc31fb2a05fa9dc49cf12c1574bc03fc503ee597d0b53c9384c69b4R74-R75>
Making the user aware of this detail is important and then have ice-roads
as the tool to assist the user in getting everything done.
Thoughts?
—
Reply to this email directly, view it on GitHub
<#20 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AA22DJKTJAMF5VFZRICD7I32L3KJJAVCNFSM6AAAAABVTLKXWCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMMBVHE2TGOBYGY>
.
You are receiving this because your review was requested.Message ID:
***@***.***>
|
|
@shadoneel But we do ask for this in the README (see below), Users input either a True or False at the start. imo a GUI may be a little over-engineered, and would not be ideal for running on Borah or other linux cluster. @jomey My main issue with pulling it out, is that it's not just GDAL , since GDAL does not have a geoid conversion. We would be asking users to compute it using Ames Stereo. Which could be less accessible. It's possible that keeping this in as is, would aid non-technical users in easily performing the geoid conversion? Alternatively, we can do what I was suggesting and just remove the -g flag. We can have an exception that looks for if reference-DEM is in geoid. And we can refer them to run another script that we make. But, this whole process could be longer, and potentially not worth the headache if we can just ask the user to be aware of what vertical datum their reference-DEM is in? |
|
I am still voting for referring the user to a separate script to properly prepare the reference DEM. Also thinking from a repeatability perspective, where for instance in our case we would convert the reference DEM every time we process a point cloud. This adds unnecessary time each time a workflow is executed. It already takes quite a while for one run and saving time should be desirable. |
|
If the software isnt user friendly its not useful.
Trying to use the software in new areas has illustrated weak points, that
can be handled by an expert but not by someone at a student or technician
level that isnt a software developer. Because 3DEP and other large lidar
products have decided to go with geoid heights, we really need capacity to
transform from geoid to ellipsoid. I was told that vertical transforms are
now possible with GDAL, without making and editing a VRT. I have not
verified this. How i did it in the past:
1. Use gdalwarp to make a vrt
gdalwarp -of vrt <source> <dest.vrt> -s_srs epsg:32611+5703 -t_srs
epsg:32611-r bilinear
e.g. Quantum provides data in NAD83(2011) UTM 11N: 26911
NAVD88(Geoid12B) = 5703
2. Use an editor like Atom to Edit the vrt along the lines of below at
the bottom just inside the closing *</*VRTDataset*>* *tag*. This is telling
gdal translate (step 3) to reference the “Vertical Shift Grids”. The last
line in the example below is the referenced closing tag. You paste above
it. <VerticalShiftGrids>
<Grids>g2012a_conus.gtx,g2012a_alaska.gtx,g2012a_guam.gtx,g2012a_hawaii.gtx,g2012a_puertorico.gtx,g2012a_samoa.gtx</Grids>
<Inverse>False</Inverse>
<ToMeterSrc>1</ToMeterSrc>
<ToMeterDest>1</ToMeterDest>
</VerticalShiftGrids>
*</*VRTDataset*>*
1. Those specific grid shift files are for NAVD88 (5703). you have to
reference different files for different vertical datums
2. if you have to deal with ft based vertical, then change the
<ToMeterSrc> or <ToMeterDest> to convert to feet/survey feet/us
feet/chains/rods/etc
3. 1. gdal_translate the vrt to a output tiff e.g.
gdal_translate -of <source.vrt> <output.tif>
Checking to see if it worked can be done using
GTIFF_REPORT_COMPD_CS=TRUE gdalinfo DryCreek_WGS84.tif -stats
Before and after the transform, and the min and max will change, following
the vertical shift you just executed.
…On Wed, Jan 22, 2025 at 7:27 AM Brent Wilder ***@***.***> wrote:
@shadoneel <https://github.com/shadoneel> But we do ask for this in the
README (see below),
-g geoid Is the reference DEM CRS orthometric (geoid height)? Will be auto set to True if you don't supply a DEM [Default: False]
Users input either a *True* or *False* at the start.
imo a GUI may be a little over-engineered, and would not be ideal for
running on Borah or other linux cluster.
@jomey <https://github.com/jomey> My main issue with pulling it out, is
that it's not just GDAL , since GDAL does not have a geoid conversion. We
would be asking users to compute it using Ames Stereo. Which could be less
accessible. It's possible that keeping this in as is, would aid
non-technical users in easily performing the geoid conversion?
Alternatively, we can do what I was suggesting and just remove the -g
flag. We can have an exception that looks for if reference-DEM is in geoid.
And we can refer them to run another script that we make. But, this whole
process could be longer, and potentially not worth the headache if we can
just ask the user to be aware of what vertical datum their reference-DEM is
in?
—
Reply to this email directly, view it on GitHub
<#20 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AA22DJPNV26RM4LFQT76KVD2L6TDVAVCNFSM6AAAAABVTLKXWCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMMBXGM4TEMRQGU>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
@shadoneel @jomey Putting these two ideas together, I would argue we need to have a separate utility script even more so. This would allow for more flexibility (perhaps allowing choice of a GDAL or Ames method), adding in if its in feet vs meter, allowing for users to assess the output stats to verify, repeatability so we are not computing the same thing over again? |
|
And I can make it clear in the documentation/README, that this geoid-conversion-tool should be used first. We can also add another Exception/Error, to catch this early on in the main pipeline. |
|
Tagging @naniciafone to chime in how she processed the 3DEP data. Think there is also a PDAL+GDAL way that uses the 3DEP point clouds. |
|
PDAL was able to perform the transformation from NAVD 1988 GEOID 12B to WGS ellipsoid with the reprojection filter... I thought I was going to need to download a gtx file but I didn't need to. |
|
Good to know about the ease of the PDAL method for point clouds @naniciafone and @jomey ... Which could also be added to this geoid-helper script? Are we in consensus then to take out the EDIT: with the idea being similar to Shad's comments that we are focusing on User input, and trying to simply bring down the technical barrier on this part? |
I would like to see Nani's workflow at least in the example. Think this is very useful to have as a reference if a user decides to go the point cloud route.
+1 from my side
|
|
Should need a little more testing perhaps? but i have exhausted all of the data I have on my computer (flipping back and forth between WGS and NAD). But I believe I have done it? It seems to be fully dummy proof and does not allow you to do any sort of geoid transformations that are impossible. It also takes in either a raster (using ASP), or it can take in a point cloud (using PDAL). I leverage the fact we have the navd88.tif already local due to the ASP install. And so this is called in the PDAL pipeline to ensure consistency when performing the transformation. |
@naniciafone - Want to give this a spin with your current use case in Kamas? |
Also, just to follow up on your earlier point @jomey , we have taken out this assumption too. And so now, we just ask the user to provide the EPSG code (e.g 32611) that they wish their data to be in. This should hopefully remove the ambiguity in which is the preferred/correct one. |
|
Nice work on moving the main logic pieces into methods. I would like to give @naniciafone a chance to test this too before we merge |
|
Y'all rock. Thanks!
…On Tue, Jan 28, 2025 at 8:23 AM Joachim Meyer ***@***.***> wrote:
Nice work on moving the main logic pieces into methods.
Consider most of my remarks as minor.
I would like to give @naniciafone <https://github.com/naniciafone> a
chance to test this too before we merge
—
Reply to this email directly, view it on GitHub
<#20 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AA22DJKM7CSCQWVE4XHTSQ32M6OIXAVCNFSM6AAAAABVTLKXWCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMMJZGMYTMNJZGQ>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
| @@ -60,6 +60,9 @@ | |||
| asp_dir = join(asp_dir, 'bin') | |||
| else: | |||
| asp_dir = abspath(join('ASP', 'bin')) | |||
| # check ASP install exists | |||
| if not exists(asp_dir): | |||
| raise Exception(f'The following is not the correct path to ASP: {asp_dir}') | |||
Fixes hard coding of CRS for geoid conversion. (Closes #19 )