-
Notifications
You must be signed in to change notification settings - Fork 32
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[5pt] PR: Incorporate Lidar-Informed elevations for bridges #1406
base: dev
Are you sure you want to change the base?
Conversation
We have a fix we need to do.. but not quite yet: data/inputs/osm/bridges/250102/huc6_dem_diff/alaska/alaska_osm_bridges_modified.gpkg but bash variables is looking only for: osm_bridges=${inputsDir}/osm/bridges/250102/huc6_dem_diff/osm_all_bridges_modified.gpkg Shouldn't there be two sets of these bash variables? one for conus and one for ak? Keep an eye on your heal_bridges_osm.py file too. It would also affect generate_pre_clip_fim_huc8.py and clip_vectors_to_wbd.py. I can help with that if you like. Just do small subset tests on pre-clip as I am going to be merging in some Matt's fixes for pre-clips. To do a full re-gen of conus pre-commits, I need both branches. I think I will approve and get Matt's 1178 merged to dev first, but hold on doing a pre-clip on his, then we can merge dev into yours and update your pre-clip. Those new pre-clips can now be used to do a full clean test on fim-pipeline and your code chagnes. I will do the new pre-clips when we get there on my prod machines. When we merge dev back to yours.. we will need to watch the delinate_hydros...sh as both of you have changes to the exact same heal calls. We will have to put this on hold until Matt's is merged in, then we can fix yours up, including pre-clips. That will affect the files being given / loaded for fim_pipeline to use. |
Rob, 2- Regarding pre-clipping of osm vector data.... we need to pre-clip below two files which are just some new input data and they should not have any conflict with anything else. I could not follow if there is any limitation for this data pre-clipping or not.
Please let me know if I missed anything else. |
Yes.
|
This lets me test Matt's DEM, WBD change, Ali's new LIDAR DEMs tool, and pre-clip which may/may not have an error. The result of that chain gives us valid exact data going into production for this PR and some retro, already merged, PR's. In a separate PR, make the fixes for get_sample_data ensuring the new DEMS, pre-clips and LIDAR DEMS are coming in correctly. Look for other changed / new inputs. Might only need the new LIDAR DEMs. TBD. |
…ation-mapping into dev-lidar-bridge2
…ation-mapping into dev-lidar-bridge2
…ation-mapping into dev-lidar-bridge2
This PR closes the issue #1242.
This PR incorporates lidar-derived elevations for OSM bridges into the FIM. The workflow consists of:
Additions
data/bridges/make_rasters_using_lidar.py
. Generates bridge elevation raster files using lidar data, as described here. This script has to be run on Windows machines. This script must be run twice, each time with a different OSM bridge-lines GPKG file: once for the CONUS and separately for Alaska. The raster files are generated using EPSG:5070 for CONUS bridges and EPSG:3338 for Alaska bridges.data/bridges/make_dem_dif_for_bridges.py
Compares lidar-elevation raster files with USGS 3DEP DEMs and generates DEM difference raster files at HUC levels, as described here. Non-bridge locations (which make up most of the HUC domain) have zero values, while OSM bridge locations typically show positive values. This script also needs to be run twice: once for the CONUS and a separate time specifically for Alaska.data/bridges/conda_fim_bridges_enviro.yml
A conda environment file to install conda env needed for running data/bridges/make_rasters_using_lidar.pydata/bridges/setup_conda_for_make_rasters.txt
Provides instructions for running the data/bridges/make_rasters_using_lidar.py script on a Windows machine using a Conda environment.Changes
data/bridges/pull_osm_bridges.py
The updates include:src/bash_variables.env
- Ingests links to two new VRT files for DEM difference rasters (CONUS and Alaska) and the new Alaska OSM bridges GPKG file.data/wbd/generate_pre_clip_fim_huc8.py
Pre-clips the new Alaska OSM bridges GPKG file.src/run_unit_wb.sh
Crops the bridge elevation difference VRT file to the buffered HUC boundary and create a copy for branch 0.src/run_by_branch.sh
Clips HUC-level bridge elevation difference rasters for branches.src/heal_bridges_osm.py
Now implements two distinct workflows for healing the REM at OSM bridge locations, depending on the availability of lidar data:src/delineate_hydros_and_produce_HAND.sh
Updated to align with the revised input arguments ofsrc/heal_bridges_osm.py
src/aggregate_by_huc.py
Applies 'threshold' HAND/discharge terminology instead of 'max' HAND/discharge terminology for determining bridges risk status.tools/bridge_inundation.py
. Same assrc/aggregate_by_huc.py
data/usgs/acquire_and_preprocess_3dep_dems.py
??Removals
Testing
Deployment Plan (For developer use)
How does the changes affect the product?
Current files / folders to be deployed
WBD_National_EPSG_5070_WBDHU8_clip_dem_domain.gpkg
(updates pending)
Diff raster files for each HUC6 (AK huc8)
udpated osm_bridges after lidar processing (each address below, adj for latest editions)
-/inputs/osm/bridges/250207/huc6_dem_diff/alaska/alaska_osm_bridges_modified.gpkg
Issuer Checklist (For developer use)
You may update this checklist before and/or after creating the PR. If you're unsure about any of them, please ask, we're here to help! These items are what we are going to look for before merging your code.
[_pt] PR: <description>
dev
branch (the default branch), you have a descriptive Feature Branch name using the format:dev-<description-of-change>
(e.g.dev-revise-levee-masking
)dev
branchpre-commit
hooks were run locally4.x.x.x
Merge Checklist (For Technical Lead use only)