Skip to content

Commit

Permalink
create conda env tools and notes
Browse files Browse the repository at this point in the history
  • Loading branch information
RobHanna-NOAA committed Feb 10, 2025
1 parent 98df252 commit 79a1484
Show file tree
Hide file tree
Showing 3 changed files with 316 additions and 1 deletion.
278 changes: 278 additions & 0 deletions data/bridges/conda_fim_bridges_enviro.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,278 @@
name: fim_bridges
channels:
- conda-forge
- defaults
- https://repo.anaconda.com/pkgs/main
- https://repo.anaconda.com/pkgs/r
- https://repo.anaconda.com/pkgs/msys2
dependencies:
- _libavif_api=1.1.1=h57928b3_2
- affine=2.3.0=pyhd3eb1b0_0
- aom=3.9.1=he0c23c2_0
- attrs=24.3.0=py312haa95532_0
- aws-c-auth=0.8.1=hd11252f_0
- aws-c-cal=0.8.1=h099ea23_3
- aws-c-common=0.10.6=h2466b09_0
- aws-c-compression=0.3.0=h099ea23_5
- aws-c-event-stream=0.5.0=h85d8506_11
- aws-c-http=0.9.2=h3888f84_4
- aws-c-io=0.15.3=hc5a9e45_6
- aws-c-mqtt=0.11.0=h2c94728_12
- aws-c-s3=0.7.9=h6a47413_1
- aws-c-sdkutils=0.2.2=h099ea23_0
- aws-checksums=0.2.2=h099ea23_4
- aws-crt-cpp=0.29.9=he488853_2
- aws-sdk-cpp=1.11.489=h7d73209_0
- azure-core-cpp=1.14.0=haf5610f_0
- azure-identity-cpp=1.10.0=hd6deed7_0
- azure-storage-blobs-cpp=12.13.0=h3241184_1
- azure-storage-common-cpp=12.8.0=hd6deed7_1
- blas=1.0=mkl
- blosc=1.21.6=hfd34d9b_1
- boost-cpp=1.85.0=ha5ead02_4
- bottleneck=1.4.2=py312h4b0e54e_0
- branca=0.6.0=py312haa95532_0
- brotli-python=1.0.9=py312h5da7b33_9
- bzip2=1.0.8=h2bbff1b_6
- c-ares=1.34.4=h2466b09_0
- ca-certificates=2025.1.31=h56e8100_0
- cairo=1.18.2=h5782bbf_1
- capnproto=1.0.2=hb5d7e06_3
- ceres-solver=2.2.0=hd842749_5
- certifi=2025.1.31=py312haa95532_0
- cfitsio=4.5.0=h2b45a09_0
- charset-normalizer=3.3.2=pyhd3eb1b0_0
- click=8.1.7=py312haa95532_0
- click-plugins=1.1.1=pyhd3eb1b0_0
- cligj=0.7.2=pyhd3eb1b0_0
- colorama=0.4.6=py312haa95532_0
- contourpy=1.3.1=py312h214f63a_0
- cycler=0.11.0=pyhd3eb1b0_0
- dav1d=1.2.1=hcfcfb64_0
- draco=1.5.7=h181d51b_0
- eigen=3.4.0=h91493d7_0
- expat=2.6.4=h8ddb27b_0
- fiona=1.10.1=py312h6e88f47_3
- fmt=11.0.2=h7f575de_0
- folium=0.14.0=py312haa95532_0
- font-ttf-dejavu-sans-mono=2.37=hab24e00_0
- font-ttf-inconsolata=3.000=h77eed37_0
- font-ttf-source-code-pro=2.038=h77eed37_0
- font-ttf-ubuntu=0.83=h77eed37_3
- fontconfig=2.15.0=h765892d_1
- fonts-conda-ecosystem=1=0
- fonts-conda-forge=1=0
- fonttools=4.55.3=py312h827c3e9_0
- freetype=2.12.1=hdaf720e_2
- freexl=2.0.0=hd7a5696_0
- gdal=3.10.1=py312hc39d689_2
- geopandas=0.14.4=pyhd8ed1ab_0
- geopandas-base=0.14.4=pyha770c72_0
- geos=3.13.0=h5a68840_0
- geotiff=1.7.3=h496ac4d_3
- gflags=2.2.2=he0c23c2_1005
- glib=2.82.2=h3d4babf_1
- glib-tools=2.82.2=h4394cf3_1
- glog=0.7.1=h3ff59bf_0
- gmp=6.3.0=h537511b_0
- hdf4=4.2.15=h5557f11_7
- hdf5=1.14.3=nompi_hb2c4d47_109
- icc_rt=2022.1.0=h6049295_2
- icu=75.1=he0c23c2_0
- idna=3.7=py312haa95532_0
- intel-openmp=2023.1.0=h59b6b97_46320
- jinja2=3.1.5=py312haa95532_0
- joblib=1.4.2=py312haa95532_0
- kealib=1.6.1=hadd4d7f_0
- kiwisolver=1.4.8=py312h5da7b33_0
- krb5=1.21.3=hdf4eb48_0
- lcms2=2.17=hbcf6048_0
- lerc=4.0.0=h5da7b33_0
- libabseil=20240722.0=cxx17_h4eb7d71_4
- libaec=1.1.3=h63175ca_0
- libamd=3.3.3=ss783_h38ac50e
- libarchive=3.7.7=h979ed78_3
- libarrow=17.0.0=hc3bbc16_47_cpu
- libarrow-acero=17.0.0=h7d8d6a5_47_cpu
- libarrow-dataset=17.0.0=h7d8d6a5_47_cpu
- libavif16=1.1.1=h4d049a7_2
- libblas=3.9.0=1_h8933c1f_netlib
- libboost=1.85.0=h444863b_4
- libboost-devel=1.85.0=h91493d7_4
- libboost-headers=1.85.0=h57928b3_4
- libbrotlicommon=1.1.0=h2466b09_2
- libbrotlidec=1.1.0=h2466b09_2
- libbrotlienc=1.1.0=h2466b09_2
- libbtf=2.3.2=ss783_hde22806
- libcamd=3.3.3=ss783_hde22806
- libcblas=3.9.0=8_h719fc58_netlib
- libccolamd=3.3.4=ss783_hde22806
- libcholmod=5.3.0=ss783_h9a56889
- libcolamd=3.3.4=ss783_hde22806
- libcrc32c=1.1.2=h0e60522_0
- libcurl=8.11.1=h88aaa65_0
- libcxsparse=4.4.1=ss783_hde22806
- libde265=1.0.15=h91493d7_0
- libdeflate=1.23=h9062f6e_0
- libevent=2.1.12=h3671451_1
- libexpat=2.6.4=he0c23c2_0
- libffi=3.4.4=hd77b12b_1
- libgdal=3.10.1=hc25ceda_2
- libgdal-arrow-parquet=3.10.1=h9c6c97a_2
- libgdal-core=3.10.1=h095903c_2
- libgdal-fits=3.10.1=h20f9414_2
- libgdal-grib=3.10.1=h9921521_2
- libgdal-hdf4=3.10.1=hf9aff8f_2
- libgdal-hdf5=3.10.1=h7df419c_2
- libgdal-jp2openjpeg=3.10.1=h768cd86_2
- libgdal-kea=3.10.1=hfc54ade_2
- libgdal-netcdf=3.10.1=h3717446_2
- libgdal-pdf=3.10.1=h33ae9eb_2
- libgdal-pg=3.10.1=h5e54256_2
- libgdal-postgisraster=3.10.1=h5e54256_2
- libgdal-tiledb=3.10.1=h8d6a7ae_2
- libgdal-xls=3.10.1=hd0c044b_2
- libglib=2.82.2=h7025463_1
- libgoogle-cloud=2.34.0=h95c5cb2_0
- libgoogle-cloud-storage=2.34.0=he5eb982_0
- libgrpc=1.67.1=h0ac93cb_1
- libheif=1.19.5=gpl_hc631cee_100
- libiconv=1.17=hcfcfb64_2
- libintl=0.22.5=h5728263_3
- libintl-devel=0.22.5=h5728263_3
- libjpeg-turbo=3.0.0=hcfcfb64_1
- libklu=2.3.5=ss783_h77d05f4
- libkml=1.3.0=h538826c_1021
- liblapack=3.9.0=8_h719fc58_netlib
- libldl=3.3.2=ss783_hde22806
- liblzma=5.6.4=h2466b09_0
- libnetcdf=4.9.2=nompi_h008f77d_116
- libparquet=17.0.0=ha850022_47_cpu
- libparu=1.0.0=ss783_h21e6e03
- libpdal=2.8.3=h7c24d9f_1
- libpdal-arrow=2.8.3=ha7d42d9_1
- libpdal-core=2.8.3=h7b54269_1
- libpdal-draco=2.8.3=ha2c8d63_1
- libpdal-e57=2.8.3=hb58253e_1
- libpdal-hdf=2.8.3=hb9e256b_1
- libpdal-icebridge=2.8.3=hb9e256b_1
- libpdal-nitf=2.8.3=hd077b48_1
- libpdal-pgpointcloud=2.8.3=ha2ce333_1
- libpdal-tiledb=2.8.3=h0741228_1
- libpdal-trajectory=2.8.3=h1c12469_1
- libpng=1.6.46=had7236b_0
- libpq=17.2=h81f3393_1
- libprotobuf=5.28.3=h8309712_1
- librbio=4.3.4=ss783_hde22806
- libre2-11=2024.07.02=h4eb7d71_2
- librttopo=1.1.0=hd4c2148_17
- libspatialindex=1.9.3=h6c2663c_0
- libspatialite=5.1.0=h939089a_12
- libspex=3.2.1=ss783_hcfd7fc7
- libspqr=4.3.4=ss783_hc35ff37
- libsqlite=3.48.0=h67fdade_1
- libssh2=1.11.1=he619c9f_0
- libsuitesparseconfig=7.8.3=ss783_ha9923ec
- libthrift=0.21.0=hbe90ef8_0
- libtiff=4.7.0=h797046b_3
- libumfpack=6.3.5=ss783_h35348e5
- libutf8proc=2.10.0=hf9b99b7_0
- libwebp-base=1.5.0=h3b0e114_0
- libxcb=1.16=h013a479_1
- libxml2=2.13.5=he286e8c_1
- libxslt=1.1.39=h3df6e99_0
- libzip=1.11.2=h3135430_0
- libzlib=1.3.1=h2466b09_2
- lz4-c=1.10.0=h2466b09_1
- lzo=2.10=hcfcfb64_1001
- m2w64-gcc-libgfortran=5.3.0=6
- m2w64-gcc-libs=5.3.0=7
- m2w64-gcc-libs-core=5.3.0=7
- m2w64-gmp=6.1.0=2
- m2w64-libwinpthread-git=5.0.0.4634.697f757=2
- mapclassify=2.5.0=py312haa95532_0
- markupsafe=3.0.2=py312h827c3e9_0
- matplotlib-base=3.10.0=py312he19b0ae_0
- metis=5.1.0=h17e2fc9_1007
- minizip=4.0.7=h9fa1bad_3
- mkl=2023.1.0=h6b88ed4_46358
- mkl-service=2.4.0=py312h827c3e9_2
- mkl_fft=1.3.11=py312h827c3e9_0
- mkl_random=1.2.8=py312h0158946_0
- mpfr=4.2.1=h56c3642_0
- msys2-conda-epoch=20160418=1
- networkx=3.4.2=py312haa95532_0
- nitro=2.7.dev8=h1537add_0
- numexpr=2.10.1=py312h4cd664f_0
- numpy=1.26.4=py312hfd52020_0
- numpy-base=1.26.4=py312h4dde369_0
- openjpeg=2.5.3=h4d64b90_0
- openssl=3.4.0=ha4e3fda_1
- orc=2.0.3=haf104fe_2
- packaging=24.2=py312haa95532_0
- pandas=2.2.3=py312h5da7b33_0
- pcre2=10.44=h3d7b363_2
- pdal=2.8.3=hd8ed1ab_0
- pillow=10.4.0=py312h381445a_1
- pip=25.0=py312haa95532_0
- pixman=0.44.2=had0cd8c_0
- poppler=24.12.0=heaa0bce_2
- poppler-data=0.4.11=haa95532_1
- postgresql=17.2=h998eeb8_1
- proj=9.5.1=h4f671f6_0
- pthread-stubs=0.4=hcd874cb_1001
- pyparsing=3.2.0=py312haa95532_0
- pyproj=3.7.0=py312ha24589b_0
- pysocks=1.7.1=py312haa95532_0
- python=3.12.8=h3f84c4b_1_cpython
- python-dateutil=2.9.0post0=py312haa95532_2
- python-tzdata=2023.3=pyhd3eb1b0_0
- python_abi=3.12=5_cp312
- pytz=2024.1=py312haa95532_0
- qhull=2020.2=h59b6b97_2
- rasterio=1.4.3=py312hc0daee4_0
- rav1e=0.6.6=h975169c_2
- re2=2024.07.02=haf4117d_2
- requests=2.32.3=py312haa95532_1
- rioxarray=0.18.1=py312haa95532_0
- rtree=1.0.1=py312h2eaa2aa_0
- scikit-learn=1.6.1=py312h585ebfc_0
- scipy=1.15.1=py312hbb039d4_0
- setuptools=75.8.0=py312haa95532_0
- shapely=2.0.7=py312h0c580ee_0
- six=1.16.0=pyhd3eb1b0_1
- snappy=1.2.1=h500f7fa_1
- snuggs=1.4.7=pyhd3eb1b0_0
- spdlog=1.15.1=hf4138ee_0
- sqlite=3.45.3=h2bbff1b_0
- suitesparse=7.8.3=ss783_hf1e1ef2
- svt-av1=2.3.0=he0c23c2_0
- tbb=2021.8.0=h59b6b97_0
- threadpoolctl=3.5.0=py312hfc267ef_0
- tiledb=2.27.0=h13fc995_8
- tk=8.6.13=h5226925_1
- tqdm=4.67.1=py312hfc267ef_0
- tzdata=2025a=h04d1e81_0
- ucrt=10.0.22621.0=h57928b3_1
- unicodedata2=15.1.0=py312h827c3e9_1
- uriparser=0.9.8=h5a68840_0
- urllib3=2.3.0=py312haa95532_0
- vc=14.42=haa95532_4
- vc14_runtime=14.42.34433=h6356254_24
- vs2015_runtime=14.42.34433=hfef2bbc_24
- wheel=0.45.1=py312haa95532_0
- win_inet_pton=1.1.0=py312haa95532_0
- x265=3.5=h2d74725_3
- xarray=2024.11.0=py312haa95532_0
- xerces-c=3.2.5=he0c23c2_2
- xorg-libxau=1.0.11=hcd874cb_0
- xorg-libxdmcp=1.1.3=hcd874cb_0
- xyzservices=2022.9.0=py312haa95532_1
- xz=5.4.6=h8cc25b3_1
- zlib=1.3.1=h2466b09_2
- zstd=1.5.6=h0ea2cb4_0
- pip:
- laspy==2.5.4
- pdal-plugins==1.6.2
- python-pdal==0.0.1
prefix: C:\Users\rdp-user\anaconda3\envs\fim_bridges
2 changes: 1 addition & 1 deletion data/bridges/make_rasters_using_lidar.py
Original file line number Diff line number Diff line change
Expand Up @@ -257,7 +257,7 @@ def process_bridges_lidar_data(OSM_bridge_file, buffer_width, raster_resolution,
os.makedirs(point_dir, exist_ok=True)
os.makedirs(tif_files_dir, exist_ok=True)

text = 'read osm bridge lines and make a polygon foortprint'
text = 'read osm bridge lines and make a polygon footprint'
print(text)
logging.info(text)
OSM_bridge_lines_gdf = gpd.read_file(OSM_bridge_file)
Expand Down
37 changes: 37 additions & 0 deletions data/bridges/setup_conda_for_make_rasters.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@

To run make_rasters_using_lidar.py, it must be run in a windows conda enviro. It might work in a linux/ubuntu enviro but has not been tested.

To setup your Conda enviro for the tool. You will only need to set it up once, then from there one, you can just turn it on/off (activate / deactivate)
1) Open Anaconda Window

2) Path to the directory where your source code is at (adjusting for path):
ie) cd C:\Users\rdp-user\Projects\dev-lidar-bridges2\inundation-mapping\data\bridges

3) run : conda env create --file=conda_fim_bridges_enviro.yml

At some point earlier one, it might ask you to continue (y/n), enter "y". It will take up to 2 - 10 mins to run.

When it is done and back to command line, you can type the following to see if the enviro is setup correctly
conda env list (and you should see somethign similar to:)

# conda environments:
#
base * C:\Users\rdp-user\anaconda3
fim_bridges C:\Users\rdp-user\anaconda3\envs\fim_bridges

(and there may be more enviros from previous projects which is fine.

To activate it (turn it on) and run it
• conda activate fim_bridges

Make sure you are in the directory you want: ie) cd C:\Users\rdp-user\Projects\dev-lidar-bridges2\inundation-mapping\data\bridges

• Now we can run it. Adjusting for pathing:

python make_rasters_using_lidar.py -i C:\Users\rdp-user\Projects\Lidar_bridges\20250207\conus_osm_bridges.gpkg -o C:\Users\rdp-user\Projects\Lidar_bridges\20250207\conus_osm_lidar_rasters\

• It may take 48 hours (wildly ish, depending on your computer specs)

When you are done, you can type:
conda deactivate

0 comments on commit 79a1484

Please sign in to comment.