Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: most_common_approx duplicated counts definition #163

Merged
merged 1 commit into from
Jan 17, 2025

Conversation

cayetanobv
Copy link
Member

@cayetanobv cayetanobv commented Jan 17, 2025

Issue

Fixes #

  • Fixed most_common_approx function: duplicated counts variable definition. Causing this error:
Preparing to upload raster file to BigQuery...
File Path: /home/cayetano/Downloads/raster/blended_output_cog.tif
File Size: 3279.5174951553345 MB
Number of bands: 3
Band types: ('uint8', 'uint8', 'uint8')
Band sizes (MB): [4096.0, 4096.0, 4096.0]
Source Band: (1, 2, 3)
Band Name: [None, None, None]
Number of Blocks: 65536
Block Dims: (256, 256)
Project: cartobq
Dataset: cayetanobv_raster_tests
Table: blended_output_compress
Number of Records Per BigQuery Append: 10000
Compress: True
Uploading Raster to BigQuery
Loading raster file to BigQuery...
Sampling raster...
Computing approximate stats...
Computing quantiles...
Computing most common values...
Computing approximate stats...
Computing quantiles...
Computing most common values...
Computing approximate stats...
Computing quantiles...
Computing most common values...
Error uploading to BigQuery. Would you like to delete the partially uploaded table? [yes/no] yes
Traceback (most recent call last):
  File "/home/cayetano/dev_projs/cartolibs/raster-loader/raster_loader/io/bigquery.py", line 135, in upload_raster
    metadata = rasterio_metadata(
               ^^^^^^^^^^^^^^^^^^
  File "/home/cayetano/dev_projs/cartolibs/raster-loader/raster_loader/io/common.py", line 273, in rasterio_metadata
    stats = raster_band_approx_stats(
            ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cayetano/dev_projs/cartolibs/raster-loader/raster_loader/io/common.py", line 556, in raster_band_approx_stats
    most_common = most_common_approx(samples_band)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cayetano/dev_projs/cartolibs/raster-loader/raster_loader/io/common.py", line 498, in most_common_approx
    return {int(bin_edges[i]): int(counts[i]) for i in idx if counts[i] > 0}
                ~~~~~~~~~^^^
IndexError: index 221 is out of bounds for axis 0 with size 61
None
Error: Error uploading to BigQuery: index 221 is out of bounds for axis 0 with size 61

Pull Request Checklist

  • I have tested the changes locally
  • I have added tests to cover my changes (if applicable)
  • I have updated the documentation (if applicable)

@cayetanobv cayetanobv requested a review from vdelacruzb January 17, 2025 13:48
Copy link
Contributor

@vdelacruzb vdelacruzb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@cayetanobv cayetanobv merged commit 2f641e9 into main Jan 17, 2025
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants