Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: dynamically size extra info buffer #1821

Merged
merged 2 commits into from
Jul 23, 2024
Merged

Conversation

koebi
Copy link
Collaborator

@koebi koebi commented Jul 23, 2024

The hardcoded extra_info buffer size of 4 is not sufficient if csv extra info has more than 4 columns.

Pull Request Checklist

  • 1. I have rebased the latest version of the main branch into my feature branch and all conflicts
    have been resolved.
  • 2. I have added information about the change/addition to functionality to the CHANGELOG.md file under the
    [Unreleased] heading.
  • 3. I have documented my code using JDocs tags.
  • 4. I have removed unnecessary commented out code, imports and System.out.println statements.
  • 5. I have written JUnit tests for any new methods/classes and ensured that they pass.
  • 6. I have created API tests for any new functionality exposed to the API.
  • 7. If changes/additions are made to the ors-config.json file, I have added these to the ors config documentation
    along with a short description of what it is for, and documented this in the Pull Request (below).
  • 8. I have built graphs with my code of the Heidelberg.osm.gz file and run the api-tests with all test passing
  • 9. I have referenced the Issue Number in the Pull Request (if the changes were from an issue).
  • 10. For new features or changes involving building of graphs, I have tested on a larger dataset
    (at least Germany), and the graphs build without problems (i.e. no out-of-memory errors).
  • 11. For new features or changes involving the graphbuilding process (i.e. changing encoders, updating the
    importer etc.), I have generated longer distance routes for the affected profiles with different options
    (avoid features, max weight etc.) and compared these with the routes of the same parameters and start/end
    points generated from the current live ORS.
    If there are differences then the reasoning for these MUST be documented in the pull request.
  • 12. I have written in the Pull Request information about the changes made including their intended usage
    and why the change was needed.
  • 13. For changes touching the API documentation, I have tested that the API playground renders correctly.

Fixes #1436

Information about the changes

The issue can be triggered by reading a csv file with 5 (or more) columns and requesting "extra_info": ["csv"].
Then, when ExtraInfoProcessor.processPathEdge calls CsvGraphStorage.getEdgeValue, which in turn calls RAMDataAccess.getBytes, the latter tries to write CsvGraphStorage.numEntries values into buffer - which was only 4 bytes long.

Examples and reasons for differences between live ORS routes, and those generated from this pull request

Required changes to ors config (if applicable)

@github-actions github-actions bot added the fix label Jul 23, 2024
@github-actions github-actions bot added fix and removed fix labels Jul 23, 2024
@koebi koebi marked this pull request as ready for review July 23, 2024 13:23
@github-actions github-actions bot added fix and removed fix labels Jul 23, 2024
koebi added 2 commits July 24, 2024 00:46
The hardcoded extra_info buffer size of 4 is not sufficient if csv extra
info has more than 4 columns.
@aoles aoles force-pushed the fix/1436-more-csv-columns branch from 5df5bd5 to 2a0cfae Compare July 23, 2024 22:46
@aoles aoles enabled auto-merge July 23, 2024 22:47
Copy link
Member

@aoles aoles left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the prompt fix! ❤️

Copy link

@aoles aoles merged commit 83c33b6 into main Jul 23, 2024
38 checks passed
@aoles aoles deleted the fix/1436-more-csv-columns branch July 23, 2024 22:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
No open projects
Status: Awaiting release
Development

Successfully merging this pull request may close these issues.

Add support for more than 4 columns in the csv file in CsvGraphStorageBuilder
2 participants