Code

Multi-threaded raster strategy#161

Code

Open

shortcutman wants to merge 27 commits intoisciences:masterisciences/exactextract:masterfrom shortcutman:parallelshortcutman/exactextract:parallelCopy head branch name to clipboard

+421-3

Commits

Commits on Jan 13, 2025

• Add libtbb as a dependency, not properly setup to be optional here.

  shortcutman

  committed
  936c126

  Copy full SHA for 936c126

• Running sequential logic through TBB pipeline with no parallelisation.

  shortcutman

  committed
  a8afe44

  Copy full SHA for a8afe44

• Refactored order of operations to be ready for more pipelining.

  shortcutman

  committed
  77dce98

  Copy full SHA for 77dce98

• Feed subgrids into tbb pipeline.

  shortcutman

  committed
  e2a5d72

  Copy full SHA for e2a5d72

• Isolate work to take subgrid and find feature hits.

  shortcutman

  committed
  68f90a9

  Copy full SHA for 68f90a9

• Make subgrid to feature intersection parallel.

  shortcutman

  committed
  63d88e7

  Copy full SHA for 63d88e7

• Addition of specific serial queue for raster read.

  shortcutman

  committed
  67eb00e

  Copy full SHA for 67eb00e

• Handle tasks where it has been determined there is no future work.

  shortcutman

  committed
  364f079

  Copy full SHA for 364f079

• Reaching end of input stream should return with dummy data that will be unused.

  shortcutman

  committed
  bae0b03

  Copy full SHA for bae0b03

• Process zonal stats in parallel into separate StatRegistry objects.

  shortcutman

  committed
  4b6f849

  Copy full SHA for 4b6f849

• Stubbed merging of stats registries for simple ops for parallel execution.

  shortcutman

  committed
  fc09454

  Copy full SHA for fc09454

• Added step to merge StatsRegistry and RasterStats. Some floating point error needs confirmation.

  shortcutman

  committed
  af49ebd

  Copy full SHA for af49ebd

• Resolve race condition errors by using thread local GEOS contexts as proscribed by the docs.

  shortcutman

  committed
  cee81ef

  Copy full SHA for cee81ef

• Clean up checks at start of pipeline filters.

  shortcutman

  committed
  cb51593

  Copy full SHA for cb51593

• Command line argument to configure parallel tokens.

  shortcutman

  committed
  d7cd157

  Copy full SHA for d7cd157

• Combine stats for all but variance.

  shortcutman

  committed
  d90d2b1

  Copy full SHA for d90d2b1

• Parallel processor OFF by default and made required dependencies optional.

  shortcutman

  committed
  f86d643

  Copy full SHA for f86d643

• Renamed parallel_raste_processor.* files for consistency.

  shortcutman

  committed
  46c5439

  Copy full SHA for 46c5439

• Restore progress functionality matching raster sequential processor output.

  shortcutman

  committed
  b1f9c4f

  Copy full SHA for b1f9c4f

• Simple clang-format fixes

  shortcutman

  committed
  68889fd

  Copy full SHA for 68889fd

• Clang format changes

  shortcutman

  committed
  b010b3d

  Copy full SHA for b010b3d

• Added basic test for combining RasterStats

  shortcutman

  committed
  9545436

  Copy full SHA for 9545436

• Refactor to support multiple operations.

  shortcutman

  committed
  391177a

  Copy full SHA for 391177a

• Fixes for clang-format

  shortcutman

  committed
  40f8512

  Copy full SHA for 40f8512

Commits on Jan 31, 2025

• Error out on unsupported weighted operations.

  shortcutman

  committed
  2c5ff4f

  Copy full SHA for 2c5ff4f

• Changed tokens to threads for easier adoption by users.

  shortcutman

  committed
  ecbcbb0

  Copy full SHA for ecbcbb0

• Error on unsupported variance operations in raster-parallel strategy.

  shortcutman

  committed
  3b2fc80

  Copy full SHA for 3b2fc80