Add CCFileProcessorSparkJob to support file-wise processing #45

Open

jt55401 wants to merge 7 commits intomainfrom sparkccfile

+282-3

Commits on Jul 31, 2024

Add sparkccfile.py to support file-wise processing in spark jobs (used in integrity job)
jt55401
committed

Commits on Aug 3, 2024

Commits on Sep 10, 2024

fix s3 functions so they work in spark environment
jt55401
committed

Commits on Sep 11, 2024

fix bug when the file was not downloaded from s3
jt55401
committed

Commits on Nov 27, 2024