Do not hold all data in memory at once: fuse List.map and List.fold

Processing 3778 bisect files required >10GiB of memory, growing as more files were processed. Now it uses almost constant memory <50MiB. Signed-off-by: Edwin Török <[email protected]>
edwintorok · Jan 11, 2022 · 7f8962e · 7f8962e
1 parent bb13aa4
commit 7f8962e
Showing 1 changed file with 3 additions and 2 deletions.
diff --git a/src/summary.ml b/src/summary.ml
@@ -68,8 +68,9 @@ let read
   : string list -> runtime
   = fun files ->
   files
-  |> List.map Bisect_common.read_runtime_data
-  |> List.fold_left merge []
+  |> List.fold_left (fun accum file ->
+    merge accum (Bisect_common.read_runtime_data file)
+  ) []
 
 (** [popcount array] returns the number of non-zero entries in an array
  * *)