forked from openxla/stablehlo
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Disable runfile tree creation (openxla#2046)
Investigating why Bazel builds, specifically hydrating from Cache have gotten so bad seems to be on account of rules_python addition. I debugged the cache step and found it caching duplication of python_rules: https://pastebin.com/ibgVB8Yk in the build tree. These are in fact sym-links however: ``` ❯ ls -l bazel-out/k8-fastbuild/bin/stablehlo/tests/vhlo/stablehlo_legalize_to_vhlo.0_9_0.mlir.test.runfiles/rules_python\~0.30.0\~python\~python_3_10_x86_64-unknown-linux-gnu/bin/ lrwxrwxrwx 175 780412 26 Feb 19:58 2to3 -> /usr/local/google/home/fmzakari/.cache/bazel/_bazel_fmzakari/17bce12c4b47a4a2fc75249afee05177/external/rules_python~0.30.0~python~python_3_10_x86_64-unknown-linux-gnu/bin/2to3 lrwxrwxrwx 180 780412 26 Feb 19:58 2to3-3.10 -> /usr/local/google/home/fmzakari/.cache/bazel/_bazel_fmzakari/17bce12c4b47a4a2fc75249afee05177/external/rules_python~0.30.0~python~python_3_10_x86_64-unknown-linux-gnu/bin/2to3-3.10 lrwxrwxrwx 176 780412 26 Feb 19:58 idle3 -> /usr/local/google/home/fmzakari/.cache/bazel/_bazel_fmzakari/17bce12c4b47a4a2fc75249afee05177/external/rules_python~0.30.0~python~python_3_10_x86_64-unknown-linux-gnu/bin/idle3 ``` However these are a ton additional inodes that `tar` has to expand and store in the GitHub action. People on the Bazel slack pointed me to "-nobuild_runfile_links" which disables the runfile **at build creation**. Our tests however still run because `bazel test` has support for on-demand runfile generation. This should make the caching much faster. --------- Co-authored-by: mlevesquedion <[email protected]>
- Loading branch information
Showing
5 changed files
with
19 additions
and
8 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters