Skip to content

Commit

Permalink
Disable runfile tree creation (openxla#2046)
Browse files Browse the repository at this point in the history
Investigating why Bazel builds, specifically hydrating from Cache have
gotten so bad seems to be on account of rules_python addition.

I debugged the cache step and found it caching duplication of
python_rules: https://pastebin.com/ibgVB8Yk in the build tree.

These are in fact sym-links however:
```
❯ ls -l bazel-out/k8-fastbuild/bin/stablehlo/tests/vhlo/stablehlo_legalize_to_vhlo.0_9_0.mlir.test.runfiles/rules_python\~0.30.0\~python\~python_3_10_x86_64-unknown-linux-gnu/bin/
lrwxrwxrwx 175 780412 26 Feb 19:58 2to3 -> /usr/local/google/home/fmzakari/.cache/bazel/_bazel_fmzakari/17bce12c4b47a4a2fc75249afee05177/external/rules_python~0.30.0~python~python_3_10_x86_64-unknown-linux-gnu/bin/2to3
lrwxrwxrwx 180 780412 26 Feb 19:58 2to3-3.10 -> /usr/local/google/home/fmzakari/.cache/bazel/_bazel_fmzakari/17bce12c4b47a4a2fc75249afee05177/external/rules_python~0.30.0~python~python_3_10_x86_64-unknown-linux-gnu/bin/2to3-3.10
lrwxrwxrwx 176 780412 26 Feb 19:58 idle3 -> /usr/local/google/home/fmzakari/.cache/bazel/_bazel_fmzakari/17bce12c4b47a4a2fc75249afee05177/external/rules_python~0.30.0~python~python_3_10_x86_64-unknown-linux-gnu/bin/idle3
```

However these are a ton additional inodes that `tar` has to expand and
store in the GitHub action.

People on the Bazel slack pointed me to "-nobuild_runfile_links" which
disables the runfile **at build creation**.
Our tests however still run because `bazel test` has support for
on-demand runfile generation.

This should make the caching much faster.

---------

Co-authored-by: mlevesquedion <[email protected]>
  • Loading branch information
fzakaria and mlevesquedion authored Feb 27, 2024
1 parent 30dac42 commit 70db15a
Show file tree
Hide file tree
Showing 5 changed files with 19 additions and 8 deletions.
11 changes: 11 additions & 0 deletions .bazelrc
Original file line number Diff line number Diff line change
Expand Up @@ -21,3 +21,14 @@ build --features=layering_check
# Set the default compiler to the `clang` binary on the `PATH`.
# TODO(fzakaria): Make this a toolchain or hermetic somehow
build --repo_env=CC=clang

# Disabling runfiles links drastically increases performance in slow disk IO situations
# Do not build runfile trees by default. If an execution strategy relies on runfile
# symlink tree, the tree is created on-demand. See: https://github.com/bazelbuild/bazel/issues/6627
# and https://github.com/bazelbuild/bazel/commit/03246077f948f2790a83520e7dccc2625650e6df
build --nobuild_runfile_links
test --nobuild_runfile_links

# https://bazel.build/reference/command-line-reference#flag--legacy_external_runfiles
build --nolegacy_external_runfiles
test --nolegacy_external_runfiles
4 changes: 2 additions & 2 deletions stablehlo/conversions/linalg/tests/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.
load("@bazel_skylib//rules:expand_template.bzl", "expand_template")
load("@llvm-project//llvm:lit_test.bzl", "lit_test", "package_path")
load("@llvm-project//llvm:lit_test.bzl", "lit_test")

package(
default_visibility = ["//visibility:public"],
Expand All @@ -28,7 +28,7 @@ expand_template(
"\"@STABLEHLO_SOURCE_DIR@\"": "os.path.join(os.environ['TEST_SRCDIR'], '_main')",
"\"@STABLEHLO_TOOLS_DIR@\"": "os.path.join(os.environ['TEST_SRCDIR'], '_main')",
"@LIT_SITE_CFG_IN_HEADER@": "# Autogenerated, do not edit.",
"@LLVM_TOOLS_DIR@": package_path("@llvm-project//llvm:BUILD"),
"@LLVM_TOOLS_DIR@": "../llvm-project/llvm",
},
template = "lit.site.cfg.py.in",
)
Expand Down
4 changes: 2 additions & 2 deletions stablehlo/conversions/tosa/tests/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.
load("@bazel_skylib//rules:expand_template.bzl", "expand_template")
load("@llvm-project//llvm:lit_test.bzl", "lit_test", "package_path")
load("@llvm-project//llvm:lit_test.bzl", "lit_test")

package(
default_visibility = ["//visibility:public"],
Expand All @@ -28,7 +28,7 @@ expand_template(
"\"@STABLEHLO_SOURCE_DIR@\"": "os.path.join(os.environ['TEST_SRCDIR'], '_main')",
"\"@STABLEHLO_TOOLS_DIR@\"": "os.path.join(os.environ['TEST_SRCDIR'], '_main')",
"@LIT_SITE_CFG_IN_HEADER@": "# Autogenerated, do not edit.",
"@LLVM_TOOLS_DIR@": package_path("@llvm-project//llvm:BUILD"),
"@LLVM_TOOLS_DIR@": "../llvm-project/llvm",
},
template = "lit.site.cfg.py.in",
)
Expand Down
4 changes: 2 additions & 2 deletions stablehlo/testdata/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.
load("@bazel_skylib//rules:expand_template.bzl", "expand_template")
load("@llvm-project//llvm:lit_test.bzl", "lit_test", "package_path")
load("@llvm-project//llvm:lit_test.bzl", "lit_test")

package(
default_visibility = ["//visibility:public"],
Expand All @@ -28,7 +28,7 @@ expand_template(
"\"@STABLEHLO_SOURCE_DIR@\"": "os.path.join(os.environ['TEST_SRCDIR'], '_main')",
"\"@STABLEHLO_TOOLS_DIR@\"": "os.path.join(os.environ['TEST_SRCDIR'], '_main')",
"@LIT_SITE_CFG_IN_HEADER@": "# Autogenerated, do not edit.",
"@LLVM_TOOLS_DIR@": package_path("@llvm-project//llvm:BUILD"),
"@LLVM_TOOLS_DIR@": "../llvm-project/llvm",
},
template = "lit.site.cfg.py.in",
)
Expand Down
4 changes: 2 additions & 2 deletions stablehlo/tests/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.
load("@bazel_skylib//rules:expand_template.bzl", "expand_template")
load("@llvm-project//llvm:lit_test.bzl", "lit_test", "package_path")
load("@llvm-project//llvm:lit_test.bzl", "lit_test")
load("@llvm-project//mlir:tblgen.bzl", "gentbl_cc_library", "td_library")
load("@rules_cc//cc:defs.bzl", "cc_library")

Expand Down Expand Up @@ -137,7 +137,7 @@ expand_template(
"\"@STABLEHLO_SOURCE_DIR@\"": "os.path.join(os.environ['TEST_SRCDIR'], '_main')",
"\"@STABLEHLO_TOOLS_DIR@\"": "os.path.join(os.environ['TEST_SRCDIR'], '_main')",
"@LIT_SITE_CFG_IN_HEADER@": "# Autogenerated, do not edit.",
"@LLVM_TOOLS_DIR@": package_path("@llvm-project//llvm:BUILD"),
"@LLVM_TOOLS_DIR@": "../llvm-project/llvm",
},
template = "lit.site.cfg.py.in",
)
Expand Down

0 comments on commit 70db15a

Please sign in to comment.