Bump ScalaPB, protoc-bridge, gRPC, and Guava deps, and add ProtobufAdapters and ScalaPBCodeGenerator wrappers
#1648
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Adds these versions to `scripts/create_repository.py`.
There are breakages still that future commits will address, but the
following are fixed in this one.
Removes the unnecessary `true` parameter from `.getPlaintext(true)` in
`test/TestServer.scala` to fix this error after the gRPC bump:
```txt
ERROR: .../test/BUILD:676:14:
scala @//test:lib_with_scala_proto_dep failed:
(Exit 1): scalac failed: error executing command
(from target //test:lib_with_scala_proto_dep)
bazel-bin/src/java/io/bazel/rulesscala/scalac/scalac
@bazel-out/darwin_arm64-fastbuild/bin/test/lib_with_scala_proto_dep.jar-0.params
test/TestServer.scala:70: error: no arguments allowed for nullary method usePlaintext: ()?0
val channel = ManagedChannelBuilder.forAddress(host, port).usePlaintext(true).build
^
```
We exclude `com.google.guava:listenablefuture` because trying to
download it breaks the build with an HTTP 404 (everything builds fine
without it anyway):
```txt
INFO: repository @io_bazel_rules_scala_listenablefuture' used the
following cache hits instead of downloading the corresponding file.
* Hash '...' for https://repo.maven.apache.org/maven2/com/google/guava/listenablefuture/9999.0-empty-to-avoid-conflict-with-guava/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar
WARNING: Download from
https://repo.maven.apache.org/maven2/com/google/guava/listenablefuture/9999.0-empty-to-avoid-conflict-with-guava/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava-sources.jar
failed: class java.io.FileNotFoundException GET returned 404 Not Found
WARNING: Download from
https://maven-central.storage-download.googleapis.com/maven2/com/google/guava/listenablefuture/9999.0-empty-to-avoid-conflict-with-guava/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava-sources.jar
failed: class java.io.FileNotFoundException GET returned 404 Not Found
WARNING: Download from
https://mirror.bazel.build/repo1.maven.org/maven2/com/google/guava/listenablefuture/9999.0-empty-to-avoid-conflict-with-guava/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava-sources.jar
failed: class java.io.FileNotFoundException GET returned 404 Not Found
WARNING: Download from
https://jcenter.bintray.com/com/google/guava/listenablefuture/9999.0-empty-to-avoid-conflict-with-guava/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava-sources.jar
failed: class java.io.FileNotFoundException GET returned 404 Not Found
ERROR: An error occurred during the fetch of repository 'io_bazel_rules_scala_listenablefuture':
Traceback (most recent call last):
File ".../scala/scala_maven_import_external.bzl",
line 130, column 32, in _jvm_import_external
repository_ctx.download(srcurls, srcpath, srcsha, auth = _get_auth(repository_ctx, srcurls))
Error in download: java.io.IOException: Error downloading
[ ...4 URLs from WARNINGs above... ]
to .../external/io_bazel_rules_scala_listenablefuture/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava-src.jar:
GET returned 404 Not Found
ERROR: /Users/mbland/src/bazelbuild/rules_scala/WORKSPACE:75:25:
fetching jvm_import_external rule
//external:io_bazel_rules_scala_listenablefuture:
[ ...Traceback, java.io.IOException, and HTTP 404 from above... ]
ERROR: .../external/io_bazel_rules_scala_guava/BUILD:9:13:
@io_bazel_rules_scala_guava//:io_bazel_rules_scala_guava depends on
@io_bazel_rules_scala_listenablefuture//:io_bazel_rules_scala_listenablefuture
in repository @io_bazel_rules_scala_listenablefuture
which failed to fetch. no such package
'@io_bazel_rules_scala_listenablefuture//':
[ ...java.io.IOException and HTTP 404 from above... ]
ERROR: Analysis of target
'//test/proto/custom_generator:failing_scala_proto_deps_toolchain_def'
failed; build aborted:
```
Since Scala 2.11 is stuck on ScalaPB 0.9.8, its dependencies don't
include the `dev.dirs:directories` artifact. Before this change,
building with `--repo_env=SCALA_VERSION=2.11.12 would die with:
```txt
$ bazel build --repo_env=SCALA_VERSION=2.11.12 \
//src/... //test/... //third_party/... //scala_proto/...
ERROR: Traceback (most recent call last):
File "/Users/mbland/src/bazelbuild/rules_scala/WORKSPACE",
line 75, column 25, in <toplevel>
scala_proto_repositories()
File "/Users/mbland/src/bazelbuild/rules_scala/scala_proto/scala_proto.bzl",
line 17, column 37, in scala_proto_repositories
scala_proto_default_repositories(**kwargs)
File "/Users/mbland/src/bazelbuild/rules_scala/scala_proto/default/repositories.bzl",
line 7, column 17, in scala_proto_default_repositories
repositories(
File "/Users/mbland/src/bazelbuild/rules_scala/third_party/repositories/repositories.bzl",
line 107, column 17, in repositories
fail("artifact %s not in third_party/repositories/scala_%s.bzl" % (
Error in fail: artifact dev_dirs_directories not in
third_party/repositories/scala_2_11.bzl
ERROR: Error computing the main repository mapping:
Encountered error while reading extension file 'go/deps.bzl':
no such package '@io_bazel_rules_go//go':
error loading package 'external': Could not load //external package
```
This is a preview of scala_proto toolchainization, given the addition of
`scala_version` and `register_toolchains` parameters to
`scala_proto_default_repositories`.
Without this, the `com.thesamet.scalapb:compilerplugin` artifact is out
of sync with other ScalaPB artifacts, leading to this error:
```txt
$ bazel build //test/proto_cross_repo_boundary/...
ERROR: .../external/proto_cross_repo_boundary/BUILD.bazel:3:14:
scala @proto_cross_repo_boundary//:sample_proto failed:
(Exit 1): scalac failed: error executing command
(from target @proto_cross_repo_boundary//:sample_proto)
bazel-out/.../bin/src/java/io/bazel/rulesscala/scalac/scalac
@bazel-out/.../bin/external/proto_cross_repo_boundary/sample_proto_scalapb.jar-0.params
bazel-out/.../bin/external/proto_cross_repo_boundary/_scalac/sample_proto/sources/1_sample_proto_scala_scalapb.srcjar/sample/sample/Sample.scala:11:
error: class Any needs to be a trait to be mixed in
) extends scalapb.GeneratedMessage with scalapb.Message[Sample] with scalapb.lenses.Updatable[Sample] {
^
Target //test/proto_cross_repo_boundary:sample_scala_proto failed to build
```
There were API changes from scalapb-runtime 0.9.7 causing the following
errors:
```txt
ERROR: .../test/src/main/scala/scalarules/test/extra_protobuf_generator/BUILD:3:14:
scala @//test/src/main/scala/scalarules/test/extra_protobuf_generator:extra_protobuf_generator
failed: (Exit 1): scalac failed: error executing Scalac command
(from target //test/src/main/scala/scalarules/test/extra_protobuf_generator:extra_protobuf_generator)
bazel-out/darwin_arm64-opt-exec-ST-a828a81199fe/bin/src/java/io/bazel/rulesscala/scalac/scalac
... (remaining 1 argument skipped)
test/src/main/scala/scalarules/test/extra_protobuf_generator/ExtraProtobufGenerator.scala:20:
error: value nameSymbol is not a member of com.google.protobuf.Descriptors.Descriptor
.add(s"final case object Custom${message.nameSymbol}{}")
^
test/src/main/scala/scalarules/test/extra_protobuf_generator/ExtraProtobufGenerator.scala:34:
error: value fileDescriptorObjectName is not a member of com.google.protobuf.Descriptors.FileDescriptor
b.setName(file.scalaDirectory + "/Custom" + file.fileDescriptorObjectName + ".scala")
^
test/src/main/scala/scalarules/test/extra_protobuf_generator/ExtraProtobufGenerator.scala:66:
error: value FileDescriptorPimp is not a member of scalapb.compiler.DescriptorImplicits
import implicits.FileDescriptorPimp
^
```
Since Scala 2.11 can't advance past scalapb-runtime 0.9.8, we use
`select_for_scala_version` to select the appropriate `ProtobufAdapter`
to maintain API compatibility.
Also, `import implicits.FileDescriptorPimp` turned out to be unnecessary
after all, even under Scala 2.11.
Fixes the `test_demonstrate_INCORRECT_scala_proto_library_stamp` test
case from `test/shell/test_strict_dependency.sh` after the ScalaPB
0.11.17 bump.
Somehow, bumping that library (presumably, instead of gRPC, et. al.)
changed the parameter list of `TestMessage` from `String` to
`String,UnknownFieldSet`. The test then failed because the following
command produced this unexpected error:
```txt
$ bazel build --verbose_failures \
//test_expect_failure/missing_direct_deps/scala_proto_deps:uses_transitive_scala_proto \
--extra_toolchains=//test/toolchains:ast_plus_one_deps_strict_deps_error
ERROR: .../test_expect_failure/missing_direct_deps/scala_proto_deps/BUILD:25:13:
Building test_expect_failure/missing_direct_deps/scala_proto_deps/libtransitive.jar
(1 source file) failed: (Exit 1): java failed: error executing command
(from target //test_expect_failure/missing_direct_deps/scala_proto_deps:transitive)
(cd ...
[ ...snip... ]
external/remotejdk11_macos_aarch64/bin/java -XX:-CompactStrings
[ ...snip... ]
external/remote_java_tools/java_tools/JavaBuilder_deploy.jar
@bazel-out/.../bin/test_expect_failure/missing_direct_deps/scala_proto_deps/libtransitive.jar-0.params
@bazel-out/.../bin/test_expect_failure/missing_direct_deps/scala_proto_deps/libtransitive.jar-1.params)
test_expect_failure/missing_direct_deps/scala_proto_deps/UseTestMessage.java:7:
error: constructor TestMessage in class TestMessage
cannot be applied to given types;
private final TestMessage m = new TestMessage("");
^
required: String,UnknownFieldSet
found: String
reason: actual and formal argument lists differ in length
```
After this change, the same command produces the following failure that
the test is expecting to catch:
```txt
ERROR: .../test_expect_failure/missing_direct_deps/scala_proto_deps/BUILD:18:14:
scala @//test_expect_failure/missing_direct_deps/scala_proto_deps:uses_transitive_scala_proto
failed: (Exit 1): scalac failed: error executing command
(from target //test_expect_failure/missing_direct_deps/scala_proto_deps:uses_transitive_scala_proto)
(cd .../execroot/io_bazel_rules_scala && \
exec env - \
bazel-out/.../bin/src/java/io/bazel/rulesscala/scalac/scalac
@bazel-out/.../bin/test_expect_failure/missing_direct_deps/scala_proto_deps/uses_transitive_scala_proto.jar-0.params)
test_expect_failure/missing_direct_deps/scala_proto_deps/UseScalaProtoIndirectly.scala:6:
error: Target '@//test_expect_failure/missing_direct_deps/scala_proto_deps:some_proto'
is used but isn't explicitly declared, please add it to the deps.
You can use the following buildozer command:
buildozer 'add deps
@//test_expect_failure/missing_direct_deps/scala_proto_deps:some_proto'
//test_expect_failure/missing_direct_deps/scala_proto_deps:uses_transitive_scala_proto
val foo: TestMessage = new UseTestMessage().getM
^
one error found
Build failure with errors.
Target //test_expect_failure/missing_direct_deps/scala_proto_deps:uses_transitive_scala_proto
failed to build
```
Already did this for `test/TestServer.scala` as part of the gRPC bump.
`test_version/version_specific_tests_dir/TestServer.scala` gets the same
treatment now to fix `test_scala_version 2.11.12` from
`test_version.sh`, which failed with:
```
ERROR: .../test_version/test_scala_version_1731027515/BUILD:161:14:
scala @//:lib_with_scala_proto_dep failed: (Exit 1): scalac failed:
error executing command (from target //:lib_with_scala_proto_dep)
bazel-out/.../bin/external/io_bazel_rules_scala/src/java/io/bazel/rulesscala/scalac/scalac
@bazel-out/.../bin/lib_with_scala_proto_dep.jar-0.params
TestServer.scala:70: error: too many arguments for method usePlaintext: ()?0
val channel = ManagedChannelBuilder.forAddress(host, port).usePlaintext(true).build
^
```
More specifically, add it to the list of core scala repository imports. It had been generated before, but somehow hadn't been required until now. With this change, all tests now pass after the ScalaPB 0.11.17, gRPC 1.68.1, and Guava 33.3.1-jre version bumps. Fixes the following error in `test_scala_version 2.12.20` from `test_version.sh`: ```txt ERROR: .../external/scala_proto_rules_scalapb_compilerplugin/BUILD:9:13: no such package '@org_scala_lang_modules_scala_collection_compat//': The repository '@org_scala_lang_modules_scala_collection_compat' could not be resolved: Repository '@org_scala_lang_modules_scala_collection_compat' is not defined and referenced by '@scala_proto_rules_scalapb_compilerplugin//:scala_proto_rules_scalapb_compilerplugin' ERROR: Analysis of target '//proto:test_proto_nogrpc' failed; build aborted: ```
Wraps `scalapb.ScalaPbCodeGenerator` to catch and return all errors.
Without this, any errors escaping from `scalapb.ScalaPbCodeGenerator`
will cause the `scala_proto` aspect workers to hang.
Also deletes some `@io_bazel_rules_scala` references, replacing them
with `Label` instances as appropriate.
The `test_scala_version 2.11.12` case from `test_version.sh` would
previously hang given the combination of ScalaPB 0.9.8 (the newest
available for Scala 2.11) and Protobuf v28.3. Running the following in a
`test_version/test_scala_version_*` repo generated by `test_version.sh`
reproduced the hang:
```txt
$ bazel build --repo_env=SCALA_VERSION=2.11.12 //proto:test_proto
[ ...wait 10 seconds, then CTRL-C... ]
INFO: Analyzed target //proto:test_proto (2 packages loaded, 22 targets configured).
INFO: Found 1 target...
[1,038 / 1,047] 4 actions running
ProtoScalaPBRule proto/test_service_scala_scalapb.srcjar; 10s worker
ProtoScalaPBRule proto/test2_scala_scalapb.srcjar; 10s worker
Target //proto:test_proto failed to build
```
With this change, the command now exposes the exact incompatibility
between these versions:
```txt
$ bazel build --repo_env=SCALA_VERSION=2.11.12 //proto:test_proto
ERROR: .../test_version/test_scala_version_1731169107/proto/BUILD:14:14:
ProtoScalaPBRule proto/test3_scala_scalapb.srcjar failed: (Exit 1):
scalapb_worker failed: error executing command
(from target //proto:test3)
bazel-out/.../bin/external/io_bazel_rules_scala/src/scala/scripts/scalapb_worker
... (remaining 2 arguments skipped)
--scala_out: java.lang.NoSuchMethodError:
'void com.google.protobuf.Descriptors$FileDescriptor.internalBuildGeneratedFileFrom(java.lang.String[],
com.google.protobuf.Descriptors$FileDescriptor[],
com.google.protobuf.Descriptors$FileDescriptor$InternalDescriptorAssigner)'
at scalapb.options.compiler.Scalapb.<clinit>(Scalapb.java:10592)
at scalapb.ScalaPbCodeGenerator$.run(ScalaPbCodeGenerator.scala:14)
at scalapb.ScalaPbCodeGenerator$.run(ScalaPbCodeGenerator.scala:10)
at scripts.ScalaPbCodeGenerator$.run(ScalaPbCodeGeneratorWrapper_2_11.scala:8)
at protocbridge.frontend.PluginFrontend$$anonfun$runWithBytes$2.apply(PluginFrontend.scala:52)
at protocbridge.frontend.PluginFrontend$$anonfun$runWithBytes$2.apply(PluginFrontend.scala:52)
at scala.util.Try$.apply(Try.scala:192)
at protocbridge.frontend.PluginFrontend$.runWithBytes(PluginFrontend.scala:51)
at protocbridge.frontend.PluginFrontend$.runWithInputStream(PluginFrontend.scala:103)
at protocbridge.frontend.PosixPluginFrontend$$anonfun$prepare$1.apply$mcV$sp(PosixPluginFrontend.scala:33)
at protocbridge.frontend.PosixPluginFrontend$$anonfun$prepare$1.apply(PosixPluginFrontend.scala:31)
at protocbridge.frontend.PosixPluginFrontend$$anonfun$prepare$1.apply(PosixPluginFrontend.scala:31)
at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
at scala.concurrent.impl.ExecutionContextImpl$AdaptedForkJoinTask.exec(ExecutionContextImpl.scala:121)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
java.lang.RuntimeException: Exit with code 1
at scala.sys.package$.error(package.scala:27)
at scripts.ScalaPBWorker$.work(ScalaPBWorker.scala:44)
at io.bazel.rulesscala.worker.Worker.persistentWorkerMain(Worker.java:96)
at io.bazel.rulesscala.worker.Worker.workerMain(Worker.java:49)
at scripts.ScalaPBWorker$.main(ScalaPBWorker.scala:39)
at scripts.ScalaPBWorker.main(ScalaPBWorker.scala)
Target //proto:test_proto failed to build
```
This will support the upcoming schema whereby `scala_toolchains` will import all artifact IDs via macros like this, and invoke `repositories` itself. This will avoid instantiating the same artifact repository multiple times for different frameworks that depend on it. Under `WORKSPACE`, this isn't a problem, but it's an error under Bzlmod, as a module extension can only instantiate a repository once. The commit message to toolchainize Scalafmt will provide more details on this.
This was referenced Nov 19, 2024
|
FYI, and FWIW, Iust filed scalapb/ScalaPB#1771 to see about applying fixes upstream to avoid |
liucijus
approved these changes
Nov 27, 2024
mbland
added a commit
to mbland/rules_scala
that referenced
this pull request
Dec 18, 2024
This version contains this update to `protocbridge.frontend.PluginFrontend.runWithInputStream()`: - https://github.com/scalapb/protoc-bridge/blob/c574d50eaee5b800fd54493fe25c4e0eed3b9def/bridge/src/main/scala/protocbridge/frontend/PluginFrontend.scala#L107-L122 This change originally came from: - https://github.com/scalapb/protoc-bridge/blob/d0d56f635d13f7efaa2755ed0d2d66bdef18b588/bridge/src/main/scala/protocbridge/frontend/PluginFrontend.scala#L107-L122 - scalapb/protoc-bridge#367 This should close scalapb/ScalaPB#1771 and obviate the need for the `try`/`catch` blocks added in bazelbuild#1630 and bazelbuild#1637, and the `ScalaPBCodeGenerator` implementation added in bazelbuild#1648. The update requires special handling in `scripts/create_repository.py` to ensure that only Scala 3.3 and up receive the `protoc-bridge_3` version. Scala 3.1 and 3.2 still receive the `protoc-bridge_2.13` version. However, the `protoc-bridge_3` version currently fails with the following (setting `RULES_SCALA_TEST_ONLY="test_scala_version 3.3.4"` fails the same way): ```txt $ RULES_SCALA_TEST_ONLY="test_scala_version 3.6.2" \ ./test_thirdparty_version.sh ERROR: third_party/test/proto/BUILD.bazel:4:14: ProtoScalaPBRule third_party/test/proto/proto_jvm_extra_protobuf_generator_scalapb.srcjar failed: (Exit 1): scalapb_worker failed: error executing command (from target //third_party/test/proto:proto) bazel-out/darwin_arm64-opt-exec-2B5CBBC6/bin/src/scala/scripts/scalapb_worker ... (remaining 2 arguments skipped) --scala_out: java.lang.NoClassDefFoundError: Could not initialize class scalapb.ScalaPbCodeGenerator$ at scripts.ScalaPbCodeGenerator$.process(ScalaPbCodeGeneratorWrapper.scala:8) at protocgen.CodeGenApp.run(CodeGenApp.scala:48) at protocgen.CodeGenApp.run$(CodeGenApp.scala:41) at scripts.ScalaPbCodeGenerator$.run(ScalaPbCodeGeneratorWrapper.scala:5) at protocgen.CodeGenApp.run(CodeGenApp.scala:33) at protocgen.CodeGenApp.run$(CodeGenApp.scala:32) at scripts.ScalaPbCodeGenerator$.run(ScalaPbCodeGeneratorWrapper.scala:5) at protocbridge.frontend.PluginFrontend$.runWithBytes(PluginFrontend.scala:48) at protocbridge.frontend.PluginFrontend$.runWithInputStream(PluginFrontend.scala:113) at protocbridge.frontend.SocketBasedPluginFrontend.prepare$$anonfun$1$$anonfun$1(SocketBasedPluginFrontend.scala:31) at protocbridge.frontend.SocketBasedPluginFrontend.prepare$$anonfun$1$$anonfun$adapted$1(SocketBasedPluginFrontend.scala:37) at scala.concurrent.impl.ExecutionContextImpl$DefaultThreadFactory$$anon$1$$anon$2.block(ExecutionContextImpl.scala:60) at java.base/java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3118) at scala.concurrent.impl.ExecutionContextImpl$DefaultThreadFactory$$anon$1.blockOn(ExecutionContextImpl.scala:71) at scala.concurrent.package$.blocking(package.scala:124) at protocbridge.frontend.SocketBasedPluginFrontend.prepare$$anonfun$1(SocketBasedPluginFrontend.scala:37) at protocbridge.frontend.SocketBasedPluginFrontend.prepare$$anonfun$adapted$1(SocketBasedPluginFrontend.scala:38) at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:687) at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:467) at java.base/java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1426) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183) java.lang.RuntimeException: Exit with code 1 at scala.sys.package$.error(package.scala:27) at scripts.ScalaPBWorker$.work(ScalaPBWorker.scala:44) at io.bazel.rulesscala.worker.Worker.persistentWorkerMain(Worker.java:96) at io.bazel.rulesscala.worker.Worker.workerMain(Worker.java:49) at scripts.ScalaPBWorker$.main(ScalaPBWorker.scala:39) at scripts.ScalaPBWorker.main(ScalaPBWorker.scala) ERROR: third_party/test/proto/BUILD.bazel:4:14 Building source jar third_party/test/proto/proto_scalapb-src.jar failed: (Exit 1): scalapb_worker failed: error executing command (from target //third_party/test/proto:proto) bazel-out/darwin_arm64-opt-exec-2B5CBBC6/bin/src/scala/scripts/scalapb_worker ... (remaining 2 arguments skipped) ```
mbland
added a commit
to mbland/rules_scala
that referenced
this pull request
Dec 19, 2024
As noted in the comment added to `select_root_artifacts()`,
`com.thesamet.scalapb:compilerplugin_3:0.11.17` depends on
`com.thesamet.scalapb:protoc-gen_2.13:0.9.7`. When using
`com.thesamet.scalapb:protoc-gen_3:0.9.8` instead, we get a crash:
```txt
$ bazel build --repo_env=SCALA_VERSION=3.6.2 //third_party/test/proto:scala
[ ...snip... ]
ERROR: third_party/test/proto/BUILD.bazel:4:14:
ProtoScalaPBRule
third_party/test/proto/proto_jvm_extra_protobuf_generator_scalapb.srcjar
failed: (Exit 1): scalapb_worker failed:
error executing command (from target //third_party/test/proto:proto)
bazel-out/darwin_arm64-opt-exec-2B5CBBC6/bin/src/scala/scripts/scalapb_worker
...
--scala_out: java.lang.NoClassDefFoundError:
Could not initialize class scalapb.ScalaPbCodeGenerator$
at scripts.ScalaPbCodeGenerator$.process(ScalaPbCodeGeneratorWrapper.scala:7)
at protocgen.CodeGenApp.run(CodeGenApp.scala:48)
at protocgen.CodeGenApp.run$(CodeGenApp.scala:23)
at scripts.ScalaPbCodeGenerator$.run(ScalaPbCodeGeneratorWrapper.scala:5)
at protocgen.CodeGenApp.run(CodeGenApp.scala:33)
at protocgen.CodeGenApp.run$(CodeGenApp.scala:23)
at scripts.ScalaPbCodeGenerator$.run(ScalaPbCodeGeneratorWrapper.scala:5)
at protocbridge.frontend.PluginFrontend$.runWithBytes(PluginFrontend.scala:48)
at protocbridge.frontend.PluginFrontend$.runWithInputStream(PluginFrontend.scala:113)
at protocbridge.frontend.SocketBasedPluginFrontend.prepare$$anonfun$1$$anonfun$1(SocketBasedPluginFrontend.scala:31)
at protocbridge.frontend.SocketBasedPluginFrontend.prepare$$anonfun$1$$anonfun$adapted$1(SocketBasedPluginFrontend.scala:37)
at scala.concurrent.impl.ExecutionContextImpl$DefaultThreadFactory$$anon$1$$anon$2.block(ExecutionContextImpl.scala:60)
at java.base/java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3118)
at scala.concurrent.impl.ExecutionContextImpl$DefaultThreadFactory$$anon$1.blockOn(ExecutionContextImpl.scala:71)
at scala.concurrent.package$.blocking(package.scala:124)
at protocbridge.frontend.SocketBasedPluginFrontend.prepare$$anonfun$1(SocketBasedPluginFrontend.scala:37)
at protocbridge.frontend.SocketBasedPluginFrontend.prepare$$anonfun$adapted$1(SocketBasedPluginFrontend.scala:38)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:687)
at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:467)
at java.base/java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1426)
at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)
java.lang.RuntimeException: Exit with code 1
at scala.sys.package$.error(package.scala:27)
at scripts.ScalaPBWorker$.work(ScalaPBWorker.scala:44)
at io.bazel.rulesscala.worker.Worker.persistentWorkerMain(Worker.java:96)
at io.bazel.rulesscala.worker.Worker.workerMain(Worker.java:49)
at scripts.ScalaPBWorker$.main(ScalaPBWorker.scala:39)
at scripts.ScalaPBWorker.main(ScalaPBWorker.scala)
Target //third_party/test/proto:scala failed to build
ERROR: test/proto/BUILD.bazel:4:14
scala @//third_party/test/proto:proto failed:
(Exit 1): scalapb_worker failed:
error executing command (from target //third_party/test/proto:proto)
```
---
As I was going back and testing some more, I couldn't reproduce the
`ProtoScalaPBRule` hanging by setting `protoc-bridge` to 0.9.7 and
`protobuf` to v26.1. I was using this command (using Bazel 7.4.1 to
avoid setting C++ compiler flags for Bazel 6.5.0):
```txt
USE_BAZEL_VERSION=7.4.1 bazel build //third_party/test/proto:scala
```
The command kept crashing every time, instead of hanging. I eventually
realized this was because of two things:
- `scalarules.test.extra_protobuf_generator.ExtraProtobufGenerator.run`
was still catching `Throwable`, since Scala 2.11 support ends with
`protoc-bridge` 0.7.14.
- Bazel kept persistent `io.bazel.rulesscala.scalac.ScalacWorker`
workers running with `ExtraProtobufGenerator` code that still had the
`try`/`catch` block after I tried removing it.
After temporarily editing out the `try`/`catch` block from
`ExtraProtobufGenerator.run` and running `bazel shutdown`, I could
reproduce the hang. I could then confirm `protoc-bridge` 0.9.8 fixed it.
---
Also, TIL that `scripts.ScalaPbCodeGenerator.process` (from
`src/scala/scripts/ScalaPbCodeGeneratorWrapper`) never needed a
`try`/`catch` block. When bumping to
`com.thesamet.scalapb:compilerplugin_2.12:0.11.17` in bazelbuild#1648
(commit 23ae356),
`scalapb.ScalaPbCodeGenerator` went from implementing
`protocbridge.ProtocCodeGenerator` to implementing
`protocgen.CodeGenApp`:
- https://github.com/scalapb/ScalaPB/blob/v0.9.7/compiler-plugin/src/main/scala/scalapb/ScalaPbCodeGenerator.scala#L9
- https://github.com/scalapb/ScalaPB/blob/v0.11.17/compiler-plugin/src/main/scala/scalapb/ScalaPbCodeGenerator.scala#L11
That commit introduced `com.thesamet.scalapb:protoc-gen_2.12:0.9.7`, and
`protocgen.CodeGenApp.run()` already had a `try`/`catch` block for
`Throwable`:
- https://github.com/scalapb/protoc-bridge/blob/v0.9.7/protoc-gen/src/main/scala/protocgen/CodeGenApp.scala#L49-L56
However, `protocbridge.ProtocCodeGenerator`, which
`protocgen.CodeGenApp` extends, only declares a `run()` method with no
implementation (and no `try`/`catch` block):
- https://github.com/scalapb/protoc-bridge/blob/v0.9.7/bridge/src/main/scala/protocbridge/ProtocCodeGenerator.scala#L5
The aforementioned `ExtraProtobufGenerator` only extends
`protocbridge.ProtocCodeGenerator`, not `protocgen.CodeGenApp`. That's
why it will still hang when we remove the `try`/`catch` block from its
`run()` method and run with `protoc-bridge` 0.9.7 and `protobuf` >
v25.5.
mbland
added a commit
to mbland/rules_scala
that referenced
this pull request
Dec 19, 2024
This required updates to `scripts/create_repository.py` to accommodate
comparisons between `_2.13` and `_3` versions of ScalaPB artifacts. The
`_3` versions of the most recent ScalaPB artifacts are only available
for Scala 3.3 and later, and `compilerplugin_3` currently still depends
on `protoc-gen_2.13`.
The `MavenCoordinates` dataclass in `scripts/create_repository.py` has
new `unversioned_artifact`, `scala_version`, and `artifact_name`
members. Updates to `MavenCoordinates.is_newer_than()` now compare
objects with matching `artifact_name` values, and take the
`scala_version` into account. These changes then precipitated:
- Replacing the `artifact_name()` call with `artifact_name` accesses.
- Replacing `ArtifactLabelMaker._remove_scala_version_suffix` with
`MavenCoordinates.unversioned_artifact`.
- Extracting `__compare_versions` from `is_newer_than` to reuse the same
code for comparing Scala versions and artifact versions. (Properly
reversed the use of `lhs` and `rhs` in the process.)
This change also removes an unnecessary `try`/`catch` block from
`scripts.ScalaPbCodeGenerator.process()` for Scala 2.12 and above.
The block is still required for the Scala 2.11 implementation of
`scripts.ScalaPbCodeGenerator.run()` and in
`scalarules.test.extra_protobuf_generator.ExtraProtobufGenerator.run()`.
Finally, this change adds notes in a few places to indicate where
support remains for Scala 2.11. This supporting code can ultimately be
removed if we ever decide to drop Scala 2.11 support.
---
Tested by setting the `protobuf` dependencies to a versions I know will
break `ScalaPB` 0.11.17:
- `abseil-cpp`: 20220623.1 => 20240722.0
- `protobuf`: v21.7 => v26.1
I temporarily removed the `try`/`catch` block for `Throwable` from
`scalarules.test.extra_protobuf_generator.ExtraProtobufGenerator.run`.
I then ran `bazel shutdown` to stop persistent `ProtoScalaPBRule`
workers.
After that, building with Bazel 7.4.1 (since Bazel 6.5.0 can't build
`abseil-cpp` 20240722.0 without setting C++14 compiler flags) crashed as
expected, instead of hanging. Undoing the `abseil-cpp` and `protobuf`
changes and building then produced a working build.
```txt
$ USE_BAZEL_VERSION=7.4.1 bazel build //third_party/test/proto:scala
[ ...snip... ]
ERROR: third_party/test/proto/BUILD.bazel:4:14:
ProtoScalaPBRule
third_party/test/proto/proto_jvm_extra_protobuf_generator_scalapb.srcjar
failed: (Exit 1):
scalapb_worker failed: error executing ProtoScalaPBRule command
(from target //third_party/test/proto:proto)
bazel-out/darwin_arm64-opt-exec-ST-a828a81199fe/bin/src/scala/scripts/scalapb_worker
... (remaining 2 arguments skipped)
--jvm_extra_protobuf_generator_out: java.lang.VerifyError: Cannot inherit from final class
at java.base/java.lang.ClassLoader.defineClass1(Native Method)
at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1022)
at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174)
at java.base/jdk.internal.loader.BuiltinClassLoader.defineClass(BuiltinClassLoader.java:800)
at java.base/jdk.internal.loader.BuiltinClassLoader$4.run(BuiltinClassLoader.java:711)
at java.base/jdk.internal.loader.BuiltinClassLoader$4.run(BuiltinClassLoader.java:706)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at java.base/jdk.internal.loader.BuiltinClassLoader.findClassOnClassPathOrNull(BuiltinClassLoader.java:719)
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:621)
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:579)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527)
at scalapb.options.Scalapb.<clinit>(Scalapb.java:24835)
at scalapb.options.compiler.Scalapb$.registerAllExtensions(Scalapb.scala:8)
at scalarules.test.extra_protobuf_generator.ExtraProtobufGenerator$.run(ExtraProtobufGenerator.scala:52)
at protocbridge.frontend.PluginFrontend$.runWithBytes(PluginFrontend.scala:48)
at protocbridge.frontend.PluginFrontend$.runWithInputStream(PluginFrontend.scala:113)
at protocbridge.frontend.SocketBasedPluginFrontend.$anonfun$prepare$2(SocketBasedPluginFrontend.scala:31)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at scala.concurrent.impl.ExecutionContextImpl$DefaultThreadFactory$$anon$1$$anon$2.block(ExecutionContextImpl.scala:75)
at java.base/java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3118)
at scala.concurrent.impl.ExecutionContextImpl$DefaultThreadFactory$$anon$1.blockOn(ExecutionContextImpl.scala:87)
at scala.concurrent.package$.blocking(package.scala:146)
at protocbridge.frontend.SocketBasedPluginFrontend.$anonfun$prepare$1(SocketBasedPluginFrontend.scala:23)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:42)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:74)
at java.base/java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1426)
at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)
java.lang.RuntimeException: Exit with code 1
at scala.sys.package$.error(package.scala:30)
at scripts.ScalaPBWorker$.work(ScalaPBWorker.scala:44)
at io.bazel.rulesscala.worker.Worker.persistentWorkerMain(Worker.java:96)
at io.bazel.rulesscala.worker.Worker.workerMain(Worker.java:49)
at scripts.ScalaPBWorker$.main(ScalaPBWorker.scala:39)
at scripts.ScalaPBWorker.main(ScalaPBWorker.scala)
Target //third_party/test/proto:scala failed to build
ERROR: third_party/test/proto/BUILD.bazel:4:14
scala @//third_party/test/proto:proto failed:
(Exit 1): scalapb_worker failed:
error executing ProtoScalaPBRule command
(from target //third_party/test/proto:proto)
bazel-out/darwin_arm64-opt-exec-ST-a828a81199fe/bin/src/scala/scripts/scalapb_worker
... (remaining 2 arguments skipped)
```
---
The initial motivation for this change was to try to eliminate the need
for the custom `scripts.ScalaPbCodeGenerator` implementation by
upgrading to `protoc-bridge` 0.9.8. This version contains a change to
`catch` any `Throwable` objects raised by a generator implementation in
`protocbridge.frontend.PluginFrontend.runWithInputStream()`.
- https://github.com/scalapb/protoc-bridge/blob/c574d50eaee5b800fd54493fe25c4e0eed3b9def/bridge/src/main/scala/protocbridge/frontend/PluginFrontend.scala#L107-L122
This change originally came from:
- https://github.com/scalapb/protoc-bridge/blob/d0d56f635d13f7efaa2755ed0d2d66bdef18b588/bridge/src/main/scala/protocbridge/frontend/PluginFrontend.scala#L107-L122
- scalapb/protoc-bridge#367
I closed scalapb/ScalaPB#1771 as a result, and reexamined the
`try`/`catch` blocks added in bazelbuild#1630 and bazelbuild#1637, and the
`scripts.ScalaPBCodeGenerator` implementations added in bazelbuild#1648.
However, the last `protoc-bridge` version to support Scala 2.11 is
0.7.14. Scala 2.11 won't be able to use `protoc-bridge` 0.9.8, so we
still need to `catch` any `Throwable`s ourselves. This includes the
`catch` blocks from the Scala 2.11 `scripts.ScalaPbCodeGenerator` and
`scalarules.test.extra_protobuf_generator.ExtraProtobufGenerator.run`.
---
In `protoc-bridge` 0.9.7 and 0.9.8, `protocbridge.ProtocCodeGenerator`
declares a `run()` method with no implementation (and no `try`/`catch`
block).
- https://github.com/scalapb/protoc-bridge/blob/v0.9.7/bridge/src/main/scala/protocbridge/ProtocCodeGenerator.scala#L5
In `protoc-gen` 0.9.7 and 0.9.8, `protogen.CodeGenApp` implements
`protocbridge.ProtocCodeGenerator`, and `CodeGenApp.run()` wraps a call
to `CodeGenApp.process()` inside a `try`/`catch` block.
- https://github.com/scalapb/protoc-bridge/blob/v0.9.7/protoc-gen/src/main/scala/protocgen/CodeGenApp.scala#L41-L56
`scalapb.ScalaPbCodeGenerator` from `compilerplugin` 0.11.17 extends
`CodeGenApp` and implements `process()`. For this version of
`ScalaPbCodeGenerator`, available in Scala 2.12 and later, there's no
need for a `try`/`catch` wrapper on our end.
- https://github.com/scalapb/ScalaPB/blob/v0.11.17/compiler-plugin/src/main/scala/scalapb/ScalaPbCodeGenerator.scala#L11
However, the last available `compilerplugin` version for Scala 2.11 is
0.9.8. `scalapb.ScalaPbCodeGenerator` from that version implements
`protocbridge.ProtocCodeGenerator` and has a `try`/`catch` in its
`run()` method. However, the `Scalapb.registerAllExtensions(registry)`
call on line 14 lies outside this block, producing the crash described
in bazelbuild#1648 (commit 23ae356). This is why
we need the Scala 2.11 implementation of `scripts.ScalaPbCodeGenerator`.
- https://github.com/scalapb/ScalaPB/blob/v0.9.8/compiler-plugin/src/main/scala/scalapb/ScalaPbCodeGenerator.scala#L14
`scalarules.test.extra_protobuf_generator.ExtraProtobufGenerator`,
also extends `ProtocCodeGenerator` directly, instead of implementing
`CodeGenApp` (so it also builds with Scala 2.11). This is why
`ExtraProtobufGenerator.run()` hangs when we remove the `try`/`catch`
block and run with `protoc-bridge` 0.9.7 and `protobuf` > v25.5, even
with later Scala versions.
---
Since `scalapb.ScalaPbCodeGenerator` from `compilerplugin` 0.11.17
implements `CodeGenApp`, `scripts.ScalaPbCodeGenerator` for Scala 2.12
should be unnecessary. The only reason we need it is to maintain the
same interface as the Scala 2.11 implementation.
---
Before the `MavenCoordinates` changes, `scripts/create_repository.py`
would ScalaPB flip ScalaPB artifacts between their `_2.13` and `_3`
versions on subsequent runs (e.g., `protoc-bridge_2.13` vs.
`protoc-bridge_3`). See the comments added within
`MavenCoordinates.new()` for details.
---
Building Scala 3.3 and later versions with `compilerplugin_3` and
`protoc-bridge_3` artifacts (instead of `protoc-bridge_2.13` artifacts)
produces the following build failure:
```txt
$ RULES_SCALA_TEST_ONLY="test_scala_version 3.6.2" \
./test_thirdparty_version.sh
[ ...snip... ]
ERROR: third_party/test/proto/BUILD.bazel:4:14:
ProtoScalaPBRule
third_party/test/proto/proto_jvm_extra_protobuf_generator_scalapb.srcjar
failed: (Exit 1): scalapb_worker failed: error executing command
(from target //third_party/test/proto:proto)
bazel-out/darwin_arm64-opt-exec-2B5CBBC6/bin/src/scala/scripts/scalapb_worker
... (remaining 2 arguments skipped)
--scala_out: java.lang.NoClassDefFoundError:
Could not initialize class scalapb.ScalaPbCodeGenerator$
at scripts.ScalaPbCodeGenerator$.process(ScalaPbCodeGeneratorWrapper.scala:8)
at protocgen.CodeGenApp.run(CodeGenApp.scala:48)
at protocgen.CodeGenApp.run$(CodeGenApp.scala:41)
at scripts.ScalaPbCodeGenerator$.run(ScalaPbCodeGeneratorWrapper.scala:5)
at protocgen.CodeGenApp.run(CodeGenApp.scala:33)
at protocgen.CodeGenApp.run$(CodeGenApp.scala:32)
at scripts.ScalaPbCodeGenerator$.run(ScalaPbCodeGeneratorWrapper.scala:5)
at protocbridge.frontend.PluginFrontend$.runWithBytes(PluginFrontend.scala:48)
at protocbridge.frontend.PluginFrontend$.runWithInputStream(PluginFrontend.scala:113)
at protocbridge.frontend.SocketBasedPluginFrontend.prepare$$anonfun$1$$anonfun$1(SocketBasedPluginFrontend.scala:31)
at protocbridge.frontend.SocketBasedPluginFrontend.prepare$$anonfun$1$$anonfun$adapted$1(SocketBasedPluginFrontend.scala:37)
at scala.concurrent.impl.ExecutionContextImpl$DefaultThreadFactory$$anon$1$$anon$2.block(ExecutionContextImpl.scala:60)
at java.base/java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3118)
at scala.concurrent.impl.ExecutionContextImpl$DefaultThreadFactory$$anon$1.blockOn(ExecutionContextImpl.scala:71)
at scala.concurrent.package$.blocking(package.scala:124)
at protocbridge.frontend.SocketBasedPluginFrontend.prepare$$anonfun$1(SocketBasedPluginFrontend.scala:37)
at protocbridge.frontend.SocketBasedPluginFrontend.prepare$$anonfun$adapted$1(SocketBasedPluginFrontend.scala:38)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:687)
at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:467)
at java.base/java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1426)
at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)
java.lang.RuntimeException: Exit with code 1
at scala.sys.package$.error(package.scala:27)
at scripts.ScalaPBWorker$.work(ScalaPBWorker.scala:44)
at io.bazel.rulesscala.worker.Worker.persistentWorkerMain(Worker.java:96)
at io.bazel.rulesscala.worker.Worker.workerMain(Worker.java:49)
at scripts.ScalaPBWorker$.main(ScalaPBWorker.scala:39)
at scripts.ScalaPBWorker.main(ScalaPBWorker.scala)
ERROR: third_party/test/proto/BUILD.bazel:4:14 Building source jar
third_party/test/proto/proto_scalapb-src.jar failed: (Exit 1):
scalapb_worker failed: error executing command
(from target //third_party/test/proto:proto)
bazel-out/darwin_arm64-opt-exec-2B5CBBC6/bin/src/scala/scripts/scalapb_worker
... (remaining 2 arguments skipped)
```
mbland
added a commit
to mbland/rules_scala
that referenced
this pull request
Jan 17, 2025
This required updates to `scripts/create_repository.py` to accommodate
comparisons between `_2.13` and `_3` versions of ScalaPB artifacts. The
`_3` versions of the most recent ScalaPB artifacts are only available
for Scala 3.3 and later, and `compilerplugin_3` currently still depends
on `protoc-gen_2.13`.
The `MavenCoordinates` dataclass in `scripts/create_repository.py` has
new `unversioned_artifact`, `scala_version`, and `artifact_name`
members. Updates to `MavenCoordinates.is_newer_than()` now compare
objects with matching `artifact_name` values, and take the
`scala_version` into account. These changes then precipitated:
- Replacing the `artifact_name()` call with `artifact_name` accesses.
- Replacing `ArtifactLabelMaker._remove_scala_version_suffix` with
`MavenCoordinates.unversioned_artifact`.
- Extracting `__compare_versions` from `is_newer_than` to reuse the same
code for comparing Scala versions and artifact versions. (Properly
reversed the use of `lhs` and `rhs` in the process.)
This change also removes an unnecessary `try`/`catch` block from
`scripts.ScalaPbCodeGenerator.process()` for Scala 2.12 and above.
The block is still required for the Scala 2.11 implementation of
`scripts.ScalaPbCodeGenerator.run()` and in
`scalarules.test.extra_protobuf_generator.ExtraProtobufGenerator.run()`.
Finally, this change adds notes in a few places to indicate where
support remains for Scala 2.11. This supporting code can ultimately be
removed if we ever decide to drop Scala 2.11 support.
---
Tested by setting the `protobuf` dependencies to a versions I know will
break `ScalaPB` 0.11.17:
- `abseil-cpp`: 20220623.1 => 20240722.0
- `protobuf`: v21.7 => v26.1
I temporarily removed the `try`/`catch` block for `Throwable` from
`scalarules.test.extra_protobuf_generator.ExtraProtobufGenerator.run`.
I then ran `bazel shutdown` to stop persistent `ProtoScalaPBRule`
workers.
After that, building with Bazel 7.4.1 (since Bazel 6.5.0 can't build
`abseil-cpp` 20240722.0 without setting C++14 compiler flags) crashed as
expected, instead of hanging. Undoing the `abseil-cpp` and `protobuf`
changes and building then produced a working build.
```txt
$ USE_BAZEL_VERSION=7.4.1 bazel build //third_party/test/proto:scala
[ ...snip... ]
ERROR: third_party/test/proto/BUILD.bazel:4:14:
ProtoScalaPBRule
third_party/test/proto/proto_jvm_extra_protobuf_generator_scalapb.srcjar
failed: (Exit 1):
scalapb_worker failed: error executing ProtoScalaPBRule command
(from target //third_party/test/proto:proto)
bazel-out/darwin_arm64-opt-exec-ST-a828a81199fe/bin/src/scala/scripts/scalapb_worker
... (remaining 2 arguments skipped)
--jvm_extra_protobuf_generator_out: java.lang.VerifyError: Cannot inherit from final class
at java.base/java.lang.ClassLoader.defineClass1(Native Method)
at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1022)
at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174)
at java.base/jdk.internal.loader.BuiltinClassLoader.defineClass(BuiltinClassLoader.java:800)
at java.base/jdk.internal.loader.BuiltinClassLoader$4.run(BuiltinClassLoader.java:711)
at java.base/jdk.internal.loader.BuiltinClassLoader$4.run(BuiltinClassLoader.java:706)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at java.base/jdk.internal.loader.BuiltinClassLoader.findClassOnClassPathOrNull(BuiltinClassLoader.java:719)
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:621)
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:579)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527)
at scalapb.options.Scalapb.<clinit>(Scalapb.java:24835)
at scalapb.options.compiler.Scalapb$.registerAllExtensions(Scalapb.scala:8)
at scalarules.test.extra_protobuf_generator.ExtraProtobufGenerator$.run(ExtraProtobufGenerator.scala:52)
at protocbridge.frontend.PluginFrontend$.runWithBytes(PluginFrontend.scala:48)
at protocbridge.frontend.PluginFrontend$.runWithInputStream(PluginFrontend.scala:113)
at protocbridge.frontend.SocketBasedPluginFrontend.$anonfun$prepare$2(SocketBasedPluginFrontend.scala:31)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at scala.concurrent.impl.ExecutionContextImpl$DefaultThreadFactory$$anon$1$$anon$2.block(ExecutionContextImpl.scala:75)
at java.base/java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3118)
at scala.concurrent.impl.ExecutionContextImpl$DefaultThreadFactory$$anon$1.blockOn(ExecutionContextImpl.scala:87)
at scala.concurrent.package$.blocking(package.scala:146)
at protocbridge.frontend.SocketBasedPluginFrontend.$anonfun$prepare$1(SocketBasedPluginFrontend.scala:23)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:42)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:74)
at java.base/java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1426)
at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)
java.lang.RuntimeException: Exit with code 1
at scala.sys.package$.error(package.scala:30)
at scripts.ScalaPBWorker$.work(ScalaPBWorker.scala:44)
at io.bazel.rulesscala.worker.Worker.persistentWorkerMain(Worker.java:96)
at io.bazel.rulesscala.worker.Worker.workerMain(Worker.java:49)
at scripts.ScalaPBWorker$.main(ScalaPBWorker.scala:39)
at scripts.ScalaPBWorker.main(ScalaPBWorker.scala)
Target //third_party/test/proto:scala failed to build
ERROR: third_party/test/proto/BUILD.bazel:4:14
scala @//third_party/test/proto:proto failed:
(Exit 1): scalapb_worker failed:
error executing ProtoScalaPBRule command
(from target //third_party/test/proto:proto)
bazel-out/darwin_arm64-opt-exec-ST-a828a81199fe/bin/src/scala/scripts/scalapb_worker
... (remaining 2 arguments skipped)
```
---
The initial motivation for this change was to try to eliminate the need
for the custom `scripts.ScalaPbCodeGenerator` implementation by
upgrading to `protoc-bridge` 0.9.8. This version contains a change to
`catch` any `Throwable` objects raised by a generator implementation in
`protocbridge.frontend.PluginFrontend.runWithInputStream()`.
- https://github.com/scalapb/protoc-bridge/blob/c574d50eaee5b800fd54493fe25c4e0eed3b9def/bridge/src/main/scala/protocbridge/frontend/PluginFrontend.scala#L107-L122
This change originally came from:
- https://github.com/scalapb/protoc-bridge/blob/d0d56f635d13f7efaa2755ed0d2d66bdef18b588/bridge/src/main/scala/protocbridge/frontend/PluginFrontend.scala#L107-L122
- scalapb/protoc-bridge#367
I closed scalapb/ScalaPB#1771 as a result, and reexamined the
`try`/`catch` blocks added in bazelbuild#1630 and bazelbuild#1637, and the
`scripts.ScalaPBCodeGenerator` implementations added in bazelbuild#1648.
However, the last `protoc-bridge` version to support Scala 2.11 is
0.7.14. Scala 2.11 won't be able to use `protoc-bridge` 0.9.8, so we
still need to `catch` any `Throwable`s ourselves. This includes the
`catch` blocks from the Scala 2.11 `scripts.ScalaPbCodeGenerator` and
`scalarules.test.extra_protobuf_generator.ExtraProtobufGenerator.run`.
---
In `protoc-bridge` 0.9.7 and 0.9.8, `protocbridge.ProtocCodeGenerator`
declares a `run()` method with no implementation (and no `try`/`catch`
block).
- https://github.com/scalapb/protoc-bridge/blob/v0.9.7/bridge/src/main/scala/protocbridge/ProtocCodeGenerator.scala#L5
In `protoc-gen` 0.9.7 and 0.9.8, `protogen.CodeGenApp` implements
`protocbridge.ProtocCodeGenerator`, and `CodeGenApp.run()` wraps a call
to `CodeGenApp.process()` inside a `try`/`catch` block.
- https://github.com/scalapb/protoc-bridge/blob/v0.9.7/protoc-gen/src/main/scala/protocgen/CodeGenApp.scala#L41-L56
`scalapb.ScalaPbCodeGenerator` from `compilerplugin` 0.11.17 extends
`CodeGenApp` and implements `process()`. For this version of
`ScalaPbCodeGenerator`, available in Scala 2.12 and later, there's no
need for a `try`/`catch` wrapper on our end.
- https://github.com/scalapb/ScalaPB/blob/v0.11.17/compiler-plugin/src/main/scala/scalapb/ScalaPbCodeGenerator.scala#L11
However, the last available `compilerplugin` version for Scala 2.11 is
0.9.8. `scalapb.ScalaPbCodeGenerator` from that version implements
`protocbridge.ProtocCodeGenerator` and has a `try`/`catch` in its
`run()` method. However, the `Scalapb.registerAllExtensions(registry)`
call on line 14 lies outside this block, producing the crash described
in bazelbuild#1648 (commit 23ae356). This is why
we need the Scala 2.11 implementation of `scripts.ScalaPbCodeGenerator`.
- https://github.com/scalapb/ScalaPB/blob/v0.9.8/compiler-plugin/src/main/scala/scalapb/ScalaPbCodeGenerator.scala#L14
`scalarules.test.extra_protobuf_generator.ExtraProtobufGenerator`,
also extends `ProtocCodeGenerator` directly, instead of implementing
`CodeGenApp` (so it also builds with Scala 2.11). This is why
`ExtraProtobufGenerator.run()` hangs when we remove the `try`/`catch`
block and run with `protoc-bridge` 0.9.7 and `protobuf` > v25.5, even
with later Scala versions.
---
Since `scalapb.ScalaPbCodeGenerator` from `compilerplugin` 0.11.17
implements `CodeGenApp`, `scripts.ScalaPbCodeGenerator` for Scala 2.12
should be unnecessary. The only reason we need it is to maintain the
same interface as the Scala 2.11 implementation.
---
Before the `MavenCoordinates` changes, `scripts/create_repository.py`
would ScalaPB flip ScalaPB artifacts between their `_2.13` and `_3`
versions on subsequent runs (e.g., `protoc-bridge_2.13` vs.
`protoc-bridge_3`). See the comments added within
`MavenCoordinates.new()` for details.
---
Building Scala 3.3 and later versions with `compilerplugin_3` and
`protoc-bridge_3` artifacts (instead of `protoc-bridge_2.13` artifacts)
produces the following build failure:
```txt
$ RULES_SCALA_TEST_ONLY="test_scala_version 3.6.2" \
./test_thirdparty_version.sh
[ ...snip... ]
ERROR: third_party/test/proto/BUILD.bazel:4:14:
ProtoScalaPBRule
third_party/test/proto/proto_jvm_extra_protobuf_generator_scalapb.srcjar
failed: (Exit 1): scalapb_worker failed: error executing command
(from target //third_party/test/proto:proto)
bazel-out/darwin_arm64-opt-exec-2B5CBBC6/bin/src/scala/scripts/scalapb_worker
... (remaining 2 arguments skipped)
--scala_out: java.lang.NoClassDefFoundError:
Could not initialize class scalapb.ScalaPbCodeGenerator$
at scripts.ScalaPbCodeGenerator$.process(ScalaPbCodeGeneratorWrapper.scala:8)
at protocgen.CodeGenApp.run(CodeGenApp.scala:48)
at protocgen.CodeGenApp.run$(CodeGenApp.scala:41)
at scripts.ScalaPbCodeGenerator$.run(ScalaPbCodeGeneratorWrapper.scala:5)
at protocgen.CodeGenApp.run(CodeGenApp.scala:33)
at protocgen.CodeGenApp.run$(CodeGenApp.scala:32)
at scripts.ScalaPbCodeGenerator$.run(ScalaPbCodeGeneratorWrapper.scala:5)
at protocbridge.frontend.PluginFrontend$.runWithBytes(PluginFrontend.scala:48)
at protocbridge.frontend.PluginFrontend$.runWithInputStream(PluginFrontend.scala:113)
at protocbridge.frontend.SocketBasedPluginFrontend.prepare$$anonfun$1$$anonfun$1(SocketBasedPluginFrontend.scala:31)
at protocbridge.frontend.SocketBasedPluginFrontend.prepare$$anonfun$1$$anonfun$adapted$1(SocketBasedPluginFrontend.scala:37)
at scala.concurrent.impl.ExecutionContextImpl$DefaultThreadFactory$$anon$1$$anon$2.block(ExecutionContextImpl.scala:60)
at java.base/java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3118)
at scala.concurrent.impl.ExecutionContextImpl$DefaultThreadFactory$$anon$1.blockOn(ExecutionContextImpl.scala:71)
at scala.concurrent.package$.blocking(package.scala:124)
at protocbridge.frontend.SocketBasedPluginFrontend.prepare$$anonfun$1(SocketBasedPluginFrontend.scala:37)
at protocbridge.frontend.SocketBasedPluginFrontend.prepare$$anonfun$adapted$1(SocketBasedPluginFrontend.scala:38)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:687)
at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:467)
at java.base/java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1426)
at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)
java.lang.RuntimeException: Exit with code 1
at scala.sys.package$.error(package.scala:27)
at scripts.ScalaPBWorker$.work(ScalaPBWorker.scala:44)
at io.bazel.rulesscala.worker.Worker.persistentWorkerMain(Worker.java:96)
at io.bazel.rulesscala.worker.Worker.workerMain(Worker.java:49)
at scripts.ScalaPBWorker$.main(ScalaPBWorker.scala:39)
at scripts.ScalaPBWorker.main(ScalaPBWorker.scala)
ERROR: third_party/test/proto/BUILD.bazel:4:14 Building source jar
third_party/test/proto/proto_scalapb-src.jar failed: (Exit 1):
scalapb_worker failed: error executing command
(from target //third_party/test/proto:proto)
bazel-out/darwin_arm64-opt-exec-2B5CBBC6/bin/src/scala/scripts/scalapb_worker
... (remaining 2 arguments skipped)
```
mbland
added a commit
to mbland/rules_scala
that referenced
this pull request
Jan 27, 2025
This required updates to `scripts/create_repository.py` to accommodate
comparisons between `_2.13` and `_3` versions of ScalaPB artifacts. The
`_3` versions of the most recent ScalaPB artifacts are only available
for Scala 3.3 and later, and `compilerplugin_3` currently still depends
on `protoc-gen_2.13`.
The `MavenCoordinates` dataclass in `scripts/create_repository.py` has
new `unversioned_artifact`, `scala_version`, and `artifact_name`
members. Updates to `MavenCoordinates.is_newer_than()` now compare
objects with matching `artifact_name` values, and take the
`scala_version` into account. These changes then precipitated:
- Replacing the `artifact_name()` call with `artifact_name` accesses.
- Replacing `ArtifactLabelMaker._remove_scala_version_suffix` with
`MavenCoordinates.unversioned_artifact`.
- Extracting `__compare_versions` from `is_newer_than` to reuse the same
code for comparing Scala versions and artifact versions. (Properly
reversed the use of `lhs` and `rhs` in the process.)
This change also removes an unnecessary `try`/`catch` block from
`scripts.ScalaPbCodeGenerator.process()` for Scala 2.12 and above.
The block is still required for the Scala 2.11 implementation of
`scripts.ScalaPbCodeGenerator.run()` and in
`scalarules.test.extra_protobuf_generator.ExtraProtobufGenerator.run()`.
Finally, this change adds notes in a few places to indicate where
support remains for Scala 2.11. This supporting code can ultimately be
removed if we ever decide to drop Scala 2.11 support.
---
Tested by setting the `protobuf` dependencies to a versions I know will
break `ScalaPB` 0.11.17:
- `abseil-cpp`: 20220623.1 => 20240722.0
- `protobuf`: v21.7 => v26.1
I temporarily removed the `try`/`catch` block for `Throwable` from
`scalarules.test.extra_protobuf_generator.ExtraProtobufGenerator.run`.
I then ran `bazel shutdown` to stop persistent `ProtoScalaPBRule`
workers.
After that, building with Bazel 7.4.1 (since Bazel 6.5.0 can't build
`abseil-cpp` 20240722.0 without setting C++14 compiler flags) crashed as
expected, instead of hanging. Undoing the `abseil-cpp` and `protobuf`
changes and building then produced a working build.
```txt
$ USE_BAZEL_VERSION=7.4.1 bazel build //third_party/test/proto:scala
[ ...snip... ]
ERROR: third_party/test/proto/BUILD.bazel:4:14:
ProtoScalaPBRule
third_party/test/proto/proto_jvm_extra_protobuf_generator_scalapb.srcjar
failed: (Exit 1):
scalapb_worker failed: error executing ProtoScalaPBRule command
(from target //third_party/test/proto:proto)
bazel-out/darwin_arm64-opt-exec-ST-a828a81199fe/bin/src/scala/scripts/scalapb_worker
... (remaining 2 arguments skipped)
--jvm_extra_protobuf_generator_out: java.lang.VerifyError: Cannot inherit from final class
at java.base/java.lang.ClassLoader.defineClass1(Native Method)
at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1022)
at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174)
at java.base/jdk.internal.loader.BuiltinClassLoader.defineClass(BuiltinClassLoader.java:800)
at java.base/jdk.internal.loader.BuiltinClassLoader$4.run(BuiltinClassLoader.java:711)
at java.base/jdk.internal.loader.BuiltinClassLoader$4.run(BuiltinClassLoader.java:706)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at java.base/jdk.internal.loader.BuiltinClassLoader.findClassOnClassPathOrNull(BuiltinClassLoader.java:719)
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:621)
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:579)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527)
at scalapb.options.Scalapb.<clinit>(Scalapb.java:24835)
at scalapb.options.compiler.Scalapb$.registerAllExtensions(Scalapb.scala:8)
at scalarules.test.extra_protobuf_generator.ExtraProtobufGenerator$.run(ExtraProtobufGenerator.scala:52)
at protocbridge.frontend.PluginFrontend$.runWithBytes(PluginFrontend.scala:48)
at protocbridge.frontend.PluginFrontend$.runWithInputStream(PluginFrontend.scala:113)
at protocbridge.frontend.SocketBasedPluginFrontend.$anonfun$prepare$2(SocketBasedPluginFrontend.scala:31)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at scala.concurrent.impl.ExecutionContextImpl$DefaultThreadFactory$$anon$1$$anon$2.block(ExecutionContextImpl.scala:75)
at java.base/java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3118)
at scala.concurrent.impl.ExecutionContextImpl$DefaultThreadFactory$$anon$1.blockOn(ExecutionContextImpl.scala:87)
at scala.concurrent.package$.blocking(package.scala:146)
at protocbridge.frontend.SocketBasedPluginFrontend.$anonfun$prepare$1(SocketBasedPluginFrontend.scala:23)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:42)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:74)
at java.base/java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1426)
at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)
java.lang.RuntimeException: Exit with code 1
at scala.sys.package$.error(package.scala:30)
at scripts.ScalaPBWorker$.work(ScalaPBWorker.scala:44)
at io.bazel.rulesscala.worker.Worker.persistentWorkerMain(Worker.java:96)
at io.bazel.rulesscala.worker.Worker.workerMain(Worker.java:49)
at scripts.ScalaPBWorker$.main(ScalaPBWorker.scala:39)
at scripts.ScalaPBWorker.main(ScalaPBWorker.scala)
Target //third_party/test/proto:scala failed to build
ERROR: third_party/test/proto/BUILD.bazel:4:14
scala @//third_party/test/proto:proto failed:
(Exit 1): scalapb_worker failed:
error executing ProtoScalaPBRule command
(from target //third_party/test/proto:proto)
bazel-out/darwin_arm64-opt-exec-ST-a828a81199fe/bin/src/scala/scripts/scalapb_worker
... (remaining 2 arguments skipped)
```
---
The initial motivation for this change was to try to eliminate the need
for the custom `scripts.ScalaPbCodeGenerator` implementation by
upgrading to `protoc-bridge` 0.9.8. This version contains a change to
`catch` any `Throwable` objects raised by a generator implementation in
`protocbridge.frontend.PluginFrontend.runWithInputStream()`.
- https://github.com/scalapb/protoc-bridge/blob/c574d50eaee5b800fd54493fe25c4e0eed3b9def/bridge/src/main/scala/protocbridge/frontend/PluginFrontend.scala#L107-L122
This change originally came from:
- https://github.com/scalapb/protoc-bridge/blob/d0d56f635d13f7efaa2755ed0d2d66bdef18b588/bridge/src/main/scala/protocbridge/frontend/PluginFrontend.scala#L107-L122
- scalapb/protoc-bridge#367
I closed scalapb/ScalaPB#1771 as a result, and reexamined the
`try`/`catch` blocks added in bazelbuild#1630 and bazelbuild#1637, and the
`scripts.ScalaPBCodeGenerator` implementations added in bazelbuild#1648.
However, the last `protoc-bridge` version to support Scala 2.11 is
0.7.14. Scala 2.11 won't be able to use `protoc-bridge` 0.9.8, so we
still need to `catch` any `Throwable`s ourselves. This includes the
`catch` blocks from the Scala 2.11 `scripts.ScalaPbCodeGenerator` and
`scalarules.test.extra_protobuf_generator.ExtraProtobufGenerator.run`.
---
In `protoc-bridge` 0.9.7 and 0.9.8, `protocbridge.ProtocCodeGenerator`
declares a `run()` method with no implementation (and no `try`/`catch`
block).
- https://github.com/scalapb/protoc-bridge/blob/v0.9.7/bridge/src/main/scala/protocbridge/ProtocCodeGenerator.scala#L5
In `protoc-gen` 0.9.7 and 0.9.8, `protogen.CodeGenApp` implements
`protocbridge.ProtocCodeGenerator`, and `CodeGenApp.run()` wraps a call
to `CodeGenApp.process()` inside a `try`/`catch` block.
- https://github.com/scalapb/protoc-bridge/blob/v0.9.7/protoc-gen/src/main/scala/protocgen/CodeGenApp.scala#L41-L56
`scalapb.ScalaPbCodeGenerator` from `compilerplugin` 0.11.17 extends
`CodeGenApp` and implements `process()`. For this version of
`ScalaPbCodeGenerator`, available in Scala 2.12 and later, there's no
need for a `try`/`catch` wrapper on our end.
- https://github.com/scalapb/ScalaPB/blob/v0.11.17/compiler-plugin/src/main/scala/scalapb/ScalaPbCodeGenerator.scala#L11
However, the last available `compilerplugin` version for Scala 2.11 is
0.9.8. `scalapb.ScalaPbCodeGenerator` from that version implements
`protocbridge.ProtocCodeGenerator` and has a `try`/`catch` in its
`run()` method. However, the `Scalapb.registerAllExtensions(registry)`
call on line 14 lies outside this block, producing the crash described
in bazelbuild#1648 (commit 23ae356). This is why
we need the Scala 2.11 implementation of `scripts.ScalaPbCodeGenerator`.
- https://github.com/scalapb/ScalaPB/blob/v0.9.8/compiler-plugin/src/main/scala/scalapb/ScalaPbCodeGenerator.scala#L14
`scalarules.test.extra_protobuf_generator.ExtraProtobufGenerator`,
also extends `ProtocCodeGenerator` directly, instead of implementing
`CodeGenApp` (so it also builds with Scala 2.11). This is why
`ExtraProtobufGenerator.run()` hangs when we remove the `try`/`catch`
block and run with `protoc-bridge` 0.9.7 and `protobuf` > v25.5, even
with later Scala versions.
---
Since `scalapb.ScalaPbCodeGenerator` from `compilerplugin` 0.11.17
implements `CodeGenApp`, `scripts.ScalaPbCodeGenerator` for Scala 2.12
should be unnecessary. The only reason we need it is to maintain the
same interface as the Scala 2.11 implementation.
---
Before the `MavenCoordinates` changes, `scripts/create_repository.py`
would ScalaPB flip ScalaPB artifacts between their `_2.13` and `_3`
versions on subsequent runs (e.g., `protoc-bridge_2.13` vs.
`protoc-bridge_3`). See the comments added within
`MavenCoordinates.new()` for details.
---
Building Scala 3.3 and later versions with `compilerplugin_3` and
`protoc-bridge_3` artifacts (instead of `protoc-bridge_2.13` artifacts)
produces the following build failure:
```txt
$ RULES_SCALA_TEST_ONLY="test_scala_version 3.6.2" \
./test_thirdparty_version.sh
[ ...snip... ]
ERROR: third_party/test/proto/BUILD.bazel:4:14:
ProtoScalaPBRule
third_party/test/proto/proto_jvm_extra_protobuf_generator_scalapb.srcjar
failed: (Exit 1): scalapb_worker failed: error executing command
(from target //third_party/test/proto:proto)
bazel-out/darwin_arm64-opt-exec-2B5CBBC6/bin/src/scala/scripts/scalapb_worker
... (remaining 2 arguments skipped)
--scala_out: java.lang.NoClassDefFoundError:
Could not initialize class scalapb.ScalaPbCodeGenerator$
at scripts.ScalaPbCodeGenerator$.process(ScalaPbCodeGeneratorWrapper.scala:8)
at protocgen.CodeGenApp.run(CodeGenApp.scala:48)
at protocgen.CodeGenApp.run$(CodeGenApp.scala:41)
at scripts.ScalaPbCodeGenerator$.run(ScalaPbCodeGeneratorWrapper.scala:5)
at protocgen.CodeGenApp.run(CodeGenApp.scala:33)
at protocgen.CodeGenApp.run$(CodeGenApp.scala:32)
at scripts.ScalaPbCodeGenerator$.run(ScalaPbCodeGeneratorWrapper.scala:5)
at protocbridge.frontend.PluginFrontend$.runWithBytes(PluginFrontend.scala:48)
at protocbridge.frontend.PluginFrontend$.runWithInputStream(PluginFrontend.scala:113)
at protocbridge.frontend.SocketBasedPluginFrontend.prepare$$anonfun$1$$anonfun$1(SocketBasedPluginFrontend.scala:31)
at protocbridge.frontend.SocketBasedPluginFrontend.prepare$$anonfun$1$$anonfun$adapted$1(SocketBasedPluginFrontend.scala:37)
at scala.concurrent.impl.ExecutionContextImpl$DefaultThreadFactory$$anon$1$$anon$2.block(ExecutionContextImpl.scala:60)
at java.base/java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3118)
at scala.concurrent.impl.ExecutionContextImpl$DefaultThreadFactory$$anon$1.blockOn(ExecutionContextImpl.scala:71)
at scala.concurrent.package$.blocking(package.scala:124)
at protocbridge.frontend.SocketBasedPluginFrontend.prepare$$anonfun$1(SocketBasedPluginFrontend.scala:37)
at protocbridge.frontend.SocketBasedPluginFrontend.prepare$$anonfun$adapted$1(SocketBasedPluginFrontend.scala:38)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:687)
at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:467)
at java.base/java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1426)
at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)
java.lang.RuntimeException: Exit with code 1
at scala.sys.package$.error(package.scala:27)
at scripts.ScalaPBWorker$.work(ScalaPBWorker.scala:44)
at io.bazel.rulesscala.worker.Worker.persistentWorkerMain(Worker.java:96)
at io.bazel.rulesscala.worker.Worker.workerMain(Worker.java:49)
at scripts.ScalaPBWorker$.main(ScalaPBWorker.scala:39)
at scripts.ScalaPBWorker.main(ScalaPBWorker.scala)
ERROR: third_party/test/proto/BUILD.bazel:4:14 Building source jar
third_party/test/proto/proto_scalapb-src.jar failed: (Exit 1):
scalapb_worker failed: error executing command
(from target //third_party/test/proto:proto)
bazel-out/darwin_arm64-opt-exec-2B5CBBC6/bin/src/scala/scripts/scalapb_worker
... (remaining 2 arguments skipped)
```
mbland
added a commit
to mbland/rules_scala
that referenced
this pull request
Jan 27, 2025
This required updates to `scripts/create_repository.py` to accommodate
comparisons between `_2.13` and `_3` versions of ScalaPB artifacts. The
`_3` versions of the most recent ScalaPB artifacts are only available
for Scala 3.3 and later, and `compilerplugin_3` currently still depends
on `protoc-gen_2.13`.
The `MavenCoordinates` dataclass in `scripts/create_repository.py` has
new `unversioned_artifact`, `scala_version`, and `artifact_name`
members. Updates to `MavenCoordinates.is_newer_than()` now compare
objects with matching `artifact_name` values, and take the
`scala_version` into account. These changes then precipitated:
- Replacing the `artifact_name()` call with `artifact_name` accesses.
- Replacing `ArtifactLabelMaker._remove_scala_version_suffix` with
`MavenCoordinates.unversioned_artifact`.
- Extracting `__compare_versions` from `is_newer_than` to reuse the same
code for comparing Scala versions and artifact versions. (Properly
reversed the use of `lhs` and `rhs` in the process.)
This change also removes an unnecessary `try`/`catch` block from
`scripts.ScalaPbCodeGenerator.process()` for Scala 2.12 and above.
The block is still required for the Scala 2.11 implementation of
`scripts.ScalaPbCodeGenerator.run()` and in
`scalarules.test.extra_protobuf_generator.ExtraProtobufGenerator.run()`.
Finally, this change adds notes in a few places to indicate where
support remains for Scala 2.11. This supporting code can ultimately be
removed if we ever decide to drop Scala 2.11 support.
---
Tested by setting the `protobuf` dependencies to a versions I know will
break `ScalaPB` 0.11.17:
- `abseil-cpp`: 20220623.1 => 20240722.0
- `protobuf`: v21.7 => v26.1
I temporarily removed the `try`/`catch` block for `Throwable` from
`scalarules.test.extra_protobuf_generator.ExtraProtobufGenerator.run`.
I then ran `bazel shutdown` to stop persistent `ProtoScalaPBRule`
workers.
After that, building with Bazel 7.4.1 (since Bazel 6.5.0 can't build
`abseil-cpp` 20240722.0 without setting C++14 compiler flags) crashed as
expected, instead of hanging. Undoing the `abseil-cpp` and `protobuf`
changes and building then produced a working build.
```txt
$ USE_BAZEL_VERSION=7.4.1 bazel build //third_party/test/proto:scala
[ ...snip... ]
ERROR: third_party/test/proto/BUILD.bazel:4:14:
ProtoScalaPBRule
third_party/test/proto/proto_jvm_extra_protobuf_generator_scalapb.srcjar
failed: (Exit 1):
scalapb_worker failed: error executing ProtoScalaPBRule command
(from target //third_party/test/proto:proto)
bazel-out/darwin_arm64-opt-exec-ST-a828a81199fe/bin/src/scala/scripts/scalapb_worker
... (remaining 2 arguments skipped)
--jvm_extra_protobuf_generator_out: java.lang.VerifyError: Cannot inherit from final class
at java.base/java.lang.ClassLoader.defineClass1(Native Method)
at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1022)
at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:174)
at java.base/jdk.internal.loader.BuiltinClassLoader.defineClass(BuiltinClassLoader.java:800)
at java.base/jdk.internal.loader.BuiltinClassLoader$4.run(BuiltinClassLoader.java:711)
at java.base/jdk.internal.loader.BuiltinClassLoader$4.run(BuiltinClassLoader.java:706)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at java.base/jdk.internal.loader.BuiltinClassLoader.findClassOnClassPathOrNull(BuiltinClassLoader.java:719)
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:621)
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:579)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:527)
at scalapb.options.Scalapb.<clinit>(Scalapb.java:24835)
at scalapb.options.compiler.Scalapb$.registerAllExtensions(Scalapb.scala:8)
at scalarules.test.extra_protobuf_generator.ExtraProtobufGenerator$.run(ExtraProtobufGenerator.scala:52)
at protocbridge.frontend.PluginFrontend$.runWithBytes(PluginFrontend.scala:48)
at protocbridge.frontend.PluginFrontend$.runWithInputStream(PluginFrontend.scala:113)
at protocbridge.frontend.SocketBasedPluginFrontend.$anonfun$prepare$2(SocketBasedPluginFrontend.scala:31)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at scala.concurrent.impl.ExecutionContextImpl$DefaultThreadFactory$$anon$1$$anon$2.block(ExecutionContextImpl.scala:75)
at java.base/java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3118)
at scala.concurrent.impl.ExecutionContextImpl$DefaultThreadFactory$$anon$1.blockOn(ExecutionContextImpl.scala:87)
at scala.concurrent.package$.blocking(package.scala:146)
at protocbridge.frontend.SocketBasedPluginFrontend.$anonfun$prepare$1(SocketBasedPluginFrontend.scala:23)
at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:659)
at scala.util.Success.$anonfun$map$1(Try.scala:255)
at scala.util.Success.map(Try.scala:213)
at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:42)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:74)
at java.base/java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1426)
at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)
java.lang.RuntimeException: Exit with code 1
at scala.sys.package$.error(package.scala:30)
at scripts.ScalaPBWorker$.work(ScalaPBWorker.scala:44)
at io.bazel.rulesscala.worker.Worker.persistentWorkerMain(Worker.java:96)
at io.bazel.rulesscala.worker.Worker.workerMain(Worker.java:49)
at scripts.ScalaPBWorker$.main(ScalaPBWorker.scala:39)
at scripts.ScalaPBWorker.main(ScalaPBWorker.scala)
Target //third_party/test/proto:scala failed to build
ERROR: third_party/test/proto/BUILD.bazel:4:14
scala @//third_party/test/proto:proto failed:
(Exit 1): scalapb_worker failed:
error executing ProtoScalaPBRule command
(from target //third_party/test/proto:proto)
bazel-out/darwin_arm64-opt-exec-ST-a828a81199fe/bin/src/scala/scripts/scalapb_worker
... (remaining 2 arguments skipped)
```
---
The initial motivation for this change was to try to eliminate the need
for the custom `scripts.ScalaPbCodeGenerator` implementation by
upgrading to `protoc-bridge` 0.9.8. This version contains a change to
`catch` any `Throwable` objects raised by a generator implementation in
`protocbridge.frontend.PluginFrontend.runWithInputStream()`.
- https://github.com/scalapb/protoc-bridge/blob/c574d50eaee5b800fd54493fe25c4e0eed3b9def/bridge/src/main/scala/protocbridge/frontend/PluginFrontend.scala#L107-L122
This change originally came from:
- https://github.com/scalapb/protoc-bridge/blob/d0d56f635d13f7efaa2755ed0d2d66bdef18b588/bridge/src/main/scala/protocbridge/frontend/PluginFrontend.scala#L107-L122
- scalapb/protoc-bridge#367
I closed scalapb/ScalaPB#1771 as a result, and reexamined the
`try`/`catch` blocks added in bazelbuild#1630 and bazelbuild#1637, and the
`scripts.ScalaPBCodeGenerator` implementations added in bazelbuild#1648.
However, the last `protoc-bridge` version to support Scala 2.11 is
0.7.14. Scala 2.11 won't be able to use `protoc-bridge` 0.9.8, so we
still need to `catch` any `Throwable`s ourselves. This includes the
`catch` blocks from the Scala 2.11 `scripts.ScalaPbCodeGenerator` and
`scalarules.test.extra_protobuf_generator.ExtraProtobufGenerator.run`.
---
In `protoc-bridge` 0.9.7 and 0.9.8, `protocbridge.ProtocCodeGenerator`
declares a `run()` method with no implementation (and no `try`/`catch`
block).
- https://github.com/scalapb/protoc-bridge/blob/v0.9.7/bridge/src/main/scala/protocbridge/ProtocCodeGenerator.scala#L5
In `protoc-gen` 0.9.7 and 0.9.8, `protogen.CodeGenApp` implements
`protocbridge.ProtocCodeGenerator`, and `CodeGenApp.run()` wraps a call
to `CodeGenApp.process()` inside a `try`/`catch` block.
- https://github.com/scalapb/protoc-bridge/blob/v0.9.7/protoc-gen/src/main/scala/protocgen/CodeGenApp.scala#L41-L56
`scalapb.ScalaPbCodeGenerator` from `compilerplugin` 0.11.17 extends
`CodeGenApp` and implements `process()`. For this version of
`ScalaPbCodeGenerator`, available in Scala 2.12 and later, there's no
need for a `try`/`catch` wrapper on our end.
- https://github.com/scalapb/ScalaPB/blob/v0.11.17/compiler-plugin/src/main/scala/scalapb/ScalaPbCodeGenerator.scala#L11
However, the last available `compilerplugin` version for Scala 2.11 is
0.9.8. `scalapb.ScalaPbCodeGenerator` from that version implements
`protocbridge.ProtocCodeGenerator` and has a `try`/`catch` in its
`run()` method. However, the `Scalapb.registerAllExtensions(registry)`
call on line 14 lies outside this block, producing the crash described
in bazelbuild#1648 (commit 23ae356). This is why
we need the Scala 2.11 implementation of `scripts.ScalaPbCodeGenerator`.
- https://github.com/scalapb/ScalaPB/blob/v0.9.8/compiler-plugin/src/main/scala/scalapb/ScalaPbCodeGenerator.scala#L14
`scalarules.test.extra_protobuf_generator.ExtraProtobufGenerator`,
also extends `ProtocCodeGenerator` directly, instead of implementing
`CodeGenApp` (so it also builds with Scala 2.11). This is why
`ExtraProtobufGenerator.run()` hangs when we remove the `try`/`catch`
block and run with `protoc-bridge` 0.9.7 and `protobuf` > v25.5, even
with later Scala versions.
---
Since `scalapb.ScalaPbCodeGenerator` from `compilerplugin` 0.11.17
implements `CodeGenApp`, `scripts.ScalaPbCodeGenerator` for Scala 2.12
should be unnecessary. The only reason we need it is to maintain the
same interface as the Scala 2.11 implementation.
---
Before the `MavenCoordinates` changes, `scripts/create_repository.py`
would ScalaPB flip ScalaPB artifacts between their `_2.13` and `_3`
versions on subsequent runs (e.g., `protoc-bridge_2.13` vs.
`protoc-bridge_3`). See the comments added within
`MavenCoordinates.new()` for details.
---
Building Scala 3.3 and later versions with `compilerplugin_3` and
`protoc-bridge_3` artifacts (instead of `protoc-bridge_2.13` artifacts)
produces the following build failure:
```txt
$ RULES_SCALA_TEST_ONLY="test_scala_version 3.6.2" \
./test_thirdparty_version.sh
[ ...snip... ]
ERROR: third_party/test/proto/BUILD.bazel:4:14:
ProtoScalaPBRule
third_party/test/proto/proto_jvm_extra_protobuf_generator_scalapb.srcjar
failed: (Exit 1): scalapb_worker failed: error executing command
(from target //third_party/test/proto:proto)
bazel-out/darwin_arm64-opt-exec-2B5CBBC6/bin/src/scala/scripts/scalapb_worker
... (remaining 2 arguments skipped)
--scala_out: java.lang.NoClassDefFoundError:
Could not initialize class scalapb.ScalaPbCodeGenerator$
at scripts.ScalaPbCodeGenerator$.process(ScalaPbCodeGeneratorWrapper.scala:8)
at protocgen.CodeGenApp.run(CodeGenApp.scala:48)
at protocgen.CodeGenApp.run$(CodeGenApp.scala:41)
at scripts.ScalaPbCodeGenerator$.run(ScalaPbCodeGeneratorWrapper.scala:5)
at protocgen.CodeGenApp.run(CodeGenApp.scala:33)
at protocgen.CodeGenApp.run$(CodeGenApp.scala:32)
at scripts.ScalaPbCodeGenerator$.run(ScalaPbCodeGeneratorWrapper.scala:5)
at protocbridge.frontend.PluginFrontend$.runWithBytes(PluginFrontend.scala:48)
at protocbridge.frontend.PluginFrontend$.runWithInputStream(PluginFrontend.scala:113)
at protocbridge.frontend.SocketBasedPluginFrontend.prepare$$anonfun$1$$anonfun$1(SocketBasedPluginFrontend.scala:31)
at protocbridge.frontend.SocketBasedPluginFrontend.prepare$$anonfun$1$$anonfun$adapted$1(SocketBasedPluginFrontend.scala:37)
at scala.concurrent.impl.ExecutionContextImpl$DefaultThreadFactory$$anon$1$$anon$2.block(ExecutionContextImpl.scala:60)
at java.base/java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3118)
at scala.concurrent.impl.ExecutionContextImpl$DefaultThreadFactory$$anon$1.blockOn(ExecutionContextImpl.scala:71)
at scala.concurrent.package$.blocking(package.scala:124)
at protocbridge.frontend.SocketBasedPluginFrontend.prepare$$anonfun$1(SocketBasedPluginFrontend.scala:37)
at protocbridge.frontend.SocketBasedPluginFrontend.prepare$$anonfun$adapted$1(SocketBasedPluginFrontend.scala:38)
at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:687)
at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:467)
at java.base/java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1426)
at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)
java.lang.RuntimeException: Exit with code 1
at scala.sys.package$.error(package.scala:27)
at scripts.ScalaPBWorker$.work(ScalaPBWorker.scala:44)
at io.bazel.rulesscala.worker.Worker.persistentWorkerMain(Worker.java:96)
at io.bazel.rulesscala.worker.Worker.workerMain(Worker.java:49)
at scripts.ScalaPBWorker$.main(ScalaPBWorker.scala:39)
at scripts.ScalaPBWorker.main(ScalaPBWorker.scala)
ERROR: third_party/test/proto/BUILD.bazel:4:14 Building source jar
third_party/test/proto/proto_scalapb-src.jar failed: (Exit 1):
scalapb_worker failed: error executing command
(from target //third_party/test/proto:proto)
bazel-out/darwin_arm64-opt-exec-2B5CBBC6/bin/src/scala/scripts/scalapb_worker
... (remaining 2 arguments skipped)
```
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Adds ScalaPB 0.11.17, protoc-bridge 0.9.8, gRPC 1.68.1, and Guava 33.3.1-jre as root artifacts in
scripts/create_repository.pyandProtobufAdapters for cross Scala version compatibility. AddsScalaPBCodeGeneratorwrappers to ensure all ScalaPB workers catch all exceptions to avoid build hangs when updating the Protobuf library.This change does not bump the Protobuf library version, which can happen in another PR and is up for discussion in #1647. As always, I'm happy to break this apart, at least into one "version bump and compatibility updates" PR and one "wrap
ScalaPBCodeGenerator" PR, if so desired.There are lots of details in the individual commit messages.
Motivation
These changes are spawned from #1482 and #1647, whereby I investigated what it would take to update the Protobuf version to accommodate future Bzlmod users that depend on newer Protobuf versions.
These changes were essential to the Protobuf experiment, yet aren't necessarily essential to the Bzlmod work, but they're generally beneficial to include for their own sake. All builds and tests are still compatible with all supported Scala versions, on Bazel 6 and Bazel 7.
This is also a good demonstration of how to add new root artifact dependencies to
scripts/create_repository.py, and how easy it is to run the script and get correct updates across all supported Scala versions.Regarding the
ScalaPBCodeGeneratorwrappers, I encountered more hanging builds in my bzlmod-update-abseil-cpp-and-protobuf working branch that the changes from #1637 didn't catch. I backported Wrap ScalaPbCodeGenerator, expose Scala 2.11 error from that branch into this one.See that commit's message and #1647 for the gory details on why the ScalaPB libs are so sensitive to the Protobuf version and why we have to use wrappers to avoid hanging the build.