Skip to content

Commit

Permalink
Stick with protoc-{bridge,gen}_2.13 under Scala 3
Browse files Browse the repository at this point in the history
As noted in the comment added to `select_root_artifacts()`,
`com.thesamet.scalapb:compilerplugin_3:0.11.17` depends on
`com.thesamet.scalapb:protoc-gen_2.13:0.9.7`. When using
`com.thesamet.scalapb:protoc-gen_3:0.9.8` instead, we get a crash:

```txt
$ bazel build --repo_env=SCALA_VERSION=3.6.2 //third_party/test/proto:scala

[ ...snip... ]
ERROR: third_party/test/proto/BUILD.bazel:4:14:
  ProtoScalaPBRule
  third_party/test/proto/proto_jvm_extra_protobuf_generator_scalapb.srcjar
  failed: (Exit 1): scalapb_worker failed:
  error executing command (from target //third_party/test/proto:proto)
  bazel-out/darwin_arm64-opt-exec-2B5CBBC6/bin/src/scala/scripts/scalapb_worker
    ...

--scala_out: java.lang.NoClassDefFoundError:
  Could not initialize class scalapb.ScalaPbCodeGenerator$
    at scripts.ScalaPbCodeGenerator$.process(ScalaPbCodeGeneratorWrapper.scala:7)
    at protocgen.CodeGenApp.run(CodeGenApp.scala:48)
    at protocgen.CodeGenApp.run$(CodeGenApp.scala:23)
    at scripts.ScalaPbCodeGenerator$.run(ScalaPbCodeGeneratorWrapper.scala:5)
    at protocgen.CodeGenApp.run(CodeGenApp.scala:33)
    at protocgen.CodeGenApp.run$(CodeGenApp.scala:23)
    at scripts.ScalaPbCodeGenerator$.run(ScalaPbCodeGeneratorWrapper.scala:5)
    at protocbridge.frontend.PluginFrontend$.runWithBytes(PluginFrontend.scala:48)
    at protocbridge.frontend.PluginFrontend$.runWithInputStream(PluginFrontend.scala:113)
    at protocbridge.frontend.SocketBasedPluginFrontend.prepare$$anonfun$1$$anonfun$1(SocketBasedPluginFrontend.scala:31)
    at protocbridge.frontend.SocketBasedPluginFrontend.prepare$$anonfun$1$$anonfun$adapted$1(SocketBasedPluginFrontend.scala:37)
    at scala.concurrent.impl.ExecutionContextImpl$DefaultThreadFactory$$anon$1$$anon$2.block(ExecutionContextImpl.scala:60)
    at java.base/java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3118)
    at scala.concurrent.impl.ExecutionContextImpl$DefaultThreadFactory$$anon$1.blockOn(ExecutionContextImpl.scala:71)
    at scala.concurrent.package$.blocking(package.scala:124)
    at protocbridge.frontend.SocketBasedPluginFrontend.prepare$$anonfun$1(SocketBasedPluginFrontend.scala:37)
    at protocbridge.frontend.SocketBasedPluginFrontend.prepare$$anonfun$adapted$1(SocketBasedPluginFrontend.scala:38)
    at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:687)
    at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:467)
    at java.base/java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1426)
    at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
    at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
    at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
    at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
    at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)

java.lang.RuntimeException: Exit with code 1
    at scala.sys.package$.error(package.scala:27)
    at scripts.ScalaPBWorker$.work(ScalaPBWorker.scala:44)
    at io.bazel.rulesscala.worker.Worker.persistentWorkerMain(Worker.java:96)
    at io.bazel.rulesscala.worker.Worker.workerMain(Worker.java:49)
    at scripts.ScalaPBWorker$.main(ScalaPBWorker.scala:39)
    at scripts.ScalaPBWorker.main(ScalaPBWorker.scala)

Target //third_party/test/proto:scala failed to build

ERROR: test/proto/BUILD.bazel:4:14
  scala @//third_party/test/proto:proto failed:
  (Exit 1): scalapb_worker failed:
  error executing command (from target //third_party/test/proto:proto)
```

---

As I was going back and testing some more, I couldn't reproduce the
`ProtoScalaPBRule` hanging by setting `protoc-bridge` to 0.9.7 and
`protobuf` to v26.1. I was using this command (using Bazel 7.4.1 to
avoid setting C++ compiler flags for Bazel 6.5.0):

```txt
USE_BAZEL_VERSION=7.4.1 bazel build //third_party/test/proto:scala
```

The command kept crashing every time, instead of hanging. I eventually
realized this was because of two things:

- `scalarules.test.extra_protobuf_generator.ExtraProtobufGenerator.run`
  was still catching `Throwable`, since Scala 2.11 support ends with
  `protoc-bridge` 0.7.14.

- Bazel kept persistent `io.bazel.rulesscala.scalac.ScalacWorker`
  workers running with `ExtraProtobufGenerator` code that still had the
  `try`/`catch` block after I tried removing it.

After temporarily editing out the `try`/`catch` block from
`ExtraProtobufGenerator.run` and running `bazel shutdown`, I could
reproduce the hang. I could then confirm `protoc-bridge` 0.9.8 fixed it.

---

Also, TIL that `scripts.ScalaPbCodeGenerator.process` (from
`src/scala/scripts/ScalaPbCodeGeneratorWrapper`) never needed a
`try`/`catch` block. When bumping to
`com.thesamet.scalapb:compilerplugin_2.12:0.11.17` in bazelbuild#1648
(commit 23ae356),
`scalapb.ScalaPbCodeGenerator` went from implementing
`protocbridge.ProtocCodeGenerator` to implementing
`protocgen.CodeGenApp`:

- https://github.com/scalapb/ScalaPB/blob/v0.9.7/compiler-plugin/src/main/scala/scalapb/ScalaPbCodeGenerator.scala#L9
- https://github.com/scalapb/ScalaPB/blob/v0.11.17/compiler-plugin/src/main/scala/scalapb/ScalaPbCodeGenerator.scala#L11

That commit introduced `com.thesamet.scalapb:protoc-gen_2.12:0.9.7`, and
`protocgen.CodeGenApp.run()` already had a `try`/`catch` block for
`Throwable`:

- https://github.com/scalapb/protoc-bridge/blob/v0.9.7/protoc-gen/src/main/scala/protocgen/CodeGenApp.scala#L49-L56

However, `protocbridge.ProtocCodeGenerator`, which
`protocgen.CodeGenApp` extends, only declares a `run()` method with no
implementation (and no `try`/`catch` block):

- https://github.com/scalapb/protoc-bridge/blob/v0.9.7/bridge/src/main/scala/protocbridge/ProtocCodeGenerator.scala#L5

The aforementioned `ExtraProtobufGenerator` only extends
`protocbridge.ProtocCodeGenerator`, not `protocgen.CodeGenApp`. That's
why it will still hang when we remove the `try`/`catch` block from its
`run()` method and run with `protoc-bridge` 0.9.7 and `protobuf` >
v25.5.
  • Loading branch information
mbland committed Dec 19, 2024
1 parent 6f32988 commit 1880f48
Show file tree
Hide file tree
Showing 5 changed files with 35 additions and 27 deletions.
14 changes: 11 additions & 3 deletions scripts/create_repository.py
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,15 @@ def select_root_artifacts(scala_version, scala_major, is_scala_3) -> List[str]:
scala_2_version = max_scala_2_version
scala_2_major = max_scala_2_major
scala_major = '3'
scalapb_major = max_scala_2_major if minor_version < 3 else scala_major
scalapb_major = scala_2_major if minor_version < 3 else scala_major

# For some reason, com.thesamet.scalapb:compilerplugin_3:0.11.17 depends on
# com.thesamet.scalapb:protoc-gen_2.13:0.9.7. Trying to use
# com.thesamet.scalapb:protoc-gen_3:0.9.8 causes a crash, saying:
# `java.lang.NoClassDefFoundError: Could not initialize class
# scalapb.ScalaPbCodeGenerator$`, even though that class is definitely in
# that jar. So we stick with protoc-gen_2.13 for now.
protoc_bridge_major = scala_2_major

scalafmt_version = SCALAFMT_VERSION
scalapb_version = SCALAPB_VERSION
Expand All @@ -103,7 +111,7 @@ def select_root_artifacts(scala_version, scala_major, is_scala_3) -> List[str]:
f'com.google.protobuf:protobuf-java:{PROTOBUF_JAVA_VERSION}',
f'com.thesamet.scalapb:compilerplugin_{scalapb_major}:' +
scalapb_version,
f'com.thesamet.scalapb:protoc-bridge_{scalapb_major}:' +
f'com.thesamet.scalapb:protoc-bridge_{protoc_bridge_major}:' +
protoc_bridge_version,
f'com.thesamet.scalapb:scalapb-runtime_{scalapb_major}:' +
scalapb_version,
Expand All @@ -123,7 +131,7 @@ def select_root_artifacts(scala_version, scala_major, is_scala_3) -> List[str]:

if scala_major != '2.11':
root_artifacts.append(
f'com.thesamet.scalapb:protoc-gen_{scalapb_major}:' +
f'com.thesamet.scalapb:protoc-gen_{protoc_bridge_major}:' +
protoc_bridge_version,
)

Expand Down
12 changes: 6 additions & 6 deletions third_party/repositories/scala_3_3.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -880,18 +880,18 @@ artifacts = {
],
},
"scala_proto_rules_scalapb_protoc_bridge": {
"artifact": "com.thesamet.scalapb:protoc-bridge_3:0.9.8",
"sha256": "6e1e38e34f3aaa14c6d46defb66b819f03edbdc4d69965011955da2a4781df9c",
"artifact": "com.thesamet.scalapb:protoc-bridge_2.13:0.9.8",
"sha256": "0b3827da2cd9bca867d6963c2a821e7eaff41f5ac3babf671c4c00408bd14a9b",
"deps": [
"@dev_dirs_directories",
"@io_bazel_rules_scala_scala_library",
"@io_bazel_rules_scala_scala_library_2",
],
},
"scala_proto_rules_scalapb_protoc_gen": {
"artifact": "com.thesamet.scalapb:protoc-gen_3:0.9.8",
"sha256": "9e5eebe35ca68a884adf3c6cade094055192d3243d29a26d32ae6b5396b39e08",
"artifact": "com.thesamet.scalapb:protoc-gen_2.13:0.9.8",
"sha256": "cf2b50721952cb4f10ca05a0ed36d7b01b88eb6505a9478556ee5a7af1a21775",
"deps": [
"@io_bazel_rules_scala_scala_library",
"@io_bazel_rules_scala_scala_library_2",
"@scala_proto_rules_scalapb_protoc_bridge",
],
},
Expand Down
12 changes: 6 additions & 6 deletions third_party/repositories/scala_3_4.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -880,18 +880,18 @@ artifacts = {
],
},
"scala_proto_rules_scalapb_protoc_bridge": {
"artifact": "com.thesamet.scalapb:protoc-bridge_3:0.9.8",
"sha256": "6e1e38e34f3aaa14c6d46defb66b819f03edbdc4d69965011955da2a4781df9c",
"artifact": "com.thesamet.scalapb:protoc-bridge_2.13:0.9.8",
"sha256": "0b3827da2cd9bca867d6963c2a821e7eaff41f5ac3babf671c4c00408bd14a9b",
"deps": [
"@dev_dirs_directories",
"@io_bazel_rules_scala_scala_library",
"@io_bazel_rules_scala_scala_library_2",
],
},
"scala_proto_rules_scalapb_protoc_gen": {
"artifact": "com.thesamet.scalapb:protoc-gen_3:0.9.8",
"sha256": "9e5eebe35ca68a884adf3c6cade094055192d3243d29a26d32ae6b5396b39e08",
"artifact": "com.thesamet.scalapb:protoc-gen_2.13:0.9.8",
"sha256": "cf2b50721952cb4f10ca05a0ed36d7b01b88eb6505a9478556ee5a7af1a21775",
"deps": [
"@io_bazel_rules_scala_scala_library",
"@io_bazel_rules_scala_scala_library_2",
"@scala_proto_rules_scalapb_protoc_bridge",
],
},
Expand Down
12 changes: 6 additions & 6 deletions third_party/repositories/scala_3_5.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -880,18 +880,18 @@ artifacts = {
],
},
"scala_proto_rules_scalapb_protoc_bridge": {
"artifact": "com.thesamet.scalapb:protoc-bridge_3:0.9.8",
"sha256": "6e1e38e34f3aaa14c6d46defb66b819f03edbdc4d69965011955da2a4781df9c",
"artifact": "com.thesamet.scalapb:protoc-bridge_2.13:0.9.8",
"sha256": "0b3827da2cd9bca867d6963c2a821e7eaff41f5ac3babf671c4c00408bd14a9b",
"deps": [
"@dev_dirs_directories",
"@io_bazel_rules_scala_scala_library",
"@io_bazel_rules_scala_scala_library_2",
],
},
"scala_proto_rules_scalapb_protoc_gen": {
"artifact": "com.thesamet.scalapb:protoc-gen_3:0.9.8",
"sha256": "9e5eebe35ca68a884adf3c6cade094055192d3243d29a26d32ae6b5396b39e08",
"artifact": "com.thesamet.scalapb:protoc-gen_2.13:0.9.8",
"sha256": "cf2b50721952cb4f10ca05a0ed36d7b01b88eb6505a9478556ee5a7af1a21775",
"deps": [
"@io_bazel_rules_scala_scala_library",
"@io_bazel_rules_scala_scala_library_2",
"@scala_proto_rules_scalapb_protoc_bridge",
],
},
Expand Down
12 changes: 6 additions & 6 deletions third_party/repositories/scala_3_6.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -889,18 +889,18 @@ artifacts = {
],
},
"scala_proto_rules_scalapb_protoc_bridge": {
"artifact": "com.thesamet.scalapb:protoc-bridge_3:0.9.8",
"sha256": "6e1e38e34f3aaa14c6d46defb66b819f03edbdc4d69965011955da2a4781df9c",
"artifact": "com.thesamet.scalapb:protoc-bridge_2.13:0.9.8",
"sha256": "0b3827da2cd9bca867d6963c2a821e7eaff41f5ac3babf671c4c00408bd14a9b",
"deps": [
"@dev_dirs_directories",
"@io_bazel_rules_scala_scala_library",
"@io_bazel_rules_scala_scala_library_2",
],
},
"scala_proto_rules_scalapb_protoc_gen": {
"artifact": "com.thesamet.scalapb:protoc-gen_3:0.9.8",
"sha256": "9e5eebe35ca68a884adf3c6cade094055192d3243d29a26d32ae6b5396b39e08",
"artifact": "com.thesamet.scalapb:protoc-gen_2.13:0.9.8",
"sha256": "cf2b50721952cb4f10ca05a0ed36d7b01b88eb6505a9478556ee5a7af1a21775",
"deps": [
"@io_bazel_rules_scala_scala_library",
"@io_bazel_rules_scala_scala_library_2",
"@scala_proto_rules_scalapb_protoc_bridge",
],
},
Expand Down

0 comments on commit 1880f48

Please sign in to comment.