Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Revert "[SPARK-42809][BUILD] Upgrade scala-maven-plugin from 4.8.0 to 4.8.1" #40482

Closed
wants to merge 1 commit into from

Conversation

LuciferYang
Copy link
Contributor

@LuciferYang LuciferYang commented Mar 20, 2023

What changes were proposed in this pull request?

This pr aims to Revert "[SPARK-42809][BUILD] Upgrade scala-maven-plugin from 4.8.0 to 4.8.1".

Why are the changes needed?

As mentioned in #40442, there are some regression with the 4.8.1:

  • Run ./build/mvn -DskipTests clean package with Java 17 will failed
[INFO] --- maven-compiler-plugin:3.11.0:compile (default-compile) @ spark-core_2.12 ---
[INFO] Not compiling main sources
[INFO]
[INFO] --- scala-maven-plugin:4.8.1:compile (scala-compile-first) @ spark-core_2.12 ---
[INFO] Compiler bridge file: /home/bjorn/.sbt/1.0/zinc/org.scala-sbt/org.scala-sbt-compiler-bridge_2.12-1.8.0-bin_2.12.17__55.0-1.8.0_20221110T195421.jar
[INFO] compiler plugin: BasicArtifact(com.github.ghik,silencer-plugin_2.12.17,1.7.10,null)
[INFO] compiling 597 Scala sources and 103 Java sources to /home/bjorn/github/spark/core/target/scala-2.12/classes ...
[ERROR] [Error] /home/bjorn/github/spark/core/src/main/scala/org/apache/spark/serializer/SerializationDebugger.scala:71: not found: value sun
[ERROR] [Error] /home/bjorn/github/spark/core/src/main/scala/org/apache/spark/storage/StorageUtils.scala:26: not found: object sun
[ERROR] [Error] /home/bjorn/github/spark/core/src/main/scala/org/apache/spark/storage/StorageUtils.scala:27: not found: object sun
[ERROR] [Error] /home/bjorn/github/spark/core/src/main/scala/org/apache/spark/storage/StorageUtils.scala:206: not found: type DirectBuffer
[ERROR] [Error] /home/bjorn/github/spark/core/src/main/scala/org/apache/spark/storage/StorageUtils.scala:210: not found: type Unsafe
[ERROR] [Error] /home/bjorn/github/spark/core/src/main/scala/org/apache/spark/storage/StorageUtils.scala:212: not found: type Unsafe
[ERROR] [Error] /home/bjorn/github/spark/core/src/main/scala/org/apache/spark/storage/StorageUtils.scala:213: not found: type DirectBuffer
[ERROR] [Error] /home/bjorn/github/spark/core/src/main/scala/org/apache/spark/storage/StorageUtils.scala:216: not found: type DirectBuffer
[ERROR] [Error] /home/bjorn/github/spark/core/src/main/scala/org/apache/spark/storage/StorageUtils.scala:236: not found: type DirectBuffer
[ERROR] [Error] /home/bjorn/github/spark/core/src/main/scala/org/apache/spark/util/ClosureCleaner.scala:452: not found: value sun
[ERROR] [Error] /home/bjorn/github/spark/core/src/main/scala/org/apache/spark/util/SignalUtils.scala:26: not found: object sun
[ERROR] [Error] /home/bjorn/github/spark/core/src/main/scala/org/apache/spark/util/SignalUtils.scala:99: not found: type SignalHandler
[ERROR] [Error] /home/bjorn/github/spark/core/src/main/scala/org/apache/spark/util/SignalUtils.scala:99: not found: type Signal
[ERROR] [Error] /home/bjorn/github/spark/core/src/main/scala/org/apache/spark/util/SignalUtils.scala:83: not found: type Signal
[ERROR] [Error] /home/bjorn/github/spark/core/src/main/scala/org/apache/spark/util/SignalUtils.scala:108: not found: type SignalHandler
[ERROR] [Error] /home/bjorn/github/spark/core/src/main/scala/org/apache/spark/util/SignalUtils.scala:108: not found: value Signal
[ERROR] [Error] /home/bjorn/github/spark/core/src/main/scala/org/apache/spark/util/SignalUtils.scala:114: not found: type Signal
[ERROR] [Error] /home/bjorn/github/spark/core/src/main/scala/org/apache/spark/util/SignalUtils.scala:116: not found: value Signal
[ERROR] [Error] /home/bjorn/github/spark/core/src/main/scala/org/apache/spark/util/SignalUtils.scala:128: not found: value Signal
[ERROR] 19 errors found
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for Spark Project Parent POM 3.5.0-SNAPSHOT:
[INFO]
[INFO] Spark Project Parent POM ........................... SUCCESS [ 3.848 s]
[INFO] Spark Project Tags ................................. SUCCESS [ 12.106 s]
[INFO] Spark Project Sketch ............................... SUCCESS [ 10.685 s]
[INFO] Spark Project Local DB ............................. SUCCESS [ 8.743 s]
[INFO] Spark Project Networking ........................... SUCCESS [ 9.362 s]
[INFO] Spark Project Shuffle Streaming Service ............ SUCCESS [ 7.828 s]
[INFO] Spark Project Unsafe ............................... SUCCESS [ 9.071 s]
[INFO] Spark Project Launcher ............................. SUCCESS [ 4.776 s]
[INFO] Spark Project Core ................................. FAILURE [ 17.228 s]
  • Run build/mvn clean install -DskipTests -Pscala-2.13 with Java8 + Scala 2.13

There are compilation errors as ERROR] -release is only supported on Java 9 and higher although it does not cause compilation failures. More, I saw davidB/scala-maven-plugin#686, So it seems that 4.8.1 and Java 8 are not compatible well.

Does this PR introduce any user-facing change?

No

How was this patch tested?

Pass GA

@github-actions github-actions bot added the BUILD label Mar 20, 2023
@LuciferYang
Copy link
Contributor Author

cc @HyukjinKwon @panbingkun @bjornjorgensen FYI

Copy link
Member

@yaooqinn yaooqinn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@yaooqinn yaooqinn closed this in 1530e8d Mar 20, 2023
@yaooqinn
Copy link
Member

Thanks, merged to master

@LuciferYang
Copy link
Contributor Author

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants