Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade Spark to 3.2.0 and Scala to 2.13.7 #333

Conversation

vincenzobaz
Copy link

No description provided.

@lbulej
Copy link
Member

lbulej commented Nov 22, 2021

Thank you for the contribution! Similar PR (#327) was already in the making and just went in, so I'm closing this. However, seeing your affiliation, I have to ask: would you be perhaps interested in cleaning up some of the dependencies to make them compatible with Scala 2.13?

@lbulej lbulej closed this Nov 22, 2021
@vincenzobaz
Copy link
Author

Hello @lbulej, my bad I did not check the other opened PRs.
I wanted to see if performances were improving with this upgrade I submitted a PR without too much thinking.
I was also looking into compatibility of (Spark 3.2.0, Scala 2.13, JDK 17). What is your experience with this triplet?

What do you mean by

some of the dependencies to make them compatible with Scala 2.13

dependencies of this benchmark or of other modules?

@lbulej
Copy link
Member

lbulej commented Nov 23, 2021

Hello @lbulej, my bad I did not check the other opened PRs. I wanted to see if performances were improving with this upgrade I submitted a PR without too much thinking. I was also looking into compatibility of (Spark 3.2.0, Scala 2.13, JDK 17). What is your experience with this triplet?

Not much yet -- we typically start systematically measuring it just before release. However, we are trying to move from older Scala versions and Spark 3.2 was a chance to move a bunch of benchmarks to Scala 2.13.

What do you mean by

some of the dependencies to make them compatible with Scala 2.13

dependencies of this benchmark or of other modules?

Dependencies of other benchmarks (see #324). The reactors benchmark uses the Reactors framework, which I already bumped to 2.12, but it needs pushing to 2.13 (https://github.com/renaissance-benchmarks/dependency-reactors). The scala-stm benchmark relies on Scala STM library, which also needs a push towards 2.13, but we don't have a forked repository for it (yet).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants