-
Notifications
You must be signed in to change notification settings - Fork 201
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[WIP] Update mantis 3rd party dependency specifications #349
base: master
Are you sure you want to change the base?
Conversation
chore: update 3rd party dependency specifications for mantis-common and corresponding changes to root build.gradle (issue Netflix#345)
mantis-common/build.gradle
Outdated
api "io.netty:netty-codec-http:$nettyVersion" | ||
api "io.netty:netty-buffer:$nettyVersion" | ||
api libraries.nettyCodec | ||
api libraries.nettyBuffer | ||
api group: 'io.netty', name: 'netty-transport-native-epoll', classifier: 'linux-x86_64', version: nettyVersion |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
would recommend moving this to libraries.nettyEpoll as well even if it's a little tricky.
It would look like this - io.netty:netty-transport-native-epoll:4.1.17.Final:linux-x86_64
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Got it, will do ASAP! Thank you.
mantis-common/build.gradle
Outdated
@@ -24,11 +24,11 @@ ext { | |||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
the ext
variable can be safely removed with this change.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This was a concern I had earlier, there were different versions of the same dependencies across different modules (so I kept this part that specifies the specific versions). Would it be better to standardize the versions and move a single version to the root build.gradle
's ext.versions
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yep, that'd be desired to realize the full benefits of this work.
The different versions would make it tricky and here's a good starting tip:
- For major version difference, Create a new library variable.
- For minor and patch, try to bump to the latest version for all subprojects (one variable per library and see if builds correctly).
You could call out the places where you are bumping the minor and patch version in the PR for an easier review.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Awesome, thank you! As some other GSoC applicants are having trouble with builds failing (including the Mantis tutorial) before we made any changes, I brought this up to Sundaram yesterday and iirc he said that right now there wasn't a good way for us to validate whether these "good first issue" PRs aren't going to break something - but that it will be fixed by end of March. If needed, I can provide the exact error on Discord so maybe we can get ./gradlew build
to work?
The only thing that succeeds at building correctly locally is the synthetic-sourcejob
example, since I found a workaround that can allow it to build temporarily. I have been using that to test but it's not ideal.
chore: update netty-transport-native-epoll dependency specification to libraries.nettyEpoll in mantis-common and corresponding changes to root build.gradle (issue Netflix#345)
refactor mantis-common build.gradle by moving ext versions to root build.gradle and removing ext variable (issue Netflix#345)
first pass at refactor 3rd party dependency specifications for mantis-connector-iceberg and corresponding changes to root build.gradle (issue Netflix#345)
first pass at refactor 3rd party dependency specifications for mantis-connector-job and corresponding changes to root build.gradle (issue Netflix#345)
first pass at refactor 3rd party dependency specifications for mantis-connector-kafka and corresponding changes to root build.gradle (issue Netflix#345)
first pass at refactor 3rd party dependency specifications for mantis-publish-netty-guice and corresponding changes to root build.gradle. (issue Netflix#345)
first pass at refactor 3rd party dependency specifications for mantis-remote-observable and corresponding changes to root build.gradle. (issue Netflix#345)
first pass at refactor 3rd party dependency specifications for mantis-server-worker and corresponding changes to root build.gradle. tbd for lines 31,32 - exclude (issue Netflix#345)
first pass at refactor 3rd party dependency specifications for mantis-server-worker-client and corresponding changes to root build.gradle. (issue Netflix#345)
first pass at refactor 3rd party dependency specifications for mantis-shaded and corresponding changes to root build.gradle. tbd - issue Netflix#194 and grouping (issue Netflix#345)
first pass at refactor 3rd party dependency specifications for mantis-source-job-kafka. (issue Netflix#345)
…ations first pass at refactor 3rd party dependency specifications for mantis-examples-groupby-sample. (issue Netflix#345)
…cifications first pass at refactor 3rd party dependency specifications for mantis-examples-jobconnector-sample and corresponding changes to root build.gradle (issue Netflix#345)
…pecifications first pass at refactor 3rd party dependency specifications for mantis-examples-mantis-publish-sample and corresponding changes to root build.gradle (issue Netflix#345)
…cy specifications first pass at refactor 3rd party dependency specifications for mantis-examples-mantis-publish-web-sample and corresponding changes to root build.gradle (issue Netflix#345)
…tions first pass at refactor 3rd party dependency specifications for mantis-examples-sine-function and corresponding changes to root build.gradle (issue Netflix#345)
…cifications first pass at refactor 3rd party dependency specifications for mantis-examples-synthetic-sourcejob and corresponding changes to root build.gradle (issue Netflix#345)
…ations first pass at refactor 3rd party dependency specifications for mantis-examples-twitter-sample and corresponding changes to root build.gradle (issue Netflix#345)
first pass at refactor 3rd party dependency specifications for mantis-examples-wordcount and corresponding changes to root build.gradle (issue Netflix#345)
Update: Finished the bulk of the refactoring, still going back over to make sure I didn't miss anything, also I had a few questions/tbd changes which I will leave comments about here soon (before requesting review again) |
@mabelbot Is this ready for review? |
Context
Changes made by module
build.gradle
ext.versions (only occurrence), nettyVersion = '4.1.34.Final' moved to rootbuild.gradle
ext.versions (bumped, requires updates to other submodules later), snappyVersion : "1.1.+" moved to rootbuild.gradle
ext.versions (only occurrence). All other dependencies made consistent.build.gradle
ext.versions (only occurrence), avroVersion : "1.10.2" moved to rootbuild.gradle
ext.versions (only occurrence), parquetVersion : "1.12.0" moved to rootbuild.gradle
ext.versions (only occurrence).build.gradle
ext.versions (only occurrence)build.gradle
'0.134.0' to match other modules using v0. Was using archaiusVersion = '2.3.+' whereas all other submodules are using latest.release so it has been bumped to use the latest.version specified in rootbuild.gradle
. kafkaVersion = '2.2.+' moved to rootbuild.gradle
ext.versions (only occurrence).mantis-control-plane:mantis-control-plane-core:
build.gradle
ext.versions: configMagicVersion = "0.11" is shared by the only 2 submodules that have it specified, so this has been moved to rootbuild.gradle
ext.versions, same with cliParserVersion = '1.1.1' and mesosVersion = '1.7.2'. hdrHistogramVersion = '2.+', jodaTimeVersion = '2.+', jsonVersion = '20180813'build.gradle
but it has been bumped to match the highest version = '2.12.+' (minor version)mantis-control-plane:mantis-control-plane-server:
build.gradle
ext.versions: akkaVersion = '2.6.15', akkaHttpVersion = '10.2.7', fenzoVersion = '0.13.8', rxJavaReactiveStreamsVersion = '1.+', scalaBinaryVersion = '2.12'mantis-examples
configurations.all
.mantisConnectorJob
.spectatorPlugin0
, move make mockneatVersion = '0.4.8' a variable in root build.gradle.javaxServletApi
, bump spectator 0 spectatorVersion = '0.96.0' to 0.134.0 (tbd making 0.134.0 a variable), create variableguiceServlet
hbcCore
mantis-network
build.gradle
ext.versions: mqlVersion = '3.4.+' (requires bumping of minor version for other modules that have this dependency).build.gradle
:mqlVersion
,mqlJvm
,nettyHandler
build.gradle
vars and/or without deletions toext
:mantis-publish:mantis-publish-core
build.gradle
ext.versions/already there from above refactors: archaiusVersion = 'latest.release' (was already present from previous refactor), versionMockito = '2.18.+' - existing rootbuild.gradle
variable bumped from 2.0.+ to 2.18.+, rootbuild.gradle
variable for v0 bumped to spectatorVersion = '0.134.0' (note if mantis-publish-core can be bumped to v1.3+ we can delete some variables, tbd)build.gradle
:build.gradle
vars and/or without deletions toext
:mantis-publish:mantis-publish-netty
build.gradle
ext.versions/already there from above refactors: spectatorVersion = 'latest.release' was found in theext
, thus I changed the rootbuild.gradle
to bump from its current 1.3.+ to latest.release.spectatorExtIpc0
build.gradle
:nettyAll
,spectatorExtIpc
build.gradle
vars and/or without deletions toext
:mantis-publish:mantis-publish-netty-guice
build.gradle
ext.versions/already there from above refactors: archaiusVersion = 'latest.release' (already there), guiceVersion = '4.2.2'build.gradle
0.134.0build.gradle
:archaius2Guice
,guice
,spectatorPlugin
build.gradle
vars and/or without deletions toext
mantis-remote-observable
build.gradle
ext.versions/already there from above refactors: rxJavaMathVersion = '0.20.6'build.gradle
"4.1.34.Final"build.gradle
:rxJavaMath
build.gradle
vars and/or without deletions toext
mantis-server:mantis-server-worker - tbd if need to fix exclude group lines if there is no version
build.gradle
ext.versions/already there from above refactors: mantisRxControlVersion = '1.3.+', mesosVersion = '1.7.2' (already there), httpComponentsVersion = '4.5.6' (bumped to 4.5.9)build.gradle
2.12.+'build.gradle
:httpClient
,sketchesCore
,moa
,hamcrestAll
build.gradle
vars and/or without deletions toext
mantis-server:mantis-server-worker-client
All fixes made with existing root
build.gradle
vars and/or without deletions toext
mantis-shaded
I grouped some but not all jackson dependencies (due to the comments), but tbd if they need to split back out due to Issue.
build.gradle
ext.versions/already there from above refactors: jacksonVersion = '2.12.+', guavaFailureAccessVersion = '1.0.1', guavaVersion = '31.1-jre', curatorVersion (already there), zookeeperVersion = '3.4.+', jlineVersion = '0.9.94'build.gradle
:jackson
(see note above),guava
,failureaccess
,curator
(grouped),zookeeper
,jline
,netty
build.gradle
vars and/or without deletions toext
mantis-source-jobs:mantis-source-job-kafka
build.gradle
ext.versions/already there from above refactors:build.gradle
'3.4.+'build.gradle
:build.gradle
vars and/or without deletions toext
No fixes to the following submodules that I observed seem to be already specified properly (list will update if version changes due to
ext
variables are necessary):Note:
./gradlew build
does not compile correctly (prior to any changes) when compiled locally, however I have checked with../../gradlew execute
onmantis-examples
which is able to build (@sundargates is aware of issue)Checklist
./gradlew build
compiles code correctly./gradlew test
passes all testsbuild.gradle
Refactor Checklist