Skip to content

Commit

Permalink
scrooge-sbt-plugin, finagle: Handle compiling for multiple languages …
Browse files Browse the repository at this point in the history
…and update Finagle Build.scala to build Java and Scala bindings

Problem:

The Scrooge SBT plugin fails to build the `finagle-thriftmux` and
`finagle-thrift` tests, because they require both Scala and Java bindings, but
Scrooge can only to generate bindings for one language. After fixing this
problem, the "ThriftMux client to Thrift server " test fails with an
OutOfMemoryError.

Solution:

Change the Scrooge language SettingKey from a `SettingKey[String]` to a
`SettingKey[Seq[String]]`, and set `scroogeLanguages` to `Seq("java", "scala")`
in Finagle's `Build.scala` file. Also, increase Finagle's SBT memory limit from
2G to 4G.

Result:

Now the Scrooge SBT plugin can generate bindings for multiple languages.

RB_ID=846198
  • Loading branch information
Stefan Lance authored and jenkins committed Jun 29, 2016
1 parent 094ce54 commit 212ac9a
Show file tree
Hide file tree
Showing 3 changed files with 30 additions and 16 deletions.
10 changes: 9 additions & 1 deletion CHANGES
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,15 @@ as it is included in Scrooge's user's guide.
4.x
-----

- Builds are now only for Java 8 and Scala 2.11. See the
Breaking API Changes
~~~~~~~~~~~~~~~~~~~~

* scrooge-sbt-plugin: Allow scrooge to build bindings for more than one
language. To reflect this, `ScroogeSBT.autoImport.scroogeLanguage` has been
renamed to `scroogeLanguages` and is now a `SettingKey[Seq[String]]`
instead of a `SettingKey[String].` ``RB_ID=846198``

* Builds are now only for Java 8 and Scala 2.11. See the
`blog post <https://finagle.github.io/blog/2016/04/20/scala-210-and-java7/>`_
for details. ``RB_ID=828898``

Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
[![Maven Central](https://img.shields.io/maven-central/v/com.twitter/scrooge_2.11.svg)](https://maven-badges.herokuapp.com/maven-central/com.twitter/scrooge_2.11)

Scrooge is a [thrift](https://thrift.apache.org/) code generator written in
Scala, which currently generates code for Scala and Java.
Scala, which currently generates code for Scala, Java, Cocoa, Android and Lua.

It's meant to be a replacement for the apache thrift code generator, and
generates conforming, compatible binary codecs by building on top of
Expand Down
34 changes: 20 additions & 14 deletions scrooge-sbt-plugin/src/main/scala/com/twitter/ScroogeSBT.scala
Original file line number Diff line number Diff line change
Expand Up @@ -140,9 +140,9 @@ object ScroogeSBT extends AutoPlugin {
"generate code from thrift files using scrooge"
)

val scroogeLanguage = SettingKey[String](
"scrooge-language",
"language to generate code in: scala, java"
val scroogeLanguages = SettingKey[Seq[String]](
"scrooge-languages",
"language(s) to generate code in: scala, java, cocoa, android, lua"
)
}

Expand Down Expand Up @@ -175,7 +175,7 @@ object ScroogeSBT extends AutoPlugin {
scroogeThriftIncludeFolders <<= (scroogeThriftSourceFolder) { Seq(_) },
scroogeThriftNamespaceMap := Map(),
scroogeThriftDependencies := Seq(),
scroogeLanguage := "scala",
scroogeLanguages := Seq("scala"),
libraryDependencies += "com.twitter" %% "scrooge-core" % com.twitter.BuildInfo.version,

// complete list of source files
Expand Down Expand Up @@ -228,20 +228,22 @@ object ScroogeSBT extends AutoPlugin {
scroogeThriftSources,
scroogeThriftOutputFolder,
scroogeThriftIncludes,
scroogeLanguage
) map { (out, sources, outputDir, inc, language) =>
scroogeLanguages
) map { (out, sources, outputDir, inc, languages) =>
// figure out if we need to actually rebuild, based on mtimes.
val allSourceDeps = sources ++ inc.foldLeft(Seq[File]()) { (files, dir) =>
val allSourceDeps: Seq[File] = sources ++ inc.foldLeft(Seq[File]()) { (files, dir) =>
files ++ (dir ** "*.thrift").get
}
val sourcesLastModified: Seq[Long] = allSourceDeps.map(_.lastModified)
val newestSource = if (sourcesLastModified.size > 0) {
val newestSource: Long = if (sourcesLastModified.nonEmpty) {
sourcesLastModified.max
} else {
Long.MaxValue
}
val outputsLastModified = (outputDir ** generatedExtensionPattern(language)).get.map(_.lastModified)
val oldestOutput = if (outputsLastModified.size > 0) {
val outputsLastModified: Seq[Long] = languages.flatMap { language =>
(outputDir ** generatedExtensionPattern(language)).get.map(_.lastModified)
}
val oldestOutput: Long = if (outputsLastModified.nonEmpty) {
outputsLastModified.min
} else {
Long.MinValue
Expand All @@ -258,17 +260,21 @@ object ScroogeSBT extends AutoPlugin {
scroogeBuildOptions,
scroogeThriftIncludes,
scroogeThriftNamespaceMap,
scroogeLanguage,
scroogeLanguages,
scroogeDisableStrict,
scroogeScalaWarnOnJavaNSFallback,
scroogeDefaultJavaNamespace
) map { (out, isDirty, sources, outputDir, opts, inc, ns, language, disableStrict, warnOnJavaNSFallback, defaultNamespace) =>
) map { (out, isDirty, sources, outputDir, opts, inc, ns, languages, disableStrict, warnOnJavaNSFallback, defaultNamespace) =>
// for some reason, sbt sometimes calls us multiple times, often with no source files.
if (isDirty && sources.nonEmpty) {
out.log.info("Generating scrooge thrift for %s ...".format(sources.mkString(", ")))
compile(out.log, outputDir, sources.toSet, inc.toSet, ns, language, opts.toSet, disableStrict, warnOnJavaNSFallback, defaultNamespace)
languages.foreach { language =>
compile(out.log, outputDir, sources.toSet, inc.toSet, ns, language, opts.toSet, disableStrict, warnOnJavaNSFallback, defaultNamespace)
}
}
languages.flatMap { language =>
(outputDir ** generatedExtensionPattern(language)).get
}
(outputDir ** generatedExtensionPattern(language)).get.toSeq
},
sourceGenerators <+= scroogeGen
)
Expand Down

0 comments on commit 212ac9a

Please sign in to comment.