Capistrano builds for various targets & sub-projects

[dazza-codes]

• systems need to have sbt installed (or the deploy must be able to install it?)
  • the deploy installs it
• configs to build specific projects for target system
  • symphony needs only a marc21 source producer
    • need advice/assistance from @jgreben on this task
  • spark needs the conversion mappings
    • done in PRs Data env settings #24 and Install sbt and assemble the spark projects #28
  • stardog needs the bibframe consumer / stardog sink
    • PR Stardog deploy #30

[atz]

Recommend compile and assembly (via sbt) happen on the dev system (wherever the cap commands initiate). The resulting jar files are all that needs to be transferred to respective deployment systems.

[dazza-codes]

The JAR uploads in PR #25 worked last night. It is too slow to do the uploads, it should be much faster to build stuff on each remote system, in parallel. It certainly is faster to build it once on the laptop, but the uploads are a killer. I'm going to start work on an alternative to PR #25 that builds on the remote systems. The first build will be expensive, but once the ivy packages are cached, it should be much faster.

[dazza-codes]

The spark streaming app is deployed and assembled, after PRs #24 and #28 were merged to master. A similar approach can be adopted for the StarDog deployment.

[dazza-codes]

The Stardog deployment is working OK in PR #30, although there are some problems with the assembly because the banana-rdf dependency is not properly published for our version of scala.

@atz submitted an issue to get it published and they did it, see

• Publish v0.8.4 jars banana-rdf/banana-rdf#328

@Maatari - we need to use that published artifact; are we using it or not?