STORM-3361: Include dependency license file in distributions, clean u…

…p license merges a bit, make storm-dist use NOTICE/LICENSE from project root

DEVELOPER.md

@@ -311,10 +311,23 @@ By default integration tests are not run in the test phase. To run Java and Cloj
 ## Listing dependency licenses
 
 You can generate a list of dependencies and their licenses by running `mvn generate-resources -Dlicense.skipAggregateAddThirdParty=false` in the project root.
-The list will be put in target/generated-sources/license/THIRD-PARTY.txt.
+The list will be put in DEPENDENCY_LICENSES.
 
 The license aggregation plugin will use the license listed in a dependency's POM. If the license is missing, or incomplete (e.g. due to multiple licenses), you can override the license by describing the dependency in the THIRD-PARTY.properties file in the project root.
 
+## Auditing licenses for LICENSE/NOTICE
+The LICENSE and NOTICE files contain licenses and notices for source distribution content. The LICENSE-binary and NOTICE-binary apply to the binary distributions.
+
+When auditing the binary LICENSE-binary and NOTICE-binary, there are a couple of helper scripts available in dev-tools. `collect_license_files` can create an aggregate NOTICE from the libraries in an extracted distribution. The aggregate NOTICE should be adjusted to remove Storm notices and duplicates, and added to the NOTICE-binary.
+
+`list_jars` can list the jars in an extracted binary distribution. Note that while listing all the jars in the binary distribution is helpful, special attention must be paid to shaded jars, as they may contain shaded dependencies that must be listed in LICENSE-binary separately.
+
+The license plugin can generate a list of dependencies with licenses for the binary distribution with the following command: `mvn generate-resources -Dlicense.skipAggregateAddThirdParty=false` in the storm-dist/binary directory. 
+
+The generated list in target/generated-sources/license/THIRD-PARTY.txt is mostly complete, and a good input to the LICENSE-binary file. The major omission in it is the storm-shaded-deps dependencies, as they are shaded. These dependencies can be manually listed with `mvn dependency:list` in the storm-shaded-deps project, and then manually added. 
+
+You can download the dependency licenses by running `mvn package -Dlicense.skipAggregateDownloadLicenses=false -DskipTests` in the project root. This will put the licenses in target/generated-resources. Keep an eye on the Maven output, as some dependencies may not have licenses configured correctly. These will have to be downloaded manually.
+
 <a name="packaging"></a>
 
 ## Create a Storm distribution (packaging)

LICENSE

@@ -203,7 +203,7 @@
 
 -----------------------------------------------------------------------
 
-For jQuery 1.11.1 (storm-core/src/ui/public/js/jquery-1.11.1.min.js)
+For jQuery 1.11.1 (storm-webapp/src/main/java/org/apache/storm/daemon/ui/WEB-INF/js/jquery-1.11.1.min.js)
 
 Includes Sizzle.js
 http://sizzlejs.com/
@@ -233,7 +233,7 @@ WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
 
 -----------------------------------------------------------------------
 
-For jQuery Cookies 2.2.0 (storm-core/src/ui/public/js/jquery.cookies.2.2.0.min.js)
+For jQuery Cookies 2.2.0 (storm-webapp/src/main/java/org/apache/storm/daemon/ui/WEB-INF/js/jquery.cookies.2.2.0.min.js)
 
 Copyright (c) 2005 - 2010, James Auldridge
 
@@ -258,32 +258,31 @@ WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
 
 -----------------------------------------------------------------------
 
-For typeahead jquery 0.10.5 (storm-core/src/ui/public/js/typeahead.jquery.min.js)
+For typeahead jquery 0.10.5 (storm-webapp/src/main/java/org/apache/storm/daemon/ui/WEB-INF/js/typeahead.jquery.min.js)
 
 Copyright (c) 2013-2014 Twitter, Inc
 
-    * Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
-    * Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
-    * Neither the name of Allan Jardine nor SpryMedia UK may be used to endorse or promote products derived from this software without specific prior written permission.
-
-THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDERS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 
-
------------------------------------------------------------------------
-
-Copyright (c) 2008-2010, Allan Jardine
-All rights reserved.
-
-Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
 
-    * Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
-    * Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
-    * Neither the name of Allan Jardine nor SpryMedia UK may be used to endorse or promote products derived from this software without specific prior written permission.
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
 
-THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDERS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE. 
 
 -----------------------------------------------------------------------
 
-For js-yaml.min.js (storm-core/src/ui/public/js/)
+For js-yaml.min.js (storm-webapp/src/main/java/org/apache/storm/daemon/ui/WEB-INF/js/js-yaml.min.js)
 
 (The MIT License)
 
@@ -309,7 +308,7 @@ THE SOFTWARE.
 
 -----------------------------------------------------------------------
 
-For dagre.min.js (storm-core/src/ui/public/js/)
+For dagre.min.js (storm-webapp/src/main/java/org/apache/storm/daemon/ui/WEB-INF/js/dagre.min.js)
 
 Copyright (c) 2012-2014 Chris Pettitt
 
@@ -333,7 +332,7 @@ THE SOFTWARE.
 
 -----------------------------------------------------------------------
 
-For cytoscape.min.js and cytoscape-dagre.js (storm-core/src/ui/public/js/)
+For cytoscape.min.js and cytoscape-dagre.js (storm-webapp/src/main/java/org/apache/storm/daemon/ui/WEB-INF/js/cytoscape.min.js and storm-webapp/src/main/java/org/apache/storm/daemon/ui/WEB-INF/js/cytoscape-dagre.js)
 
 Copyright (c) 2016 The Cytoscape Consortium
 
@@ -357,7 +356,7 @@ SOFTWARE.
 
 -----------------------------------------------------------------------
 
-For esprima.js (storm-core/src/ui/public/js/)
+For esprima.js (storm-webapp/src/main/java/org/apache/storm/daemon/ui/WEB-INF/js/esprima.min.js)
 
 Copyright JS Foundation and other contributors, https://js.foundation/
 
@@ -383,7 +382,7 @@ THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 
 -----------------------------------------------------------------------
 
-For mustache.js (storm-core/src/ui/public/js/jquery.mustache.js)
+For mustache.js (storm-webapp/src/main/java/org/apache/storm/daemon/ui/WEB-INF/js/jquery.mustache.js)
 
 The MIT License
 
@@ -410,7 +409,7 @@ THE SOFTWARE.
 
 -----------------------------------------------------------------------
 
-For moment.js (storm-core/src/ui/public/js/moment.min.js)
+For moment.js (storm-webapp/src/main/java/org/apache/storm/daemon/ui/WEB-INF/js/moment.min.js)
 
 Copyright (c) 2011-2014 Tim Wood, Iskren Chernev, Moment.js contributors
 
@@ -438,7 +437,7 @@ OTHER DEALINGS IN THE SOFTWARE.
 
 -----------------------------------------------------------------------
 
-For Jquery url plugin (storm-core/src/ui/public/js/url.min.js)
+For Jquery url plugin (storm-webapp/src/main/java/org/apache/storm/daemon/ui/WEB-INF/js/url.min.js)
 
 Jquery Url (A Jquery plugin for URL parser) v1.8.6
 Source repository: https://github.com/websanova/js-url
@@ -447,7 +446,7 @@ Licensed under an MIT-style license. Seehttps://github.com/websanova/js-url#lice
 -----------------------------------------------------------------------
 
 
-For jquery.blockUI.min.js (storm-core/src/ui/public/js/jquery.blockUI.min.js)
+For jquery.blockUI.min.js (storm-webapp/src/main/java/org/apache/storm/daemon/ui/WEB-INF/js/jquery.blockUI.min.js)
 
 jQuery BlockUI; v20131009
 http://jquery.malsup.com/block/
@@ -474,10 +473,10 @@ use the BlockUI plugin in commercial projects as long as the copyright header is
 
 For jquery dataTables v1.10.4
 
-(storm-core/src/ui/public/js/jquery.dataTables.1.10.4.min.js
-storm-core/src/ui/public/css/jquery.dataTables.1.10.4.min.css
-storm-core/src/ui/public/images/{{back, forward}_{disabled, enabled, enabled_hover}}.png
-storm-core/src/ui/public/images/{sort_{asc, asc_disabled, both, desc, desc_disabled}}.png)
+(storm-webapp/src/main/java/org/apache/storm/daemon/ui/WEB-INF/js/jquery.dataTables.1.10.4.min.js
+storm-webapp/src/main/java/org/apache/storm/daemon/ui/WEB-INF/css/jquery.dataTables.1.10.4.min.css
+storm-webapp/src/main/java/org/apache/storm/daemon/ui/WEB-INF/images/{{back, forward}_{disabled, enabled, enabled_hover}}.png
+storm-webapp/src/main/java/org/apache/storm/daemon/ui/WEB-INF/images/{sort_{asc, asc_disabled, both, desc, desc_disabled}}.png)
 
 Copyright (c) 2008-2013 SpryMedia Limited
 http://datatables.net
@@ -504,8 +503,9 @@ THE SOFTWARE.
 
 For jquery dataTables bootstrap integration
 
-(storm-core/src/ui/public/js/jquery.dataTables.1.10.4.min.js
-torm-core/src/ui/public/css/dataTables.bootstrap.css)
+(storm-webapp/src/main/java/org/apache/storm/daemon/ui/WEB-INF/js/jquery.dataTables.1.10.4.min.js
+storm-webapp/src/main/java/org/apache/storm/daemon/ui/WEB-INF/js/dataTables.bootstrap.min.js
+storm-webapp/src/main/java/org/apache/storm/daemon/ui/WEB-INF/css/dataTables.bootstrap.css)
 
 Copyright (c) 2013-2014 SpryMedia Limited
 http://datatables.net/license
@@ -533,8 +533,8 @@ THE SOFTWARE.
 -----------------------------------------------------------------------
 
 For bootstrap v3.3.1
-(storm-core/src/ui/public/js/bootstrap-3.3.1.min.js
-storm-core/src/ui/public/css/bootstrap-3.3.1.min.css)
+(storm-webapp/src/main/java/org/apache/storm/daemon/ui/WEB-INF/js/bootstrap-3.3.1.min.js
+storm-webapp/src/main/java/org/apache/storm/daemon/ui/WEB-INF/css/bootstrap-3.3.1.min.css)
 
 Bootstrap v3.3.1 (http://getbootstrap.com)
 Copyright 2011-2014 Twitter, Inc.
@@ -543,8 +543,8 @@ Licensed under MIT (https://github.com/twbs/bootstrap/blob/master/LICENSE)
 -----------------------------------------------------------------------
 
 For jQuery JSONFormatter 1.0.1 2015-02-28
-(storm-core/src/ui/public/js/jsonFormatter.min.js
-storm-core/src/ui/public/css/jsonFormatter.min.css)
+(storm-webapp/src/main/java/org/apache/storm/daemon/ui/WEB-INF/js/jsonFormatter.min.js
+storm-webapp/src/main/java/org/apache/storm/daemon/ui/WEB-INF/css/jsonFormatter.min.css)
 http://www.jqueryscript.net/other/jQuery-Plugin-For-Pretty-JSON-Formatting-jsonFormatter.htmlA
 
 The MIT License (MIT)
@@ -573,7 +573,7 @@ SOFTWARE.
 
 For statistic image
 
-storm-core/src/ui/public/images/statistic.png
+storm-webapp/src/main/java/org/apache/storm/daemon/ui/WEB-INF/images/statistic.png
 
 Copyright (c) 2015 Github, Inc.
 
@@ -598,16 +598,21 @@ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
 OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
 THE SOFTWARE.
 
+-----------------------------------------------------------------------
+
+For PMML sample files
 
+examples/storm-pmml-examples/src/main/resources/KNIME_PMML_4.1_Examples_single_audit_logreg.xml
+examples/storm-pmml-examples/src/main/resources/Audit.50.csv
 
 This product bundles PMML Sample Files, which are available under a
-"3-clause BSD" license.  For details, see http://dmg.org/documents/dmg-pmml-license-2016.pdf.
+"3-clause BSD" license.  For details, see http://dmg.org/documents/dmg-pmml-license-2016.pdf
 
 -----------------------------------------------------------------------
 
 For vis.js 4.16.1 2016-04-18
-(storm-core/src/ui/public/js/vis.min.js
-storm-core/src/ui/public/css/vis.min.css)
+(storm-webapp/src/main/java/org/apache/storm/daemon/ui/WEB-INF/js/vis.min.js
+storm-webapp/src/main/java/org/apache/storm/daemon/ui/WEB-INF/css/vis.min.css)
 
 vis.js
 https://github.com/almende/vis