Skip to content

Commit

Permalink
Dependency upgrades, fix #1144
Browse files Browse the repository at this point in the history
Signed-off-by: Julien Nioche <[email protected]>
  • Loading branch information
jnioche committed Feb 21, 2024
1 parent 953a8c5 commit 2f80e7a
Show file tree
Hide file tree
Showing 11 changed files with 59 additions and 56 deletions.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ NOTE: These instructions assume that you have [Apache Maven](https://maven.apach

StormCrawler requires Java 11 or above.

The version of Apache Storm to install must match the one defined in the pom.xml file of your topology. The major version of StormCrawler mirrors the one from Apache Storm, i.e whereas StormCrawler 1.x used Storm 1.2.3, the current version now requires Storm 2.6.0. Our [Ansible-Storm](https://github.com/DigitalPebble/ansible-storm) repository contains resources to install Apache Storm using Ansible. Alternatively, our [stormCrawler-docker](https://github.com/DigitalPebble/stormcrawler-docker) project should help you run Apache Storm on Docker.
The version of Apache Storm to install must match the one defined in the pom.xml file of your topology. The major version of StormCrawler mirrors the one from Apache Storm, i.e whereas StormCrawler 1.x used Storm 1.2.3, the current version now requires Storm 2.6.1. Our [Ansible-Storm](https://github.com/DigitalPebble/ansible-storm) repository contains resources to install Apache Storm using Ansible. Alternatively, our [stormCrawler-docker](https://github.com/DigitalPebble/stormcrawler-docker) project should help you run Apache Storm on Docker.

Once Storm is installed, the easiest way to get started is to generate a brand new StormCrawler project using \:

Expand Down
80 changes: 41 additions & 39 deletions THIRD-PARTY.txt

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion archetype/src/main/resources/archetype-resources/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ Have a look at the code and resources and modify them to your heart's content.

# Prerequisites

You need to install Apache Storm. The instructions on [setting up a Storm cluster](https://storm.apache.org/releases/2.6.0/Setting-up-a-Storm-cluster.html) should help. Alternatively,
You need to install Apache Storm. The instructions on [setting up a Storm cluster](https://storm.apache.org/releases/2.6.1/Setting-up-a-Storm-cluster.html) should help. Alternatively,
the [stormcrawler-docker](https://github.com/DigitalPebble/stormcrawler-docker) project contains resources for running Apache Storm on Docker.

You also need to have an instance of URLFrontier running. See [the URLFrontier README](https://github.com/crawler-commons/url-frontier/tree/master/service); the easiest way is to use Docker, like so:
Expand Down
2 changes: 1 addition & 1 deletion archetype/src/main/resources/archetype-resources/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<stormcrawler.version>${project.version}</stormcrawler.version>
<storm.version>2.6.0</storm.version>
<storm.version>2.6.1</storm.version>
</properties>

<build>
Expand Down
4 changes: 2 additions & 2 deletions core/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -24,9 +24,9 @@
<httpclient.version>4.5.14</httpclient.version>
<snakeyaml.version>2.2</snakeyaml.version>
<commons.lang.version>2.6</commons.lang.version>
<wiremock.version>3.3.1</wiremock.version>
<wiremock.version>3.4.1</wiremock.version>
<rometools.version>2.1.0</rometools.version>
<selenium.version>4.16.1</selenium.version>
<selenium.version>4.18.1</selenium.version>
<cli.version>1.6.0</cli.version>
<okhttp.version>4.12.0</okhttp.version>
<caffeine.version>3.1.8</caffeine.version>
Expand Down
2 changes: 1 addition & 1 deletion external/aws/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
<description>AWS resources for StormCrawler</description>

<properties>
<aws.version>1.12.585</aws.version>
<aws.version>1.12.663</aws.version>
</properties>

<dependencies>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<stormcrawler.version>${StormCrawlerVersion}</stormcrawler.version>
<storm.version>2.6.0</storm.version>
<storm.version>2.6.1</storm.version>
</properties>

<build>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<stormcrawler.version>${StormCrawlerVersion}</stormcrawler.version>
<storm.version>2.6.0</storm.version>
<storm.version>2.6.1</storm.version>
</properties>

<build>
Expand Down
2 changes: 1 addition & 1 deletion external/solr/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
<description>Solr resources for StormCrawler</description>

<properties>
<solr.version>9.4.0</solr.version>
<solr.version>9.5.0</solr.version>
</properties>

<dependencies>
Expand Down
2 changes: 1 addition & 1 deletion external/warc/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@

<properties>
<commons-codec.version>1.10</commons-codec.version>
<jwarc.version>0.28.5</jwarc.version>
<jwarc.version>0.29.0</jwarc.version>
</properties>

<dependencies>
Expand Down
15 changes: 8 additions & 7 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -78,15 +78,16 @@
<additionalparam>-Xdoclint:none</additionalparam>
<!-- dependency versions -->
<junit.version>4.13.2</junit.version>
<storm-client.version>2.6.0</storm-client.version>
<jackson.version>2.15.2</jackson.version>
<storm-client.version>2.6.1</storm-client.version>
<jackson.version>2.16.1</jackson.version>
<tika.version>2.9.1</tika.version>
<mockito.version>5.8.0</mockito.version>
<mockito.version>5.10.0</mockito.version>
<jetbrains.annotations.version>24.1.0</jetbrains.annotations.version>
<log4j2.version>2.21.1</log4j2.version>
<git-code-format-maven-plugin.version>5.1</git-code-format-maven-plugin.version>
<testcontainers.version>1.19.3</testcontainers.version>
<log4j2.version>2.23.0</log4j2.version>
<git-code-format-maven-plugin.version>5.3</git-code-format-maven-plugin.version>
<testcontainers.version>1.19.5</testcontainers.version>
<maven.mojo.license.plugin>2.4.0</maven.mojo.license.plugin>
<jacoco.maven.plugin>0.8.11</jacoco.maven.plugin>
</properties>

<distributionManagement>
Expand Down Expand Up @@ -256,7 +257,7 @@
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>0.8.11</version>
<version>${jacoco.maven.plugin}</version>
<executions>
<execution>
<id>prepare-agent</id>
Expand Down

0 comments on commit 2f80e7a

Please sign in to comment.