forked from apache/storm
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
set '* text=auto' in .gitattributes in order to avoid merge work beca…
…use of line feed changes
- Loading branch information
Karl-Philipp Richter
committed
Mar 23, 2018
1 parent
0fdad2c
commit e9c427c
Showing
48 changed files
with
3,136 additions
and
3,127 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,2 +1,11 @@ | ||
# Some storm-webapp logviewer tests require input files to have LF line endings due to byte counting. | ||
storm-webapp/src/test/resources/*.log.test text eol=lf | ||
storm-webapp/src/test/resources/*.log.test text eol=lf | ||
|
||
# Convert the test on check-in and check-out (the conversion of all files has been done once on master and should be enforced from now on) | ||
* text=auto | ||
|
||
# There're reports of EOL conversion messing up PNG files, but that might have been a bug in git 2.10 only (see https://github.com/git/git/blob/master/Documentation/RelNotes/2.10.0.txt#L248 for details) | ||
*.png binary | ||
*.tar.gz binary | ||
*.zip binary | ||
*.tgz binary |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,40 +1,40 @@ | ||
--- | ||
title: Azue Event Hubs Integration | ||
layout: documentation | ||
documentation: true | ||
--- | ||
|
||
Storm spout and bolt implementation for Microsoft Azure Eventhubs | ||
|
||
### build ### | ||
mvn clean package | ||
|
||
### run sample topology ### | ||
To run the sample topology, you need to modify the config.properties file with | ||
the eventhubs configurations. Here is an example: | ||
|
||
eventhubspout.username = [username: policy name in EventHubs Portal] | ||
eventhubspout.password = [password: shared access key in EventHubs Portal] | ||
eventhubspout.namespace = [namespace] | ||
eventhubspout.entitypath = [entitypath] | ||
eventhubspout.partitions.count = [partitioncount] | ||
|
||
# if not provided, will use storm's zookeeper settings | ||
# zookeeper.connectionstring=zookeeper0:2181,zookeeper1:2181,zookeeper2:2181 | ||
|
||
eventhubspout.checkpoint.interval = 10 | ||
eventhub.receiver.credits = 1024 | ||
|
||
Then you can use storm.cmd to submit the sample topology: | ||
storm jar {jarfile} com.microsoft.eventhubs.samples.EventCount {topologyname} {spoutconffile} | ||
where the {jarfile} should be: eventhubs-storm-spout-{version}-jar-with-dependencies.jar | ||
|
||
### Run EventHubSendClient ### | ||
We have included a simple EventHubs send client for testing purpose. You can run the client like this: | ||
java -cp .\target\eventhubs-storm-spout-{version}-jar-with-dependencies.jar com.microsoft.eventhubs.client.EventHubSendClient | ||
[username] [password] [entityPath] [partitionId] [messageSize] [messageCount] | ||
If you want to send messages to all partitions, use "-1" as partitionId. | ||
|
||
### Windows Azure Eventhubs ### | ||
http://azure.microsoft.com/en-us/services/event-hubs/ | ||
|
||
--- | ||
title: Azue Event Hubs Integration | ||
layout: documentation | ||
documentation: true | ||
--- | ||
|
||
Storm spout and bolt implementation for Microsoft Azure Eventhubs | ||
|
||
### build ### | ||
mvn clean package | ||
|
||
### run sample topology ### | ||
To run the sample topology, you need to modify the config.properties file with | ||
the eventhubs configurations. Here is an example: | ||
|
||
eventhubspout.username = [username: policy name in EventHubs Portal] | ||
eventhubspout.password = [password: shared access key in EventHubs Portal] | ||
eventhubspout.namespace = [namespace] | ||
eventhubspout.entitypath = [entitypath] | ||
eventhubspout.partitions.count = [partitioncount] | ||
|
||
# if not provided, will use storm's zookeeper settings | ||
# zookeeper.connectionstring=zookeeper0:2181,zookeeper1:2181,zookeeper2:2181 | ||
|
||
eventhubspout.checkpoint.interval = 10 | ||
eventhub.receiver.credits = 1024 | ||
|
||
Then you can use storm.cmd to submit the sample topology: | ||
storm jar {jarfile} com.microsoft.eventhubs.samples.EventCount {topologyname} {spoutconffile} | ||
where the {jarfile} should be: eventhubs-storm-spout-{version}-jar-with-dependencies.jar | ||
|
||
### Run EventHubSendClient ### | ||
We have included a simple EventHubs send client for testing purpose. You can run the client like this: | ||
java -cp .\target\eventhubs-storm-spout-{version}-jar-with-dependencies.jar com.microsoft.eventhubs.client.EventHubSendClient | ||
[username] [password] [entityPath] [partitionId] [messageSize] [messageCount] | ||
If you want to send messages to all partitions, use "-1" as partitionId. | ||
|
||
### Windows Azure Eventhubs ### | ||
http://azure.microsoft.com/en-us/services/event-hubs/ | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,45 +1,45 @@ | ||
storm-eventhubs | ||
===================== | ||
|
||
Storm spout and bolt implementation for Microsoft Azure Eventhubs | ||
|
||
### build ### | ||
mvn clean package | ||
|
||
### run sample topology ### | ||
To run the sample topology, you need to modify the config.properties file with | ||
the eventhubs configurations. Here is an example: | ||
|
||
eventhubspout.username = [username: policy name in EventHubs Portal] | ||
eventhubspout.password = [password: shared access key in EventHubs Portal] | ||
eventhubspout.namespace = [namespace] | ||
eventhubspout.entitypath = [entitypath] | ||
eventhubspout.partitions.count = [partitioncount] | ||
|
||
# if not provided, will use storm's zookeeper settings | ||
# zookeeper.connectionstring=zookeeper0:2181,zookeeper1:2181,zookeeper2:2181 | ||
|
||
eventhubspout.checkpoint.interval = 10 | ||
eventhub.receiver.credits = 1024 | ||
|
||
Then you can use storm.cmd to submit the sample topology: | ||
storm jar {jarfile} com.microsoft.eventhubs.samples.EventCount {topologyname} {spoutconffile} | ||
where the {jarfile} should be: eventhubs-storm-spout-{version}-jar-with-dependencies.jar | ||
|
||
### Run EventHubSendClient ### | ||
We have included a simple EventHubs send client for testing purpose. You can run the client like this: | ||
java -cp .\target\eventhubs-storm-spout-{version}-jar-with-dependencies.jar com.microsoft.eventhubs.client.EventHubSendClient | ||
[username] [password] [entityPath] [partitionId] [messageSize] [messageCount] | ||
If you want to send messages to all partitions, use "-1" as partitionId. | ||
|
||
### Serialization Scheme ### | ||
By default the serialization scheme is StringEventDataScheme where only the body of the eventdata is being sent. | ||
To have a more comprehensive Eventhub metadata exposure look into the BinaryEventDataScheme. | ||
|
||
### Windows Azure Eventhubs ### | ||
http://azure.microsoft.com/en-us/services/event-hubs/ | ||
|
||
## Committer Sponsors | ||
|
||
* P. Taylor Goetz ([[email protected]](mailto:[email protected])) | ||
|
||
storm-eventhubs | ||
===================== | ||
|
||
Storm spout and bolt implementation for Microsoft Azure Eventhubs | ||
|
||
### build ### | ||
mvn clean package | ||
|
||
### run sample topology ### | ||
To run the sample topology, you need to modify the config.properties file with | ||
the eventhubs configurations. Here is an example: | ||
|
||
eventhubspout.username = [username: policy name in EventHubs Portal] | ||
eventhubspout.password = [password: shared access key in EventHubs Portal] | ||
eventhubspout.namespace = [namespace] | ||
eventhubspout.entitypath = [entitypath] | ||
eventhubspout.partitions.count = [partitioncount] | ||
|
||
# if not provided, will use storm's zookeeper settings | ||
# zookeeper.connectionstring=zookeeper0:2181,zookeeper1:2181,zookeeper2:2181 | ||
|
||
eventhubspout.checkpoint.interval = 10 | ||
eventhub.receiver.credits = 1024 | ||
|
||
Then you can use storm.cmd to submit the sample topology: | ||
storm jar {jarfile} com.microsoft.eventhubs.samples.EventCount {topologyname} {spoutconffile} | ||
where the {jarfile} should be: eventhubs-storm-spout-{version}-jar-with-dependencies.jar | ||
|
||
### Run EventHubSendClient ### | ||
We have included a simple EventHubs send client for testing purpose. You can run the client like this: | ||
java -cp .\target\eventhubs-storm-spout-{version}-jar-with-dependencies.jar com.microsoft.eventhubs.client.EventHubSendClient | ||
[username] [password] [entityPath] [partitionId] [messageSize] [messageCount] | ||
If you want to send messages to all partitions, use "-1" as partitionId. | ||
|
||
### Serialization Scheme ### | ||
By default the serialization scheme is StringEventDataScheme where only the body of the eventdata is being sent. | ||
To have a more comprehensive Eventhub metadata exposure look into the BinaryEventDataScheme. | ||
|
||
### Windows Azure Eventhubs ### | ||
http://azure.microsoft.com/en-us/services/event-hubs/ | ||
|
||
## Committer Sponsors | ||
|
||
* P. Taylor Goetz ([[email protected]](mailto:[email protected])) | ||
|
94 changes: 47 additions & 47 deletions
94
...storm-eventhubs/src/main/java/org/apache/storm/eventhubs/bolt/DefaultEventDataFormat.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,47 +1,47 @@ | ||
/******************************************************************************* | ||
* Licensed to the Apache Software Foundation (ASF) under one | ||
* or more contributor license agreements. See the NOTICE file | ||
* distributed with this work for additional information | ||
* regarding copyright ownership. The ASF licenses this file | ||
* to you under the Apache License, Version 2.0 (the | ||
* "License"); you may not use this file except in compliance | ||
* with the License. You may obtain a copy of the License at | ||
* | ||
* http://www.apache.org/licenses/LICENSE-2.0 | ||
* | ||
* Unless required by applicable law or agreed to in writing, software | ||
* distributed under the License is distributed on an "AS IS" BASIS, | ||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
* See the License for the specific language governing permissions and | ||
* limitations under the License. | ||
*******************************************************************************/ | ||
package org.apache.storm.eventhubs.bolt; | ||
|
||
import org.apache.storm.tuple.Tuple; | ||
|
||
/** | ||
* A default implementation of IEventDataFormat that converts the tuple | ||
* into a delimited string. | ||
*/ | ||
public class DefaultEventDataFormat implements IEventDataFormat { | ||
private static final long serialVersionUID = 1L; | ||
private String delimiter = ","; | ||
|
||
public DefaultEventDataFormat withFieldDelimiter(String delimiter) { | ||
this.delimiter = delimiter; | ||
return this; | ||
} | ||
|
||
@Override | ||
public byte[] serialize(Tuple tuple) { | ||
StringBuilder sb = new StringBuilder(); | ||
for(Object obj : tuple.getValues()) { | ||
if(sb.length() != 0) { | ||
sb.append(delimiter); | ||
} | ||
sb.append(obj.toString()); | ||
} | ||
return sb.toString().getBytes(); | ||
} | ||
|
||
} | ||
/******************************************************************************* | ||
* Licensed to the Apache Software Foundation (ASF) under one | ||
* or more contributor license agreements. See the NOTICE file | ||
* distributed with this work for additional information | ||
* regarding copyright ownership. The ASF licenses this file | ||
* to you under the Apache License, Version 2.0 (the | ||
* "License"); you may not use this file except in compliance | ||
* with the License. You may obtain a copy of the License at | ||
* | ||
* http://www.apache.org/licenses/LICENSE-2.0 | ||
* | ||
* Unless required by applicable law or agreed to in writing, software | ||
* distributed under the License is distributed on an "AS IS" BASIS, | ||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
* See the License for the specific language governing permissions and | ||
* limitations under the License. | ||
*******************************************************************************/ | ||
package org.apache.storm.eventhubs.bolt; | ||
|
||
import org.apache.storm.tuple.Tuple; | ||
|
||
/** | ||
* A default implementation of IEventDataFormat that converts the tuple | ||
* into a delimited string. | ||
*/ | ||
public class DefaultEventDataFormat implements IEventDataFormat { | ||
private static final long serialVersionUID = 1L; | ||
private String delimiter = ","; | ||
|
||
public DefaultEventDataFormat withFieldDelimiter(String delimiter) { | ||
this.delimiter = delimiter; | ||
return this; | ||
} | ||
|
||
@Override | ||
public byte[] serialize(Tuple tuple) { | ||
StringBuilder sb = new StringBuilder(); | ||
for(Object obj : tuple.getValues()) { | ||
if(sb.length() != 0) { | ||
sb.append(delimiter); | ||
} | ||
sb.append(obj.toString()); | ||
} | ||
return sb.toString().getBytes(); | ||
} | ||
|
||
} |
Oops, something went wrong.