Skip to content

Commit

Permalink
set '* text=auto' in .gitattributes in order to avoid merge work beca…
Browse files Browse the repository at this point in the history
…use of line feed changes
  • Loading branch information
Karl-Philipp Richter committed Mar 23, 2018
1 parent 0fdad2c commit e9c427c
Show file tree
Hide file tree
Showing 48 changed files with 3,136 additions and 3,127 deletions.
11 changes: 10 additions & 1 deletion .gitattributes
Original file line number Diff line number Diff line change
@@ -1,2 +1,11 @@
# Some storm-webapp logviewer tests require input files to have LF line endings due to byte counting.
storm-webapp/src/test/resources/*.log.test text eol=lf
storm-webapp/src/test/resources/*.log.test text eol=lf

# Convert the test on check-in and check-out (the conversion of all files has been done once on master and should be enforced from now on)
* text=auto

# There're reports of EOL conversion messing up PNG files, but that might have been a bug in git 2.10 only (see https://github.com/git/git/blob/master/Documentation/RelNotes/2.10.0.txt#L248 for details)
*.png binary
*.tar.gz binary
*.zip binary
*.tgz binary
80 changes: 40 additions & 40 deletions docs/storm-eventhubs.md
Original file line number Diff line number Diff line change
@@ -1,40 +1,40 @@
---
title: Azue Event Hubs Integration
layout: documentation
documentation: true
---

Storm spout and bolt implementation for Microsoft Azure Eventhubs

### build ###
mvn clean package

### run sample topology ###
To run the sample topology, you need to modify the config.properties file with
the eventhubs configurations. Here is an example:

eventhubspout.username = [username: policy name in EventHubs Portal]
eventhubspout.password = [password: shared access key in EventHubs Portal]
eventhubspout.namespace = [namespace]
eventhubspout.entitypath = [entitypath]
eventhubspout.partitions.count = [partitioncount]

# if not provided, will use storm's zookeeper settings
# zookeeper.connectionstring=zookeeper0:2181,zookeeper1:2181,zookeeper2:2181

eventhubspout.checkpoint.interval = 10
eventhub.receiver.credits = 1024

Then you can use storm.cmd to submit the sample topology:
storm jar {jarfile} com.microsoft.eventhubs.samples.EventCount {topologyname} {spoutconffile}
where the {jarfile} should be: eventhubs-storm-spout-{version}-jar-with-dependencies.jar

### Run EventHubSendClient ###
We have included a simple EventHubs send client for testing purpose. You can run the client like this:
java -cp .\target\eventhubs-storm-spout-{version}-jar-with-dependencies.jar com.microsoft.eventhubs.client.EventHubSendClient
[username] [password] [entityPath] [partitionId] [messageSize] [messageCount]
If you want to send messages to all partitions, use "-1" as partitionId.

### Windows Azure Eventhubs ###
http://azure.microsoft.com/en-us/services/event-hubs/

---
title: Azue Event Hubs Integration
layout: documentation
documentation: true
---

Storm spout and bolt implementation for Microsoft Azure Eventhubs

### build ###
mvn clean package

### run sample topology ###
To run the sample topology, you need to modify the config.properties file with
the eventhubs configurations. Here is an example:

eventhubspout.username = [username: policy name in EventHubs Portal]
eventhubspout.password = [password: shared access key in EventHubs Portal]
eventhubspout.namespace = [namespace]
eventhubspout.entitypath = [entitypath]
eventhubspout.partitions.count = [partitioncount]

# if not provided, will use storm's zookeeper settings
# zookeeper.connectionstring=zookeeper0:2181,zookeeper1:2181,zookeeper2:2181

eventhubspout.checkpoint.interval = 10
eventhub.receiver.credits = 1024

Then you can use storm.cmd to submit the sample topology:
storm jar {jarfile} com.microsoft.eventhubs.samples.EventCount {topologyname} {spoutconffile}
where the {jarfile} should be: eventhubs-storm-spout-{version}-jar-with-dependencies.jar

### Run EventHubSendClient ###
We have included a simple EventHubs send client for testing purpose. You can run the client like this:
java -cp .\target\eventhubs-storm-spout-{version}-jar-with-dependencies.jar com.microsoft.eventhubs.client.EventHubSendClient
[username] [password] [entityPath] [partitionId] [messageSize] [messageCount]
If you want to send messages to all partitions, use "-1" as partitionId.

### Windows Azure Eventhubs ###
http://azure.microsoft.com/en-us/services/event-hubs/

90 changes: 45 additions & 45 deletions external/storm-eventhubs/README.md
Original file line number Diff line number Diff line change
@@ -1,45 +1,45 @@
storm-eventhubs
=====================

Storm spout and bolt implementation for Microsoft Azure Eventhubs

### build ###
mvn clean package

### run sample topology ###
To run the sample topology, you need to modify the config.properties file with
the eventhubs configurations. Here is an example:

eventhubspout.username = [username: policy name in EventHubs Portal]
eventhubspout.password = [password: shared access key in EventHubs Portal]
eventhubspout.namespace = [namespace]
eventhubspout.entitypath = [entitypath]
eventhubspout.partitions.count = [partitioncount]

# if not provided, will use storm's zookeeper settings
# zookeeper.connectionstring=zookeeper0:2181,zookeeper1:2181,zookeeper2:2181

eventhubspout.checkpoint.interval = 10
eventhub.receiver.credits = 1024

Then you can use storm.cmd to submit the sample topology:
storm jar {jarfile} com.microsoft.eventhubs.samples.EventCount {topologyname} {spoutconffile}
where the {jarfile} should be: eventhubs-storm-spout-{version}-jar-with-dependencies.jar

### Run EventHubSendClient ###
We have included a simple EventHubs send client for testing purpose. You can run the client like this:
java -cp .\target\eventhubs-storm-spout-{version}-jar-with-dependencies.jar com.microsoft.eventhubs.client.EventHubSendClient
[username] [password] [entityPath] [partitionId] [messageSize] [messageCount]
If you want to send messages to all partitions, use "-1" as partitionId.

### Serialization Scheme ###
By default the serialization scheme is StringEventDataScheme where only the body of the eventdata is being sent.
To have a more comprehensive Eventhub metadata exposure look into the BinaryEventDataScheme.

### Windows Azure Eventhubs ###
http://azure.microsoft.com/en-us/services/event-hubs/

## Committer Sponsors

* P. Taylor Goetz ([[email protected]](mailto:[email protected]))

storm-eventhubs
=====================

Storm spout and bolt implementation for Microsoft Azure Eventhubs

### build ###
mvn clean package

### run sample topology ###
To run the sample topology, you need to modify the config.properties file with
the eventhubs configurations. Here is an example:

eventhubspout.username = [username: policy name in EventHubs Portal]
eventhubspout.password = [password: shared access key in EventHubs Portal]
eventhubspout.namespace = [namespace]
eventhubspout.entitypath = [entitypath]
eventhubspout.partitions.count = [partitioncount]

# if not provided, will use storm's zookeeper settings
# zookeeper.connectionstring=zookeeper0:2181,zookeeper1:2181,zookeeper2:2181

eventhubspout.checkpoint.interval = 10
eventhub.receiver.credits = 1024

Then you can use storm.cmd to submit the sample topology:
storm jar {jarfile} com.microsoft.eventhubs.samples.EventCount {topologyname} {spoutconffile}
where the {jarfile} should be: eventhubs-storm-spout-{version}-jar-with-dependencies.jar

### Run EventHubSendClient ###
We have included a simple EventHubs send client for testing purpose. You can run the client like this:
java -cp .\target\eventhubs-storm-spout-{version}-jar-with-dependencies.jar com.microsoft.eventhubs.client.EventHubSendClient
[username] [password] [entityPath] [partitionId] [messageSize] [messageCount]
If you want to send messages to all partitions, use "-1" as partitionId.

### Serialization Scheme ###
By default the serialization scheme is StringEventDataScheme where only the body of the eventdata is being sent.
To have a more comprehensive Eventhub metadata exposure look into the BinaryEventDataScheme.

### Windows Azure Eventhubs ###
http://azure.microsoft.com/en-us/services/event-hubs/

## Committer Sponsors

* P. Taylor Goetz ([[email protected]](mailto:[email protected]))

Original file line number Diff line number Diff line change
@@ -1,47 +1,47 @@
/*******************************************************************************
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*******************************************************************************/
package org.apache.storm.eventhubs.bolt;

import org.apache.storm.tuple.Tuple;

/**
* A default implementation of IEventDataFormat that converts the tuple
* into a delimited string.
*/
public class DefaultEventDataFormat implements IEventDataFormat {
private static final long serialVersionUID = 1L;
private String delimiter = ",";

public DefaultEventDataFormat withFieldDelimiter(String delimiter) {
this.delimiter = delimiter;
return this;
}

@Override
public byte[] serialize(Tuple tuple) {
StringBuilder sb = new StringBuilder();
for(Object obj : tuple.getValues()) {
if(sb.length() != 0) {
sb.append(delimiter);
}
sb.append(obj.toString());
}
return sb.toString().getBytes();
}

}
/*******************************************************************************
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*******************************************************************************/
package org.apache.storm.eventhubs.bolt;

import org.apache.storm.tuple.Tuple;

/**
* A default implementation of IEventDataFormat that converts the tuple
* into a delimited string.
*/
public class DefaultEventDataFormat implements IEventDataFormat {
private static final long serialVersionUID = 1L;
private String delimiter = ",";

public DefaultEventDataFormat withFieldDelimiter(String delimiter) {
this.delimiter = delimiter;
return this;
}

@Override
public byte[] serialize(Tuple tuple) {
StringBuilder sb = new StringBuilder();
for(Object obj : tuple.getValues()) {
if(sb.length() != 0) {
sb.append(delimiter);
}
sb.append(obj.toString());
}
return sb.toString().getBytes();
}

}
Loading

0 comments on commit e9c427c

Please sign in to comment.