-
Notifications
You must be signed in to change notification settings - Fork 11
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* Remove stop of daemon thread option * Make consumers and producers public to allow list topics for both of them * Extend tests with List Topics keyword * Temp possible fix of list topics * Change return of reference in decode_data to new list * Update test for debug * Update producer flush * Usage of OFFSET vars in robot tests * FIx return only copy of messages to not overwrite * Remove unused args * Update get_messages_from_thread args * Update assign_to_topic_partition * Add topic partition related keywords * Add seek keyword * pep8 updates * Make decode_data private * Remove remove_zero_bytes from _decode_data * Allow poll full message object via only_value arg * Temp tests for travis validation * Bump requirements versions * Bump requirements versions * Revert Poll error handling * Revert Poll error handling * Add partition for produce * Change only_value default to True * Minor test updates * Raise poll error * Refactor is_assigned * Introduce unassign * Add get_position keyword * Fix indent * Update confluent-kafka-library * Run enterprise kafka for testing * Fix arg order in GetMessagesThread * Add Create Topic Partition keyword * Refactoring and bugfixing * Update examples * Update readme * Bump version
- Loading branch information
Showing
14 changed files
with
538 additions
and
161 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,56 @@ | ||
| --- | ||
| version: '2' | ||
| services: | ||
| zookeeper: | ||
| image: confluentinc/cp-zookeeper:5.3.1 | ||
| hostname: zookeeper | ||
| container_name: zookeeper | ||
| ports: | ||
| - "2181:2181" | ||
| environment: | ||
| ZOOKEEPER_CLIENT_PORT: 2181 | ||
|
|
||
| broker: | ||
| image: confluentinc/cp-enterprise-kafka:5.3.1 | ||
| hostname: broker | ||
| container_name: broker | ||
| depends_on: | ||
| - zookeeper | ||
| ports: | ||
| - "9092:9092" | ||
| environment: | ||
| KAFKA_BROKER_ID: 1 | ||
| KAFKA_ZOOKEEPER_CONNECT: 'zookeeper:2181' | ||
| KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT | ||
| KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://broker:29092,PLAINTEXT_HOST://localhost:9092 | ||
| KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1 | ||
| KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0 | ||
|
|
||
| schema-registry: | ||
| image: confluentinc/cp-schema-registry:5.3.1 | ||
| hostname: schema-registry | ||
| container_name: schema-registry | ||
| depends_on: | ||
| - zookeeper | ||
| - broker | ||
| ports: | ||
| - "8081:8081" | ||
| environment: | ||
| SCHEMA_REGISTRY_HOST_NAME: schema-registry | ||
| SCHEMA_REGISTRY_KAFKASTORE_CONNECTION_URL: 'zookeeper:2181' | ||
|
|
||
| rest-proxy: | ||
| image: confluentinc/cp-kafka-rest:5.3.1 | ||
| depends_on: | ||
| - zookeeper | ||
| - broker | ||
| - schema-registry | ||
| ports: | ||
| - 8082:8082 | ||
| hostname: rest-proxy | ||
| container_name: rest-proxy | ||
| environment: | ||
| KAFKA_REST_HOST_NAME: rest-proxy | ||
| KAFKA_REST_BOOTSTRAP_SERVERS: 'broker:29092' | ||
| KAFKA_REST_LISTENERS: "http://0.0.0.0:8082" | ||
| KAFKA_REST_SCHEMA_REGISTRY_URL: 'http://schema-registry:8081' |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| { | ||
| "namespace": "example.avro", | ||
| "type": "record", | ||
| "name": "User", | ||
| "fields": [ | ||
| { | ||
| "name": "name", | ||
| "type": "string" | ||
| } | ||
| ] | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| { | ||
| "namespace": "example.avro", | ||
| "type": "record", | ||
| "name": "User", | ||
| "fields": [ | ||
| { | ||
| "name": "name", | ||
| "type": "string" | ||
| }, | ||
| { | ||
| "name": "number", | ||
| "type": [ | ||
| "int", | ||
| "null" | ||
| ] | ||
| } | ||
| ] | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,74 @@ | ||
| *** Settings *** | ||
| Library ConfluentKafkaLibrary | ||
| Library Collections | ||
| Library String | ||
|
|
||
| Suite Setup Starting Test | ||
|
|
||
|
|
||
| *** Test Cases *** | ||
| Avro Producer With Schemas As String Argument | ||
| [Setup] Clear Messages From Thread ${MAIN_THREAD} | ||
| ${value_schema}= Set Variable {"namespace": "example.avro","type": "record","name": "User","fields": [{"name": "name","type": "string"},{"name": "number","type": ["int","null"]}]} | ||
| ${key_schema}= Set Variable {"namespace": "example.avro","type": "record","name": "User","fields": [{"name": "name","type": "string"}]} | ||
| ${producer_id}= Create Producer schema_registry_url=http://127.0.0.1:8081 | ||
| ... value_schema=${value_schema} key_schema=${key_schema} | ||
| ${value}= Create Dictionary name=Robot number=${10} | ||
| Produce group_id=${producer_id} topic=avro_testing1 partition=${0} value=${value} key=${KEY} | ||
| Wait Until Keyword Succeeds 10x 0.5s All Messages Are Delivered ${producer_id} | ||
| Sleep 1s | ||
|
|
||
| ${consumer_group_id}= Create Consumer auto_offset_reset=earliest schema_registry_url=http://127.0.0.1:8081 | ||
| Subscribe Topic group_id=${consumer_group_id} topics=avro_testing1 | ||
| ${messages}= Poll group_id=${consumer_group_id} | ||
| Should Be Equal ${TEST_DATA} ${messages} | ||
| ${thread_messages}= Get Messages From Thread ${MAIN_THREAD} | ||
| Should Be Equal ${TEST_DATA} ${thread_messages} | ||
| [Teardown] Basic Teardown ${consumer_group_id} | ||
|
|
||
| Avro Producer With Path To Schemas | ||
| [Setup] Clear Messages From Thread ${MAIN_THREAD} | ||
| ${value_schema_file_path}= Set Variable examples/schema/producer/ValueSchema.avsc | ||
| ${key_schema_file_path}= Set Variable examples/schema/producer/KeySchema.avsc | ||
| ${producer_id}= Create Producer schema_registry_url=http://127.0.0.1:8081 | ||
| ... value_schema=${value_schema_file_path} key_schema=${key_schema_file_path} | ||
| ${value}= Create Dictionary name=Robot number=${10} | ||
| Produce group_id=${producer_id} topic=avro_testing2 partition=${0} value=${value} key=${KEY} | ||
| Wait Until Keyword Succeeds 10x 0.5s All Messages Are Delivered ${producer_id} | ||
| Sleep 1s | ||
|
|
||
| ${consumer_group_id}= Create Consumer auto_offset_reset=earliest schema_registry_url=http://127.0.0.1:8081 | ||
| Subscribe Topic group_id=${consumer_group_id} topics=avro_testing2 | ||
| ${messages}= Poll group_id=${consumer_group_id} | ||
| Should Be Equal ${TEST_DATA} ${messages} | ||
| ${thread_messages}= Get Messages From Thread ${MAIN_THREAD} | ||
| Should Be Equal ${TEST_DATA} ${thread_messages} | ||
| [Teardown] Basic Teardown ${consumer_group_id} | ||
|
|
||
|
|
||
| *** Keywords *** | ||
| Starting Test | ||
| Set Suite Variable @{TEST_TOPIC} avro_testing1 avro_testing2 | ||
| Set Suite Variable &{KEY} name=testkey | ||
| ${value}= Create Dictionary name=Robot number=${10} | ||
| ${data}= Create List ${value} | ||
| Set Suite Variable ${TEST_DATA} ${data} | ||
|
|
||
| ${thread}= Start Consumer Threaded topics=${TEST_TOPIC} schema_registry_url=http://127.0.0.1:8081 auto_offset_reset=latest | ||
| Set Suite Variable ${MAIN_THREAD} ${thread} | ||
|
|
||
| All Messages Are Delivered | ||
| [Arguments] ${producer_id} | ||
| ${count}= Flush ${producer_id} | ||
| Log Reaming messages to be delivered: ${count} | ||
| Should Be Equal As Integers ${count} 0 | ||
|
|
||
| Basic Teardown | ||
| [Arguments] ${group_id} | ||
| Unsubscribe ${group_id} | ||
| Close Consumer ${group_id} | ||
|
|
||
| Unassign Teardown | ||
| [Arguments] ${group_id} | ||
| Unassign ${group_id} | ||
| Close Consumer ${group_id} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,3 +1,3 @@ | ||
| robotframework==3.1.1 | ||
| confluent-kafka==1.0.0 | ||
| robotframework==3.1.2 | ||
| confluent-kafka==1.2.0 | ||
| uuid==1.30 |
Oops, something went wrong.