[#19] Add functionality for:

- handle commands and configs - use pubsub as internal message - updating db for device configs Signed-off-by: Georgios Dimitropoulos <[email protected]>
sotec-iot · Aug 9, 2023 · c97a44d · c97a44d
1 parent c01bccb
commit c97a44d
Show file tree

Hide file tree

Showing 32 changed files with 1,818 additions and 369 deletions.
diff --git a/device-communication/README.md b/device-communication/README.md
@@ -1,6 +1,6 @@
 # Device Communication API
 
-Device communication API enables users and applications to send configurations and commands to devices via HTTP
+Device communication API enables users and applications to send configurations and commands to devices via HTTP(S)
 endpoints.
 
 ![img.png](img.png)
@@ -18,10 +18,14 @@ router.
 
 #### commands/{tenantId}/{deviceId}
 
-- POST : post a command for a specific device (NOT IMPLEMENTED YET)
+- POST : post a command for a specific device
 
 <p>
 
+#### states/{tenantId}/{deviceId}?numStates=(int 0 - 10)
+
+- GET : list of device states
+
 #### configs/{tenantId}/{deviceId}?numVersion=(int 0 - 10)
 
 - GET : list of device config versions
@@ -30,24 +34,98 @@ router.
 
 For more information please see resources/api/openApi file.
 
+## Pub/Sub - Internal Messaging
+
+API communicates with hono components via the internal messaging interface (implemented from Google's PubSub).
+All the settings for the InternalMessaging component are in the application.yaml file. By publish/subscribe to a topic
+application sends or expects some message attributes.
+
+### Events
+
+API will subscribe to all tenants' event topic at startup.
+
+Expected message Attributes:
+
+- deviceId
+- tenantId
+- content-type
+
+Application will <b>proceed only empty Notifications events (content-type is
+application/vnd.eclipse-hono-empty-notification)</b>.
+
+### States
+
+API will subscribe to all tenants' state topic at startup.
+
+Expected message Attributes:
+
+- deviceId
+- tenantId
+
+States are read only.
+
+### Configs
+
+Application will publish the latest device configuration when:
+
+- an empty Notifications event was received
+- a new device config was created
+
+Message will be published with the following attributes:
+
+- deviceId
+- tenantId
+- config-version
+
+The Body will be a JSON object with the device config object.
+
+After publishing device configs, application subscribes to config_response topic and waits for the device to ack the
+configs.
+
+### Config ACK
+
+Expected message attributes:
+
+- deviceId
+- tenantId
+- configVersion (the config version received from device)
+
+If configVersion is not set, application will ack always the latest config.
+
+### Commands
+
+A command will be published from API to the command topic.
+
+Attributes:
+
+- deviceId
+- tenantId
+- subject (always set to "command")
+
+Body:
+
+The command as string.
+
 ## Database
 
-Application uses PostgreSQL database. All the database configurations can be found in application.yaml file.
+Application uses PostgresSQL database. All the database configurations can be found in application.yaml file.
 
 ### Tables
 
-- DeviceConfig <br>
+- device_configs <br>
   Is used for saving device config versions
-- DeviceRegistration <br>
+- device_registrations <br>
   Is used for validating if a device exist
+- device_status <br>
+  Is used for saving device states
 
 ### Migrations
 
-When Application starts, tables will be created by the DatabaseSchemaCreator service.
+When Applications starts tables will be created by the DatabaseSchemaCreator service.
 
-### Running PostgreSQL container locally
+### Running postgresSQL container local
 
-For running the PostgreSQL Database locally with docker, run:
+For running the PostgresSQL Database local with docker run:
 
 ``````
 
@@ -58,14 +136,14 @@ docker run -p 5432:5432 --name some-postgres -e POSTGRES_PASSWORD=mysecretpasswo
 After the container is running, log in to the container and with psql create the database. Then we have
 to set the application settings.
 
-Default PostgreSQL values:
+Default postgresSQl values:
 
 - userName = postgres
 - password = mysecretpassword
 
 ## Build and Push API Docker Image
 
-Mavens auto build and push functionality can be enabled from application.yaml settings:
+Mavens auto build and push functionality ca be enabled from application.yaml settings:
 
 ````
 
@@ -74,17 +152,19 @@ quarkus:
   builder: docker
   build: true
   push: true
-  image: "<registry>/<organization>/hono-device-communication"
+  image: "gcr.io/sotec-iot-core-dev/hono-device-communication"
 
 ````
 
-By running maven package, install or deploy, this will automatically build the docker image and if push is enabled it will
-push the image to the given registry.
+By running maven package, install or deploy will automatically build the docker image and if push is enabled it will
+push the image
+to the given registry.
 
 ## OpenApi Contract-first
 
 For creating the endpoints, Vertx takes the openApi definition file and maps every endpoint operation-ID with a specific
-Handler function.
+Handler
+function.
 
 ## Handlers
 
@@ -99,4 +179,22 @@ Adding new Endpoint steps:
 2. Use an existing const Class or create a new one under /config and set the operation id name
 3. Implement an HttpEndpointHandler and set the Routes
 
+## PubSub Events
+
+Application subscribes and uses to the following topics:
+
+1. TENANT_ID.command
+2. TENANT_ID.command_response
+3. TENANT_ID.event
+4. TENANT_ID.event.state
+5. registry-tenant.notification
+
+## Automatically create PubSub topics and subscriptions
+
+Application creates all tenants topics and subscriptions when:
+
+1. Application starts if are not exist
+2. New tenant is created
+
+
 
diff --git a/device-communication/pom.xml b/device-communication/pom.xml
@@ -7,17 +7,24 @@
     <artifactId>device-communication</artifactId>
     <version>1.0-SNAPSHOT</version>
     <properties>
-        <compiler-plugin.version>3.10.1</compiler-plugin.version>
+        <compiler-plugin.version>3.10.0</compiler-plugin.version>
         <maven.compiler.release>17</maven.compiler.release>
         <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
         <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
         <quarkus.platform.artifact-id>quarkus-bom</quarkus.platform.artifact-id>
         <quarkus.platform.group-id>io.quarkus.platform</quarkus.platform.group-id>
         <quarkus.platform.version>2.15.3.Final</quarkus.platform.version>
         <skipITs>true</skipITs>
-        <surefire-plugin.version>3.0.0-M7</surefire-plugin.version>
+        <surefire-plugin.version>3.0.0-M9</surefire-plugin.version>
         <org.mapstruct.version>1.5.3.Final</org.mapstruct.version>
         <hono.version>2.1.0</hono.version>
+        <vertx-pg-client.version>4.3.7</vertx-pg-client.version>
+        <jakarta.persistence-api.version>2.2.3</jakarta.persistence-api.version>
+        <com.ongres.scram.client.version>2.1</com.ongres.scram.client.version>
+        <quarkus-container-image-docker.version>2.16.0.Final</quarkus-container-image-docker.version>
+        <quarkus-google-cloud-pubsub.version>1.3.0</quarkus-google-cloud-pubsub.version>
+        <maven-surefire-plugin.version>3.0.0-M9</maven-surefire-plugin.version>
+        <com.google.guava.version>31.1-jre</com.google.guava.version>
     </properties>
     <dependencyManagement>
         <dependencies>
@@ -50,10 +57,6 @@
             <groupId>io.quarkus</groupId>
             <artifactId>quarkus-vertx</artifactId>
         </dependency>
-        <dependency>
-            <groupId>io.vertx</groupId>
-            <artifactId>vertx-core</artifactId>
-        </dependency>
         <dependency>
             <groupId>io.vertx</groupId>
             <artifactId>vertx-web</artifactId>
@@ -87,17 +90,17 @@
         <dependency>
             <groupId>io.vertx</groupId>
             <artifactId>vertx-pg-client</artifactId>
-            <version>4.3.7</version>
+            <version>${vertx-pg-client.version}</version>
         </dependency>
         <dependency>
             <groupId>jakarta.persistence</groupId>
             <artifactId>jakarta.persistence-api</artifactId>
-            <version>2.2.3</version>
+            <version>${jakarta.persistence-api.version}</version>
         </dependency>
         <dependency>
             <groupId>com.ongres.scram</groupId>
             <artifactId>client</artifactId>
-            <version>2.1</version>
+            <version>${com.ongres.scram.client.version}</version>
         </dependency>
         <!-- https://mvnrepository.com/artifact/io.vertx/vertx-sql-client-templates -->
         <dependency>
@@ -115,13 +118,58 @@
         <dependency>
             <groupId>io.quarkus</groupId>
             <artifactId>quarkus-container-image-docker</artifactId>
-            <version>2.16.0.Final</version>
+            <version>${quarkus-container-image-docker.version}</version>
         </dependency>
         <dependency>
             <groupId>io.vertx</groupId>
             <artifactId>vertx-health-check</artifactId>
         </dependency>
+        <!-- https://mvnrepository.com/artifact/io.quarkiverse.googlecloudservices/quarkus-google-cloud-pubsub -->
+        <dependency>
+            <groupId>io.quarkiverse.googlecloudservices</groupId>
+            <artifactId>quarkus-google-cloud-pubsub</artifactId>
+            <version>${quarkus-google-cloud-pubsub.version}</version>
+        </dependency>
+
+        <dependency>
+            <groupId>org.apache.maven.plugins</groupId>
+            <artifactId>maven-surefire-plugin</artifactId>
+            <version>${maven-surefire-plugin.version}</version>
+            <type>maven-plugin</type>
+        </dependency>
+
+        <dependency>
+            <groupId>com.google.guava</groupId>
+            <artifactId>guava</artifactId>
+            <version>${com.google.guava.version}</version>
+        </dependency>
+        <dependency>
+            <groupId>io.vertx</groupId>
+            <artifactId>vertx-core</artifactId>
+            <version>4.3.7</version>
+        </dependency>
 
+        <dependency>
+            <groupId>org.eclipse.hono</groupId>
+            <artifactId>hono-core</artifactId>
+            <version>2.4.0-SNAPSHOT</version>
+        </dependency>
+        <dependency>
+            <groupId>org.eclipse.hono</groupId>
+            <artifactId>hono-client-pubsub-common</artifactId>
+            <version>2.4.0-SNAPSHOT</version>
+        </dependency>
+        <dependency>
+            <groupId>org.eclipse.hono</groupId>
+            <artifactId>hono-client-notification</artifactId>
+            <version>2.4.0-SNAPSHOT</version>
+        </dependency>
+        <dependency>
+            <groupId>commons-codec</groupId>
+            <artifactId>commons-codec</artifactId>
+            <version>1.15</version>
+        </dependency>
+
 
     </dependencies>
     <build>

diff --git a/device-communication/src/main/docker/Dockerfile.legacy-jar b/device-communication/src/main/docker/Dockerfile.legacy-jar
@@ -81,6 +81,7 @@ ENV LANGUAGE='en_US:en'
 
 RUN mkdir api/
 RUN mkdir db/
+RUN mkdir creds/
 
 COPY target/classes/api/* /api/
 COPY target/classes/db/* /db/

diff --git a/...ation/src/main/java/org/eclipse/hono/communication/api/config/DeviceCommandConstants.java b/...ation/src/main/java/org/eclipse/hono/communication/api/config/DeviceCommandConstants.java
@@ -27,6 +27,5 @@ public final class DeviceCommandConstants {
     public static final String POST_DEVICE_COMMAND_OP_ID = "postCommand";
 
     private DeviceCommandConstants() {
-        // avoid instantiation
     }
 }
diff --git a/...ommunication/src/main/java/org/eclipse/hono/communication/api/config/PubSubConstants.java b/...ommunication/src/main/java/org/eclipse/hono/communication/api/config/PubSubConstants.java
@@ -0,0 +1,48 @@
+/*
+ * ***********************************************************
+ *  Copyright (c) 2023 Contributors to the Eclipse Foundation
+ *  <p>
+ *  See the NOTICE file(s) distributed with this work for additional
+ *  information regarding copyright ownership.
+ *  <p>
+ *  This program and the accompanying materials are made available under the
+ *  terms of the Eclipse Public License 2.0 which is available at
+ *  http://www.eclipse.org/legal/epl-2.0
+ *  <p>
+ *  SPDX-License-Identifier: EPL-2.0
+ * **********************************************************
+ *
+ */
+
+package org.eclipse.hono.communication.api.config;
+
+import java.util.List;
+
+/**
+ * Constant values for PubSub.
+ */
+public final class PubSubConstants {
+
+    public static final String TENANT_NOTIFICATIONS = "registry-tenant.notification";
+    public static final String TELEMETRY_ENDPOINT = "telemetry";
+    public static final String EVENT_ENDPOINT = "event";
+    public static final String EVENT_STATES_SUBTOPIC_ENDPOINT = "event.state";
+    public static final String COMMAND_ENDPOINT = "command";
+    public static final String COMMAND_RESPONSE_ENDPOINT = "command_response";
+
+    private PubSubConstants() {
+    }
+
+    /**
+     * Gets the list of all topics need to be created per tenant.
+     *
+     * @return List of all topics.
+     */
+    public static List<String> getTenantTopics() {
+        return List.of(EVENT_ENDPOINT,
+                COMMAND_ENDPOINT,
+                COMMAND_RESPONSE_ENDPOINT,
+                EVENT_STATES_SUBTOPIC_ENDPOINT,
+                TELEMETRY_ENDPOINT);
+    }
+}