This repository includes a Docker-based accountability solution based on Sysdig, Librdkafka producer, Kafka and MongoDB. This approach aims to identify the causes that have triggered a set of specific events, thanks to the use of the syscalls run by the monitored system. Features such as being completely decoupled from the monitored system, real-time analysis and optimized querying make this solution an optimal choice when it comes to understanding the root causes of a system's behaviour. Different assessment scenarios have been developed to define the best strategy to reduce the impact of the audit process and logging tasks.
Sysdig (version 0.28.0)
Librdkafka (version 1.7.0)
Zookeeper (version 7.0.1)
Kafka (version 7.0.1)
Kafka-connect (version 7.0.1)
MongoDB (version 5.0.5)
MongoDB Atlas (version 5.0.6 Enterprise)
Docker-compose (version 1.26.0)
Dependencies can be installed with setup.sh. The kernel headers must be installed in the host operating system, before running sysdig.
Host IP must be set in the Docker environment variable BROKER_KAFKA_ADVERTISED_HOST_NAME, defined in .env.
To enable TLS support, self-signed certificates, keystores and truststores can be generated by running the script create-secrets.sh.
To study autonomous systems different from ROS framework, audited processes and syscalls can be specified in settings.lua
ROS Docker image and workspace folder must be created by running init_ros.sh. Calls to loginfo() method should be uncommented in talker.py and in listener.py. ROS execution can be started from ROS folder by running.
docker-compose up
Scenario II. Zookeeper, Kafka broker, Kafka connect, Librdkafka producer with Sysdig and MongoDB (local)
Replace producer.cpp with producer-nossl.cpp to avoid TLS configuration.
MongoDB connection URI value must be assigned to the connection.uri property in MongoSinkConnector.properties from Kafka connect, and in sink-connect.sh for the Kafka-MongoDB connector creation. For this scenario, this value should be equal to
mongodb://root:admin@mongo:27017
The scenario can be deployed by running
docker-compose -f docker-compose-notls.yml up -d
ROS Docker image and workspace folder must be created by running init_ros.sh. Calls to loginfo() method should be commented in talker.py and in listener.py. ROS execution can be started from ROS folder by running.
docker-compose up
Scenario III. Zookeeper, Kafka broker, Kafka connect, Librdkafka producer with Sysdig and Atlas MongoDB
Replace producer.cpp with producer-nossl.cpp to avoid TLS configuration.
MongoDB connection URI value must be assigned to the connection.uri property in MongoSinkConnector.properties from Kafka connect, and in sink-connect.sh for the Kafka-MongoDB connector creation. For this scenario, this value should be equal to
mongodb+srv://root:[email protected]
The scenario can be deployed by running
docker-compose -f docker-compose-notls-atlas.yml up -d
ROS Docker image and workspace folder must be created by running init_ros.sh. Calls to loginfo() method should be commented in talker.py and in listener.py. ROS execution can be started from ROS folder by running.
docker-compose up
Scenario IV. Zookeeper, Kafka broker, Kafka connect, Librdkafka producer with Sysdig and MongoDB (local) with TLSv1.3
MongoDB connection URI value must be assigned to the connection.uri property in MongoSinkConnector.properties from Kafka connect, and in sink-connect.sh for the Kafka-MongoDB connector creation. For this scenario, this value should be equal to
mongodb://root:admin@mongo:27017/admin?ssl=true
The scenario can be deployed by running
docker-compose -f docker-compose-tls.yml up -d
ROS Docker image and workspace folder must be created by running init_ros.sh. Calls to loginfo() method should be commented in talker.py and in listener.py. ROS execution can be started from ROS folder by running.
docker-compose up
Scenario V. Zookeeper, Kafka broker, Kafka connect, Librdkafka producer with Sysdig and Atlas MongoDB with TLSv1.3
MongoDB connection URI value must be assigned to the connection.uri property in MongoSinkConnector.properties from Kafka connect, and in sink-connect.sh for the Kafka-MongoDB connector creation. For this scenario, this value should be equal to
mongodb+srv://root:[email protected]/admin?ssl=true
The scenario can be deployed by running
docker-compose -f docker-compose-tls-atlas.yml up -d
ROS Docker image and workspace folder must be created by running init_ros.sh. Calls to loginfo() method should be commented in talker.py and in listener.py. ROS execution can be started from ROS folder by running.
docker-compose up
DMARCE (EDMAR+CASCAR) Project: EDMAR PID2021-126592OB-C21 -- CASCAR PID2021-126592OB-C22 funded by MCIN/AEI/10.13039/501100011033 and by ERDF A way of making Europe
