STORY: Digital twin deployment on ZOS 3.0

[despiegk]

to be further defined

deploy using ZOS primitives & RMB

make script in vlang

implementation

• VM primitive
• use flist
• planetary network only first

related issues

• flist: https://github.com/threefoldtech/digitaltwin/issues/270
• integration of the publishingtools server https://github.com/threefoldtech/digitaltwin/issues/269
• #consolidate digital twin hub consolidate (June end) #1062

[xmonader]

as we don't allow custom flists for VMs there will be a longer starting time always in that case.

changed, we support flists since today (-:
we need to create a docker builder or so, to build all needed and create the flist (can use publishtools)

[despiegk]

AM NOT SURE SPECS ARE STILL GOOD, FIRST THING TO DO CREATE SPECS,
KRISTOF TO SIGN OFF

[LennertDefauw1]

Current status

Currently, a PR has been made in the Grid Weblets repository (threefoldtecharchive/grid_weblets#1033). The PR contains a weblet which is supporting FreeFlow to be deployed on top of Grid3.

There are a few parameters needed:

• Virtual Machine name
• 3bot name: to provide a full dns name: eg: lennertapp2.gent01.grid.tf
• Virtual machine capacity + specs
• Gateway node
• Node to deploy on

We are using a gateway component to make the FreeFlow instance more accessible so we can just visit eg: lennertapp2.gent01.grid.tf. The gateway is communicating with the backend of FreeFlow over IPv6.

We decided to communicate over IPv6 because:

• Public IPv4 is overkill since this only extra billing
• The public Yggdrasil IP was not an option: see next section

Gateway communication via Yggdrasil is not possible

Since everything within FreeFlow relies on Yggdrasil locations, gateway communication via Yggdrasil was not an option. All data is stored in a Redis Database instance inside your chat container referenced with a Yggdrasil location (Yggdrasil IPv6 IP)

Eg: I am container A and I want to see the avatar of container B: inside the Redis instance, we keep a list of all contacts.

Important to know is that every chat container exposes its own API; eg: http://[my-yggdrasil-ip]/avatar with a payload of the avatar.

A contact is a key-value pair with the name as the key and a Yggdrasil IP as the value. When we want to retrieve the avatar of container B, we send a curl request to http://[container-b-yggdrasil-ip]/avatar over Yggdrasil to obtain its avatar. In this way, everything within FreeFlow relies on Yggdrasil communication.

Why can't we use the Yggdrasil IP?

Since the location is the most important part of communication. We always want to ensure this location is correct. Therefore, we want to have a static Yggdrasil IP address, since we never want to change a Yggdrasil IP afterward. We use the derivedSeed scope inside the TF Connect Authenticator. This secret key is derived from the main seed inside the ThreeFold Connect app, in combination with the appId of the application: eg demo.freeflow.life.

When the user wants to access its chat container, he has to authenticate first. When the authentication is done, we extract the derivedSeed and create Yggdrasil keys based on that seed. In this way, if the container ever would be deleted, the same Yggdrasil IP will be regenerated since it's based on the derived seed which is always the same for the user. In this way, we can provide a static Yggdrasil location for each user.

But when the Yggdrasil IP got changed based on the derived seed, the gateway is not able to communicate anymore to the backend since it would redirect to the old Yggdrasil IP. That's why we can't use Yggdrasil as communication between the gateway and our FreeFlow backend.

DNS

When deploying a FreeFlow container is deployed, a DNS will be provided based on their 3botname in the following format:

{{3botname}}.{{deployed grid node}}.grid.tf

eg: lennertapp2.gent01.grid.tf

Conclusion

To avoid the cost of a public IPv4 address, and due to the Yggdrasil communication issue: we used IPv6 as the communication protocol. This also requires that the gateway where we want to deploy our container needs to support IPv6. An extra filter is needed.

Deployment

The deployment part contains several parts:

zinit

We are using zinit to launch our startup script, provide OpenSSH with the user their ssh key, ..

qsfs

We will need to mount our data outside the container for updating the environments. eg: a new FreeFlow version is QA validated and is ready to set on production. We don't want the user force to create a new container so we will need to be able to update the existing container. Therefore, we will use zqfs (Still needs to be discussed/implemented)

flist

Since FreeFlow is built with docker, this needs to be converted to a custom flist to deploy the chat container on the grid. Currently, the latest version of the flist is available at my hub.grid.tf profile.

Currently, updating the flist is a manual action by pushing the docker image to docker hub, and converting it to a flist in the UI manually. Maybe this part can still be automated.

Future plans

It would be cool to deploy the 3rd party apps within FreeFlow on your node. Those components would be the following:

• Only-Office document server (for file manager)
• Kutana (for conferences)
• Jimber browser (for safe internetting)

When everything went well in step 1, this could be the next step. (Still has to be discussed)

Extras

When researching the grid client, flists, zinit, etc. I made some scripts deploying FreeFlow containers manually / deleting them which you can find here.