Telemetry

The telemetry role allows users to set up iDRAC telemetry support and visualizations.

To initiate telemetry support, fill out the following parameters in omnia/input/telemetry_config.yml:

Name	Description
idrac_telemetry_support boolean Required	Enables iDRAC telemetry support and visualizations. Values * true <- Default * false
slurm_telemetry_support boolean Required	Enables slurm telemetry support and visualizations. Values * true <- Default * false
timescaledb_name string Optional	Postgres DB name with timescale extension is used for storing iDRAC and slurm telemetry metrics. Default values: telemetry_metrics
mysqldb_name string Optional	MySQL DB name used to store IPs and credentials of iDRACs having datacenter license Default values: idrac_telemetrysource_services_db
timezone string Optional	This is the timezone that will be set during provisioning of OS. Accepted values are listed in telemetry/common/files/timezone.txt. Default values: GMT
timescaledb_user string Required	Username used for to authenticate to timescale db. The username must not contain -,, ',". The Length of the username should be at least 2 characters.
timescaledb_password string Required	Password used for to authenticate to timescale db. The username must not contain -,, ',". The Length of the username should be at least 2 characters.
mysqldb_user string Required	Username used for to authenticate to mysql db. The username must not contain -,, ',". The Length of the username should be at least 2 characters.
mysqldb_password string Required	Password used for to authenticate to mysql db. The username must not contain -,, ',". The Length of the username should be at least 2 characters.
mysqldb_root_password string Required	Root password used for to authenticate to mysql db. The username must not contain -,, ',". The Length of the username should be at least 2 characters.
idrac_username string Optional	The username for iDRAC. The username must not contain -,, ',". Required only if idrac_telemetry_support is true.
idrac_password string Optional	The password for iDRAC. The username must not contain -,, ',". Required only if idrac_telemetry_support is true.
grafana_username string Required	The username for grafana UI. The length of username should be at least 5. The username must not contain -,, ',".
grafana_password string Required	The password for grafana UI. The length of username should be at least 5. The username must not contain -,, ',". 'admin' is not an accepted value.
node_password string Optional	Password of manager node. Required only if slurm_telemetry_support is true.

Once control_plane.yml and omnia.yml are executed, run the following commands from omnia/telemetry:

ansible-playbook telemetry.yml -i inventory

Note

The passed inventory should have 3 groups: idrac, manager, compute.

After initiation, new nodes can be added to telemetry by running the following commands from omnia/telemetry:

ansible-playbook add_idrac_node.yml -i inventory

Note

• The passed inventory should have an idrac group.
• telemetry_config.yml is encrypted upon executing telemetry.yml. To edit the file, use ansible-vault edit telemetry_config.yml --vault-password-file .telemetry_vault_key.
• If idrac_telemetry is true while executing telemetry.yml, or while running add_idrac_node.yml, if the inventory passed does not contain an idrac group, idrac telemetry will run on IP’s present under /opt/omnia/provisioned_idrac_inventory of control plane.

Viewing Performance Stats on Grafana

Using Texas Technical University data visualization lab, data polled from iDRAC and Slurm can be processed to generate live graphs. These Graphs can be accessed on the Grafana UI.

Once provision.yml is executed and Grafana is set up, use telemetry.yml to initiate the Graphs. Data polled via Slurm and iDRAC is streamed into internal databases. This data is processed to create the 4 graphs listed below.

Note

This feature only works on Nodes using iDRACs with a datacenter license running a minimum firmware of 4.0.

All your data in a glance

Using the following graphs, data can be visualized to gather correlational information.

• ParallelCoordinates
• SankeyLayout
• SpiralLayout
• PowerMap

Note

The timestamps used for the time metric are based on the timezone set in input/provision_config.yml. In the event of a mismatch between the timezone on the browser being used to access Grafana UI and the timezone in input/provision_config.yml, the time range being used to filter information on the Grafana UI will have to be adjusted per the timezone in input/provision_config.yml.