Version 2: Changing from CLI arguments to a config.ini file

[robertsinfosec]

Hello ProxMoxians!

Background

This started off as a simple project to solve an annoying problem: how to configure your ProxMox server to use cloneable cloud images. With ideas people have had and with Pull Requests, there are:

• Things that are universally valuable to everyone. For example, supporting more distributions or accounting for a ProxMox bug.
• Personal preferences. For example, how much RAM or how many vCPU's the templates should have, or what software should be pre-installed on the image.

Problem

Until now, I've pretty much used my own personal preferences dictate what is in this repo, because that is what we use in our lab. However, that is not convenient if you have different preferences or requirements for YOUR lab.

Solution

One way to solve this is to move everything we can from being hard-coded in the bash scripts, and moving them into a config.ini file for example, in the root of each OS folder. As an example, imagine (to start) a /ubuntu/config.ini file with something like:

[VM]
CORES=4
MEMORY=4096

[User]
USERNAME=admin
PASSWORD=pass123

And then in the bash script, something like:

function get_ini_value {
    local section=$1
    local key=$2
    local default_value=$3
    local value=$(awk -F'=' -v section="[$section]" -v key="$key" '
        $1 == section {in_section = true}
        in_section && $1 ~ key {print $2; exit}
        /^\[/ && $1 != section {in_section = false}
    ' config.ini)
    echo "${value:-$default_value}"
}

CORES=$(get_ini_value "VM" "CORES" "1")
MEMORY=$(get_ini_value "VM" "MEMORY" "512")

echo "Cores: $CORES, Memory: $MEMORY"

Similarly, for pre-installed software, maybe there is a section in the INI file like this:

[Software]
PACKAGES=qemu-guest-agent,watchdog,figlet,neofetch,ufw,fail2ban

That way, you could specify whatever software you want pre-installed on the images in YOUR environment.

Planning

Similar to me, I assume there are people that use the scripts as-is and are happy with them. I want to make sure not to unnecessarily break anyone's setup. So, my thought was to make a v1 Release/branch/tag, so that bug-fixes could continue on. Then, this new approach would live-on in the master branch.

Anyone have any opinions about that?

Otherwise, I was thinking that as time permits, I might create a v2 branch, and start retro-fitting these scripts and documentation to start using config files instead of command-line arguments. Once working, completed, tested, and documented, we pick a day, carve-off the v1 version as a release/tag, and then deliver these new changes to master as the new way forward.

Any thoughts on any of this? Please let me know - thanks!

[slazarov]

Hi Rob, another idea is to also add an option to copy ssh keys from a local file?

[robertsinfosec]

Great idea!! I wrote up a feature request here with some starting ideas - feel free to add to it: #14

[aduzsardi]

wouldn't it make sense to just use libguestfs to modify your template to your liking without needing cloud-init at all ?
i just did that with a vanilla cloud image and it works a treat ... added a few packages , removed a few packages including cloud-init , added a default user , configured authorized_keys for the user , a few changes to sshd_config , set network interface to dhcp and other tweaks.

then i imported the image in proxmox , and the rest of the configuration if needed are made either manually or some configuration management tool like ansible,chef,salt etc

[aduzsardi]

Note: it is true that i'm the sole system administrator with ssh access to the VMs created from the template

[robertsinfosec]

In what way is this different? With this repo, on a new proxmox server, I clone this repo, copy and modify the various "_rebuild..." scripts with my settings, and then kick off main script. I come back in :20 minutes and I have all of the cloud images production-ready. Being a fundamentally lazy person, this was the least painful way that I've found to do this.

Can you please expand on your idea?

[aduzsardi]

I was trying to say that you don't really need cloud-init at all to create working templates for proxmox or , you can achieve the same thing using libguestfs tools.
That's all, i don't know if one way is better than the other ... unless you want to allow other people to launch VMs out of your template, then you should use cloud-init for sure.

[robertsinfosec]

UPDATE: 09-May-2024

All,

I dug into this a bit. The more I thought about it, the more complex this is getting. Primarily there are two difficulties:

• I would want the ability to have "defaults" for all templates, and distro-specific defaults too.
• I would want to list all of the templates I want to load, but there is nuance and detail needed for each, which needs to be written down SOMEwhere.

So, as I fleshed this out, I came up with an example JSON data structure like below.

JSON Format

See below for the JSON format example.

Click to expand

{
    "defaults": {
        "storage": "HDD_01A",
        "user": "sysadmin",
        "password": "P4zzw0rd!",
        "searchdomain": "lab.example.com",
        "sshkeyid": "jdoe",
        "preinstalled_software": [
            "qemu-guest-agent",
            "watchdog"
        ]
    },
    "distros": [
        {
            "name": "AlmaLinux",
            "defaults": {
                "storage": "SSD01",
                "script_path": "./almalinux/",
                "preinstalled_software": [
                    "qemu-guest-agent",
                    "watchdog",
                    "epel-release"
                ]
            },
            "versions": [
                {
                    "distro_version": "8",
                    "vm_template_id": "101800",
                    "vm_template_name": "almalinux-cloud-8",
                    "storage": "SSD02"
                },
                {
                    "distro_version": "9",
                    "vm_template_id": "101900",
                    "vm_template_name": "almalinux-cloud-9",
                    "storage": "SSD02"
                }
            ]
        },
        {
            "name": "CentOS",
            "defaults": {
                "storage": "SSD01",
                "script_path": "./centos/",
                "preinstalled_software": [
                    "qemu-guest-agent",
                    "watchdog",
                    "epel-release"
                ]
            },
            "versions": [
                {
                    "distro_version": "8",
                    "vm_template_id": "102800",
                    "vm_template_name": "centos-cloud-8",
                    "storage": "SSD02"
                },
                {
                    "distro_version": "9",
                    "vm_template_id": "102900",
                    "vm_template_name": "centos-cloud-9",
                    "storage": "SSD02"
                }
            ]
        },
        {
            "name": "Debian",
            "defaults": {
                "storage": "SSD01",
                "script_path": "./debian/",
                "preinstalled_software": [
                    "qemu-guest-agent",
                    "watchdog",
                    "figlet",
                    "neofetch",
                    "ufw",
                ]
            },
            "versions": [
                {
                    "distro_version": "Buster",
                    "distro_name": "10",
                    "vm_template_id": "1110000",
                    "vm_template_name": "debian-cloud-buster",
                    "storage": "SSD01"
                },
                {
                    "distro_version": "Bullseye",
                    "distro_name": "11",
                    "vm_template_id": "1111000",
                    "vm_template_name": "debian-cloud-bullseye",
                    "storage": "SSD01"
                },
                {
                    "distro_version": "Bookworm",
                    "distro_name": "12",
                    "vm_template_id": "1112000",
                    "vm_template_name": "debian-cloud-bookworm",
                    "storage": "SSD01"
                }
            ]
        },
        {
            "name": "openSUSE",
            "defaults": {
                "storage": "SSD01",
                "script_path": "./opensuse/",
                "preinstalled_software": [
                    "qemu-guest-agent",
                    "watchdog",
                    "epel-release"
                ]
            },
            "versions": [
                {
                    "distro_version": "Leap",
                    "distro_name": "15.2",
                    "vm_template_id": "101520",
                    "vm_template_name": "opensuse-cloud-leap15-1",
                    "storage": "SSD01"
                },
                {
                    "distro_version": "Leap",
                    "distro_name": "15.3",
                    "vm_template_id": "101530",
                    "vm_template_name": "opensuse-cloud-leap15-3",
                    "storage": "SSD01"
                },
                {
                    "distro_version": "Leap",
                    "distro_name": "15.4",
                    "vm_template_id": "101540",
                    "vm_template_name": "opensuse-cloud-leap15-4",
                    "storage": "SSD01"
                }
            ]
        },
        {
            "name": "Rocky Linux",
            "defaults": {
                "storage": "SSD01",
                "script_path": "./rockylinux/",
                "preinstalled_software": [
                    "qemu-guest-agent",
                    "watchdog",
                    "epel-release"
                ]
            },
            "versions": [
                {
                    "distro_version": "8",
                    "vm_template_id": "103800",
                    "vm_template_name": "rockylinux-cloud-8",
                    "storage": "SSD02"
                },
                {
                    "distro_version": "9",
                    "vm_template_id": "103900",
                    "vm_template_name": "rockylinux-cloud-9",
                    "storage": "SSD02"
                }
            ]
        },
        {
            "name": "Ubuntu",
            "defaults": {
                "storage": "SSD01",
                "script_path": "./ubuntu/",
                "preinstalled_software": [
                    "qemu-guest-agent",
                    "watchdog",
                    "figlet",
                    "neofetch",
                    "ufw",
                    "fail2ban"
                ]
            },
            "versions": [
                {
                    "distro_version": "24.04",
                    "distro_name": "noble",
                    "vm_template_id": "102404",
                    "vm_template_name": "ubuntu-cloud-noble",
                    "storage": "SSD01"
                },
                {
                    "distro_version": "22.04",
                    "distro_name": "jammy",
                    "vm_template_id": "102204",
                    "vm_template_name": "ubuntu-cloud-jammy",
                    "storage": "SSD01"
                },
                {
                    "distro_version": "20.04",
                    "distro_name": "focal",
                    "vm_template_id": "102004",
                    "vm_template_name": "ubuntu-cloud-focal",
                    "storage": "SSD01"
                },
                {
                    "distro_version": "18.04",
                    "distro_name": "bionic",
                    "vm_template_id": "101804",
                    "vm_template_name": "ubuntu-cloud-bionic",
                    "storage": "SSD01"
                },
                {
                    "distro_version": "16.04",
                    "distro_name": "xenial",
                    "vm_template_id": "101604",
                    "vm_template_name": "ubuntu-cloud-xenial",
                    "storage": "SSD01"
                }
            ]
        }
    ]
}

Which to my eyes is VERY unwieldy and difficult to manage, even though it has the data elements I want.

YAML Format

Alternatively, here is the same data in YAML format:

Click to expand

defaults:
  storage: HDD_01A
  user: sysadmin
  password: P4zzw0rd!
  searchdomain: lab.example.com
  sshkeyid: jdoe
  preinstalled_software:
    - qemu-guest-agent
    - watchdog

distros:
  - name: AlmaLinux
    defaults:
      storage: SSD01
      script_path: ./almalinux/
      preinstalled_software:
        - qemu-guest-agent
        - watchdog
        - epel-release
    versions:
      - distro_version: 8
        vm_template_id: 101800
        vm_template_name: almalinux-cloud-8
        storage: SSD02
      - distro_version: 9
        vm_template_id: 101900
        vm_template_name: almalinux-cloud-9
        storage: SSD02
  - name: CentOS
    defaults:
      storage: SSD01
      script_path: ./centos/
      preinstalled_software:
        - qemu-guest-agent
        - watchdog
        - epel-release
    versions:
      - distro_version: 8
        vm_template_id: 102800
        vm_template_name: centos-cloud-8
        storage: SSD02
      - distro_version: 9
        vm_template_id: 102900
        vm_template_name: centos-cloud-9
        storage: SSD02
  - name: Debian
    defaults:
      storage: SSD01
      script_path: ./debian/
      preinstalled_software:
        - qemu-guest-agent
        - watchdog
        - figlet
        - neofetch
        - ufw
    versions:
      - distro_version: Buster
        distro_name: 10
        vm_template_id: 1110000
        vm_template_name: debian-cloud-buster
        storage: SSD01
      - distro_version: Bullseye
        distro_name: 11
        vm_template_id: 1111000
        vm_template_name: debian-cloud-bullseye
        storage: SSD01
      - distro_version: Bookworm
        distro_name: 12
        vm_template_id: 1112000
        vm_template_name: debian-cloud-bookworm
        storage: SSD01
  - name: openSUSE
    defaults:
      storage: SSD01
      script_path: ./opensuse/
      preinstalled_software:
        - qemu-guest-agent
        - watchdog
        - epel-release
    versions:
      - distro_version: Leap
        distro_name: 15.2
        vm_template_id: 101520
        vm_template_name: opensuse-cloud-leap15-1
        storage: SSD01
      - distro_version: Leap
        distro_name: 15.3
        vm_template_id: 101530
        vm_template_name: opensuse-cloud-leap15-3
        storage: SSD01
      - distro_version: Leap
        distro_name: 15.4
        vm_template_id: 101540
        vm_template_name: opensuse-cloud-leap15-4
        storage: SSD01
  - name: Rocky Linux
    defaults:
      storage: SSD01
      script_path: ./rockylinux/
      preinstalled_software:
        - qemu-guest-agent
        - watchdog
        - epel-release
    versions:
      - distro_version: 8
        vm_template_id: 103800
        vm_template_name: rockylinux-cloud-8
        storage: SSD02
      - distro_version: 9
        vm_template_id: 103900
        vm_template_name: rockylinux-cloud-9
        storage: SSD02
  - name: Ubuntu
    defaults:
      storage: SSD01
      script_path: ./ubuntu/
      preinstalled_software:
        - qemu-guest-agent
        - watchdog
        - figlet
        - neofetch
        - ufw
        - fail2ban
    versions:
      - distro_version: 24.04
        distro_name: noble
        vm_template_id: 102404
        vm_template_name: ubuntu-cloud-noble
        storage: SSD01
      - distro_version: 22.04
        distro_name: jammy
        vm_template_id: 102204
        vm_template_name: ubuntu-cloud-jammy
        storage: SSD01
      - distro_version: 20.04
        distro_name: focal
        vm_template_id: 102004
        vm_template_name: ubuntu-cloud-focal
        storage: SSD01
      - distro_version: 18.04
        distro_name: bionic
        vm_template_id: 101804
        vm_template_name: ubuntu-cloud-bionic
        storage: SSD01
      - distro_version: 16.04
        distro_name: xenial
        vm_template_id: 101604
        vm_template_name: ubuntu-cloud-xenial
        storage: SSD01

INI Format

Some might say "couldn't we just do this as a simple config file"? however this same kind of information is ridiculously difficult to work with in that format. Typical Linux-style config files don't really support the concept of a hierarchy. Everything is just flat. Here's the same data in an INI file-like format:

Click to expand

[defaults]
storage = HDD_01A
user = sysadmin
password = P4zzw0rd!
searchdomain = lab.example.com
sshkeyid = jdoe
preinstalled_software = qemu-guest-agent,watchdog

[AlmaLinux]
storage = SSD01
script_path = ./almalinux/
preinstalled_software = qemu-guest-agent,watchdog,epel-release

[AlmaLinux-8]
vm_template_id = 101800
vm_template_name = almalinux-cloud-8
storage = SSD02

[AlmaLinux-9]
vm_template_id = 101900
vm_template_name = almalinux-cloud-9
storage = SSD02

[CentOS]
storage = SSD01
script_path = ./centos/
preinstalled_software = qemu-guest-agent,watchdog,epel-release

[CentOS-8]
vm_template_id = 102800
vm_template_name = centos-cloud-8
storage = SSD02

[CentOS-9]
vm_template_id = 102900
vm_template_name = centos-cloud-9
storage = SSD02

[Debian]
storage = SSD01
script_path = ./debian/
preinstalled_software = qemu-guest-agent,watchdog,figlet,neofetch,ufw

[Debian-Buster]
distro_name = 10
vm_template_id = 1110000
vm_template_name = debian-cloud-buster
storage = SSD01

[Debian-Bullseye]
distro_name = 11
vm_template_id = 1111000
vm_template_name = debian-cloud-bullseye
storage = SSD01

[Debian-Bookworm]
distro_name = 12
vm_template_id = 1112000
vm_template_name = debian-cloud-bookworm
storage = SSD01

[openSUSE]
storage = SSD01
script_path = ./opensuse/
preinstalled_software = qemu-guest-agent,watchdog,epel-release

[openSUSE-Leap-15.2]
distro_name = 15.2
vm_template_id = 101520
vm_template_name = opensuse-cloud-leap15-1
storage = SSD01

[openSUSE-Leap-15.3]
distro_name = 15.3
vm_template_id = 101530
vm_template_name = opensuse-cloud-leap15-3
storage = SSD01

[openSUSE-Leap-15.4]
distro_name = 15.4
vm_template_id = 101540
vm_template_name = opensuse-cloud-leap15-4
storage = SSD01

[RockyLinux]
storage = SSD01
script_path = ./rockylinux/
preinstalled_software = qemu-guest-agent,watchdog,epel-release

[RockyLinux-8]
vm_template_id = 103800
vm_template_name = rockylinux-cloud-8
storage = SSD02

[RockyLinux-9]
vm_template_id = 103900
vm_template_name = rockylinux-cloud-9
storage = SSD02

[Ubuntu]
storage = SSD01
script_path = ./ubuntu/
preinstalled_software = qemu-guest-agent,watchdog,figlet,neofetch,ufw,fail2ban

[Ubuntu-24.04]
distro_name = noble
vm_template_id = 102404
vm_template_name = ubuntu-cloud-noble
storage = SSD01

[Ubuntu-22.04]
distro_name = jammy
vm_template_id = 102204
vm_template_name = ubuntu-cloud-jammy
storage = SSD01

[Ubuntu-20.04]
distro_name = focal
vm_template_id = 102004
vm_template_name = ubuntu-cloud-focal
storage = SSD01

[Ubuntu-18.04]
distro_name = bionic
vm_template_id = 101804
vm_template_name = ubuntu-cloud-bionic
storage = SSD01

[Ubuntu-16.04]
distro_name = xenial
vm_template_id = 101604
vm_template_name = ubuntu-cloud-xenial
storage = SSD01

Comparing JSON to YAML

I think the YAML is probably the easiest to read. But still:

File Format	Lines of Config
JSON	217
YAML	149
INI	133

Question

Does anyone have any other ideas for file formats, or perhaps a more concise way to store this kind of data? All feedback welcome!