Set correct SELinux context options on container & VM mounts

[stgraber]

Per https://discuss.linuxcontainers.org/t/unable-to-add-lxc-rootfs-mount-options-context/20943/14 we should try and at least set the correct context mount options whenever we mount a container or VM data directory.

Ideally we'd have something more robust where we can handle per-instance labeling but that would require someone with more SELinux knowledge than I have to get this done properly, put testing in place and keep an eye on it so it doesn't regress :)

So for now, we'll try to go for low hanging fruits and just make things better where feasible.

[mschiff]

Hi Stéphane,

in the meantime I have developed a (mostly) working SELinux-Module for incus! Getting containers and VMs running on a SELinux system is not yet straight forward, but it is doable and working with one major issue which I am not able to solve without some basic incus support here.

It would we great to have some very basic support for this in incus as it would make running qemu VMs much better:

The missing piece is the SELinux-labels of all related files for a specific VM.

A possibility to pass a context to incus which it will use to label VM specific dirs before creating supporting files and starting a VM would eliminate this last big roadblock.

Currently its like that:

after creating/starting a VM I get:

(sysadm_r)@host ~ # ls -lZ /var/lib/incus/devices/vm1/
dr-x------. 6 nobody root system_u:object_r:unlabeled_t:s0 14  6. Dez 17:59 config.mount

(sysadm_r)@host ~ # ls -lZ /var/lib/incus/devices/vm1/config.mount/
-r--------. 1 nobody root system_u:object_r:unlabeled_t:s0        2  6. Dez 17:59 agent-mounts.json
-rw-r--r--. 1 nobody root system_u:object_r:unlabeled_t:s0      819  6. Dez 17:59 agent.conf
-r--------. 1 nobody root system_u:object_r:unlabeled_t:s0      725  6. Dez 17:59 agent.crt
-r--------. 1 nobody root system_u:object_r:unlabeled_t:s0      288  6. Dez 17:59 agent.key
dr-x------. 2 nobody root system_u:object_r:unlabeled_t:s0        3  6. Dez 17:59 files
-r-x------. 1 nobody root system_u:object_r:unlabeled_t:s0 21721508 19. Nov 14:24 incus-agent
-rwx------. 1 nobody root system_u:object_r:unlabeled_t:s0     1302  6. Dez 17:59 install.sh
lrwxrwxrwx. 1 root   root system_u:object_r:unlabeled_t:s0       11  6. Dez 17:59 lxd-agent
dr-x------. 2 nobody root system_u:object_r:unlabeled_t:s0        2  6. Dez 17:59 nics
-r--------. 1 nobody root system_u:object_r:unlabeled_t:s0      688  6. Dez 17:59 server.crt
dr-x------. 2 nobody root system_u:object_r:unlabeled_t:s0        4  6. Dez 17:59 systemd
dr-x------. 2 nobody root system_u:object_r:unlabeled_t:s0        3  6. Dez 17:59 udev

(sysadm_r)@host ~ # ls -lZ /var/lib/incus/devices/vm1/config.mount/
-r--------. 1 nobody root system_u:object_r:unlabeled_t:s0        2  6. Dez 17:59 agent-mounts.json
-rw-r--r--. 1 nobody root system_u:object_r:unlabeled_t:s0      819  6. Dez 17:59 agent.conf
-r--------. 1 nobody root system_u:object_r:unlabeled_t:s0      725  6. Dez 17:59 agent.crt
-r--------. 1 nobody root system_u:object_r:unlabeled_t:s0      288  6. Dez 17:59 agent.key
dr-x------. 2 nobody root system_u:object_r:unlabeled_t:s0        3  6. Dez 17:59 files
-r-x------. 1 nobody root system_u:object_r:unlabeled_t:s0 21721508 19. Nov 14:24 incus-agent
-rwx------. 1 nobody root system_u:object_r:unlabeled_t:s0     1302  6. Dez 17:59 install.sh
lrwxrwxrwx. 1 root   root system_u:object_r:unlabeled_t:s0       11  6. Dez 17:59 lxd-agent
dr-x------. 2 nobody root system_u:object_r:unlabeled_t:s0        2  6. Dez 17:59 nics
-r--------. 1 nobody root system_u:object_r:unlabeled_t:s0      688  6. Dez 17:59 server.crt
dr-x------. 2 nobody root system_u:object_r:unlabeled_t:s0        4  6. Dez 17:59 systemd
dr-x------. 2 nobody root system_u:object_r:unlabeled_t:s0        3  6. Dez 17:59 udev

The unlabeled_t is the problem here and could be solved by labeleing the root node of the filesystem before creating files and dirs inside.

A solution would be if incusd labeled the root dir of the mount /var/lib/incus/devices/vm1/config.mount to system_u:object_r:container_var_lib_t:s0 after mounting and before creating the files inside.

Or is there any hook I could use for that or maybe one can be added so I could specify the command to use myself?

Many thanks in advance for considering.

-Marc

[stgraber]

We got some funding coming in now to tackle the SELinux issue and try to label things correctly for both containers and VMs. Ideally we'd want Incus to automatically:

• Detect that it's on a SELinux system
• Detect the correct set of labels on that system for:
  • container rootfs
  • container config
  • VM disks
  • VM config
• Apply correct labels on instance creation

I believe you gave us all the right pointers for the labels to use in the forum thread, so it's mostly going to be a matter of getting a test system up and running and then iterate on the logic until we get it behaving.

If you could provide instructions on how to set up a suitable SELinux test environment in a VM, including your new SELinux module, that'd be pretty useful.

PS: The funding I mentioned is aimed for the Incus project at large and can be used to sub-contract some part of this work, so if you have time, interest and the ability to take on some contract work, we may be able to compensate some of your work on this.

[mschiff]

We got some funding coming in now to tackle the SELinux issue and try to label things correctly for both containers and VMs.

This is GREAT news ;-)

Ideally we'd want Incus to automatically:

* Detect that it's on a SELinux system

* Detect the correct set of labels on that system for:
  
  * container rootfs
  * container config
  * VM disks
  * VM config

* Apply correct labels on instance creation

For best security the SELinux MLS/MCS Policy is being used most of the time. Then every container is running in the same domain (with same process label) but with a different category. That way the containers are segregated from each other.

One way to achive that is if the user sets a different category for each container. If containers need to share data, they need to share at least one category (You can specifiy single multiple categories per label). This is what I do now by adding something like

  raw.lxc: |
    lxc.selinux.context = system_u:system_r:container_t:s0:c100

to the container config (c100 being the category)

The other (more automatic) way would be, that a user can enable "automatic container segregation" and then incusd would use random categories for each container. Propably in the for :cX,cY and X being 512-1024 and cY being 0-1024, so there still is room fixed cats.

I believe you gave us all the right pointers for the labels to use in the forum thread, so it's mostly going to be a matter of getting a test system up and running and then iterate on the logic until we get it behaving.

Sounds good!

If you could provide instructions on how to set up a suitable SELinux test environment in a VM, including your new SELinux module, that'd be pretty useful.

My module is currently in "PR pending state". It is usable already, but still needs some tweaks. For example currently all containers are runnning in the "spc_t" type which is a "super privileged container". With this, MCS categories will not be enforced. The correct type will be "container_t".

What you basically need:

• Gentoo system with SELinux enabled hardened profile, using systemd. -> default/linux/amd64/23.0/hardened/selinux/systemd ( https://www.gentoo.org/downloads/#amd64-advanced -> "H/SELinux stage 3" (this uses openrc though, which is great and maybe not really relevant for developing but you can switch to systemd after installing this)
• a git clone of the hardened-refpolicy git repo -> https://github.com/gentoo/hardened-refpolicy
• my module files (incus.fc, incus.if, incus.te) -> Add new incus module, add required rules to other modules gentoo/hardened-refpolicy#6

I might also work with a Red Hat based system, which will be easier to setup, but I did not test the module on redhat yet, although, the types being used etc. should be the same. That being said, there are interfaces in the policy that might differ and so in a future version there might be some conditionals in the module based on target distro.

I am happy to help setting up such an envirenment.

PS: The funding I mentioned is aimed for the Incus project at large and can be used to sub-contract some part of this work, so if you have time, interest and the ability to take on some contract work, we may be able to compensate some of your work on this.

Thank you for this! I woud be very interested, but unfortunately I am fully "booked" and cannot take more projects at the moment :-/