Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support loading external kernels #256

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Conversation

slp
Copy link
Contributor

@slp slp commented Jan 21, 2025

This PR makes libkrunfw optional (by dynamically loading it) and adds
support for loading external kernels from multiple image formats. The
ones currently implemented are:

  • ELF: A kernel binary in ELF format (vmlinux).
  • PeGz: A PE binary embedding a kernel image compressed with GZIP.
  • ImageBz2: An Image file embedding a kernel compressed with BZIP2.
  • ImageGz: An Image file embedding a kernel compressed with GZIP.
  • ImageZstd: An Image file embedding a kernel compressed with ZSTD.

Adding new kernel formats should be quite straightforward.

Please note this change doesn't implement support for loading an
external initramfs. The main reason is that we can't guarantee to
maintain the control of the VM boot when using an arbitrary
initramfs.

This means that the external kernel must be built with, at least,
the following driver built-in:

  • virtio-mmio
  • virtio-console
  • virtio-fs

Depending on the use case, more drivers might be required.

Instead of linking statically against libkrunfw, load it as a dynamic
library. This will enable us to make its presence optional, which will
come handy when we support loading external payloads.

Signed-off-by: Sergio Lopez <[email protected]>
@slp slp force-pushed the external-kernel branch 6 times, most recently from 4707ac6 to 9e362bd Compare January 21, 2025 16:00
Introduce initial support for external kernels by dealing with the
easier case: raw images on aarch64.

This commit adds a new function, "krun_set_kernel", which receives a
path to the external kernel.

Future commits will add support for more image formats and x86_64.

Signed-off-by: Sergio Lopez <[email protected]>
@slp slp force-pushed the external-kernel branch from 9e362bd to 95e72d6 Compare January 21, 2025 16:07
@sbrivio-rh
Copy link

Funny, I was trying (very much in the background) to do something like this, but making libkrunfw load kernel images instead. My use case is testing kernels and especially userspace tools (passt, seitan, nft) against different kernels/kernel changes.

This looks way simpler and it looks like you're almost done (or at least much closer than I was), so I'm relieved. :)

@slp
Copy link
Contributor Author

slp commented Jan 23, 2025

Funny, I was trying (very much in the background) to do something like this, but making libkrunfw load kernel images instead. My use case is testing kernels and especially userspace tools (passt, seitan, nft) against different kernels/kernel changes.

This looks way simpler and it looks like you're almost done (or at least much closer than I was), so I'm relieved. :)

Please note that we don't intend to support an external initramfs, at least for the moment. This implies you still need a custom kernel config that has, at least, virtio-mmio, virtio-fs and virtio-console built-in.

@sbrivio-rh
Copy link

Please note that we don't intend to support an external initramfs, at least for the moment.

Yeah, I would actually like to get rid of the initramfs in my current workflow (based on mbuto), for simplicity.

This implies you still need a custom kernel config that has, at least, virtio-mmio, virtio-fs and virtio-console built-in.

Custom configuration, sure, but not patched, right?

@slp
Copy link
Contributor Author

slp commented Jan 23, 2025

This implies you still need a custom kernel config that has, at least, virtio-mmio, virtio-fs and virtio-console built-in.

Custom configuration, sure, but not patched, right?

Nope, no downstream patches are required, unless you want to use TSI, which I'm pretty sure you don't ;-P

@slp slp force-pushed the external-kernel branch 5 times, most recently from d62666b to c242bfd Compare January 24, 2025 12:47
@slp slp marked this pull request as ready for review January 24, 2025 12:59
@slp slp force-pushed the external-kernel branch from c242bfd to 15c336b Compare January 24, 2025 16:11
#[allow(clippy::missing_safety_doc)]
#[no_mangle]
pub unsafe extern "C" fn krun_set_kernel(_ctx_id: u32, _c_kernel_path: *const c_char) -> i32 {
-libc::EOPNOTSUPP
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's holding us back from supporting krun_set_kernel with the tee feature?

Copy link
Contributor Author

@slp slp Jan 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We still need custom kernel (at least for SEV/SNP, not so sure about TDX), a custom qboot (for the initial memory initialization), and a custom init (for bundling an agent and opening the LUKS volume).

We should get rid of those custom components. A possible plan for that could be:

  • Adopt OVMF/EDK2 as FW. It would need to be a custom built, since we aren't 100% compatible with QEMU, but would have the facilities required to boot an unmodified kernel in a TEE.
  • Find a way to generate an initramfs on the fly in a reproducible way, so it can be part of the attestation envelope.

Both tasks seem doable, but require a significant amount of effort. Once we get there, we'll be able to support external kernels with the TEE flavor.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Find a way to generate an initramfs on the fly in a reproducible way, so it can be part of the attestation envelope.

Can you expand on this a bit more?

In the previous commit we added support for the simplest type of
external kernel, a raw image that can be directly copied into the
VM's memory.

This commit builds on that to add support for multiple kernel
formats. The ones currently implemented are:

 - ELF: A kernel binary in ELF format (vmlinux).
 - PeGz: A PE binary embedding a kernel image compressed with GZIP.
 - ImageBz2: An Image file embedding a kernel compressed with BZIP2.
 - ImageGz: An Image file embedding a kernel compressed with GZIP.
 - ImageZstd: An Image file embedding a kernel compressed with ZSTD.

Adding new kernel formats should be quite straightforward.

Please note this change doesn't implement support for loading an
external initramfs. The main reason is that we can't guarantee to
maintain the control of the VM boot when using an arbitrary
initramfs.

This means that the external kernel must be built with, at least,
the following driver built-in:

- virtio-mmio
- virtio-console
- virtio-fs

Depending on the use case, more drivers might be required.

Signed-off-by: Sergio Lopez <[email protected]>
@slp slp force-pushed the external-kernel branch from 15c336b to 16adb4f Compare January 27, 2025 11:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants