IBM HPCS/Key Protect KMS integration #2723

humblec · 2021-12-20T08:22:09Z

rbd: Implement Key Protect KMS integration for Ceph CSI

This commit adds the support for HPCS/Key Protect IBM KMS service
to Ceph CSI service. EncryptDEK() and DecryptDEK() of RBD volumes are
done with the help of key protect KMS server by wrapping and unwrapping
the DEK and by using the DEKStoreMetadata.

Fixes: # #2585

What is covered:

KMS integration interface
Documentation about the design consideration
Dependency update
Additionally this has been tested against Key Protect service hosted in an IBM cloud and below is the integration test result logs:

   

--------------- Application POD Mounting ------------
...
/pvc/name":"testkp","csi.storage.k8s.io/pvc/namespace":"openshift-storage","encrypted":"true","encryptionKMSID":"kp-metadata-test","imageFeatures":"layering","imageName":"csi-vol-948fe374-5f0d-11ec-a42d-0a580a84001c","journalPool":"ocs-storagecluster-cephblockpool","pool":"ocs-storagecluster-cephblockpool","storage.kubernetes.io/csiProvisionerIdentity":"1639727227436-8081-openshift-storage.rbd.csi.ceph.com","thickProvision":"false"},"volume_id":"0001-0011-...
.......
I1217 07:59:58.080408  257478 crypto.go:199] ID: 6 Req-ID: 0001-0011-openshift-storage-0000000000000001-948fe374-5f0d-11ec-a42d-0a580a84001c Encrypting device "/dev/rbd0"	 with LUKS
I1217 08:00:04.518933  257478 crypto.go:258] ID: 6 Req-ID: 0001-0011-openshift-storage-0000000000000001-948fe374-5f0d-11ec-a42d-0a580a84001c "/dev/mapper/luks-rbd-0001-0011-openshift-storage-0000000000000001-948fe374-5f0d-11ec-a42d-0a580a84001c" is not an active LUKS device (an error (exit status 4) occurred while running cryptsetup args: [status luks-rbd-0001-0011-openshift-storage-0000000000000001-948fe374-5f0d-11ec-a42d-0a580a84001c]):
I1217 08:00:04.518963  257478 crypto.go:210] ID: 6 Req-ID: 0001-0011-openshift-storage-0000000000000001-948fe374-5f0d-11ec-a42d-0a580a84001c Opening device "/dev/rbd0" with LUKS on "luks-rbd-0001-0011-openshift-storage-0000000000000001-948fe374-5f0d-11ec-a42d-0a580a84001c"
..
....



----- Inside Application POD -------------
...
/dev/mapper/luks-rbd-0001-0011-openshift-storage-0000000000000001-948fe374-5f0d-11ec-a42d-0a580a84001c  977M  2.5M  958M   1% /usr/share/nginx/html
tmpfs                                                                                                    15G   20K   15G   1% /run/secrets/kubernetes.io/serviceaccount

# 


----------- Unmounting -----------
...

I1217 08:25:42.631344  257478 crypto.go:232] ID: 68 Req-ID: 0001-0011-openshift-storage-0000000000000001-948fe374-5f0d-11ec-a42d-0a580a84001c Closing LUKS device "luks-rbd-0001-0011-openshift-storage-0000000000000001-948fe374-5f0d-11ec-a42d-0a580a84001c"
.....


------------------------ RBD image metadata which holds the Blob - -------------------------


bash-4.4$ rbd image-meta list -p ocs-storagecluster-cephblockpool csi-vol-6ec9281a-5f07-11ec-8323-0a580a84000f
There are 0 metadata on this image.
bash-4.4$ rbd image-meta list -p ocs-storagecluster-cephblockpool csi-vol-948fe374-5f0d-11ec-a42d-0a580a84001c\ 
> ;
There are 3 metadata on this image:

Key                                 Value                                                                                                                                                                                                                                                                           
rbd.csi.ceph.com/dek                ZXlKamFYQm9aWEowWlhoMElqb2ljRU40V1ZNd09VSjJXakp1TXpKaGMxTlVka1E1ZVU1SGRERlFORFZTU0RkRE5sQjFjVmhFYTJ0MWN6MGlMQ0pwZGlJNklsVm9TRVF5ZW5oQ1pGQk9jRVl5WkM4aUxDSjJaWEp6YVc5dUlqb2lOQzR3TGpBaUxDSm9ZVzVrYkdVaU9pSmpOMkU1WVdFNU1TMDFZMkkxTFRRNFpHRXRZVGd5TVMxbE9EVmpNamRpT1Rsa09USWlmUT09
rbd.csi.ceph.com/encrypted          encrypted                                                                                                                                                                                                                                                                       
rbd.csi.ceph.com/thick-provisioned  false

Signed-off-by: Humble Chirammal [email protected]

docs/design/proposals/encryped-with-keyprotect.md

internal/kms/keyprotect.go

humblec · 2021-12-20T09:56:17Z

@Madhu-1 thanks.. I have addressed the comments. ptal.. thanks 👍

internal/kms/keyprotect.go

examples/kms/vault/kms-config.yaml

nixpanic · 2021-12-20T11:28:24Z

internal/kms/keyprotect.go

+})
+
+// KeyProtectKMS store the KMS connection information retrieved from the kms configmap
+type KeyProtectKMS struct {


keyProtectKMS can probably be lowercase?

All the kms structs are exported atm, so thought of keeping it as exported for now and mark a todo for revisitng all other KMS and its references ..etc.

Where did you mark this TODO?

#2725 @nixpanic this covers it.

internal/kms/keyprotect.go

docs/design/proposals/encryped-with-keyprotect.md

nixpanic · 2021-12-20T11:39:47Z

Please also run make containerized-tests before updating the PR, make sure the simple formatting/linting CI jobs succeed.

humblec · 2021-12-20T13:05:43Z

Please also run make containerized-tests before updating the PR, make sure the simple formatting/linting CI jobs succeed.

yeah. mostly I verify that, but recently , my system started to report out of space before completing the run, so I was skipping it for last few times and kept running only static-check :) .. but will do the cleanup of the system and run it :)

humblec · 2021-12-20T13:09:45Z

@nixpanic comments are addressed.. Thanks.. PTAL.

This commit adds the support for HPCS/Key Protect IBM KMS service to Ceph CSI service. EncryptDEK() and DecryptDEK() of RBD volumes are done with the help of key protect KMS server by wrapping and unwrapping the DEK and by using the DEKStoreMetadata. Signed-off-by: Humble Chirammal <[email protected]>

Signed-off-by: Humble Chirammal <[email protected]>

This commit add the design considerations of IBM Key protect KMS service to the Ceph CSI integration. Signed-off-by: Humble Chirammal <[email protected]>

This commit adds the Key protect client SDK for the Key Protect KMS integration to the Ceph CSI driver. Signed-off-by: Humble Chirammal <[email protected]>

humblec requested a review from nixpanic December 20, 2021 08:22

humblec added this to the release-3.5.0 milestone Dec 20, 2021

humblec added component/rbd Issues related to RBD enhancement New feature or request Priority-0 highest priority issue labels Dec 20, 2021

humblec requested a review from a team December 20, 2021 08:24

humblec force-pushed the kp branch 2 times, most recently from de3585f to 37fe217 Compare December 20, 2021 08:37

humblec requested a review from Madhu-1 December 20, 2021 08:37

humblec force-pushed the kp branch from 37fe217 to 1ee1ff9 Compare December 20, 2021 08:51

Madhu-1 requested changes Dec 20, 2021

View reviewed changes

humblec force-pushed the kp branch from 1ee1ff9 to a6f4335 Compare December 20, 2021 09:51

humblec requested a review from Madhu-1 December 20, 2021 09:56

Madhu-1 reviewed Dec 20, 2021

View reviewed changes

internal/kms/keyprotect.go Outdated Show resolved Hide resolved

internal/kms/keyprotect.go Outdated Show resolved Hide resolved

internal/kms/keyprotect.go Show resolved Hide resolved

humblec force-pushed the kp branch 2 times, most recently from 696a48c to adaca60 Compare December 20, 2021 10:30

nixpanic requested changes Dec 20, 2021

View reviewed changes

humblec force-pushed the kp branch 2 times, most recently from fd9224d to e547a21 Compare December 20, 2021 13:02

humblec requested review from Madhu-1 and nixpanic December 20, 2021 13:09

humblec mentioned this pull request Dec 20, 2021

Reverify all the KMS struct exports and unexport if not needed #2725

Closed

nixpanic requested a review from a team December 20, 2021 15:28

Madhu-1 approved these changes Dec 20, 2021

View reviewed changes

nixpanic approved these changes Dec 21, 2021

View reviewed changes

humblec added 3 commits December 21, 2021 14:53

rbd: add kms unit test for key protect server

a993d34

Signed-off-by: Humble Chirammal <[email protected]>

doc: IBM key protect/HPCS design doc

805a441

This commit add the design considerations of IBM Key protect KMS service to the Ceph CSI integration. Signed-off-by: Humble Chirammal <[email protected]>

rebase: IBM key protect integration module dependency update

b027dc6

This commit adds the Key protect client SDK for the Key Protect KMS integration to the Ceph CSI driver. Signed-off-by: Humble Chirammal <[email protected]>

humblec force-pushed the kp branch from e547a21 to b027dc6 Compare December 21, 2021 14:53

mergify bot merged commit 93e43d1 into ceph:devel Dec 21, 2021

humblec mentioned this pull request Jan 25, 2022

rbd: unexport existing KMS structs for vault, AWS and KeyProtect #2750

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

IBM HPCS/Key Protect KMS integration #2723

IBM HPCS/Key Protect KMS integration #2723

humblec commented Dec 20, 2021

humblec commented Dec 20, 2021

nixpanic Dec 20, 2021

humblec Dec 20, 2021

nixpanic Dec 20, 2021

humblec Dec 20, 2021

nixpanic commented Dec 20, 2021

humblec commented Dec 20, 2021 •

edited

Loading

humblec commented Dec 20, 2021

IBM HPCS/Key Protect KMS integration #2723

IBM HPCS/Key Protect KMS integration #2723

Conversation

humblec commented Dec 20, 2021

humblec commented Dec 20, 2021

nixpanic Dec 20, 2021

Choose a reason for hiding this comment

humblec Dec 20, 2021

Choose a reason for hiding this comment

nixpanic Dec 20, 2021

Choose a reason for hiding this comment

humblec Dec 20, 2021

Choose a reason for hiding this comment

nixpanic commented Dec 20, 2021

humblec commented Dec 20, 2021 • edited Loading

humblec commented Dec 20, 2021

humblec commented Dec 20, 2021 •

edited

Loading