From dfdc8848075abc1d3954bb98ad857a44f6aa9ee8 Mon Sep 17 00:00:00 2001 From: Abdul Hameed Date: Thu, 20 Feb 2025 14:04:33 -0500 Subject: [PATCH] feat: Add Feast Operator RBAC example with Kubernetes Authentication type. Signed-off-by: Abdul Hameed --- examples/README.md | 3 +- examples/operator-rbac/README.md | 6 + .../operator-rbac/client/feature_store.yaml | 15 + examples/operator-rbac/k8s-rbac.ipynb | 1669 +++++++++++++++++ examples/operator-rbac/permissions_apply.py | 21 + ...v1alpha1_featurestore_kubernetes_auth.yaml | 15 +- 6 files changed, 1725 insertions(+), 4 deletions(-) create mode 100644 examples/operator-rbac/README.md create mode 100644 examples/operator-rbac/client/feature_store.yaml create mode 100644 examples/operator-rbac/k8s-rbac.ipynb create mode 100644 examples/operator-rbac/permissions_apply.py diff --git a/examples/README.md b/examples/README.md index ab5288dd5a..6dac867be4 100644 --- a/examples/README.md +++ b/examples/README.md @@ -18,4 +18,5 @@ The following examples illustrate various **Feast** use cases to enhance underst The examples below showcase how to deploy and manage **Feast on Kubernetes** using the **Feast Go Operator**. 1. **[Operator Quickstart](operator-quickstart)**: Demonstrates how to install and use Feast on Kubernetes with the Feast Go Operator. -1. **[Operator Quickstart with Postgres in TLS](operator-postgres-tls-demo)**: Demonstrates installing and configuring Feast with PostgreSQL in TLS mode on Kubernetes using the Feast Go Operator, with an emphasis on volumes and VolumeMounts support. +1. **[Operator Quickstart with Postgres in TLS](operator-postgres-tls-demo)**: Demonstrates installing and configuring Feast with PostgreSQL in TLS mode on Kubernetes using the Feast Go Operator, with an emphasis on volumes and VolumeMounts support. +1. **[Operator RBAC with Kubernetes](operator-rbac)**: Demonstrates the Feast RBAC example on Kubernetes using the Feast Operator. diff --git a/examples/operator-rbac/README.md b/examples/operator-rbac/README.md new file mode 100644 index 0000000000..ca2a6ca30c --- /dev/null +++ b/examples/operator-rbac/README.md @@ -0,0 +1,6 @@ +# Running the Feast RBAC example on Kubernetes using the Feast Operator. + +The [k8s-rbac.ipynb](k8s-rbac.ipynb) will guide you through how to enable Role-Based Access Control (RBAC) for Feast using [Feast Operator](../../infra/feast-operator/) with the Kubernetes Authentication type. + + + diff --git a/examples/operator-rbac/client/feature_store.yaml b/examples/operator-rbac/client/feature_store.yaml new file mode 100644 index 0000000000..49a4c42636 --- /dev/null +++ b/examples/operator-rbac/client/feature_store.yaml @@ -0,0 +1,15 @@ +project: feast_rbac +provider: local +offline_store: + host: localhost + type: remote + port: 8081 +online_store: + path: http://localhost:8082 + type: remote +registry: + path: localhost:8083 + registry_type: remote +auth: + type: kubernetes +entity_key_serialization_version: 3 diff --git a/examples/operator-rbac/k8s-rbac.ipynb b/examples/operator-rbac/k8s-rbac.ipynb new file mode 100644 index 0000000000..3cad6b31e6 --- /dev/null +++ b/examples/operator-rbac/k8s-rbac.ipynb @@ -0,0 +1,1669 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Feast Operator with RBAC Configuration\n", + "## Objective\n", + "\n", + "This demo provides a reference implementation of a runbook on how to enable Role-Based Access Control (RBAC) for Feast using the Feast Operator with the Kubernetes authentication type.\n", + "\n", + "The demo steps include deploying the Feast Operator, creating Feast instances with server components (registry, offline store, online store), and client examples within a Kubernetes environment. The goal is to ensure secure access control for Feast instances deployed by the Feast Operator.\n", + " \n", + "Please read these reference documents for understanding the Feast RBAC framework.\n", + "- [RBAC Architecture](https://docs.feast.dev/v/master/getting-started/architecture/rbac) \n", + "- [RBAC Permission](https://docs.feast.dev/v/master/getting-started/concepts/permission).\n", + "- [RBAC Authorization Manager](https://docs.feast.dev/v/master/getting-started/components/authz_manager)\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Prerequisites\n", + "* Kubernetes Cluster\n", + "* [kubectl](https://kubernetes.io/docs/tasks/tools/#kubectl) Kubernetes CLI tool." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Install Prerequisites\n", + "\n", + "The following commands install and configure all the prerequisites on a MacOS environment. You can find the\n", + "equivalent instructions on the offical documentation pages:\n", + "* Install the `kubectl` cli.\n", + "* Install Kubernetes and Container runtime (e.g. [Colima](https://github.com/abiosoft/colima)).\n", + " * Alternatively, authenticate to an existing Kubernetes or OpenShift cluster.\n", + " \n", + "```bash\n", + "brew install colima kubectl\n", + "colima start -r containerd -k -m 3 -d 100 -c 2 --cpu-type max -a x86_64\n", + "colima list\n", + "```" + ] + }, + { + "cell_type": "code", + "metadata": { + "ExecuteTime": { + "end_time": "2025-02-27T19:38:37.216114Z", + "start_time": "2025-02-27T19:38:36.897921Z" + } + }, + "source": [ + "!kubectl create ns feast\n", + "!kubectl config set-context --current --namespace feast" + ], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "namespace/feast created\r\n", + "Context \"kind-kind\" modified.\r\n" + ] + } + ], + "execution_count": 120 + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Validate the cluster setup:" + ] + }, + { + "cell_type": "code", + "metadata": { + "ExecuteTime": { + "end_time": "2025-02-26T21:42:40.198657Z", + "start_time": "2025-02-26T21:42:40.025647Z" + } + }, + "source": [ + "!kubectl get ns feast" + ], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "NAME STATUS AGE\r\n", + "feast Active 8s\r\n" + ] + } + ], + "execution_count": 30 + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Install the Feast Operator" + ] + }, + { + "cell_type": "code", + "metadata": { + "ExecuteTime": { + "end_time": "2025-02-27T19:38:59.548716Z", + "start_time": "2025-02-27T19:38:47.448069Z" + } + }, + "source": [ + "## Use this install command from a release branch (e.g. 'v0.43-branch')\n", + "!kubectl apply -f ../../infra/feast-operator/dist/install.yaml\n", + "\n", + "## OR, for the latest code/builds, use one the following commands from the 'master' branch\n", + "# !make -C ../../infra/feast-operator install deploy IMG=quay.io/feastdev-ci/feast-operator:develop FS_IMG=quay.io/feastdev-ci/feature-server:develop\n", + "# !make -C ../../infra/feast-operator install deploy IMG=quay.io/feastdev-ci/feast-operator:$(git rev-parse HEAD) FS_IMG=quay.io/feastdev-ci/feature-server:$(git rev-parse HEAD)\n", + "\n", + "!kubectl wait --for=condition=available --timeout=5m deployment/feast-operator-controller-manager -n feast-operator-system" + ], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "namespace/feast-operator-system created\r\n", + "customresourcedefinition.apiextensions.k8s.io/featurestores.feast.dev created\r\n", + "serviceaccount/feast-operator-controller-manager created\r\n", + "role.rbac.authorization.k8s.io/feast-operator-leader-election-role created\r\n", + "clusterrole.rbac.authorization.k8s.io/feast-operator-featurestore-editor-role created\r\n", + "clusterrole.rbac.authorization.k8s.io/feast-operator-featurestore-viewer-role created\r\n", + "clusterrole.rbac.authorization.k8s.io/feast-operator-manager-role created\r\n", + "clusterrole.rbac.authorization.k8s.io/feast-operator-metrics-auth-role created\r\n", + "clusterrole.rbac.authorization.k8s.io/feast-operator-metrics-reader created\r\n", + "rolebinding.rbac.authorization.k8s.io/feast-operator-leader-election-rolebinding created\r\n", + "clusterrolebinding.rbac.authorization.k8s.io/feast-operator-manager-rolebinding created\r\n", + "clusterrolebinding.rbac.authorization.k8s.io/feast-operator-metrics-auth-rolebinding created\r\n", + "service/feast-operator-controller-manager-metrics-service created\r\n", + "deployment.apps/feast-operator-controller-manager created\r\n", + "deployment.apps/feast-operator-controller-manager condition met\r\n" + ] + } + ], + "execution_count": 121 + }, + { + "metadata": {}, + "cell_type": "markdown", + "source": [ + "## Deployment Architecture\n", + "The primary objective of this runbook is to guide the deployment of Feast services with RBAC\n", + "\n", + "In this notebook, we will deploy a distributed topology of Feast services, which includes:\n", + "\n", + "* `Registry Server`: Handles metadata storage for feature definitions.\n", + "* `Online Store Server`: Uses the `Registry Server` to query metadata and is responsible for low-latency serving of features.\n", + "* `Offline Store Server`: Uses the `Registry Server` to query metadata and provides access to batch data for historical feature retrieval.\n", + "* `Kubernetes` Authentication types for RBAC Configuration for Feast resources.\n", + "* Setting update client RBAC examples to test Feast RBAC based on roles assigned.\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Install the Feast services via FeatureStore CR\n", + "Next, we'll use the running Feast Operator to install the feast services with Server components online, offline, registry with kubernetes Authorization set. Apply the included [reference deployment](../../infra/feast-operator/config/samples/v1alpha1_featurestore_kubernetes_auth.yaml) to install and configure Feast with kubernetes Authorization ." + ] + }, + { + "cell_type": "code", + "metadata": { + "ExecuteTime": { + "end_time": "2025-02-27T19:46:37.579711Z", + "start_time": "2025-02-27T19:46:37.151072Z" + } + }, + "source": [ + "!cat ../../infra/feast-operator/config/samples/v1alpha1_featurestore_kubernetes_auth.yaml\n", + "!kubectl apply -f ../../infra/feast-operator/config/samples/v1alpha1_featurestore_kubernetes_auth.yaml -n feast" + ], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "apiVersion: feast.dev/v1alpha1\r\n", + "kind: FeatureStore\r\n", + "metadata:\r\n", + " name: sample-kubernetes-auth\r\n", + "spec:\r\n", + " feastProject: feast_rbac\r\n", + " authz:\r\n", + " kubernetes:\r\n", + " roles:\r\n", + " - feast-writer\r\n", + " - feast-reader\r\n", + " services:\r\n", + " offlineStore:\r\n", + " server: {}\r\n", + " onlineStore:\r\n", + " server: {}\r\n", + " registry:\r\n", + " local:\r\n", + " server: {}\r\n", + " ui: {}\r\n", + "featurestore.feast.dev/sample-kubernetes-auth created\r\n" + ] + } + ], + "execution_count": 122 + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Validate the running FeatureStore deployment\n", + "Validate the deployment status." + ] + }, + { + "cell_type": "code", + "metadata": { + "ExecuteTime": { + "end_time": "2025-02-27T19:47:03.114473Z", + "start_time": "2025-02-27T19:46:50.078710Z" + } + }, + "source": [ + "!kubectl get all\n", + "!kubectl wait --for=condition=available --timeout=8m deployment/feast-sample-kubernetes-auth" + ], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "NAME READY STATUS RESTARTS AGE\r\n", + "pod/feast-sample-kubernetes-auth-774f6df8df-q7v7f 0/4 Running 0 13s\r\n", + "\r\n", + "NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE\r\n", + "service/feast-sample-kubernetes-auth-offline ClusterIP 10.96.75.204 80/TCP 13s\r\n", + "service/feast-sample-kubernetes-auth-online ClusterIP 10.96.234.54 80/TCP 13s\r\n", + "service/feast-sample-kubernetes-auth-registry ClusterIP 10.96.130.144 80/TCP 13s\r\n", + "service/feast-sample-kubernetes-auth-ui ClusterIP 10.96.165.234 80/TCP 13s\r\n", + "\r\n", + "NAME READY UP-TO-DATE AVAILABLE AGE\r\n", + "deployment.apps/feast-sample-kubernetes-auth 0/1 1 0 13s\r\n", + "\r\n", + "NAME DESIRED CURRENT READY AGE\r\n", + "replicaset.apps/feast-sample-kubernetes-auth-774f6df8df 1 1 0 13s\r\n", + "deployment.apps/feast-sample-kubernetes-auth condition met\r\n" + ] + } + ], + "execution_count": 123 + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Validate that the FeatureStore CR is in a `Ready` state." + ] + }, + { + "cell_type": "code", + "metadata": { + "ExecuteTime": { + "end_time": "2025-02-26T21:44:22.317774Z", + "start_time": "2025-02-26T21:44:22.141848Z" + } + }, + "source": [ + "!kubectl get feast" + ], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "NAME STATUS AGE\r\n", + "sample-kubernetes-auth Ready 81s\r\n" + ] + } + ], + "execution_count": 35 + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Configure the RBAC Permissions\n", + "we have defined permission in `permissions_apply.py`." + ] + }, + { + "metadata": { + "ExecuteTime": { + "end_time": "2025-02-26T21:44:30.332020Z", + "start_time": "2025-02-26T21:44:30.209058Z" + } + }, + "cell_type": "code", + "source": [ + "#view the permissions \n", + "!cat permissions_apply.py" + ], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "from feast.feast_object import ALL_RESOURCE_TYPES\r\n", + "from feast.permissions.action import READ, AuthzedAction, ALL_ACTIONS\r\n", + "from feast.permissions.permission import Permission\r\n", + "from feast.permissions.policy import RoleBasedPolicy\r\n", + "\r\n", + "admin_roles = [\"feast-writer\"]\r\n", + "user_roles = [\"feast-reader\"]\r\n", + "\r\n", + "user_perm = Permission(\r\n", + " name=\"feast_user_permission\",\r\n", + " types=ALL_RESOURCE_TYPES,\r\n", + " policy=RoleBasedPolicy(roles=user_roles),\r\n", + " actions=[AuthzedAction.DESCRIBE] + READ\r\n", + ")\r\n", + "\r\n", + "admin_perm = Permission(\r\n", + " name=\"feast_admin_permission\",\r\n", + " types=ALL_RESOURCE_TYPES,\r\n", + " policy=RoleBasedPolicy(roles=admin_roles),\r\n", + " actions=ALL_ACTIONS\r\n", + ")\r\n" + ] + } + ], + "execution_count": 36 + }, + { + "metadata": { + "ExecuteTime": { + "end_time": "2025-02-27T19:49:48.630139Z", + "start_time": "2025-02-27T19:49:48.089115Z" + } + }, + "cell_type": "code", + "source": [ + "# Copy the Permissions to the pods under feature_repo directory\n", + "!kubectl cp permissions_apply.py $(kubectl get pods -l 'feast.dev/name=sample-kubernetes-auth' -ojsonpath=\"{.items[*].metadata.name}\"):/feast-data/feast_rbac/feature_repo -c online" + ], + "outputs": [], + "execution_count": 124 + }, + { + "cell_type": "code", + "metadata": { + "ExecuteTime": { + "end_time": "2025-02-27T19:49:52.653102Z", + "start_time": "2025-02-27T19:49:52.375440Z" + } + }, + "source": [ + "#view the feature_store.yaml configuration \n", + "!kubectl exec deploy/feast-sample-kubernetes-auth -itc online -- cat feature_store.yaml" + ], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "E0227 14:49:52.453894 50390 websocket.go:296] Unknown stream id 1, discarding message\n", + "project: feast_rbac\r\n", + "provider: local\r\n", + "offline_store:\r\n", + " type: dask\r\n", + "online_store:\r\n", + " path: /feast-data/online_store.db\r\n", + " type: sqlite\r\n", + "registry:\r\n", + " path: /feast-data/registry.db\r\n", + " registry_type: file\r\n", + "auth:\r\n", + " type: kubernetes\r\n", + "entity_key_serialization_version: 3\r\n" + ] + } + ], + "execution_count": 125 + }, + { + "metadata": {}, + "cell_type": "markdown", + "source": "## Apply the Permissions and Feast Object to Registry" + }, + { + "metadata": { + "ExecuteTime": { + "end_time": "2025-02-27T19:50:08.586896Z", + "start_time": "2025-02-27T19:49:57.964674Z" + } + }, + "cell_type": "code", + "source": "!kubectl exec deploy/feast-sample-kubernetes-auth -itc online -- feast apply", + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + ": MADV_DONTNEED does not work (memset will be used instead)\r\n", + ": (This is the expected behaviour if you are running under QEMU)\r\n", + "/opt/app-root/lib64/python3.11/site-packages/feast/feature_view.py:48: DeprecationWarning: Entity value_type will be mandatory in the next release. Please specify a value_type for entity '__dummy'.\r\n", + " DUMMY_ENTITY = Entity(\r\n", + "/opt/app-root/lib64/python3.11/site-packages/pydantic/_internal/_fields.py:192: UserWarning: Field name \"vector_enabled\" in \"SqliteOnlineStoreConfig\" shadows an attribute in parent \"VectorStoreConfig\"\r\n", + " warnings.warn(\r\n", + "/opt/app-root/lib64/python3.11/site-packages/pydantic/_internal/_fields.py:192: UserWarning: Field name \"vector_len\" in \"SqliteOnlineStoreConfig\" shadows an attribute in parent \"VectorStoreConfig\"\r\n", + " warnings.warn(\r\n", + "/feast-data/feast_rbac/feature_repo/example_repo.py:27: DeprecationWarning: Entity value_type will be mandatory in the next release. Please specify a value_type for entity 'driver'.\r\n", + " driver = Entity(name=\"driver\", join_keys=[\"driver_id\"])\r\n", + "Applying changes for project feast_rbac\r\n", + "/opt/app-root/lib64/python3.11/site-packages/feast/feature_store.py:579: RuntimeWarning: On demand feature view is an experimental feature. This API is stable, but the functionality does not scale well for offline retrieval\r\n", + " warnings.warn(\r\n", + "Created project \u001B[1m\u001B[32mfeast_rbac\u001B[0m\r\n", + "Created entity \u001B[1m\u001B[32mdriver\u001B[0m\r\n", + "Created feature view \u001B[1m\u001B[32mdriver_hourly_stats\u001B[0m\r\n", + "Created feature view \u001B[1m\u001B[32mdriver_hourly_stats_fresh\u001B[0m\r\n", + "Created on demand feature view \u001B[1m\u001B[32mtransformed_conv_rate\u001B[0m\r\n", + "Created on demand feature view \u001B[1m\u001B[32mtransformed_conv_rate_fresh\u001B[0m\r\n", + "Created feature service \u001B[1m\u001B[32mdriver_activity_v2\u001B[0m\r\n", + "Created feature service \u001B[1m\u001B[32mdriver_activity_v1\u001B[0m\r\n", + "Created feature service \u001B[1m\u001B[32mdriver_activity_v3\u001B[0m\r\n", + "Created permission \u001B[1m\u001B[32mfeast_admin_permission\u001B[0m\r\n", + "Created permission \u001B[1m\u001B[32mfeast_user_permission\u001B[0m\r\n", + "\r\n", + "Created sqlite table \u001B[1m\u001B[32mfeast_rbac_driver_hourly_stats_fresh\u001B[0m\r\n", + "Created sqlite table \u001B[1m\u001B[32mfeast_rbac_driver_hourly_stats\u001B[0m\r\n", + "\r\n" + ] + } + ], + "execution_count": 126 + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": "**List the applied permission details permissions on Feast Resources.**" + }, + { + "cell_type": "code", + "metadata": { + "ExecuteTime": { + "end_time": "2025-02-27T19:51:14.808688Z", + "start_time": "2025-02-27T19:50:24.288337Z" + } + }, + "source": [ + "!kubectl exec deploy/feast-sample-kubernetes-auth -itc online -- feast permissions list-roles\n", + "!kubectl exec deploy/feast-sample-kubernetes-auth -itc online -- feast permissions list\n", + "!kubectl exec deploy/feast-sample-kubernetes-auth -itc online -- feast permissions describe feast_admin_permission\n", + "!kubectl exec deploy/feast-sample-kubernetes-auth -itc online -- feast permissions describe feast_user_permission" + ], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + ": MADV_DONTNEED does not work (memset will be used instead)\r\n", + ": (This is the expected behaviour if you are running under QEMU)\r\n", + "/opt/app-root/lib64/python3.11/site-packages/feast/feature_view.py:48: DeprecationWarning: Entity value_type will be mandatory in the next release. Please specify a value_type for entity '__dummy'.\r\n", + " DUMMY_ENTITY = Entity(\r\n", + "/opt/app-root/lib64/python3.11/site-packages/pydantic/_internal/_fields.py:192: UserWarning: Field name \"vector_enabled\" in \"SqliteOnlineStoreConfig\" shadows an attribute in parent \"VectorStoreConfig\"\r\n", + " warnings.warn(\r\n", + "/opt/app-root/lib64/python3.11/site-packages/pydantic/_internal/_fields.py:192: UserWarning: Field name \"vector_len\" in \"SqliteOnlineStoreConfig\" shadows an attribute in parent \"VectorStoreConfig\"\r\n", + " warnings.warn(\r\n", + "+--------------+\r\n", + "| ROLE NAME |\r\n", + "+==============+\r\n", + "| feast-reader |\r\n", + "+--------------+\r\n", + "| feast-writer |\r\n", + "+--------------+\r\n", + ": MADV_DONTNEED does not work (memset will be used instead)\r\n", + ": (This is the expected behaviour if you are running under QEMU)\r\n", + "/opt/app-root/lib64/python3.11/site-packages/feast/feature_view.py:48: DeprecationWarning: Entity value_type will be mandatory in the next release. Please specify a value_type for entity '__dummy'.\r\n", + " DUMMY_ENTITY = Entity(\r\n", + "/opt/app-root/lib64/python3.11/site-packages/pydantic/_internal/_fields.py:192: UserWarning: Field name \"vector_enabled\" in \"SqliteOnlineStoreConfig\" shadows an attribute in parent \"VectorStoreConfig\"\r\n", + " warnings.warn(\r\n", + "/opt/app-root/lib64/python3.11/site-packages/pydantic/_internal/_fields.py:192: UserWarning: Field name \"vector_len\" in \"SqliteOnlineStoreConfig\" shadows an attribute in parent \"VectorStoreConfig\"\r\n", + " warnings.warn(\r\n", + "NAME TYPES NAME_PATTERNS ACTIONS ROLES REQUIRED_TAGS\r\n", + "feast_admin_permission Project - CREATE feast-writer -\r\n", + " FeatureView DESCRIBE\r\n", + " OnDemandFeatureView UPDATE\r\n", + " BatchFeatureView DELETE\r\n", + " StreamFeatureView READ_ONLINE\r\n", + " Entity READ_OFFLINE\r\n", + " FeatureService WRITE_ONLINE\r\n", + " DataSource WRITE_OFFLINE\r\n", + " ValidationReference\r\n", + " SavedDataset\r\n", + " Permission\r\n", + "feast_user_permission Project - DESCRIBE feast-reader -\r\n", + " FeatureView READ_OFFLINE\r\n", + " OnDemandFeatureView READ_ONLINE\r\n", + " BatchFeatureView\r\n", + " StreamFeatureView\r\n", + " Entity\r\n", + " FeatureService\r\n", + " DataSource\r\n", + " ValidationReference\r\n", + " SavedDataset\r\n", + " Permission\r\n", + ": MADV_DONTNEED does not work (memset will be used instead)\r\n", + ": (This is the expected behaviour if you are running under QEMU)\r\n", + "/opt/app-root/lib64/python3.11/site-packages/feast/feature_view.py:48: DeprecationWarning: Entity value_type will be mandatory in the next release. Please specify a value_type for entity '__dummy'.\r\n", + " DUMMY_ENTITY = Entity(\r\n", + "/opt/app-root/lib64/python3.11/site-packages/pydantic/_internal/_fields.py:192: UserWarning: Field name \"vector_enabled\" in \"SqliteOnlineStoreConfig\" shadows an attribute in parent \"VectorStoreConfig\"\r\n", + " warnings.warn(\r\n", + "/opt/app-root/lib64/python3.11/site-packages/pydantic/_internal/_fields.py:192: UserWarning: Field name \"vector_len\" in \"SqliteOnlineStoreConfig\" shadows an attribute in parent \"VectorStoreConfig\"\r\n", + " warnings.warn(\r\n", + "spec:\r\n", + " name: feast_admin_permission\r\n", + " types:\r\n", + " - PROJECT\r\n", + " - FEATURE_VIEW\r\n", + " - ON_DEMAND_FEATURE_VIEW\r\n", + " - BATCH_FEATURE_VIEW\r\n", + " - STREAM_FEATURE_VIEW\r\n", + " - ENTITY\r\n", + " - FEATURE_SERVICE\r\n", + " - DATA_SOURCE\r\n", + " - VALIDATION_REFERENCE\r\n", + " - SAVED_DATASET\r\n", + " - PERMISSION\r\n", + " actions:\r\n", + " - CREATE\r\n", + " - DESCRIBE\r\n", + " - UPDATE\r\n", + " - DELETE\r\n", + " - READ_ONLINE\r\n", + " - READ_OFFLINE\r\n", + " - WRITE_ONLINE\r\n", + " - WRITE_OFFLINE\r\n", + " policy:\r\n", + " roleBasedPolicy:\r\n", + " roles:\r\n", + " - feast-writer\r\n", + "meta:\r\n", + " createdTimestamp: '2025-02-27T19:50:08.178925Z'\r\n", + " lastUpdatedTimestamp: '2025-02-27T19:50:08.178925Z'\r\n", + "\r\n", + ": MADV_DONTNEED does not work (memset will be used instead)\r\n", + ": (This is the expected behaviour if you are running under QEMU)\r\n", + "/opt/app-root/lib64/python3.11/site-packages/feast/feature_view.py:48: DeprecationWarning: Entity value_type will be mandatory in the next release. Please specify a value_type for entity '__dummy'.\r\n", + " DUMMY_ENTITY = Entity(\r\n", + "/opt/app-root/lib64/python3.11/site-packages/pydantic/_internal/_fields.py:192: UserWarning: Field name \"vector_enabled\" in \"SqliteOnlineStoreConfig\" shadows an attribute in parent \"VectorStoreConfig\"\r\n", + " warnings.warn(\r\n", + "/opt/app-root/lib64/python3.11/site-packages/pydantic/_internal/_fields.py:192: UserWarning: Field name \"vector_len\" in \"SqliteOnlineStoreConfig\" shadows an attribute in parent \"VectorStoreConfig\"\r\n", + " warnings.warn(\r\n", + "spec:\r\n", + " name: feast_user_permission\r\n", + " types:\r\n", + " - PROJECT\r\n", + " - FEATURE_VIEW\r\n", + " - ON_DEMAND_FEATURE_VIEW\r\n", + " - BATCH_FEATURE_VIEW\r\n", + " - STREAM_FEATURE_VIEW\r\n", + " - ENTITY\r\n", + " - FEATURE_SERVICE\r\n", + " - DATA_SOURCE\r\n", + " - VALIDATION_REFERENCE\r\n", + " - SAVED_DATASET\r\n", + " - PERMISSION\r\n", + " actions:\r\n", + " - DESCRIBE\r\n", + " - READ_OFFLINE\r\n", + " - READ_ONLINE\r\n", + " policy:\r\n", + " roleBasedPolicy:\r\n", + " roles:\r\n", + " - feast-reader\r\n", + "meta:\r\n", + " createdTimestamp: '2025-02-27T19:50:08.179716Z'\r\n", + " lastUpdatedTimestamp: '2025-02-27T19:50:08.179716Z'\r\n", + "\r\n" + ] + } + ], + "execution_count": 127 + }, + { + "metadata": {}, + "cell_type": "markdown", + "source": [ + "## Feast Client with RBAC\n", + "### RBAC Kubernetes Authentication\n", + "This Feast **Role-Based Access Control (RBAC)** in Kubernetes support authentication **inside a Kubernetes pod** and **outside a pod** when running a local script.\n", + "### Inside a Kubernetes Pod\n", + "Feast automatically retrieves the Kubernetes ServiceAccount token from:\n", + "```\n", + "/var/run/secrets/kubernetes.io/serviceaccount/token\n", + "```\n", + "This means:\n", + "- No manual configuration is needed inside a pod.\n", + "- The token is mounted automatically and used for authentication.\n", + "- Developer just need create the binding with role and service account accordingly.\n", + "- Code Reference: \n", + "[Feast Kubernetes Auth Client Manager (Pod Token Usage)](https://github.com/feast-dev/feast/blob/master/sdk/python/feast/permissions/client/kubernetes_auth_client_manager.py#L15) \n", + "- See the example will use service account from pod [Example](https://github.com/feast-dev/feast/blob/master/examples/rbac-remote/client/k8s/)\n", + "\n", + "### Outside a Kubernetes Pod (Local Machine)\n", + "If running Feast outside of Kubernetes, authentication requires setting the token manually:\n", + "```sh\n", + "export LOCAL_K8S_TOKEN=\"your-service-account-token\"\n", + "```\n", + "Feast will use this token for authentication.\n", + "\n", + "Reference: \n", + "[Feast Authentication via `LOCAL_K8S_TOKEN`](https://github.com/feast-dev/feast/blob/master/sdk/python/feast/permissions/client/kubernetes_auth_client_manager.py#L50)" + ] + }, + { + "metadata": {}, + "cell_type": "markdown", + "source": "### Feature Store settings" + }, + { + "metadata": {}, + "cell_type": "code", + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "project: feast_rbac\r\n", + "provider: local\r\n", + "offline_store:\r\n", + " host: feast-sample-kubernetes-auth-offline.feast.svc.cluster.local\r\n", + " type: remote\r\n", + " port: 80\r\n", + "online_store:\r\n", + " path: http://feast-sample-kubernetes-auth-online.feast.svc.cluster.local:80\r\n", + " type: remote\r\n", + "registry:\r\n", + " path: feast-sample-kubernetes-auth-registry.feast.svc.cluster.local:80\r\n", + " registry_type: remote\r\n", + "auth:\r\n", + " type: kubernetes\r\n", + "entity_key_serialization_version: 3\r\n" + ] + } + ], + "execution_count": 43, + "source": "!cat client/feature_store.yaml" + }, + { + "metadata": {}, + "cell_type": "markdown", + "source": "**The Operator creates the client ConfigMap containing the feature_store.yaml. We can retrieve it and port froward to local**" + }, + { + "metadata": { + "ExecuteTime": { + "end_time": "2025-02-27T19:55:11.430758Z", + "start_time": "2025-02-27T19:55:11.104085Z" + } + }, + "cell_type": "code", + "source": "!kubectl get configmap feast-sample-kubernetes-auth-client -n feast -o jsonpath='{.data.feature_store\\.yaml}' ", + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "project: feast_rbac\r\n", + "provider: local\r\n", + "offline_store:\r\n", + " host: feast-sample-kubernetes-auth-offline.feast.svc.cluster.local\r\n", + " type: remote\r\n", + " port: 80\r\n", + "online_store:\r\n", + " path: http://feast-sample-kubernetes-auth-online.feast.svc.cluster.local:80\r\n", + " type: remote\r\n", + "registry:\r\n", + " path: feast-sample-kubernetes-auth-registry.feast.svc.cluster.local:80\r\n", + " registry_type: remote\r\n", + "auth:\r\n", + " type: kubernetes\r\n", + "entity_key_serialization_version: 3\r\n" + ] + } + ], + "execution_count": 128 + }, + { + "metadata": {}, + "cell_type": "markdown", + "source": "### The function below is executed to support the preparation of client testing." + }, + { + "metadata": {}, + "cell_type": "markdown", + "source": "Run Port Forwarding for All Services for local testing " + }, + { + "metadata": { + "ExecuteTime": { + "end_time": "2025-02-27T20:02:42.161665Z", + "start_time": "2025-02-27T20:02:42.127601Z" + } + }, + "cell_type": "code", + "source": [ + "import subprocess\n", + "\n", + "# Define services and their local ports\n", + "services = {\n", + " \"offline_store\": (\"feast-sample-kubernetes-auth-offline\", 8081),\n", + " \"online_store\": (\"feast-sample-kubernetes-auth-online\", 8082),\n", + " \"registry\": (\"feast-sample-kubernetes-auth-registry\", 8083),\n", + "}\n", + "\n", + "# Start port-forwarding for each service\n", + "port_forward_processes = {}\n", + "for name, (service, local_port) in services.items():\n", + " cmd = f\"kubectl port-forward svc/{service} -n feast {local_port}:80\"\n", + " process = subprocess.Popen(cmd, shell=True)\n", + " port_forward_processes[name] = process\n", + " print(f\"Port forwarding {service} -> localhost:{local_port}\")" + ], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Port forwarding feast-sample-kubernetes-auth-offline -> localhost:8081\n", + "Port forwarding feast-sample-kubernetes-auth-online -> localhost:8082\n", + "Port forwarding feast-sample-kubernetes-auth-registry -> localhost:8083\n" + ] + } + ], + "execution_count": 129 + }, + { + "metadata": {}, + "cell_type": "markdown", + "source": "Function to retrieve a Kubernetes service account token and set it as an environment variable" + }, + { + "metadata": { + "ExecuteTime": { + "end_time": "2025-02-27T20:03:36.819299Z", + "start_time": "2025-02-27T20:03:36.816847Z" + } + }, + "cell_type": "code", + "source": [ + "import subprocess\n", + "import os\n", + "\n", + "def get_k8s_token(service_account):\n", + " namespace = \"feast\"\n", + "\n", + " if not service_account:\n", + " raise ValueError(\"Service account name is required.\")\n", + "\n", + " result = subprocess.run(\n", + " [\"kubectl\", \"create\", \"token\", service_account, \"-n\", namespace],\n", + " capture_output=True, text=True, check=True\n", + " )\n", + "\n", + " token = result.stdout.strip()\n", + "\n", + " if not token:\n", + " return None # Silently return None if token retrieval fails\n", + "\n", + " os.environ[\"LOCAL_K8S_TOKEN\"] = token\n", + " return \"Token Retrieved: ***** (hidden for security)\"\n" + ], + "outputs": [], + "execution_count": 130 + }, + { + "metadata": {}, + "cell_type": "markdown", + "source": [ + "**Generating training data. The following test functions were copied from the `test_workflow.py` template but we added `try` blocks to print only \n", + "the relevant error messages, since we expect to receive errors from the permission enforcement modules.**" + ] + }, + { + "metadata": { + "ExecuteTime": { + "end_time": "2025-02-27T20:06:26.881545Z", + "start_time": "2025-02-27T20:06:26.876870Z" + } + }, + "cell_type": "code", + "source": [ + "from feast import FeatureStore\n", + "\n", + "def fetch_historical_features_entity_df(store: FeatureStore, for_batch_scoring: bool):\n", + " try:\n", + " entity_df = pd.DataFrame.from_dict(\n", + " {\n", + " \"driver_id\": [1001, 1002, 1003],\n", + " \"event_timestamp\": [\n", + " datetime(2021, 4, 12, 10, 59, 42),\n", + " datetime(2021, 4, 12, 8, 12, 10),\n", + " datetime(2021, 4, 12, 16, 40, 26),\n", + " ],\n", + " \"label_driver_reported_satisfaction\": [1, 5, 3],\n", + " # values we're using for an on-demand transformation\n", + " \"val_to_add\": [1, 2, 3],\n", + " \"val_to_add_2\": [10, 20, 30],\n", + " }\n", + " )\n", + " if for_batch_scoring:\n", + " entity_df[\"event_timestamp\"] = pd.to_datetime(\"now\", utc=True)\n", + "\n", + " training_df = store.get_historical_features(\n", + " entity_df=entity_df,\n", + " features=[\n", + " \"driver_hourly_stats:conv_rate\",\n", + " \"driver_hourly_stats:acc_rate\",\n", + " \"driver_hourly_stats:avg_daily_trips\",\n", + " \"transformed_conv_rate:conv_rate_plus_val1\",\n", + " \"transformed_conv_rate:conv_rate_plus_val2\",\n", + " ],\n", + "\n", + " ).to_df()\n", + " print(training_df.head())\n", + "\n", + " except Exception as e:\n", + " print(f\"An error occurred while fetching historical features: {e}\")\n", + "\n", + "\n", + "def fetch_online_features(store, source: str = \"\"):\n", + " try:\n", + " entity_rows = [\n", + " # {join_key: entity_value}\n", + " {\n", + " \"driver_id\": 1001,\n", + " \"val_to_add\": 1000,\n", + " \"val_to_add_2\": 2000,\n", + " },\n", + " {\n", + " \"driver_id\": 1002,\n", + " \"val_to_add\": 1001,\n", + " \"val_to_add_2\": 2002,\n", + " },\n", + " ]\n", + " if source == \"feature_service\":\n", + " features_to_fetch = store.get_feature_service(\"driver_activity_v1\")\n", + " elif source == \"push\":\n", + " features_to_fetch = store.get_feature_service(\"driver_activity_v3\")\n", + " else:\n", + " features_to_fetch = [\n", + " \"driver_hourly_stats:acc_rate\",\n", + " \"transformed_conv_rate:conv_rate_plus_val1\",\n", + " \"transformed_conv_rate:conv_rate_plus_val2\",\n", + " ]\n", + " returned_features = store.get_online_features(\n", + " features=features_to_fetch,\n", + " entity_rows=entity_rows,\n", + " ).to_dict()\n", + " for key, value in sorted(returned_features.items()):\n", + " print(key, \" : \", value)\n", + "\n", + " except Exception as e:\n", + " print(f\"An error occurred while fetching online features: {e}\")" + ], + "outputs": [], + "execution_count": 135 + }, + { + "metadata": {}, + "cell_type": "markdown", + "source": [ + "## Setting Up and Testing Feast Users\n", + "The steps below will:\n", + "- Create **three different ServiceAccounts** for Feast.\n", + "- Assign appropriate **RoleBindings** for access control.\n", + "- Retrieve and use the **ServiceAccount token** for authentication.\n", + "- Run the test cases to validate RBAC." + ] + }, + { + "metadata": {}, + "cell_type": "markdown", + "source": [ + "## Test Cases\n", + "| User Type | ServiceAccount | RoleBinding Assigned | Expected Behavior in output |\n", + "|----------------|-----------------------------|----------------------|------------------------------------------------------------|\n", + "| **Read-Only** | `feast-user-sa` | `feast-reader` | Can **read** from the feature store, but **cannot write**. |\n", + "| **Unauthorized** | `feast-unauthorized-user-sa` | _None_ | **Access should be denied** in `test.py`. |\n", + "| **Admin** | `feast-admin-sa` | `feast-writer` | Can **read and write** feature store data. |" + ] + }, + { + "metadata": {}, + "cell_type": "markdown", + "source": "### Setup & Test Read-Only Feast User (serviceaccount: feast-user-sa, role: feast-reader)" + }, + { + "metadata": {}, + "cell_type": "markdown", + "source": "**Step 1: Create the ServiceAccount and Role Binding**" + }, + { + "metadata": {}, + "cell_type": "code", + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Creating ServiceAccount: feast-user-sa\r\n", + "serviceaccount/feast-user-sa created\r\n", + "Assigning Read-Only RoleBinding: feast-user-rolebinding\r\n", + "rolebinding.rbac.authorization.k8s.io/feast-user-rolebinding created\r\n" + ] + } + ], + "execution_count": 131, + "source": [ + "# Step 1: Create the ServiceAccount\n", + "!echo \"Creating ServiceAccount: feast-user-sa\"\n", + "!kubectl create serviceaccount feast-user-sa -n feast\n", + "\n", + "# Step 2: Assign RoleBinding (Read-Only Access for Feast)\n", + "!echo \"Assigning Read-Only RoleBinding: feast-user-rolebinding\"\n", + "!kubectl create rolebinding feast-user-rolebinding --role=feast-reader --serviceaccount=feast:feast-user-sa -n feast" + ] + }, + { + "metadata": {}, + "cell_type": "markdown", + "source": "**Step 2: Set the Token**" + }, + { + "metadata": {}, + "cell_type": "code", + "outputs": [ + { + "data": { + "text/plain": [ + "'Token Retrieved: ***** (hidden for security)'" + ] + }, + "execution_count": 132, + "metadata": {}, + "output_type": "execute_result" + } + ], + "execution_count": 132, + "source": "get_k8s_token(\"feast-user-sa\")" + }, + { + "metadata": {}, + "cell_type": "markdown", + "source": "**Step 3:Test function features from offline, online and materialize_incremental etc**" + }, + { + "metadata": {}, + "cell_type": "code", + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Running Feast RBAC test for Read only User...\r\n", + "\n", + "--- Historical features for training ---\n", + "Handling connection for 8083\n", + "Handling connection for 8081\n", + " driver_id event_timestamp label_driver_reported_satisfaction \\\n", + "0 1001 2021-04-12 10:59:42+00:00 1 \n", + "1 1002 2021-04-12 08:12:10+00:00 5 \n", + "2 1003 2021-04-12 16:40:26+00:00 3 \n", + "\n", + " val_to_add val_to_add_2 conv_rate acc_rate avg_daily_trips \\\n", + "0 1 10 0.274426 0.314969 971 \n", + "1 2 20 0.742959 0.215825 723 \n", + "2 3 30 0.466147 0.412917 247 \n", + "\n", + " conv_rate_plus_val1 conv_rate_plus_val2 \n", + "0 1.274426 10.274426 \n", + "1 2.742959 20.742959 \n", + "2 3.466147 30.466147 \n", + "\n", + "--- Historical features for batch scoring ---\n", + "Handling connection for 8081\n", + " driver_id event_timestamp \\\n", + "0 1002 2025-02-27 20:09:28.354218+00:00 \n", + "1 1001 2025-02-27 20:09:28.354218+00:00 \n", + "2 1003 2025-02-27 20:09:28.354218+00:00 \n", + "\n", + " label_driver_reported_satisfaction val_to_add val_to_add_2 conv_rate \\\n", + "0 5 2 20 0.824729 \n", + "1 1 1 10 0.732856 \n", + "2 3 3 30 0.672832 \n", + "\n", + " acc_rate avg_daily_trips conv_rate_plus_val1 conv_rate_plus_val2 \n", + "0 0.251341 20 2.824729 20.824729 \n", + "1 0.468609 830 1.732856 10.732856 \n", + "2 0.280995 904 3.672832 30.672832 \n", + "\n", + "--- Load features into online store/materialize_incremental ---\n", + "Materializing \u001B[1m\u001B[32m2\u001B[0m feature views to \u001B[1m\u001B[32m2025-02-27 15:09:28-05:00\u001B[0m into the \u001B[1m\u001B[32mremote\u001B[0m online store.\n", + "\n", + "\u001B[1m\u001B[32mdriver_hourly_stats\u001B[0m from \u001B[1m\u001B[32m2025-02-26 15:09:28-05:00\u001B[0m to \u001B[1m\u001B[32m2025-02-27 15:09:28-05:00\u001B[0m:\n", + "Handling connection for 8081\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + " 0%| | 0/5 [00:00