sdk/python/endpoints/batch/deploy-pipelines/batch-scoring-with-preprocessing/sdk-deploy-and-test.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "source": [
    "# How to deploy a pipeline to perform batch scoring with preprocessing\n",
    "\n",
    "This example shows you how to reuse preprocessing code and the parameters learned during preprocessing before you use your model for inferencing. By reusing the preprocessing code and learned parameters, we can ensure that the same transformations (such as normalization and feature encoding) that were applied to the input data during training are also applied during inferencing. The model used for inference will perform predictions on tabular data from the [UCI Heart Disease Data Set](https://archive.ics.uci.edu/ml/datasets/Heart+Disease)."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Connect to Azure Machine Learning Workspace\n",
    "\n",
    "The [workspace](https://docs.microsoft.com/en-us/azure/machine-learning/concept-workspace) is the top-level resource for Azure Machine Learning, providing a centralized place to work with all the artifacts you create when you use Azure Machine Learning. In this section we will connect to the workspace in which the job will be run.\n",
    "\n",
    "### 1.1. Import the required libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azure.ai.ml import MLClient, Input\n",
    "from azure.ai.ml import load_component\n",
    "from azure.ai.ml.entities import (\n",
    "    BatchEndpoint,\n",
    "    PipelineComponentBatchDeployment,\n",
    "    AmlCompute,\n",
    "    Environment,\n",
    "    Model,\n",
    ")\n",
    "from azure.ai.ml.constants import AssetTypes\n",
    "from azure.ai.ml.dsl import pipeline\n",
    "from azure.identity import DefaultAzureCredential"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.2. Configure workspace details and get a handle to the workspace\n",
    "\n",
    "To connect to a workspace, we need identifier parameters - a subscription, resource group and workspace name. We will use these details in the `MLClient` from `azure.ai.ml` to get a handle to the required Azure Machine Learning workspace. We use the default [default azure authentication](https://docs.microsoft.com/en-us/python/api/azure-identity/azure.identity.defaultazurecredential?view=azure-python) for this tutorial. Check the [configuration notebook](../../jobs/configuration.ipynb) for more details on how to configure credentials and connect to a workspace."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "jupyter": {
     "outputs_hidden": false,
     "source_hidden": false
    },
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "outputs": [],
   "source": [
    "subscription_id = \"<SUBSCRIPTION_ID>\"\n",
    "resource_group = \"<RESOURCE_GROUP>\"\n",
    "workspace = \"<AML_WORKSPACE_NAME>\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ml_client = MLClient(\n",
    "    DefaultAzureCredential(), subscription_id, resource_group, workspace\n",
    ")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you are working in a Azure Machine Learning compute, you can simply:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ml_client = MLClient.from_config(DefaultAzureCredential())"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "source": [
    "## 2. Create the pipeline component\n",
    "\n",
    "In this section, we'll create all the assets required for our inference pipeline. We'll begin by creating an environment that includes necessary libraries for the pipeline's components. Next, we'll create a compute cluster on which the batch deployment will run. Afterwards, we'll register the components, models, and transformations we need to build our inference pipeline. Finally, we'll build and test the pipeline."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.1 Create the environment\n",
    "\n",
    "The components in this example will use an environment with the `XGBoost` and `scikit-learn` libraries. The `environment/conda.yml` file contains the environment's configuration:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "name": "configure_environment"
   },
   "outputs": [],
   "source": [
    "environment = Environment(\n",
    "    name=\"xgboost-sklearn-py38\",\n",
    "    description=\"An environment for models built with XGBoost and Scikit-learn.\",\n",
    "    image=\"mcr.microsoft.com/azureml/openmpi4.1.0-ubuntu20.04:latest\",\n",
    "    conda_file=\"environment/conda.yml\",\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "name": "create_environment"
   },
   "outputs": [],
   "source": [
    "ml_client.environments.create_or_update(environment)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2 Create a compute cluster"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "name": "create_compute"
   },
   "outputs": [],
   "source": [
    "compute_name = \"batch-cluster\"\n",
    "if not any(filter(lambda m: m.name == compute_name, ml_client.compute.list())):\n",
    "    compute_cluster = AmlCompute(\n",
    "        name=compute_name,\n",
    "        description=\"Batch endpoints compute cluster\",\n",
    "        min_instances=0,\n",
    "        max_instances=5,\n",
    "    )\n",
    "    ml_client.begin_create_or_update(compute_cluster).result()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.3 Register components and models"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Register the model to use for prediction:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "name": "register_model"
   },
   "outputs": [],
   "source": [
    "model_name = \"heart-classifier\"\n",
    "model_local_path = \"model\"\n",
    "\n",
    "model = ml_client.models.create_or_update(\n",
    "    Model(name=model_name, path=model_local_path, type=AssetTypes.MLFLOW_MODEL)\n",
    ")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The registered model wasn't trained directly on input data. Instead, the input data was preprocessed (or transformed) before training, using a prepare component. We'll also need to register this component. Register the prepare component:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "name": "register_preprocessing_component"
   },
   "outputs": [],
   "source": [
    "prepare_data = load_component(source=\"components/prepare/prepare.yml\")\n",
    "\n",
    "ml_client.components.create_or_update(prepare_data)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As part of the data transformations in the prepare component, the input data was normalized to center the predictors and limit their values in the range of [-1, 1]. The transformation parameters were captured in a scikit-learn transformation that we can also register to apply later when we have new data. Register the transformation as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "name": "register_transformation"
   },
   "outputs": [],
   "source": [
    "transformation_name = \"heart-classifier-transforms\"\n",
    "transformation_local_path = \"transformations\"\n",
    "\n",
    "transformations = ml_client.models.create_or_update(\n",
    "    Model(\n",
    "        name=transformation_name,\n",
    "        path=transformation_local_path,\n",
    "        type=AssetTypes.CUSTOM_MODEL,\n",
    "    )\n",
    ")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.4 Create the pipeline\n",
    "\n",
    "The pipeline we want to operationalize has takes 1 input, the training data, and produces 3 outputs, the trained model, the evaluation results, and the data transformations applied as preprocessing. It is composed of 2 components:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "name": "load_component"
   },
   "outputs": [],
   "source": [
    "prepare_data = ml_client.components.get(\"uci_heart_prepare\", label=\"latest\")\n",
    "score_data = load_component(source=\"components/score/score.yml\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Construct the pipeline:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "name": "configure_pipeline"
   },
   "outputs": [],
   "source": [
    "@pipeline()\n",
    "def uci_heart_classifier_scorer(\n",
    "    input_data: Input(type=AssetTypes.URI_FOLDER), score_mode: str\n",
    "):\n",
    "    \"\"\"This pipeline demonstrates how to make batch inference using a model from the Heart Disease Data Set problem, where pre and post processing is required as steps. The pre and post processing steps can be components reusable from the training pipeline.\"\"\"\n",
    "    prepared_data = prepare_data(\n",
    "        data=input_data,\n",
    "        transformations=Input(type=AssetTypes.CUSTOM_MODEL, path=transformations.id),\n",
    "    )\n",
    "    scored_data = score_data(\n",
    "        data=prepared_data.outputs.prepared_data,\n",
    "        model=Input(type=AssetTypes.MLFLOW_MODEL, path=model.id),\n",
    "        score_mode=score_mode,\n",
    "    )\n",
    "\n",
    "    return {\"scores\": scored_data.outputs.scores}"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.5 Test the pipeline\n",
    "\n",
    "Let's test the pipeline with some sample data. To do that, we'll create a job using the pipeline and the `batch-cluster` compute cluster created previously."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "name": "configure_pipeline_job"
   },
   "outputs": [],
   "source": [
    "pipeline_job = uci_heart_classifier_scorer(\n",
    "    input_data=Input(type=\"uri_folder\", path=\"data/unlabeled/\"), score_mode=\"append\"\n",
    ")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, we'll configure some run settings to run the test:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "name": "configure_pipeline_job_defaults"
   },
   "outputs": [],
   "source": [
    "pipeline_job.settings.default_datastore = \"workspaceblobstore\"\n",
    "pipeline_job.settings.default_compute = \"batch-cluster\""
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create the test job:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "name": "test_pipeline"
   },
   "outputs": [],
   "source": [
    "pipeline_job_run = ml_client.jobs.create_or_update(\n",
    "    pipeline_job, experiment_name=\"uci-heart-score-pipeline\"\n",
    ")\n",
    "pipeline_job_run"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "source": [
    "## 3 Create Batch Endpoint\n",
    "\n",
    "Batch endpoints are endpoints that are used batch inferencing on large volumes of data over a period of time. Batch endpoints receive pointers to data and run jobs asynchronously to process the data in parallel on compute clusters. Batch endpoints store outputs to a data store for further analysis.\n",
    "\n",
    "### 3.1 Configure the endpoint\n",
    "\n",
    "First, let's create the endpoint that is going to host the batch deployments. To ensure that our endpoint name is unique, let's create a random suffix to append to it. \n",
    "\n",
    "> In general, you won't need to use this technique but you will use more meaningful names. Please skip the following cell if your case:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "name": "name_endpoint"
   },
   "outputs": [],
   "source": [
    "endpoint_name = \"uci-classifier-score\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import random\n",
    "import string\n",
    "\n",
    "# Creating a unique endpoint name by including a random suffix\n",
    "allowed_chars = string.ascii_lowercase + string.digits\n",
    "endpoint_suffix = \"\".join(random.choice(allowed_chars) for x in range(5))\n",
    "endpoint_name = f\"{endpoint_name}-{endpoint_suffix}\"\n",
    "\n",
    "print(f\"Endpoint name: {endpoint_name}\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's configure the endpoint:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "jupyter": {
     "outputs_hidden": false,
     "source_hidden": false
    },
    "name": "configure_endpoint",
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "outputs": [],
   "source": [
    "endpoint = BatchEndpoint(\n",
    "    name=endpoint_name,\n",
    "    description=\"Batch scoring endpoint of the Heart Disease Data Set prediction task\",\n",
    ")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.2 Create the endpoint\n",
    "Using the `MLClient` created earlier, we will now create the Endpoint in the workspace. This command will start the endpoint creation and return a confirmation response while the endpoint creation continues."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "jupyter": {
     "outputs_hidden": false,
     "source_hidden": false
    },
    "name": "create_endpoint",
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "outputs": [],
   "source": [
    "ml_client.batch_endpoints.begin_create_or_update(endpoint).result()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can see the endpoint as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "name": "query_endpoint"
   },
   "outputs": [],
   "source": [
    "endpoint = ml_client.batch_endpoints.get(name=endpoint_name)\n",
    "print(endpoint)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. Deploy the pipeline component\n",
    "\n",
    "To deploy the pipeline component, we have to create a batch deployment. A deployment is a set of resources required for hosting the asset that does the actual work."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "source": [
    "### 4.1 Creating the component\n",
    "\n",
    "Our pipeline is defined in a function. We are going to create a component out of it. Pipeline components are reusable compute graphs that can be included in batch deployments or used to compose more complex pipelines."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "jupyter": {
     "outputs_hidden": false,
     "source_hidden": false
    },
    "name": "build_pipeline",
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "outputs": [],
   "source": [
    "pipeline_component = ml_client.components.create_or_update(\n",
    "    uci_heart_classifier_scorer().component\n",
    ")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.2 Configuring the deployment"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "jupyter": {
     "outputs_hidden": false,
     "source_hidden": false
    },
    "name": "configure_deployment",
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "outputs": [],
   "source": [
    "deployment = PipelineComponentBatchDeployment(\n",
    "    name=\"uci-classifier-prepros-xgb\",\n",
    "    description=\"A sample deployment with pre and post processing done before and after inference.\",\n",
    "    endpoint_name=endpoint.name,\n",
    "    component=pipeline_component,\n",
    "    settings={\"continue_on_step_failure\": False, \"default_compute\": compute_name},\n",
    ")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.3 Create the deployment\n",
    "Using the `MLClient` created earlier, we will now create the deployment in the workspace. This command will start the deployment creation and return a confirmation response while the deployment creation continues."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "jupyter": {
     "outputs_hidden": false,
     "source_hidden": false
    },
    "name": "create_deployment",
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "outputs": [],
   "source": [
    "ml_client.batch_deployments.begin_create_or_update(deployment).result()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once created, let's configure this new deployment as the default one:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "name": "update_default_deployment"
   },
   "outputs": [],
   "source": [
    "endpoint = ml_client.batch_endpoints.get(endpoint_name)\n",
    "endpoint.defaults.deployment_name = deployment.name\n",
    "ml_client.batch_endpoints.begin_create_or_update(endpoint).result()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see the endpoint URL as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(f\"The default deployment is {endpoint.defaults.deployment_name}\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "source": [
    "### 4.6 Testing the deployment\n",
    "\n",
    "Once the deployment is created, it is ready to recieve jobs."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 4.6.1 Run a batch job "
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The input data asset definition:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "name": "configure_inputs"
   },
   "outputs": [],
   "source": [
    "input_data = Input(type=\"uri_folder\", path=\"data/unlabeled/\")\n",
    "score_mode = Input(type=\"string\", default=\"append\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can invoke the default deployment as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "jupyter": {
     "outputs_hidden": false,
     "source_hidden": false
    },
    "name": "invoke_deployment",
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "outputs": [],
   "source": [
    "job = ml_client.batch_endpoints.invoke(\n",
    "    endpoint_name=endpoint.name,\n",
    "    inputs={\"input_data\": input_data, \"score_mode\": score_mode},\n",
    ")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "source": [
    "#### 4.6.2 Get the details of the invoked job\n",
    "\n",
    "Let us get details and logs of the invoked job:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "jupyter": {
     "outputs_hidden": false,
     "source_hidden": false
    },
    "name": "get_job",
    "nteract": {
     "transient": {
      "deleting": false
     }
    }
   },
   "outputs": [],
   "source": [
    "ml_client.jobs.get(job.name)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can wait for the job to finish using the following code:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "name": "stream_job_logs"
   },
   "outputs": [],
   "source": [
    "ml_client.jobs.stream(name=job.name)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 4.6.3 Access job outputs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Pipelines can define multiple outputs. When defined, use the name of the output to access its results. In our case, the pipeline contains an output called \"scores\":"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "name": "download_outputs"
   },
   "outputs": [],
   "source": [
    "ml_client.jobs.download(name=job.name, download_path=\".\", output_name=\"scores\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Read the scored data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "name": "read_outputs"
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import glob\n",
    "\n",
    "output_files = glob.glob(\"named-outputs/scores/*.csv\")\n",
    "score = pd.concat((pd.read_csv(f) for f in output_files))\n",
    "score"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice that the outputs of the pipeline may be different from the outputs of intermediate steps. In pipelines, each step is executed as child job. You can use `parent_job_name` to find all the child jobs from a given job:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pipeline_job_steps = {\n",
    "    step.properties[\"azureml.moduleName\"]: step\n",
    "    for step in ml_client.jobs.list(parent_job_name=job.name)\n",
    "}"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This dictonary contains the module name as key, and the job as values. This makes easier to work with them:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "preprocessing_job = pipeline_job_steps[\"uci_heart_prepare\"]\n",
    "score_job = pipeline_job_steps[\"uci_heart_score\"]"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Confirm the jobs' statuses using the following:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(f\"Preprocessing job: {preprocessing_job.status}\")\n",
    "print(f\"Scoring job: {score_job.status}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can also access the outputs of each of those intermediate steps as we did for the pipeline job."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5. Clean up resources\n",
    "\n",
    "Clean-up the resources created. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "name": "delete_endpoint"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "."
     ]
    }
   ],
   "source": [
    "ml_client.batch_endpoints.begin_delete(endpoint_name)"
   ]
  }
 ],
 "metadata": {
  "kernel_info": {
   "name": "amlv2"
  },
  "kernelspec": {
   "display_name": "previews",
   "language": "python",
   "name": "previews"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  },
  "nteract": {
   "version": "nteract-front-end@1.0.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}