Skip to content

Commit

Permalink
azure account reqs, magic edits, misc.
Browse files Browse the repository at this point in the history
  • Loading branch information
MEM4GH committed May 13, 2024
1 parent 5fd0896 commit 6957c33
Show file tree
Hide file tree
Showing 14 changed files with 72 additions and 107 deletions.
16 changes: 9 additions & 7 deletions docs/explore-and-analyze-data/magic-commands.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,9 +26,9 @@ JupyterLab with the AI Unlimited kernel supports these magic commands in additio
```
Where:

- host: Name or IP address of the engine service.
- host: Name or IP address of the AI Unlimited manager.

- apikey: API Key value from the AI Unlimited setup **Profile** page.
- apikey: API key value from the AI Unlimited setup **Profile** page.

- **`[Optional]`** withTLS: If False (F), the default client-server communication does not use TLS.

Expand All @@ -49,7 +49,7 @@ Where:

- project: Name of the project to be created.

- env: Cloud environment where the project is hosted. The value can be aws, azure, gcp, or vsphere. For the current release, AWS and Azure are supported.
- env: Cloud environment where the project is hosted. For the current release, AWS and Azure are supported.

- **`[Optional]`** team: Name of the team collaborating on the project.

Expand Down Expand Up @@ -78,7 +78,9 @@ Where:
- **`[Optional]`** team: Name of the team collaborating on the project.

**Output**:
```Project `Project_Name` deleted```
```bash
Project `Project_Name` deleted
```

## %project_list

Expand Down Expand Up @@ -255,7 +257,7 @@ Where:

- **`[Optional]`** tags: The key-value pairs that are assigned to the engine for quick identification.

- **`[Optional]`** iamrole: The IAM Role used for the engine.
- **`[Optional]`** iamrole: The IAM role used for the engine.

- **`[Optional]`** roleprefix: The string appended to the beginning of the IAM role assigned to the engine.

Expand All @@ -266,7 +268,7 @@ Where:

- **`[Optional]`** network: The network to which you want to deploy the engine.

- **`[Optional]`** keyvault: The Key Vault used by the engine where sensitive information such as passwords can be securely stored.
- **`[Optional]`** keyvault: The key vault used by the engine where sensitive information such as passwords can be securely stored.

- **`[Optional]`** keyvaultresourcegroup:

Expand Down Expand Up @@ -425,7 +427,7 @@ Require input

## %help

**Description**: View the list of magics provided with AI-Unlimited-Teradata SQL CE Kernel.
**Description**: View the list of magic commands provided by the AI Unlimited kernel.

**Usage**:
```bash
Expand Down
2 changes: 1 addition & 1 deletion docs/explore-and-analyze-data/project-flow.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ pagination_next: null

You might be the owner of one or more projects and a collaborator on others.

AI Unlimited provides [magic commands](/docs/explore-and-analyze-data/magic-commands.md) (`%magic_command`) for creating and managing projects in your JupyterLab notebooks. While project owners and collaborators can use the same magic commands, their involvement is typically different.
AI Unlimited provides [magic commands](/docs/explore-and-analyze-data/magic-commands.md) (`%magic_command`) for creating and managing projects in Jupyter notebooks. While project owners and collaborators can use the same magic commands, their involvement is typically different.

Here is the project lifecycle from both points of view.

Expand Down
41 changes: 0 additions & 41 deletions docs/glossary.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,22 +23,6 @@ The manager orchestrates the deployment of the AI/ML engine on your cloud servic
Load balancers distribute incoming traffic across servers. An application load balancer inspects incoming traffic content, such as HTTP headers and paths, to make routing decisions. You can use an application load balancer as a way to expose AI Unlimited and its engine to the public IP network.


### term

Definition


## B

### term

Definition

### term

Definition


## C

<a id="glo-cidr"></a>
Expand Down Expand Up @@ -66,22 +50,13 @@ A group of interconnected computers that work together as a single system for en
A central repository of data for data analytics, machine learning, and data exploration. A data lake is typically built on [object storage](#glo-object-storage) and is designed for data accessibility for future data analysis.


### term

Definition


## I

<a id="glo-iam-role"></a>
### IAM role

An [Identity and Access Management (IAM) role](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles.html) is an identity you can create in your AWS account. It has permissions policies that determine what it can and cannot do in AWS. An IAM user is unique to one user, but an IAM role can be shared by users.

### term

Definition


## L

Expand All @@ -98,10 +73,6 @@ A load balancer distributes incoming traffic evenly across servers, which improv

A concept in containerized platforms such as Docker. You can map a directory on the machine running the containers (the host) to a directory in a container to share data and files between the host and container.

### term

Definition


## N

Expand All @@ -123,11 +94,6 @@ An OAuth app allows a user to grant access to their account on one website or se
A scalable, highly effective way to store large amounts of structured, semi-structured, or unstructured data in the cloud. Amazon S3 is object storage on AWS. Azure Blob Storage is object storage on Azure. Object storage is the basis for a [data lake](#glo-data-lake).


### term

Definition


## S

<a id="glo-systemd"></a>
Expand All @@ -136,18 +102,11 @@ Definition
Linux foundational software for system and service management. When you deploy the AI Unlimited or JupyterLab CloudFormation template on AWS, it runs on a server instance in a container controlled by systemd.


### term

Definition


## T

<a id="glo-tls"></a>
### Transport Layer Security (TLS)

A protocol that creates a secure connection between two parties communiating over the internet. TLS encrypts data during transmission, making it unreadable to anyone who intercepts it, and it verifies the identities of both parties.

### term

Definition
Original file line number Diff line number Diff line change
Expand Up @@ -21,20 +21,20 @@ For installation support, ask the [community](https://support.teradata.com/commu
:::


## Download the manager template
## Choose a manager template

CloudFormation templates for the manager are here in the AI Unlimited GitHub repository you cloned:

`deployments/aws/templates/ai-unlimited/`

1. Choose a template based on the type of [load balancer](/docs/glossary.md#glo-load-balancer) you want to use.<br />
:::note
You might want to ask a cloud admin at your organization for guidance.
:::
Choose a template based on the type of [load balancer](/docs/glossary.md#glo-load-balancer) you want to use.<br />
:::note
You might want to ask a cloud admin at your organization for guidance.
:::
- `ai-unlimited-with-alb.yaml`&mdash;Hosts the manager behind an [application load balancer](/docs/glossary.md#glo-application-load-balancer)
- `ai-unlimited-with-nlb.yaml`&mdash;Hosts the manager behind a [network load balancer](/docs/glossary.md#glo-network-load-balancer)
- `ai-unlimited-without-lb.yaml`&mdash;No load balancer. If you're unsure about which template to use, we recommend this one.
2. Download the template.


## Load the template

Expand All @@ -47,7 +47,7 @@ We recommend selecting the region closest to your primary work location.
3. Search for and go to **CloudFormation**.
4. Select **Create Stack**, then **With new resources (standard)**.
5. Select **Choose an existing template**, then **Upload a template file**.
6. Choose the template file you downloaded, and click **Next**.
6. Select the template file you chose to use, and click **Next**.

<a id="aws-parms"></a>
## Specify stack details and options
Expand Down
20 changes: 10 additions & 10 deletions docs/install-ai-unlimited/prod-azure-portal-deploy-manager.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,19 +21,19 @@ For installation support, ask the [community](https://support.teradata.com/commu
:::


## Download the manager template
## Choose a manager template

Azure Resource Manager (ARM) templates for the manager are here in the AI Unlimited GitHub repository you cloned:
ARM templates for the manager are here in the AI Unlimited GitHub repository you cloned:

`develop/deployments/azure/templates/arm/ai-unlimited`
`deployments/azure/templates/arm/ai-unlimited`

Choose a template based on the type of [load balancer](/docs/glossary.md#glo-load-balancer) you want to use.<br />
:::note
You might want to ask a cloud admin at your organization for guidance.
:::
- `ai-unlimited-with-alb.yaml`&mdash;Hosts the manager behind an [application load balancer](/docs/glossary.md#glo-application-load-balancer)
- `ai-unlimited-with-nlb.yaml`&mdash;Hosts the manager behind a [network load balancer](/docs/glossary.md#glo-network-load-balancer)
- `ai-unlimited-without-lb.yaml`&mdash;No load balancer. If you're unsure about which template to use, we recommend this one.
:::note
You might want to ask a cloud admin at your organization for guidance.
:::
- `ai-unlimited-with-alb.json`&mdash;Hosts the manager behind an [application load balancer](/docs/glossary.md#glo-application-load-balancer)
- `ai-unlimited-with-nlb.json`&mdash;Hosts the manager behind a [network load balancer](/docs/glossary.md#glo-network-load-balancer)
- `ai-unlimited-without-lb.json`&mdash;No load balancer. If you're unsure about which template to use, we recommend this one.


## Load the template
Expand All @@ -44,7 +44,7 @@ Choose a template based on the type of [load balancer](/docs/glossary.md#glo-loa
:::
2. Select **Deploy a custom template**.
3. Select **Build your own template in the editor**, then **Load file**.
4. Choose the template file you downloaded, then select **Save**.
4. Select the template file you chose to use, then select **Save**.

<a id="azure-parms"></a>
## Specify instance details
Expand Down
2 changes: 1 addition & 1 deletion docs/install-ai-unlimited/setup-ai-unlimited.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ By setting up AI Unlimited, you become the AI Unlimited admin at your organizati
For setup support, ask the [community](https://support.teradata.com/community?id=community_forum&sys_id=b0aba91597c329d0e6d2bd8c1253affa).
:::


<a id="setup-fields"></a>
## Complete the fields

***(changes to the UI - info in these sections is back to WIP)***
Expand Down
4 changes: 2 additions & 2 deletions docs/manage-ai-unlimited/add-collaborators.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,12 +18,12 @@ If you are the project owner, you can add collaborators to your project. The ste
1. In GitHub or GitLab, add users to the project repository.<br/>
See details about adding repo users for [GitHub](https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/managing-repository-settings/managing-teams-and-people-with-access-to-your-repository) or [GitLab](https://docs.gitlab.com/ee/user/project/members/#add-users-to-a-project).

2. When you deploy the engine from your JupyterLab notebook, the users are added to the engine automatically.
2. When you deploy the engine from your Jupyter notebook, the users are added to the engine automatically.


## After the engine is deployed

1. In GitHub or GitLab, add users to the project repository.<br/>
See details about adding repo users for [GitHub](https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/managing-repository-settings/managing-teams-and-people-with-access-to-your-repository) or [GitLab](https://docs.gitlab.com/ee/user/project/members/#add-users-to-a-project).
2. In your JupyterLab notebook, run the magic command `%project_engine_update_users` to add the users to the engine.
2. In your Jupyter notebook, run the magic command `%project_engine_update_users` to add the users to the engine.

4 changes: 2 additions & 2 deletions docs/manage-ai-unlimited/suspend-and-restore-project.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,15 +16,15 @@ All project information, including user and object store authorizations and data

## Suspend

In the project, run this:
In your notebook, run this:

```bash
%project_engine_suspend <Project_Name>
```

## Restore

In the project, run this:
In your notebook, run this:

```bash
%project_restore project=<Project_Name>, gitref=<Git_Reference>
Expand Down
17 changes: 8 additions & 9 deletions docs/resources/aws-requirements.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,30 +23,29 @@ pagination_next: null

## Allow AI Unlimited to create roles

The manager needs a role that allows it to deploy the engine. The engine needs a role that allows the engine nodes to communicate.
The manager needs a role that allows it to deploy the engine. The engine needs a role that allows the engine nodes to communicate. You can let AI Unlimited create both of these roles.

:::note
If your security does not allow AI Unlimited to create roles, [create the roles](#provide-roles) yourself or with the help of your cloud admin.
:::

**Role for the manager**

To let AI Unlimited create this role for you, when you [specify the stack details](/docs/install-ai-unlimited/prod-aws-console-deploy-ai-unlimited.md#aws-parms), provide these values for these parameters:
- `IamRole`: **New**
- `IamRoleName`: leave blank


**Role for the engine**

Let AI Unlimited create a new role for the engine each time the engine is deployed.
AI Unlimited can create a new role for the engine each time the engine is deployed.

The engine-specific policies AI Unlimited creates are restricted this way:

```bash
"Resource": ["arn:aws:secretsmanager:`REGION`:`ACCOUNT_ID`:secret:compute-engine/`CLUSTER_NAME`/`SECRET_NAME`"]
```

To allow this, when you [set up AI Unlimited](/docs/install-ai-unlimited/setup-ai-unlimited), leave the **Default IAM role** field blank.

:::note
If your security does not allow AI Unlimited to create roles, create the roles yourself or with the help of your cloud admin.
:::
To allow AI Unlimited to create a new role for each engine deployment, when you configure the [cloud integration](/docs/install-ai-unlimited/setup-ai-unlimited#setup-fields) as part of [AI Unlimited setup](/docs/install-ai-unlimited/setup-ai-unlimited), leave the **Default IAM role** field blank.


<a id="provide-roles"></a>
Expand Down Expand Up @@ -102,7 +101,7 @@ If you are providing the manager's role, use the samples you need in an attached
If you use [ai-unlimited-engine.json](https://github.com/Teradata/ai-unlimited/blob/develop/deployments/aws/policies/ai-unlimited-engine.json) for the engine's role, use [ai-unlimited-without-iam-role-permissions.json](https://github.com/Teradata/ai-unlimited/blob/develop/deployments/aws/policies/ai-unlimited-workspaces-without-iam-role-permissions.json) for the manager's role.
:::

- When you [set up AI Unlimited](/docs/install-ai-unlimited/setup-ai-unlimited.md), put the role name in the **Default IAM role** field.
- When you configure the [cloud integration](/docs/install-ai-unlimited/setup-ai-unlimited#setup-fields) as part of [AI Unlimited setup](/docs/install-ai-unlimited/setup-ai-unlimited), put the role name in the **Default IAM role** field.



Expand Down
22 changes: 8 additions & 14 deletions docs/resources/azure-requirements.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,30 +10,24 @@ pagination_next: null

# Azure account requirements

***(This topic is still being worked - please disregard for now.)***

Prepare your Azure account:

- Work with your cloud admin to ensure your Azure account has the permissions needed to create the cloud resources definined in the [AI Unlimited template](https://github.com/Teradata/ai-unlimited/blob/develop/deployments/azure/ai-unlimited.json).
- Work with your cloud admin to ensure your Azure account has the permissions needed to create the cloud resources definined in the [AI Unlimited template](https://github.com/Teradata/ai-unlimited/tree/develop/deployments/azure/templates/arm/ai-unlimited).

- Networking requirements: Your Azure [resource group](https://learn.microsoft.com/en-us/azure/azure-resource-manager/management/manage-resource-groups-portal) must have an [Azure Virtual Network (VNet)](https://learn.microsoft.com/en-us/azure/virtual-network/quick-create-portal) configured with a [subnet](https://learn.microsoft.com/en-us/azure/virtual-network/virtual-network-manage-subnet?tabs=azure-portal). Use an existing VNet or subnet, or create your own, depending on your account permissions.

***(This roll link was included, how does it fit with the context? [role](https://learn.microsoft.com/en-us/azure/role-based-access-control/custom-roles))***

- If you'll need to access the manager instance, after it is installed, to run commands or debug, you can use a [key pair](https://learn.microsoft.com/en-us/azure/virtual-machines/linux/mac-create-ssh-keys) that you generate to securely connect using Secure Shell (SSH). You'll need the key pair when you [specify the instance details](/docs/install-ai-unlimited/prod-azure-portal-deploy-manager.md#azure-parms).


## Create the role required by the manager
- If you'll need to access the manager instance, after it is installed, to run commands or debug, you can use a [key pair](https://learn.microsoft.com/en-us/azure/virtual-machines/ssh-keys-portal) that you generate to securely connect using Secure Shell (SSH). You'll need the key pair when you [specify the instance details](/docs/install-ai-unlimited/prod-azure-portal-deploy-manager.md#azure-parms).

The manager's role allows it to deploy the engine. Each time the manager deploys the engine, the manager creates a new role for the engine to enable its nodes to communicate.

***(Jack is looking into whether we provide the role or if the user has to provide it.)***
## Create the role for the manager

If you have the permissions needed to create roles, create the role for the manager using the [role-policy](https://github.com/Teradata/ai-unlimited/blob/develop/deployments/azure/role-policy.json) template. ***(with the [ai-unlimited.json](https://github.com/Teradata/ai-unlimited/blob/develop/deployments/azure/policies/ai-unlimited.json) policy or is that baked in? verify)***
The manager needs a role that allows it to do the following:
- Deploy the engine.
- Each time it deploys the engine, create a role for the engine that allows the engine's nodes to communicate.

See [Create or update Azure custom roles using the Azure portal](https://learn.microsoft.com/en-us/azure/role-based-access-control/custom-roles-portal) for details.
If you have the permissions needed to create roles, create the role for the manager using the [role-policy](https://github.com/Teradata/ai-unlimited/tree/develop/deployments/azure/templates/arm/init/role-policy.json) ARM template. See [Create or update Azure custom roles using an ARM template](https://learn.microsoft.com/en-us/azure/role-based-access-control/custom-roles-template).

Or share [ai-unlimited.json](https://github.com/Teradata/ai-unlimited/blob/develop/deployments/azure/policies/ai-unlimited.json) with your cloud admin they can create the custom role for you.
Or share the [ai-unlimited.json](https://github.com/Teradata/ai-unlimited/blob/develop/deployments/azure/policies/ai-unlimited.json) policy, which contains the permissions the manager needs, with your cloud admin so they can create the role for you.

You'll need the `RoleDefinitionId` when you [specify the instance details](/docs/install-ai-unlimited/prod-azure-portal-deploy-manager.md#azure-parms).

Expand Down
12 changes: 6 additions & 6 deletions docs/resources/jupyterlab/install-jupyterlab-aws.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,20 +34,20 @@ References to the AWS Management Console are accurate as of April 11, 2024.



## Download the Jupyter template
## Choose a Jupyter template

CloudFormation templates for JupyterLab are here in the AI Unlimited GitHub repository you cloned:

`deployments/aws/templates/jupyter/`

1. Choose a template based on whether you intend to use a [load balancer](/docs/glossary.md#glo-load-balancer) and what type.

Choose a template based on whether you intend to use a [load balancer](/docs/glossary.md#glo-load-balancer) and what type.
:::note
You might want to ask a cloud admin at your organization for guidance.
:::
- `jupyter-alb.yaml`&mdash;Hosts JupyterLab behind an [application load balancer](/docs/glossary.md#glo-application-load-balancer)
- `jupyter-with-nlb.yaml`&mdash;Hosts JupyterLab behind a [network load balancer](/docs/glossary.md#glo-network-load-balancer)
- `jupyter-without-lb.yaml`&mdash;No load balancer

2. Download the template.


## Upload the template
Expand All @@ -56,7 +56,7 @@ CloudFormation templates for JupyterLab are here in the AI Unlimited GitHub repo
3. Search for and go to **CloudFormation**.
4. Select **Create Stack**, then **With new resources (standard)**.
5. Select **Choose an existing template** and **Upload a template file**.
6. Choose the template file you downloaded, and click **Next**.
6. Select the template file you chose to use, and click **Next**.


<a id="jup-aws-parms"></a>
Expand Down
Loading

0 comments on commit 6957c33

Please sign in to comment.