terraform: add AWS/EKS deployment for ChatQnA

[poussa]

Description

Add AWS/EKS terraform deployment for OPEA applications. This PR add support for ChatQnA.

Followup PRs add more cloud providers and OPEA applications.

Issues

opea-project/GenAIExamples#427
opea-project/GenAIExamples#551
opea-project/GenAIExamples#550
opea-project/GenAIExamples#549
opea-project/GenAIExamples#510

Type of change

List the type of change like below. Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds new functionality)
• Breaking change (fix or feature that would break existing design and interface)

Dependencies

None

Tests

None

[poussa]

@mkbhanda @daisy-ycguo PTAL.

This has now been tested on AWS EKS.

[lianhao]

@poussa Could you pls fix this DCO issue by sing-off your commit? i.e. git commit -s

[poussa]

@poussa Could you pls fix this DCO issue by sing-off your commit? i.e. git commit -s

This is now fixed.

@lianhao @mkbhanda

[mkbhanda]

minor commends and a few questions. Thank you!

[poussa]

@mkbhanda thanks for the review. I fixed most of your suggestions and commented the rest.

[lucasmelogithub]

@mkbhanda

Thoughts on a folder structure that is more scalable? I also have Terraform examples and Ansible code to commit from opea-project/GenAIExamples#970

Suggestion: GenAIExamples/ChatQnA/terraform/intel/cpu/xeon/aws/eks/main.tf
This follows the docker_compose already defined structure.

We'll also have Ansible, so it would scale to GenAIExamples/ChatQnA/ansible/intel/cpu/xeon/ubuntu/single-node/playbook.yaml

[poussa]

My idea of the terraform structure is:

cloud-service-provider
|--aws
    |--aks
        |---[terraform files] and <opea-app.tfvars> # instance type etc. defined in .tfvars
    |--fargate
        |---[terraform files] and <opea-app.tfvars> # instance type etc. defined in .tfvars
    |--- [other relevant AWS services]
|--gcp
    |--gke
        |---[terraform files] and <opea-app.tfvars> # instance type etc. defined in .tfvars
    |--vertex-ai
        |---[terraform files] and <opea-app.tfvars> # instance type etc. defined in .tfvars
    |--- [other relevant GCP services]
|--azure
    |--aks
        |---[terraform files] and <opea-app.tfvars> # instance type etc. defined in .tfvars
    |--- [other relevant Azure services]

This way we don't need to expose app name (e.g. chatqna), vendor name (e.g. intel), product names (e.g., xeon) or accelerator type (e.g. gaudi) in the directory structure. They are hidden in .tfvars config files or terraform command line arguments (-var foo=bar). There can be multiple .tfvars files per directory to capture multiple apps, accelerators, and cpu configurations.

NOTE: the top level directory can be in any repo but that is a separate topic.

[lucasmelogithub]

My idea of the terraform structure is:

cloud-service-provider
|--aws
    |--aks
        |---[terraform files] and <opea-app.tfvars> # instance type etc. defined in .tfvars
    |--fargate
        |---[terraform files] and <opea-app.tfvars> # instance type etc. defined in .tfvars
    |--- [other relevant AWS services]
|--gcp
    |--gke
        |---[terraform files] and <opea-app.tfvars> # instance type etc. defined in .tfvars
    |--vertex-ai
        |---[terraform files] and <opea-app.tfvars> # instance type etc. defined in .tfvars
    |--- [other relevant GCP services]
|--azure
    |--aks
        |---[terraform files] and <opea-app.tfvars> # instance type etc. defined in .tfvars
    |--- [other relevant Azure services]

This way we don't need to expose app name (e.g. chatqna), vendor name (e.g. intel), product names (e.g., xeon) or accelerator type (e.g. gaudi) in the directory structure. They are hidden in .tfvars config files or terraform command line arguments (-var foo=bar). There can be multiple .tfvars files per directory to capture multiple apps, accelerators, and cpu configurations.

NOTE: the top level directory can be in any repo but that is a separate topic.

Ok, I think we are looking to solve two different problems. I think you are going for generic infrastructure for any use case.
I would at least suggest a more flexible pattern, use GenAIInfra/scripts/terraform/<aws|azure|gcp>

[lucasmelogithub]

A suggestion, remove the default so that an input is required and it prompts the user if missing.

[mkbhanda]

should the region also not have a default. Alternately a default region establishes that we can find the instance type mentioned in that region.

[poussa]

Ok, I think we are looking to solve two different problems. I think you are going for generic infrastructure for any use case. I would at least suggest a more flexible pattern, use GenAIInfra/scripts/terraform/<aws|azure|gcp>

terraform we should add, since there will be other deployment methods like aws/cloudformation, maybe ansible. So that should have a place in the path structure.

[poussa]

Of course, this is inspired by https://github.com/NVIDIA/nim-deploy/tree/main/cloud-service-providers

[lucasmelogithub]

Ok, I think we are looking to solve two different problems. I think you are going for generic infrastructure for any use case. I would at least suggest a more flexible pattern, use GenAIInfra/scripts/terraform/<aws|azure|gcp>

terraform we should add, since there will be other deployment methods like aws/cloudformation, maybe ansible. So that should have a place in the path structure.

Right, that was the idea. And GenAIInfra/scripts... supports Ansible, Bash, etc which should not be under /cloud-service-providers/...

[mkbhanda]

My idea of the terraform structure is:

cloud-service-provider
|--aws
    |--aks
        |---[terraform files] and <opea-app.tfvars> # instance type etc. defined in .tfvars
    |--fargate
        |---[terraform files] and <opea-app.tfvars> # instance type etc. defined in .tfvars
    |--- [other relevant AWS services]
|--gcp
    |--gke
        |---[terraform files] and <opea-app.tfvars> # instance type etc. defined in .tfvars
    |--vertex-ai
        |---[terraform files] and <opea-app.tfvars> # instance type etc. defined in .tfvars
    |--- [other relevant GCP services]
|--azure
    |--aks
        |---[terraform files] and <opea-app.tfvars> # instance type etc. defined in .tfvars
    |--- [other relevant Azure services]

This way we don't need to expose app name (e.g. chatqna), vendor name (e.g. intel), product names (e.g., xeon) or accelerator type (e.g. gaudi) in the directory structure. They are hidden in .tfvars config files or terraform command line arguments (-var foo=bar). There can be multiple .tfvars files per directory to capture multiple apps, accelerators, and cpu configurations.

NOTE: the top level directory can be in any repo but that is a separate topic.

I am in favor of this organization for the same reasons that @poussa cites and also agree keeping terraform as part of the path to differentiate between cloud formation etc. Thank you @lucasmelogithub for reviewing and providing comments. I am OK with a default name of OPEA for the cluster, keep it easy. For a more savvy user with multiple clusters, they can change the vars file.

[mkbhanda]

LGTM

[mkbhanda]

should we just call it opea?

[mkbhanda]

should the region also not have a default. Alternately a default region establishes that we can find the instance type mentioned in that region.