Deployment with Kubernetes

Prerequisites)

Engine Version:
Check Releases.

Helm:
Helm 3.8.0+ installed. Helm charts hosted as OCI packages:

helm show chart oci://ghcr.io/arthur-ai/arthur-engine/charts/arthur-genai-engine:<version_number>

OpenAI GPT Model:
- OpenAI/Azure endpoint(s) with appropriate token limits.
DNS:
- DNS URL with SSL certificate for Engine.
Kubernetes Environment:
- Cluster tested on AWS EKS v1.31.
- kubectl admin access.
- Nginx ingress controller.
- Dedicated namespace, e.g. arthur.
- For CPU: at least 2x m8g.large nodes.
- For GPU: at least 2x g4dn.2xlarge nodes and metrics server.
- Managed PostgreSQL 15 database (pre-create, e.g., arthur_genai_engine).
Container Image Repository:
- Access to Docker Hub or configure private registry in values.yaml.

(Optional) GPU Node Group on AWS EKS

For GPU production deployment, configure your EKS cluster with a GPU node group.
Instructions for preparing user data, IAM, AMI, and autoscaling are provided above (see detailed guide).

Installation Steps (GenAI Engine)

Prepare a values.yaml configuration file based on values.yaml.template.
Create Kubernetes secrets (see below).

Install the Helm chart:

helm upgrade --install -n arthur -f values.yaml arthur-genai-engine oci://ghcr.io/arthur-ai/arthur-engine/charts/arthur-genai-engine --version <version_number>

Configure DNS: Create an A record pointing to the GenAI Engine's ingress load balancer.
Verify deployment:
```
kubectl get pods -n arthur
```
You should see GenAI Engine pods running. Inspect logs as needed.

Creating Kubernetes Secrets (example):

kubectl -n arthur create secret generic postgres-secret \
    --from-literal=username='<username>' \
    --from-literal=password='<password>'

kubectl -n arthur create secret docker-registry arthur-repository-credentials \
    --docker-server='registry-1.docker.io' \
    --docker-username='<username>' \
    --docker-password='<password>' \
    --docker-email=''

kubectl -n arthur create secret generic genai-engine-secret-admin-key \
    --from-literal=key='<key>'

kubectl -n arthur create secret generic genai-engine-secret-open-ai-gpt-model-names-endpoints-keys \
    --from-literal=keys='<your_gpt_keys>'

For production, use secure methods for secrets management (e.g. Sealed Secrets or external secret stores).

ML Engine Helm Chart Deployment

Create secrets for client credentials:

kubectl -n arthur create secret generic ml-engine-client-secret \
    --from-literal=client_id='<client_id>' \
    --from-literal=client_secret='<client_secret>'

Prepare a values.yaml file for ML Engine.

Install the chart:

helm upgrade --install -n arthur -f values.yaml arthur-ml-engine oci://ghcr.io/arthur-ai/arthur-engine/charts/arthur-ml-engine --version <version_number>

Verify ML Engine pods are running with:
```
kubectl get pods -n arthur
```

FAQ (Kubernetes)

Azure OpenAI Quota:
Use multiple endpoints for round-robin, or request a quota increase from Azure.