openbox-backend

Purpose

openbox-backend is the control-plane REST API for the OpenBox platform. It exposes endpoints for user management, agent CRUD, workflow triggers, audit log retrieval, and admin operations. The openbox-fe frontend and any external integrations talk to it directly.

Plane: Control plane.

Ports & endpoints

Port	Protocol	Purpose	Exposed via
3000	HTTP/REST	Platform API	Istio Gateway / ALB / NGINX Ingress
9464	HTTP	Prometheus /metrics scrape	Cluster-internal only (not routed via ingress)
3000/health	HTTP	Liveness probe	K8s kubelet
3000/readyz	HTTP	Readiness probe	K8s kubelet

Public endpoints follow /api/v1/* naming. Full OpenAPI spec at /api/v1/openapi.json (once running).

Runtime

Property	Value
Container count per pod	1
Resource requests	500m CPU / 1 Gi RAM
Resource limits	1500m CPU / 2 Gi RAM
HPA min → max replicas	1 → 8 (target CPU 70%)
Node placement	General-purpose CPU pool (no GPU / DB / memory taint)
StatefulSet?	No — stateless Deployment (all state in Postgres + Redis)

Bin-packs cleanly on m6i.large and larger. On dev clusters, minReplicas: 1 is safe.

Configuration

Chart values keys (top-level openbox-backend.*)

Authoritative reference: values-reference. Most-configured keys:

Key	Purpose
image.repository / image.tag	Container image + pinned version
replicaCount	Static replica count (ignored when HPA is on)
autoscaling.{enabled,minReplicas,maxReplicas,targetCPUUtilizationPercentage}	HPA tuning
resources	Requests/limits
serviceAccount.annotations	IRSA / Workload Identity annotations
env.KMS_KEY_ARN	AWS KMS CMK ARN for envelope encryption
env.OPA_BUNDLE_BUCKET	S3 bucket holding OPA policy bundles
db.host / db.port / db.name / db.userSecretRef	External DB (overrides in-cluster Bitnami PG)

Runtime env vars (populated by chart from values)

• DATABASE_URL — computed from db.* values or in-cluster Postgres service
• KEYCLOAK_URL — points at identity-service.<ns>.svc:8080
• OPA_URL — points at opa-app.<ns>.svc:8181
• GUARDRAILS_URL — points at guardrails-api.<ns>.svc:8000
• KMS_KEY_ARN, OPA_BUNDLE_BUCKET — from values
• NODE_ENV=production

IRSA / IAM permissions required (AWS)

If deploying on EKS, the ServiceAccount must be annotated with an IAM role that grants:

• kms:Encrypt / kms:Decrypt / kms:GenerateDataKey / kms:DescribeKey on your KMS CMK
• s3:GetObject / s3:PutObject / s3:DeleteObject / s3:ListBucket on your OPA bundles bucket

See Terraform snippet 04-iam-irsa-roles.tf or the CloudFormation BackendIrsaRole in openbox-infra.yaml.

Dependencies

Downstream (what openbox-backend calls)

Target	Protocol	Purpose
identity-service (Keycloak)	HTTPS OIDC	User authentication + JWT verification
opa-app	HTTP/JSON :8181	Authorization decisions
guardrails-api	HTTP/REST :8000	ML guardrail checks
openbox-postgresql	JDBC :5432	App transactional storage
AWS KMS	HTTPS AWS API	Envelope encryption
AWS S3	HTTPS AWS API	Occasional bundle reads

Upstream (what calls openbox-backend)

• openbox-fe (via ingress) — frontend UI
• openbox-core-server (cluster-internal) — for admin operations
• External customer integrations (via ingress) — B2B APIs

Deployment guidelines

Prod

• Set autoscaling.minReplicas: 2 (never single-pod in prod — avoid single-pod restart downtime)
• Enable PodDisruptionBudget with maxUnavailable: 1
• Pin image.tag explicitly (no :latest)
• Wire IRSA — do NOT bake AWS static credentials into the pod
• Use external RDS/Aurora for DB — not in-cluster Bitnami PG

Local PoC (S8)

• autoscaling.minReplicas: 1, maxReplicas: 1 — laptops don't need scale
• In-cluster Bitnami PG is fine
• Skip IRSA — chart uses default ServiceAccount

Troubleshooting

Pod CrashLoopBackOff with AccessDeniedException in logs → IRSA missing / wrong. Verify SA annotation: kubectl get sa openbox-backend -n openbox -o yaml | grep eks.amazonaws.com

Backend returns 401 for all requests → Keycloak realm/client mismatch. Log into Keycloak admin → verify realm openbox exists and has a openbox-backend client with correct redirect URIs.

HPA reports <unknown> target CPU → metrics-server not installed. Deploy from official manifests.

High p99 latency on /api/v1/* → Check downstream latency: OPA + Keycloak + guardrails-api. See Data flows → latency targets. Backend has minimal own logic — most latency is external.

Pod Ready but request returns 502 through ingress → Istio sidecar not injected. Verify namespace label: kubectl get ns openbox -o yaml | grep istio-injection. Restart pods after adding the label.