openbox-backend

Purpose

openbox-backend is the control-plane REST API for the OpenBox platform. It exposes endpoints for user management, agent CRUD, workflow triggers, audit log retrieval, and admin operations. The openbox-fe frontend and any external integrations talk to it directly.

Plane: Control plane.

Ports & endpoints

Port	Protocol	Purpose	Exposed via
3000	HTTP/REST	Platform API	Istio Gateway / ALB / NGINX Ingress
9464	HTTP	Prometheus /metrics scrape	Cluster-internal only (not routed via ingress)
3000/health	HTTP	Liveness probe	K8s kubelet
3000/readyz	HTTP	Readiness probe	K8s kubelet

Public endpoints follow /api/v1/* naming. Full OpenAPI spec at /api/v1/openapi.json (once running).

Runtime

Property	Value
Container count per pod	1
Resource requests	500m CPU / 1 Gi RAM
Resource limits	1500m CPU / 2 Gi RAM
HPA min → max replicas	1 → 8 (target CPU 70%)
Node placement	General-purpose CPU pool (no GPU / DB / memory taint)
StatefulSet?	No — stateless Deployment (all state in Postgres + Redis)

Bin-packs cleanly on m6i.large and larger. On dev clusters, minReplicas: 1 is safe.

Configuration

Chart values keys (top-level openbox-backend.*)

Authoritative reference: values-reference. Most-configured keys:

Key	Purpose
image.repository / image.tag	Container image + pinned version
replicaCount	Static replica count (ignored when HPA is on)
autoscaling.{enabled,minReplicas,maxReplicas,targetCPUUtilizationPercentage}	HPA tuning
resources	Requests/limits
serviceAccount.annotations	IRSA / Workload Identity annotations
env.KMS_KEY_ARN	AWS KMS CMK ARN for envelope encryption
env.OPA_BUNDLE_BUCKET	S3 bucket holding OPA policy bundles
db.host / db.port / db.name / db.userSecretRef	External DB (overrides in-cluster Bitnami PG)

Runtime env vars (populated by chart from values)

• DATABASE_URL — computed from db.* values or in-cluster Postgres service
• KEYCLOAK_URL — points at identity-service.<ns>.svc:8080
• OPA_URL — points at opa-app.<ns>.svc:8181
• GUARDRAILS_URL — points at guardrails-api.<ns>.svc:8000
• KMS_KEY_ARN, OPA_BUNDLE_BUCKET — from values
• NODE_ENV=production

IRSA / IAM permissions required (AWS)

If deploying on EKS, the ServiceAccount must be annotated with an IAM role that grants:

• kms:Encrypt / kms:Decrypt / kms:GenerateDataKey / kms:DescribeKey on your KMS CMK
• s3:GetObject / s3:PutObject / s3:DeleteObject / s3:ListBucket on your OPA bundles bucket

The CDK path creates the key, the bundle bucket and these roles for you. On other paths, grant them to the backend's service account yourself.

Dependencies

Downstream (what openbox-backend calls)

Target	Protocol	Purpose
identity-service (Keycloak)	HTTPS OIDC	User authentication + JWT verification
opa-app	HTTP/JSON :8181	Authorization decisions
guardrails-api	HTTP/REST :8000	ML guardrail checks
openbox-postgresql	JDBC :5432	App transactional storage
AWS KMS	HTTPS AWS API	Envelope encryption
AWS S3	HTTPS AWS API	Occasional bundle reads

Upstream (what calls openbox-backend)

• openbox-fe (via ingress) — frontend UI
• openbox-core-server (cluster-internal) — for admin operations
• External customer integrations (via ingress) — B2B APIs

Deployment guidelines

Prod

• Set autoscaling.minReplicas: 2 (never single-pod in prod — avoid single-pod restart downtime)
• Enable PodDisruptionBudget with maxUnavailable: 1
• Pin image.tag explicitly (no :latest)
• Wire IRSA — do NOT bake AWS static credentials into the pod
• Use external RDS/Aurora for DB — not in-cluster Bitnami PG

Local PoC

• autoscaling.minReplicas: 1, maxReplicas: 1 — laptops don't need scale
• In-cluster Bitnami PG is fine
• Skip IRSA — chart uses default ServiceAccount

Troubleshooting

Pod CrashLoopBackOff with AccessDeniedException in logs → IRSA missing / wrong. Verify SA annotation: kubectl get sa openbox-backend -n openbox -o yaml | grep eks.amazonaws.com

Backend returns 401 for all requests → Keycloak realm/client mismatch. Log into Keycloak admin → verify realm openbox exists and has a openbox-backend client with correct redirect URIs.

HPA reports <unknown> target CPU → metrics-server not installed. Deploy from official manifests.

High p99 latency on /api/v1/* → Check downstream latency: OPA + Keycloak + guardrails-api. See Data flows → latency targets. Backend has minimal own logic — most latency is external.

Pod Ready but request returns 502 through ingress → Istio sidecar not injected. Verify namespace label: kubectl get ns openbox -o yaml | grep istio-injection. Restart pods after adding the label.