One Helm chart, every environment
ArgoCD and ephemeral envs included

Twenty-plus services, half a dozen long-lived environments, and tickets that deserve a cluster namespace of their own for a week. This is the Helm practice that carries that: one generic chart that works as a resource compiler, small per-environment value files, ArgoCD applications as the only deployment unit — and ephemeral environments that cost one values file and one Application manifest.

The Helm chart problem

Helm's happy path is one chart per service. At twenty services that means twenty copies of the same Deployment/Service/Ingress boilerplate drifting apart, twenty places to fix a pod-anti-affinity bug, and an "environment" that exists only as a wall of --set flags. Two deeper problems make naive layouts worse:

• Helm merges maps but replaces lists. An overlay that wants to add one app to a list must restate the entire list — so every environment file silently owns a full copy of the world, and forgetting one entry deletes a service from an environment.
• Environment ≠ set of services. Dev runs experimental services prod has never seen; an ephemeral ticket env needs six services out of twenty. If the chart hard-codes what exists, every difference becomes a fork.

The solution: catalog + manifest

The chart's values.yaml plays two roles at once. It is a catalog: one x-app-* block per service holding its full configuration (image repo, env files, resources, node placement). And it is a manifest: plain lists (apps, cronjobs, jobs, persistentVolumes) of references into the catalog. Templates never iterate services directly — they iterate the manifest and resolve each reference:

{{- range (concat .Values.apps .Values.appsExtension) }}
{{- with (index $.Values .appRef) }}
{{- if not .ignore }}
# ... one Deployment/Service/SecretProviderClass/... per resolved app
{{- end }}
{{- end }}
{{- end }}

Three consequences fall out. An environment that wants fewer services overrides the manifest, not the catalog — six lines, not six hundred. An environment that wants to add services appends to appsExtension — the extension-list twin every list has, which sidesteps Helm's list-replacement without restating the base list. And tuning one service in one environment is a partial override of its catalog block, because catalog blocks are maps and maps deep-merge.

# chart values.yaml — the catalog entry (defaults for every env)
x-app-backoffice:
  name: backoffice
  applyPodResourcesQuota: true
  commonEnv:
    - file: common.env
    - file: service-discovery.env
  appEnv:
    - file: backoffice.env
  deployment:
    nodeGroup: none
    imageRepo: backoffice
    tag: not-found            # sentinel: a real tag MUST come from the env values
    resources:
      requests: { cpu: 500m, memory: 600Mi }

# values-stage.yml — the overlay touches only what differs
x-app-backoffice:
  deployment:
    tag: stage-1707129600     # CI-written build tag
    replicas: 2
  appEnvExtension:
    - file: stage/backoffice.env   # adds, never restates appEnv

How the chart renders resources

The logical schema: value files merge left to right (maps deep-merge, later wins), the compiler loop resolves the manifest against the catalog, and each resolved entry emits a fixed bundle of resources. Dashed boxes are per-app; one app in the manifest ≈ six Kubernetes objects out.

The per-app bundle is where platform policy lives once: node placement via nodeGroup/tier affinity and Karpenter taints, zone-spread anti-affinity, an IRSA service account per app, and a CSI SecretProviderClass so secrets come from the cloud secret store at mount time — the runtime sibling of the build-time secret rule from the Docker article: credentials live in neither layers nor manifests.

Chart + real values = ArgoCD app

Nothing deploys the chart by hand. An environment is an ArgoCD Application: the chart path, an ordered stack of value files, and a destination. The value-file order is the whole configuration model — base first, then each overlay, later files winning:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: core-stage
spec:
  project: stage
  source:
    repoURL: 'git@…/orchestrator.git'
    path: k8s/charts/core
    targetRevision: master
    helm:
      valueFiles:
        - values.yaml                            # catalog + default manifest
        - ../../helm-values/values-prod.yml      # prod-shaped baseline
        - ../../helm-values/values-stage.yml     # stage's deltas win last
  destination:
    name: stage
    namespace: stage
  syncPolicy:
    syncOptions:
      - CreateNamespace=true

Two details do a lot of work. Stage stacks prod's values under its own — so stage is "prod plus deltas", and a config drift between them is a visible diff in one small file, not an archaeology project. And because the Application references a git path and revision, the deployed state of every environment is a commit — the GitOps audit trail the SOC 2 series wants for free: who changed which environment is git log on a values file.

Ephemeral environments: one file and one manifest

The payoff of the whole design. A ticket needs its own environment — a feature branch of a few services, isolated data, a URL to click. That is: a new values file that overrides the current configuration, plus a new Application pointing the same chart at a new namespace.

namespace: ticket-589            # the namespace IS the ticket
envBucket: https://env-vars-dev.s3.….amazonaws.com/
envNamespace: dev                # reuse dev's env-file tree…
replicas: 1

apps:                            # …but run only what the ticket needs:
  - appRef: x-app-webui          # the manifest is REPLACED — six services,
  - appRef: x-app-api-gateway    # not twenty
  - appRef: x-app-auth
  - appRef: x-app-config-server
  - appRef: x-app-entity-relations
  - appRef: x-app-ehb-server
cronjobs:
  - jobRef: x-cronjob-send-notification-emails
jobs: []                         # no migrations in a throwaway env
persistentVolumes: []

x-app-api-gateway:               # per-app deltas: feature-branch builds
  deployment:
    tag: dev-1668052714924       # the branch build under test
  appEnvExtension:
    - file: ticket-589/api-gateway.env   # ticket-specific config, added not restated

x-app-ehb-server:
  deployment:
    tag: ehb-589-a41c3f0

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: ticket-589
spec:
  project: default
  source:
    repoURL: 'git@…/orchestrator.git'
    path: k8s/charts/core                      # the SAME chart as prod
    targetRevision: feature/ticket-589         # can track the feature branch
    helm:
      valueFiles:
        - values.yaml                          # full catalog available
        - ../../helm-values/values-ticket-589.yml
  destination:
    server: https://<dev-cluster>
    namespace: ticket-589
  syncPolicy:
    syncOptions:
      - CreateNamespace=true
    automated:
      prune: true        # delete the file → ArgoCD deletes the resources
      selfHeal: true

The cost model is the point: environment number seven did not add a chart, a pipeline, or a copy of anything — it added ~40 lines of values and ~25 lines of Application. Everything else is the same compiler.

Caveats

• The extension convention is a convention. Nothing stops an overlay from redefining apps when it meant appsExtension — and because lists replace, that silently removes services. Rule: base lists are written once in the chart; overlays may only replace deliberately (ephemeral manifests) or append via extensions. Reviews enforce it; a rendered-manifest diff in CI catches it.
• The chart's own values.yaml must stay environment-neutral in spirit. It doubles as the catalog, so it inevitably carries defaults — keep them inert (sentinel tags like not-found, quota-safe resources) so rendering the bare chart produces something that visibly refuses to run, rather than something that quietly runs like dev.
• YAML anchors don't cross value files. Anchors (&migrationSecrets) are file-local; they resolve at parse time, before Helm merges. Handy within one env file, silently absent if you expect them from another.
• Image tags in git are noise and audit trail at once. CI writing build tags into env values makes every deploy a commit — perfect traceability, busy history. Keep tag bumps as their own commits so humans can filter them out and auditors can filter them in.
• An ephemeral env is cheap in YAML, not in cloud. Namespace-per-ticket still means nodes, load balancers and IRSA roles per namespace (the per-app service accounts need their OIDC roles to exist — that part is infrastructure, not values). Track ephemeral namespaces with an owner and an expiry, or the cluster becomes a museum of closed tickets.
• Branch-tracking Applications drift by design. An ephemeral app pinned to a feature branch is exactly right for its purpose and exactly wrong the moment the branch merges — part of tear-down is retargeting or deleting the app, not just merging the code.

Before you template: the decision checklist

• One generic chart or chart-per-service. (my choice — one chart as a resource compiler; per-service charts only for things that are genuinely not shaped like the platform's services)
• Where the catalog lives. In the chart's values.yaml, or per environment? (my choice — in the chart: one place where a service's shape is defined, environments own only deltas and manifests)
• List semantics. How overlays interact with lists. (my choice — every list ships with an *Extension twin; replacing a base list is legal only for ephemeral manifests, and a rendered diff in CI is the guard)
• Where image tags live. CI --set, a registry watcher, or committed values? (my choice — committed values: the deployed state of every env is a commit, which is the audit story)
• Value stacking order. What each env layers on. (my choice — base catalog → shared baseline → env deltas; stage stacks on prod's values so their diff is one readable file)
• Sync policy per tier. (my choice — production syncs manually behind a human; ephemeral envs run automated prune + selfHeal, because their whole point is zero ceremony)
• Ephemeral lifecycle. Who creates, who deletes, when. (my choice — created by the ticket's engineer via the two-file recipe, owner and expiry noted in the values file, deleted with the Application; git history is the archive)

The chart is a compiler, values are the program, and an environment is just a program variant — which is why a new environment costs a values file, not an architecture meeting.

Related

The images this chart deploys are built by the pipeline from DevOps for a SOC 2 audit — Part 1: Docker as an evidence factory — the same get_env_vars.sh that fetches env files at container start is the one the chart's envBucket serves.