TL;DR
GitOps is not just for Kubernetes manifests; it is a way to run everything that can be declared—cloud infrastructure, policies, even app configs—through Git and reconciliation loops. Beyond Kubernetes, the challenge is less about tools and more about deciding what “desired state” means for each layer and how far you want automation to go.
“If your GitOps story stops at deployments but your infrastructure still lives in tribal knowledge, you have not finished the job—you just moved the mess.”
Desired State for More Than Clusters
Once teams see the benefit of GitOps on clusters, the natural next step is to ask: what else should look like this?
- Infrastructure repos: networks, IAM, databases, and queues managed through IaC.
- Application repos: deployments, services, configuration, secrets references, separated per domain or team.
- Promotion via pull requests: overlays for dev/stage/prod so environments differ in data and scale, not in structure.
The rule of thumb: if changing it via the console scares you, it probably belongs in a desired-state repo.
Git as the Control Plane (Without Becoming a Bottleneck)
- Single front door: All change requests enter via Git. Emergency changes still flow through PRs, even if post-facto, with an audit trail.
- Separation of repos: Platform vs app vs policy; clear OWNERS and codeowners for reviews.
- Fast feedback: CI must be fast (lint, policy, diff) so PRs do not queue for hours; reconcile speed matters for trust.
“Git is the source of truth, not the source of latency. If your GitOps cycle is slow, people will route around it.”
Policy-as-Code Across the Organization
As GitOps expands, policy-as-code becomes mandatory, not optional.
- Use Rego policies (with OPA) to enforce:
- Image signatures and trusted registries.
- Resource limits and quotas.
- Tagging and naming conventions.
- Run policies:
- In CI/CD to block bad changes before they merge.
- At admission time to prevent risky resources from being created.
The point is not to make life hard; it is to encode decisions once and apply them consistently.
Policy tips:
- Make errors actionable (“set tag
owner”) instead of “policy failed.”
- Version policies; test in CI against fixtures before promotion.
- Cover security, cost, and reliability (resource limits, network egress, image signing).
Multi-Cloud Reconciliation Without Losing Your Mind
Going multi-cloud multiplies the opportunity for drift and surprise.
- Separate state per cloud and region: Avoid hidden coupling; each environment should have its own source of truth and controller.
- Shared patterns, not shared state: Reuse modules and policies, but keep each cloud’s resources declared separately, with clear boundaries.
- Drift detection: Regularly run plan/diff jobs to see where reality diverges from Git, and choose consciously whether to reconcile or accept the drift.
- Fail-open vs fail-closed: Decide how controllers behave when Git or policy backends are unreachable—silence is worse than a loud fail.
GitOps here is less about “multi-cloud symmetry” and more about making sure you know what is running where and why.
Runbook: Expanding GitOps Beyond Clusters
- Split repos by intent: Platform (clusters, networks, IAM), apps (services/configs), and org-wide policy. Assign owners.
- Define overlays: Make environment differences explicit (scale, endpoints, secrets refs). Review them in PRs.
- Wire policy gates: CI policy checks + admission controls; actionable errors; versioned policies.
- Add drift checks: Scheduled plan/diff per env/cloud; notify teams; reconcile or document accepted drift.
- Bootstrap/rollback drills: Practice restoring controllers from clean clones; rehearse disabling/reverting policies that misfire.
Actionable Patterns
- Repository scaffolding that distinguishes platform, applications, and policy repos.
- Environment overlays that keep differences explicit and reviewable.
- A policy starter pack with guardrails for security, networking, and cost.
- Promotion workflows documented with diagrams that match how your teams actually work, not how a vendor slide looks.
- Controllers with health checks and alerts—“is reconciliation working?” should be visible.
Metrics That Show GitOps Is Working (Beyond Clusters)
- Drift: Count and age of drift findings per env/cloud; time to reconcile or accept.
- Promotion health: Lead time and failure rate for promotions; time to rollback when policy or apply fails.
- Policy efficacy: Violations blocked with actionable errors; reduction in out-of-band console changes.
- Coverage: % infra/app/policy managed via desired-state repos; % controllers with health alerts.
- Developer experience: Time to provision a new service/env via Git; support tickets about “which repo controls this?”
30/60/90 Plan for GitOps Beyond Kubernetes
- 30 days: Separate repos (platform/app/policy); add CI checks (lint/policy/diff). Define overlays for dev/stage/prod. Start drift jobs in non-prod.
- 60 days: Add admission policies; controllers with health alerts. Pilot multi-cloud drift detection in one secondary region. Document emergency/retroactive change path.
- 90 days: Enforce policy gates on prod; canary reconciliations for risky changes. Expand drift checks to all regions. Run quarterly bootstrap/rollback drills and publish results.
- Controllers: Flux/Argo CD for clusters; Atlantis/Spacelift/TF Cloud for infra; keep a small set.
- Policy: OPA/Gatekeeper/Kyverno for clusters; OPA/Conftest or Sentinel for IaC; shared policy repo with tests.
- Observability: Health alerts on controllers (sync lag, failures); drift job results to chat; dashboards per env/cloud.
- Security and provenance: Signed artifacts and manifests; admit only trusted registries; short-lived credentials for controllers.
Small, well-known tools beat an endless stack that nobody masters.
Common Pitfalls (and How to Dodge Them)
- Console creep: Manual changes never codified. Fix with drift alerts and a “codify within hours” policy.
- Slow pipelines: PR checks take forever; people bypass Git. Fix by slimming CI and caching providers/modules.
- Policy theater: Vague errors or silent skips. Fix by testing policies and surfacing clear remediation hints.
- Single repo to rule them all: Everything jammed together; reviews stall. Fix with scoped repos and owners.
- Multi-cloud mirroring: Forcing symmetry where requirements differ. Fix with shared patterns, distinct desired state per cloud, and explicit deltas.
Case Study (Hypothetical, But Useful)
A payments platform splits repos into platform (clusters, networks, IAM), apps (per domain), and policy. They add CI lint/policy/diff checks and admission policies for image signing and network egress. Drift jobs run daily per region; alerts go to chat. A promotion flow applies to dev → stage → prod via PRs with canaries for risky changes. During a policy rollout, a tagging rule blocks prod applies; the team rolls back via Git, reconciles within minutes, and updates the policy with clearer remediation text.
Result: fewer console hotfixes, faster onboarding for new services (one repo + overlay), and predictable rollbacks when policies misfire.
Conclusion
GitOps scales beyond Kubernetes when you apply the same principles—declarative configurations, policy guardrails, and promotion via code review—to everything that matters: infrastructure, policies, and applications across clouds and environments.
It is as much a governance model as it is a delivery model. When Git really is the place you go to understand and change the system, “what is running where?” stops being a guessing game.
Stay tuned for more GitOps content that focuses on entire platforms, not just cluster YAML.