marstack-govern — every fact derived from the platform

Watch it run

Unedited, against a real cluster. A division is already reconciled; a quota request is filed claiming to be someone else, and the API server writes who actually sent it. The controller then gathers the evidence an approver will decide against — preflight through the real admission chain, and a simulation that packs real nodes.

A terminal session filing a Kubernetes quota request under one identity while the manifest claims another, and the API server overwriting the claim — Recorded by site/demo.sh into demo.cast, an asciinema recording that is plain text you can read line by line. Point the script at your own cluster and you get your own copy, with your own numbers:

The preflight and simulation numbers in that recording come from the cluster it ran against — two nodes with 20 cores allocatable. Against your cluster they will be different numbers, which is the point.

The console

The same cluster, through the web console. It is a client of the public API, not a privileged path into it: every list is filtered by what Kubernetes says the signed-in person may see, confirmed with a SelfSubjectAccessReview under their own identity.

The margov console moving through services, divisions, requests, cost, policies, topology and preview environments of a live cluster — Recorded by site/ui-demo.sh, which drives a real browser against a running `margov` and renders the video to a gif. It needs `npm install playwright` and `ffmpeg`:

Watch what the Topology view says when there is no trace pipeline: no trace metrics are available, so the graph would be a guess. A missing dependency is named rather than drawn around.

The one rule

Humans supply two kinds of input. Everything else is derived.

Input	Examples	Shape
Intent	raise my quota, grant me access, approve, reject	a custom resource carrying requester, reason and an expiry
Policy	the rupiah rate per vCPU, Kyverno rules, how idle cost is allocated	a versioned, approved object that later records cite

Facts — what is running, who owns it, how much it uses, who changed what — are read from the Kubernetes API, the Kubernetes audit log, the OCI registry, Mimir, Loki, Tempo, Kyverno, Trivy, Argo CD and Hubble. If a fact cannot be derived, it is not shown. There is no free-text field describing reality.

The rule is enforced mechanically, not by discipline. PostgreSQL holds only a projection, and a test drops it, replays from the platform and compares. A column whose only source is a form cannot survive that replay, so the violation fails CI rather than waiting to be noticed.

What it is

One Go binary, margov, with the web console embedded in it. It watches the cluster through informers, projects what it sees into PostgreSQL, and serves that back over ConnectRPC — gRPC, REST and the browser from one set of protobuf definitions.

It composes rather than reimplements. Capsule holds the cross-namespace quota, Kyverno is the admission policy engine, Argo CD is the desired-versus-live authority, Trivy supplies vulnerability reports, Prometheus or Mimir supplies usage. Where one of those is missing, the affected feature says so by name instead of guessing.

There is no mandatory gateway. The console and kubectl reach the same API server, are stopped by the same RBAC, and land in the same audit log. The console can never show more than the same person's kubectl would.

Quickstart

Build the binary, point it at a database and a cluster:

make build     # ./bin/margov, with the console embedded

createdb govern

./bin/margov serve \
  --database-url postgres://localhost:5432/govern \
  --kube-context kind-govern \
  --secure-cookies=false \
  --insecure-dev-identity "you@example.test:payments-admins"

--insecure-dev-identity signs every visitor in as that subject with those group claims, and says so loudly in the log and across the top of the page. Against a real cluster, use OIDC instead — group claims are matched against the access grants on each Division, and what a signed-in person actually sees is confirmed with Kubernetes through a SelfSubjectAccessReview under their own identity.

Divisions and quota

A Division is reconciled into namespaces, limit ranges, default-deny network policies and role bindings from group claims. How the quota is enforced depends on what the cluster actually has, decided by a RESTMapper lookup rather than by configuration:

Capsule	Enforcement	`status.quotaBackend`
installed	one `GlobalResourceQuota` over every namespace labelled with the division	`capsule`
absent	a `ResourceQuota` in each namespace	`resourcequota`

The two are mutually exclusive. When Capsule appears the controller deletes the per-namespace quotas it wrote earlier — leaving them would cap each namespace at the whole division total on its own, which is the failure the global quota exists to prevent.

The condition quotes the shared ledger rather than claiming success: "Capsule enforces the division total across 4 namespaces; 26500m of 40 cpu is committed." A gap visible in kubectl get division is a gap; a gap hidden behind a green checkmark is a lie.

Requests and decisions

Every request is a custom resource in the requester's namespace, so its progress can be watched with kubectl. Three things happen before a human sees it:

Step	What it produces
Recommendation	p95 and p99 over thirty days, a proposed number, and when the current quota runs out
Preflight	server-side apply with `dryRun=All` through the real admission chain, including Kyverno
Simulation	bin packing against real node capacity: which pods stop fitting, and the commitment before and after

The decision records the evidence the approver saw at that moment, not merely that someone clicked Approve. Deciding against stale evidence is refused: the approval carries a digest of the recommendation, preflight and simulation, and a mismatch comes back as Aborted.

Approvals are time-bound. Each carries an expiry and a reason, and a controller revokes it when it lapses.

Chargeback

Cost is usage times a rate, and the rate is policy: a PricingPolicy with an effective date, an approver, and a revision that every invoice cites. Billing is on requests, not usage — requested capacity is locked away from other divisions, so that is what a division actually consumes. Usage is still shown, as the measure of idle.

An invoice is generated once the month has ended and never recalculated. A period still running is refused as a forecast; a month with no policy in force is refused rather than billed at zero. Each invoice carries a digest over its own inputs, so a nightly job can tell "already correct" from "an input changed" instead of rewriting finished invoices forever.

Money is math/big.Rat end to end. No float ever touches a rupiah.

Audit

The Kubernetes audit webhook is the single source. Because every platform mutation is a custom resource applied under the caller's own identity through impersonation, "when did this division change its quota" is already in the cluster audit log. There is no second trail that could disagree with the first.

Each event is hashed onto the one before it and written to both PostgreSQL and an append-only archive. Verification rehashes the exact bytes that were hashed — stored alongside the row, because jsonb reorders keys and timestamptz truncates to microseconds, and either would break a chain over untouched data.

Attribution is enforced at admission. A mutating webhook writes every requestedBy, decidedBy and grantedBy from userInfo.username on the admission request itself. A claimed value is overwritten, an empty one filled in, and on update the field is immutable. Renewals are append-only, so a lease extension cannot be disowned or erased.

Preview environments

An EphemeralEnvironment holds a real namespace with its own quota and the division's default-deny, and a controller takes it back when the lease ends.

The expiry is creationTimestamp plus the granted duration, recomputed to the same value on every pass. A controller that writes now + ttl each reconcile gives a preview an unbounded life while still reporting 48 hours left. Extending is a recorded act: a renewal carries the extension, a reason and who granted it.

Reclamation deletes by owner reference, never by name. The namespace name is derived from the division and the change number, so a collision with something created by hand is possible; such a namespace is reported as orphaned and left alone.

How it is verified

Unit tests check that code does what its author meant. They cannot check whether the author understood what Kyverno, Capsule, Argo CD or Prometheus actually emit — and that is where the expensive mistakes have been. Every module that reads a third-party API carries a cluster test that runs the same code against the real component and skips when it is absent:

GOVERN_TEST_KUBE_CONTEXT=kind-margov \
GOVERN_TEST_METRICS_URL=http://127.0.0.1:9090 \
  go test ./...

Two of them earned their place immediately. Against a live Kyverno the platform reported zero violations while kubectl showed six — Kyverno names the offending resource in the report's scope, and only aggregated reports use the per-result array the code was reading. Against a live Prometheus, a window shorter than the subquery resolution returned nothing at all rather than a small number.

What it will not do

It is not an orchestrator. Crossplane, Kratix and KubeVela fulfil requests; this decides them.
It is not a replacement for Grafana. Free-form investigation stays there; deep links carry context.
It is not CI/CD. It reads pipeline results; it does not run pipelines.
It is not a hand-maintained catalog. A feature that needs a YAML file kept in sync by humans is designed wrong.
It is not a SIEM.
It is not the system of record for who belongs to which division. The identity provider is.