marstack-cloud — containers, VMs and microVMs as one resource

What it is

A control plane, an agent that runs on every node, and a client — all three in one binary. You describe an instance; the node makes it exist. Whether that instance is a container under runc, a virtual machine under QEMU, a microVM under Cloud Hypervisor or a sandbox under Firecracker is a field on the object, not a different product.

The split matters because the agent reconciles rather than takes orders. It asks the control plane what should be running on it and makes that true. Nothing dials into a node, so a node needs no listening port of its own, and a control plane that dies does not take running workloads down with it.

State lives in SQLite next to the control plane. That is one process holding the authority, which is a deliberate trade for this stage: simple to run and to reason about, not yet a highly available control plane.

Install

Take the archive for your platform from the releases page, check it against the published checksums, and put the binary on your path.

shasum -a 256 -c SHA256SUMS
tar -xzf marstack_v0.2.0_linux_amd64.tar.gz
sudo install -m 0755 marstack_v0.2.0_linux_amd64/marstack /usr/local/bin/marstack

marstack version

Linux on amd64 or arm64 for anything that runs a workload. macOS on arm64 runs the control plane and the client, which is enough to drive a cluster from a laptop.

First run

The control plane writes its first admin token to bootstrap-token in its data directory on the first start. That token is how the first person gets in and creates everyone else.

marstack server --data-dir ~/marstack-data &
sudo marstack agent --name bm-1 --zone rack-a &

export MARSTACK_ENDPOINT=http://127.0.0.1:7443
export MARSTACK_TOKEN=$(cat ~/marstack-data/bootstrap-token)

marstack instance create --name api-1 --image alpine:3.20
marstack instance list

NAME    ID                ISOLATION   IMAGE          VCPU   MEMORY   DESIRED   OBSERVED   NODE
api-1   i-php2q13mwt3qy   container   alpine:3.20    1      512      running   running    bm-1

Adding a second machine is the same command with a different name. Nothing about the first node changes, and the two are one network from the moment the second one reports.

Without --tls-cert and --tls-key the control plane serves plain HTTP, and every bearer token crosses the network in the clear. It says so on every start for a reason: that is fine on a loopback address and wrong anywhere else.

Instances

Four isolations, one object. The field decides which runtime starts it; everything else — networks, volumes, firewalls, names, placement — behaves the same way whichever you picked.

Isolation	Runs on	Boots from	For
`container`	runc	an OCI image	the common case, shared kernel
`vm`	QEMU	a registered disk image, or an ISO you install yourself	a whole operating system, its own kernel, PCI passthrough
`microvm`	Cloud Hypervisor	an OCI image and a chosen kernel	a kernel boundary at close to container speed
`sandbox`	Firecracker	an OCI image and a chosen kernel	the smallest surface, for code you do not trust

marstack instance create --name db-1 --isolation vm \
    --image ubuntu-24.04 --vcpu 4 --memory-mib 4096 --disk-gib 40

marstack instance create --name fn-1 --isolation sandbox \
    --image alpine:3.20 --kernel vmlinux-6.6

marstack instance resize db-1 --vcpu 8 --memory-mib 8192

An instance gets an internal DNS name the moment it is created, so nothing has to wait for an address to be handed out before it can be referred to. Every command that takes an id takes the name instead.

Networking

Each network is a range, and each node is given a slice of it. Traffic between nodes is routed, not encapsulated: no VXLAN, no MTU arithmetic, nothing to packet-capture through a tunnel. A container on one node reaches a VM on another because the route exists, not because a tunnel carries it.

marstack network create --name prod --cidr 10.42.0.0/16 --cidr6 fd00:42::/48

marstack instance create --name web-1 --image nginx:1.27 \
    --network prod --network mgmt

Networks can carry a range per family, so an instance can be IPv4-only, IPv6-only or dual stack. The first network given is eth0 and carries the default route; the rest are extra interfaces.

Reaching in

A published port opens on the node that happens to run the instance, and follows it if the instance is placed somewhere else. A balancer spreads one port across many instances, and can route by host and path to different services behind the same port.

marstack forward create --instance web-1 --port 8080 --target 80

marstack balancer create --name edge --port 443 --tls-acme shop.example.com
marstack balancer route set edge \
    --route 'shop.example.com/api=api' \
    --route 'shop.example.com/=web'

Certificates can come from an ACME authority over HTTP-01 and are renewed before they expire. A certificate that is close to expiring and not being renewed becomes an event, so it is visible before it is an outage.

Volumes and backups

A volume outlives the instance it is attached to. It can be detached from a running guest, snapshotted, rolled back, cloned into a new volume, and carried off a node that is being drained.

marstack volume create --name pgdata --size-gib 100
marstack volume attach pgdata --instance db-1

marstack volume snapshot create pgdata --name before-upgrade
marstack snapshot restore snap-3k9x2m
marstack snapshot clone snap-3k9x2m --name pgdata-copy

Backups can be written to this machine's disk, which is the single point of failure they exist to survive, or to anything speaking S3 — including MinIO. Content is sealed at rest when the control plane is started with a backup key.

Lose a backup key and the backups it sealed are gone for good. Keep it somewhere other than the data directory it protects.

Services and balancers

A service holds a number of replicas of one template. It replaces a replica the node has given up on, rolls onto a new template one replica at a time, and can hold its own count against a target load rather than a fixed number.

marstack service create --name api --replicas 3 \
    --image ghcr.io/example/api:1.4 --network prod

marstack service autoscale set api --min 3 --max 12 --cpu-target 60
marstack service template set api --image ghcr.io/example/api:1.5

Replicas are spread across nodes and zones when they share a placement group, and can be pinned to nodes carrying a label when they need particular hardware.

marstack instance create --name train-1 --isolation vm \
    --node-selector gpu=h100 --device 0000:65:00.0

A GPU or accelerator is handed to a guest by passthrough, from an inventory each node reports. A card already claimed by one guest cannot be given to another.

Projects and tokens

Everything belongs to a project: instances, networks, volumes, snapshots, firewalls, ports and images. A token is bound to one project and carries a role, so a token that leaks is bounded by both.

Role	May
`admin`	everything, including creating people and projects
`member`	read and change what its project holds
`viewer`	read what its project holds, and nothing else
`node`	only the endpoints an agent needs — it cannot schedule

marstack project create --name staging
marstack user create --email ada@example.test --role member --project staging
marstack token create --name ci --role member --project staging --expires-in 720h

marstack quota set --project staging --instances 20 --volumes-gib 500

Every change is recorded with who made it, not only which token — and so is every refusal. A rate limit applies per caller and per project, so one client cannot spend the platform's attention on its own.

The console

Optional, and not inside the binary. It is published as its own archive per release, so a control plane that only answers the API never holds one.

marstack ui install
#   fetching marstack_console_v0.2.0.tar.gz
#   checksum ok
#   installed 3 files to /var/lib/marstack/console

marstack server --ui-dir /var/lib/marstack/console

The archive is checked against the release's SHA256SUMS, and one holding a symlink or a path that reaches outside the directory it unpacks into is refused. Without --ui-dir, nothing is served at / — which is what a control plane with no console looks like.

The console signs in against the same API everything else uses and keeps the session in a cookie a script in the page cannot read. What the API refuses, it refuses in the browser too: a viewer sees what a viewer may see.

Before production

Four things the platform will warn you about, in the order they will hurt.

Do this	Or else
Pass `--tls-cert` and `--tls-key`	Every bearer token crosses the network in the clear, and the console's session cookie cannot be marked secure
Pass `--backup-key-file`	Backup content is stored unencrypted, so a copy of every volume sits in the data directory in the clear
Pass `--object-store-endpoint`	Backups live on the one machine whose loss they exist to survive
Replace the bootstrap token	The first admin token has no expiry and sits in a file on disk

What it will not do

Stated plainly, because finding out later is worse.

The control plane is one process. State is SQLite beside it. Workloads survive its loss because the agent pulls, but there is no highly available control plane yet, and no failover story to configure.

The serial console of a VM is not reachable through the control plane. It reads a socket on the node itself, so that one view of the console needs a node-local bridge that does not exist yet. The interactive shell into a running container does work, because that is a control plane feature.

Release artifacts are not signed. marstack ui install checks the console archive against the checksums published with the same release, which means it trusts the release host rather than a signature it can verify on its own.

There is no anti-spoof filtering on the datapath. An instance is stopped from using an address it was not given, but a node is trusted about what it reports.

It is pre-1.0. The API, the CLI and the on-disk state may change between minor versions. Read the changelog before upgrading.