--- title: "Kubernetes: a cluster per project, on your servers · Frazer" description: "Each project gets its own Kubernetes cluster on your servers: VMs built by Frazer’s cluster controller from one resource you plan and apply, with GitOps, eBPF networking, gateways, certificates, secrets and monitoring installed. No per-cluster fee." url: https://frazer.ai/kubernetes index: https://frazer.ai/llms.txt tokens: 5154 --- > For agents: every page on frazer.ai is indexed at https://frazer.ai/llms.txt. # Kubernetes: a cluster per project, on your servers · Frazer **Kubernetes** frazer.ai # A cluster per project. On your servers. No cluster fee. Each project gets its own Kubernetes cluster: virtual machines spread over your servers, built by Frazer’s cluster controller from one resource you plan and apply. It arrives with GitOps, eBPF networking, gateways, certificates, secrets and monitoring installed, and deploys what you keep in Git. Clusters have no fee of their own; their VMs use cores you already pay for. [Create a cluster](https://frazer.ai/start?product=kubernetes) [Read the docs](https://frazer.ai/docs) *The overview of the cluster acme-production-1 in the Frazer console: healthy, Kubernetes v1.36.4, 7 of 7 nodes ready, 41 % of 44 vCPU and 56 % of 164 GiB requested. v1.36.5 is available and scheduled for the maintenance window, control plane first. Platform components are healthy, GitOps is synced and the control-plane datastore has 3 of 3 members. Two node pools run on three servers, and recent activity shows a rollout from Git, a live restart and three plans.* A production cluster’s overview product UI, demo organization 01 · How it works Fig. 1 ## A cluster is a set of VMs. Spread over your own servers. Your servers form a pool substrate that Frazer runs: Kubernetes and Frazer’s VM layer on bare metal. A cluster is a set of virtual machines on it, each one a Kubernetes node. Frazer renders the cluster objects from your Cluster resource and sends them signed; the substrate creates, repairs and replaces the VMs. **Fig. 1** Two clusters from two Acme projects on 3 servers demo organization - ### One node, one VM Each node is a virtual machine running Ubuntu and Kubernetes. Your containers never run on the bare host. - ### Spread over servers A production cluster’s three control-plane VMs sit on three different servers. - ### Your network Each VM’s card sits in your project’s VPC subnet, and pods use its secondary ranges. - ### Keeps running If Frazer is unreachable, the cluster’s own GitOps keeps applying the last values. 02 · Classes ## Three classes. The same stack in each. - ### small Staging and small projects Control plane One VM Survives A failed worker, replaced automatically Upgrades Machines replaced Needs One server - ### production Recommended Production services Control plane Three VMs on three servers Survives The loss of one server Upgrades Machines replaced, one surge VM; control plane first Needs Three servers; dedicated-CPU workers - ### dedicated Large databases, GPUs, full NIC speed Nodes Whole servers from a dedicated-node pool Survives The loss of one server Upgrades In place, so local disks survive Needs Servers you give to one cluster ### Capacity rules, by default - Control-plane VMs get their full CPU and the highest priority. - Workers share CPU 1:1 in production, up to 2:1 in staging, up to 4:1 for previews. Memory is never overcommitted. - Each server keeps 8 cores and 32 GiB for itself and the substrate. - Quota is checked when you plan, with one spare VM per pool held for upgrades and repairs. - A node not ready for 10 minutes is replaced; 15 for database pools, and never two of three at once. 03 · Plan and provision Fig. 2 ## Plan first. Then apply. Nothing exists until you approve. Create a cluster in the console or the CLI and the plan shows what will exist: quota used on the shared pool, the spare VM held for upgrades, the servers that host the control plane, ranges and addresses, the monthly cost, and the rendered objects as a diff. In a production project, a second person approves. *The plan for the production cluster acme-production-2 in the Frazer console, waiting for approval: 11 cluster objects and one Application to add, nothing changed or destroyed; 9 VMs on 3 servers with one control-plane VM per server, needing 48 vCPU and 180 GiB; 7 checks passed and 1 warning that losing a server would leave no spare room; a reviewer other than the requester approves before it applies.* **Fig. 2** The plan for a second production cluster product UI, demo organization ### Ten phases to Ready 1. Requested 2. Accepted 3. Creating machines 4. Control plane starting 5. Control plane ready 6. Nodes joining 7. Seeding 8. Platform syncing 9. Environments syncing 10. Ready Designed to reach Ready in 4 minutes at the median and 6 at the 95th percentile, from pre-built node images and a warm pool of VMs. ### After Ready A change has the same shape: a pool’s replicas, the Kubernetes version, the profile version or an add-on switch, each a plan you apply. Rolling back is applying an earlier revision. Deleting is soft, with a restore window: you type the project ID, a final database backup is taken, and production clusters carry a deletion lock. 04 · Workloads ## Workloads come from your Git. Nobody presses “new app”. A new service is a folder in your environment repository. Connect any number of environments to a cluster, from any number of repositories; a namespace belongs to exactly one of them. 1. ### Bring servers Servers in a substrate pool become the pool substrate. Its host project shares capacity with your projects. 2. ### Create a project Members, quota and an environment label. Its secrets namespace and registry project come with it. 3. ### Create a cluster Plan, then apply. Frazer builds the VMs and installs the platform set. 4. ### Connect an environment A repository, a branch and a path. The cluster’s GitOps deploys everything in it. 5. ### Add a gateway and a domain An address from the site, a certificate and DNS records. 6. ### Push code Your CI builds, signs and pushes to Frazer Registry, then writes the digest to the environment repository. ``` $ frazer kubernetes clusters create acme-production-2 \ --project acme-production --location us-virginia1 \ --class production --dry-run Pool vint-hill-default · shared from acme-infra Quota 48 vCPU, 180 GiB of your share · surge VM held Servers a control-plane VM on each of Atlas, Borealis, Cobalt Objects + 11 cluster objects · + Application frazer-platform Network pods 10.64.192.0/18 · services 10.96.48.0/20 Cost $0 a month on your own servers Approval production: a reviewer other than you ✓ Plan pl-0412 saved. Nothing is created until it is approved. ``` ``` $ frazer releases environments connect data --project acme-production \ --cluster acme-production-1 \ --repo acme/data-platform-envs --path envs/production \ --namespaces data-airflow,data-clickhouse --profile platform --dry-run Plan: connect environment "data" to cluster acme-production-1 + Namespaces env-data, data-airflow, data-clickhouse + AppProject env-data · profile platform (elevated) + Applications 4 from commit 3f9c2e1 ~ Git token the cluster also reads acme/data-platform-envs ✓ Namespaces no conflict with environment "platform" ✓ Policy dry run: 0 violations · schemas valid Cost +6 vCPU, +24 GiB requested · no fee Approval org admin (profile platform) ``` ``` $ frazer kubernetes clusters get-credentials acme-production-1 ✓ kubeconfig written · context acme-production-1 · no secret in the file $ kubectl get nodes # a 10-minute token from frazer, through Frazer Link NAME STATUS ROLES AGE VERSION cp-4k2xq Ready control-plane 12d v1.36.4 cp-9wd7m Ready control-plane 12d v1.36.4 cp-h3n5t Ready control-plane 12d v1.36.4 general-7hk2p Ready 12d v1.36.4 general-b8s4v Ready 12d v1.36.4 general-m2q9c Ready 12d v1.36.4 general-x5r1d Ready 3d v1.36.4 # 7 nodes: 3 control plane, 4 workers, all VMs on your servers ``` A rollout, as the console follows it by image digest 1. Built 2. Written to Git 3. Synced 4. Rolled out 5. Verified Every cluster page shows nodes, workloads, the GitOps resource tree, live and desired YAML, and the diff. Scale and Edit open a pull request when you install the Frazer writer App; restart, cordon and drain act live, as you, and land in the audit log. Environments, permission profiles and the pull-request check in depth: [Environments and GitOps](https://frazer.ai/gitops) 05 · The platform set ## Arrives with the platform set. Each piece has exactly one owner. A ### The Kubernetes distribution Upgraded with the Kubernetes version - Kubernetes and its datastore 1.36 - eBPF networking - Cluster DNS, resource metrics - Volume snapshots - No ingress controller B ### The seed, once Then handed to the cluster’s GitOps - frazer-cloud-controller-manager - Secret sync - Cluster keys registered with Frazer Secrets - frazer-secrets store, Git and registry credentials - Frazer GitOps - Application frazer-platform C ### The cluster’s own GitOps Installs the rest by sync wave, then upgrades it with the profile version 1. −30 Frazer GitOps, self-managed 2. −28 Secret sync, adopted 3. −26 frazer-clusterlet 4. −25 Default-deny network policy 5. −20 CRDs: Gateway API, gateway, monitoring 6. −18 Frazer Policy, in audit 7. −15 Certificate manager and issuers 8. −12 Local NVMe volumes 9. −10 Cluster gateway 10. −8 Gateways 11. −7 frazer-xds 12. −6 Metrics and log agents 13. −5 Node lifecycle 14. −4 Config reload, policies enforced 15. −3 Optional add-ons 16. 0 Your environments ### Optional add-ons Each one is a switch on the cluster and a plan. Anything else comes through your environment. - Managed Postgres with backups - Cluster backups - Event-driven autoscaling - Progressive rollouts - DNS records at your provider - GPU drivers - Sandbox runtimes - Runtime threat detection - OpenTelemetry collector - Guaranteed components (eBPF networking, the policies, the certificate manager, the agent, the monitoring agents and the security sensors: the runtime sensor, the vulnerability scanner and CIS benchmark checks) change through the cluster’s settings. Environments cannot edit them. - Profile versions roll by ring: Frazer’s canary clusters, then staging-labeled projects, then production-labeled ones, each after Synced, Healthy and a soak. Production moves only inside its maintenance window. - A project can pin a profile version for up to one release cycle. 06 · Upgrades ## Upgrades inside your window. Rings first, your production last. Two channels Rapid and Regular, the default. A cluster runs one of the three newest minors Frazer certifies. Rings first A version reaches your production after Frazer’s canary clusters and staging projects ran it healthy. Any failure holds it everywhere. Your window Windows and exclusions per project, one calendar for clusters, databases, VMs and hosts, notices 7 days, 72 hours and 1 hour ahead. Rehearse first Restore the cluster’s state at the target version, dry-run every object and call the webhooks before a minor upgrade. ### A minor upgrade, step by step 1. **Preflight** Names the client still calling a deprecated API, and flags disruption budgets that allow no eviction, single-replica workloads and node-local volumes. 2. **Anchor** A control-plane snapshot before anything changes. 3. **Control plane** Replaced one VM at a time with a surge VM, then a soak: 24 hours in production, 1 hour in staging. 4. **Node pools** Surge, or blue-green in batches with the old VMs parked, disks kept, so a pool rollback starts them again. Drains wait for disruption budgets, then hold the node; nothing is force-evicted by default. 5. **Profile** The platform profile moves too, when the new minor needs it. 6. **Postflight** Conformance checks and your own synthetic checks. ### Plan it, or rehearse it first Every upgrade is a plan first, then an operation you can pause, resume or roll back. Rehearse a minor upgrade before you plan it, then let it run in your window. ``` $ frazer kubernetes clusters upgrade acme-production-1 \ --to 1.37 --rehearse # restores the cluster’s state at the target version, dry-runs every # object and calls the webhooks; reports what would break ``` ``` $ frazer kubernetes clusters upgrade acme-production-1 \ --to 1.37 --dry-run # steps, preflight results, the risk to each workload, the duration, # the surge cost and the slot it will run in ``` ``` $ frazer kubernetes clusters upgrade acme-production-1 \ --to 1.37 --at next-window # runs inside your maintenance window as an operation # you can pause, resume or roll back ``` ### Limits we state - VM workers are replaced at each upgrade, so their local volumes are created again and a database replica clones from its primary. Large databases belong on the dedicated class, which upgrades in place. - Rolling a control plane back during the soak is limited to staging and opt-in clusters. Elsewhere, recovery is a forward fix, or a restore to the snapshot, with the writes it would lose listed for you to approve first. 07 · Attached clusters ## Bring clusters you already run. Outbound only. Your RBAC decides. Attach a cluster you run elsewhere, managed by a cloud provider or on premises, through the same outbound-only agent. Pick how much Frazer may do; change it any time, and its lifecycle stays with you. - ### Observed See it in Frazer Frazer can Inventory, health, metrics, logs, cost; read views in the console and MCP Rights A read-only ClusterRole, no Secrets Workloads Yours, untouched Price Free - ### Connected Recommended Work with it from Frazer Frazer can Also the console and kubectl through the tunnel Rights Impersonation of mapped users and groups; your RBAC decides Workloads Yours, untouched Price Free - ### Managed Deliver to it from Git Frazer can Also environments: Frazer GitOps from your Git, or adopt the GitOps you run Rights GitOps per namespace or per cluster, your choice Workloads From your Git, through Frazer GitOps Price $0.75 per node vCPU / month, max $150 per node - ### Outbound only The agent dials out. No inbound port, no load balancer. - ### Double opt-in The install token only makes the cluster appear as pending until you accept it. - ### Never cluster-admin Connected, each request runs as the person who made it; your RBAC decides and your audit log records it. - ### Detach any time Detach removes the agent’s namespace and nothing else. 08 · Everything around the cluster ## The parts you expect. None you have to assemble. - ### kubectl without a VPN `get-credentials` writes a kubeconfig with no secret. Each call gets a 10-minute token for one cluster. - ### GitOps and dashboards Open them from the cluster page, behind Frazer sign-in. - ### Gateways, DNS, certificates Addresses from the site, and your domain three ways: delegate it, keep your provider, or set records yourself. - ### Secrets stay out of Git Git holds ExternalSecret objects. Values live in Frazer Secrets, write-only for people. - ### Signed images only Frazer Registry signs each digest on push; admission checks the signature. - ### gRPC without a sidecar `frazer-xds` serves only ready endpoints; clients balance per request. - ### Metrics and logs Agents in every cluster; dashboards per organization and per cluster. - ### Previews per pull request A namespace per pull request, deleted on merge, close or expiry. 09 · Controls ## Who may do what. Every action leaves a record. ### Who may do what Your Frazer role Kubernetes viewer, developer and admin map to RBAC groups in every cluster; kubectl acts with your role. Developers Connect environments with the `standard` profile: namespaced kinds in their own namespaces. Org admins Approve the `platform` profile for environments that bring CRDs and operator webhooks. A second person Approves production plans. Nobody approves their own; agents and service accounts never approve. Org owners Hold a break-glass kubeconfig; using it marks the cluster tainted until its CA is rotated. ### What is logged - Every plan and apply, with the revision it made. - Every kubectl call through Frazer Link, as the person who made it. - Live actions from the console: restart, cordon, drain, replace machine. - Kubernetes events, kept in the project’s logs for **30 days**. - Any use of break-glass access. ### At the cap - A capped budget **never stops** workloads on your own servers; it refuses only new billable items. - Your guaranteed share of a pool is held for you and never lent; bursts above it are never preempted. - A full pool fails the plan, never a running cluster, and the plan offers to request more. - A plan without spare capacity for losing one server asks you to acknowledge it. - Production clusters carry a **deletion lock**; deletion is soft, with a restore window and a final backup. 10 · Pricing ## No cluster fee. A cluster per project stays free. A cluster has no fee of its own. Its VMs run on your servers, which pay the per-core server fee, and the platform set comes with every cluster. [See every price](https://frazer.ai/pricing#lines) Per cluster $0 per cluster Your servers $1.50 per physical core / month · min $15, max $150 per server · first server ≤ 16 cores free Included Kubernetes clusters, GitOps, gateways, DNS, certificates Attached Observed and connected: free · managed: $0.75 per node vCPU / month, no minimum, max $150 per node Questions ## Kubernetes, answered. Do I run the substrate? No. Frazer runs the pool substrate on your servers: Kubernetes, the VM layer, the cluster controller and an image cache. You see your clusters and your share of the pool’s capacity. The substrate runs only VMs and platform controllers, never your containers. What happens to my cluster if Frazer is unreachable? It keeps running. The cluster’s own GitOps keeps applying the last values and chart, and each cluster encrypts its Secrets with a local key, so they stay readable. Frazer keeps only an escrow copy of that key. Can I install my own operators and tools? Yes, through your environment: raw manifests or a Kustomization under `manifests/`. With the `platform` profile, which an org admin approves, the environment may also add CRDs and operator webhooks. The guaranteed components stay under the cluster’s settings. How do I reach the API server? Run `frazer kubernetes clusters get-credentials`. The kubeconfig holds no secret: kubectl asks `frazer` for a 10-minute token for that one cluster, and the traffic goes through Frazer Link. There is no VPN to install, and your Frazer role maps to Kubernetes RBAC groups. Where should a large database run? On the dedicated class. VM workers are replaced at each upgrade, so their local disks are created again. Dedicated clusters use whole servers and upgrade in place. Either way, durability comes from three instances on three servers and backups, not from the disk. Can I bring a cluster I already run? Yes. It attaches through the same outbound-only agent, as observed, connected or managed. Detaching removes the agent’s namespace and nothing else. ## More in Compute - [**Servers and pools** Bring, rent or buy servers](https://frazer.ai/servers) - [**Serverless** Deploy a repo, DB included](https://frazer.ai/serverless) - [**VMs** Images, disks, groups](https://frazer.ai/vms) - [**Devboxes** Real machines per engineer](https://frazer.ai/devboxes) ## Bring your servers. Get a cluster per project. [Create a cluster](https://frazer.ai/start?product=kubernetes) [Read the docs](https://frazer.ai/docs)