My canary analysis aborted a good release and promoted a bad one
A Prometheus canary analysis aborted a good release and promoted a bad one before it caught 6 of 6. Judge against stable, refuse empty windows, count requests.
11 min read
Topic
Kubernetes
Stateful data workloads on Kubernetes: scheduling, operators, resource shaping, disruption, and the operational price of explicit control.
I have operated data workloads on Kubernetes since 2022. Kubernetes does not make stateful systems simple; it makes scheduling, disruption, resource pressure, and ownership explicit. These notes examine that trade through mechanics you can reproduce: requests and limits, operators, eviction, autoscaling signals, rollout safety, and the difference between a busy cluster and an efficient one.
2 posts
All writingThe user's laptop is not a server. Make it a queue consumer.
A cloud agent that drives a browser on a user's Mac kept reporting it offline. The fix was not sticky routing: model the laptop as a queue consumer.
13 min read