A slimmer Dragonfly for teams who just want faster image pulls

The pipeline I care about most this month spends about a third of its wall time pulling images. Not building, not testing. Pulling. Every runner in the pool fetches the same base layers from the same registry, and every time we double the pool we double the pain. So when a CNCF post landed on August 13 describing a lightweight Dragonfly deployment that skips the database stack, I read it twice on the same coffee.

Dragonfly is a CNCF project that turns image and file pulls into peer-to-peer traffic. The standard install is a real system: a Scheduler, a Seed Client, a Client on every node, plus a Manager control plane backed by MySQL and Redis. That is a lot to run if all you want is for your CI runners to stop hammering the registry. The lightweight model is Dragonfly with the control plane and the databases lifted out. What remains is the part that actually moves bytes.

Three components, no database

The lightweight install keeps three things:

• Scheduler as a StatefulSet, coordinating who pulls what from whom.
• Seed Client as a StatefulSet, the peer that owns the "authoritative" copy of an artifact for its cluster.
• Client as a DaemonSet, one per node, doing the actual pull on behalf of the container runtime.

Gone are the Manager, MySQL and Redis. That is the whole trick. You lose the web console, the OpenAPI endpoints, multi-cluster management and API-triggered preheat jobs. You keep the P2P distribution.

How the pieces still find each other

Without a Manager, two things had to move. The first is configuration. If manager.addr is left unset, the Scheduler and Client load their dynamic config from a mounted /etc/dragonfly/dynconfig.yaml file. It ships as a ConfigMap. You edit it, changes propagate on the refresh interval (default one minute) without restarting Pods. That is not a hardship for a CI cluster where "the operator" is one platform engineer who does not want to babysit a control plane.

The second is discovery. Clients need to know which Schedulers are alive. The post uses a headless Kubernetes Service so a Client resolves the hostname to a set of Pod IPs and filters unhealthy ones on its own. Native primitive, no extra service to run. This is the part I liked most, because it is the kind of change that stops being interesting the moment it works, which is exactly what you want.

Where this fits in a CI/CD day

The post is explicit about the target: single-cluster deployments, edge locations and CI/CD pipelines that just want to stop overloading their registry during image pulls. That matches the mental model of most build clusters I have worked in. One region, one blast radius, one team on the hook. If you also run Dragonfly across regions with API-orchestrated preheats, you are not the audience for this variant. You want the full stack.

For a runner pool, the deploy is a single Helm command against a values file:

helm install --wait --create-namespace \
  --namespace dragonfly-system \
  dragonfly dragonfly/dragonfly \
  -f charts-config.yaml

Once it is up, your container runtime pulls through the Client on the node, the Client asks the Scheduler where to fetch pieces, and the Seed Client covers the first pull that would otherwise miss. From the pipeline's point of view, nothing changes. From the registry's point of view, ninety-something percent of the traffic stops arriving.

The rough edges to know about

The tradeoffs are the ones the post names, and I want to repeat them so nobody is surprised.

No console, so observability lives in whatever you already run for Prometheus and logs. No OpenAPI, so anything you were driving from a script now needs a different hook. No multi-cluster stitching, so if two clusters need to share seed data you are back to the full deployment. No API preheat, so warming a new base image before a big rollout is a dfctl call from a job you schedule yourself.

None of these are dealbreakers for a CI pool. They are also not nothing. Write down which of them you actually use today before you cut the Manager, and make sure the answer is honest.

What I am watching next

The thing I want to run through my own runners this quarter is the preheat story from a scheduled job. If a nightly cron can pull the ten base images our morning PRs will need, the p95 pull time on the first push of the day should drop noticeably. That is the kind of change engineers feel without reading a changelog. If it works, I will tell you. If it does not, I will tell you that too.