Understanding Docker caching

Understanding caching is fundamental to working with containers. If you're constantly working with Docker images — building them, rebuilding them, shipping them — it's important to have a firm grasp on how Docker caching actually works: which layers get cached, what invalidates them, and how caching behaves across machines.

This post is not going to cover the basics of the container image layer architecture — if you need to brush up, the official guides on image layers and the build cache are the right starting points. This post is focused on the nuances of builds as a developer lives them day to day.

First, how a layer becomes a cache hit

One piece of groundwork, because everything below depends on it. When BuildKit evaluates an instruction, it reuses the cached layer only if the instruction itself is unchanged, every parent layer above it was a cache hit, and — for COPY and ADD — the checksums of the copied files are identical. The moment one layer misses, every layer after it rebuilds, no matter how innocent.

That chain rule is why instruction order is the single biggest caching lever you have: copy your dependency manifests and install dependencies before you copy your source code, so an ordinary code edit doesn't invalidate the expensive dependency layer.

The invisible cache-killer: your build context

Before any instruction runs, Docker ships the build context — everything under the directory you pass to docker build — to the builder. Two problems hide in there. The obvious one is size: a bloated context makes every build slower before caching even gets a say. The sneaky one is invalidation: COPY . . checksums every file in the context, so a rotated log file, editor swap file, or .git churn from an unrelated branch switch silently invalidates the layer — and everything after it — even though no source file changed.

The fix is a deliberate .dockerignore. Treat it like .gitignore's stricter sibling: if a file isn't needed to build the image, keep it out of the context.

.git
target/
node_modules/
*.log
.idea/ .vscode/
Dockerfile docker-compose*.yml
# anything else the build never reads

If your cache "randomly" misses on machines where nothing changed, the build context is the first suspect.

ARG, ENV, and surprise invalidation

Build arguments and environment variables are part of the cache key too, and they invalidate more than people expect. The rule: a changed ENV — or a changed ARG value — invalidates every instruction from the point where it's first consumed, because each subsequent RUN executes with that value in its environment. Declaring an ARG is free; using it is what taints the chain.

The classic self-inflicted wound is stamping a volatile value — a git SHA, a build timestamp — near the top of the Dockerfile. Every single build gets a new value, so every single build misses everything below it:

# Bad — volatile ARG consumed at the top: every build invalidates
# the dependency install and everything after it
ARG GIT_SHA
LABEL git-sha=$GIT_SHA
RUN ./mvnw -q dependency:go-offline
# …

# Good — consume volatile ARGs last: only the label layer rebuilds
RUN ./mvnw -q dependency:go-offline
# …
ARG GIT_SHA
LABEL git-sha=$GIT_SHA

The same reasoning applies to ENV: keep stable settings (like JAVA_TOOL_OPTIONS) high in the file, and anything per-build as low as it can go. And remember that the base image is an input too — a mutable tag like :latest resolving to a new digest invalidates the entire build.

The two scopes of caching

Docker caching splits into two distinct scenarios, and most confusion comes from mixing them up:

• Single-machine caching — the cache lives on one machine and is used on that same machine. This is the developer scenario: you're changing code and rebuilding on your laptop all day. It works great out of the box.
• Multi-machine caching — a fleet of machines builds and consumes the same images, so the cache has to live somewhere central. This is your CI/CD system, and it's what your production deployment pipeline cares about.

Single-machine caching

The built-in layer cache

By default, Docker has a built-in cache: every layer produced by a build is kept on local disk, and when you build a new version of the image, Docker automatically reuses cached layers and only re-executes the instructions whose inputs changed. You don't configure anything — it's why the second build of the day takes seconds while the first took minutes.

Cache mounts — the "run cache"

There's also a special, very interesting kind of cache: cache mounts (RUN --mount=type=cache). Use them for files that are generated or needed temporarily during the build but don't belong in the final image.

The classic example is dependency downloads. In a Spring build, Maven downloads all your dependency jars — but the final product is a fat jar, where the classes have been extracted and repackaged; the downloaded jars themselves aren't needed in the image. Without help, every build that invalidates the dependency layer re-downloads every jar. A cache mount designates a directory that persists on the local filesystem across builds while staying out of the image:

# syntax=docker/dockerfile:1
FROM eclipse-temurin:21-jdk AS build
WORKDIR /app
COPY mvnw pom.xml ./
COPY .mvn .mvn
# ~/.m2 persists across builds but never enters the image
RUN --mount=type=cache,target=/root/.m2 \
    ./mvnw -q dependency:go-offline
COPY src src
RUN --mount=type=cache,target=/root/.m2 \
    ./mvnw -q package -DskipTests

Now a change to src/ re-runs the package step, but Maven finds every common library already sitting in the mounted cache — nothing gets re-downloaded. The same pattern applies to npm's cache directory, pip's wheel cache, Go's module cache, and apt package lists.

A cache-mount cookbook

The pattern is the same everywhere — only the target= path changes. Here are the paths for the common ecosystems (adjust /root if your build runs as a non-root user):

# npm
RUN --mount=type=cache,target=/root/.npm npm ci

# pip
RUN --mount=type=cache,target=/root/.cache/pip \
    pip install -r requirements.txt

# Go — build cache and module cache are separate
RUN --mount=type=cache,target=/root/.cache/go-build \
    --mount=type=cache,target=/go/pkg/mod \
    go build ./...

# Gradle
RUN --mount=type=cache,target=/root/.gradle \
    gradle build --no-daemon

# Maven
RUN --mount=type=cache,target=/root/.m2 ./mvnw -q package

# apt — locked: dpkg can't tolerate concurrent access
RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
    --mount=type=cache,target=/var/lib/apt,sharing=locked \
    apt-get update && apt-get install -y build-essential

The sharing= flag matters once builds run concurrently on the same machine (parallel CI jobs, multiple images sharing one builder). The default, sharing=shared, lets every build touch the mount at once — fine for content-addressed stores like Maven's or Go's. sharing=locked serializes access, which is what package databases like apt/dpkg need. sharing=private gives each build its own copy. One apt-specific footnote: Debian-based images ship a docker-clean hook that deletes downloaded packages after install — remove it (or set keep-downloaded-packages) for the apt cache mount to actually accumulate anything.

Multi-machine caching

A single-machine cache is useless the moment your builds run on a fleet — CI runners are often ephemeral, and each fresh machine starts with an empty cache. The images (and their cache) need to be stored centrally.

Registry-based cache

The most common form of multi-machine caching is the registry-based cache: build cache is stored in your OCI registry, right next to the images themselves. This makes it very simple — no extra infrastructure or workflow is needed. Conventionally you use the same repository with a dedicated tag for the cache:

$ docker buildx build \
    --cache-from type=registry,ref=registry.example.com/api:buildcache \
    --cache-to   type=registry,ref=registry.example.com/api:buildcache,mode=max \
    -t registry.example.com/api:0.4.2 --push .

The two directives split the responsibilities:

• --cache-from tells Docker where to retrieve cached layers from — layers whose inputs haven't changed are pulled from the OCI registry instead of being rebuilt.
• --cache-to tells Docker where to store the layers that did change — they're pushed back into the registry cache tag so the next machine gets the hit.

This gives you a standardized cache that works across any number of machines. mode=max is worth calling out: the default (min) only caches layers that end up in the final image, while max also caches intermediate multi-stage layers — usually what you want for CI, at the cost of more registry storage.

A lighter-weight cousin is the inline cache (--cache-to type=inline), which embeds the cache metadata into the image itself so a plain --cache-from your-image:latest works with no separate cache tag — simpler, but limited to min-style caching. BuildKit also supports purpose-built backends (type=gha for GitHub Actions, type=s3, type=local) when your platform provides one.

The trade-off

• Pros: very simple; based on the standardized OCI image format and registry you already run; no extra workflow — cache tags live beside your images.
• Cons: you're pulling layers over the network. A registry round-trip is never as fast as layers already sitting on local SSD — for a warm single machine, the local cache still wins.

Multi-stage builds and the cache

Multi-stage builds interact with caching in ways that are easy to get wrong. Each stage caches independently — a miss in your build stage doesn't invalidate the runtime stage's base layers, and a COPY --from=build layer is invalidated by the checksum of the copied artifact, not by the build stage rebuilding. If the rebuilt jar is byte-identical, the final stage still hits.

FROM eclipse-temurin:21-jdk AS build
# …dependency + package steps from earlier…

FROM eclipse-temurin:21-jre
# invalidated by the jar's checksum, not by the build stage re-running
COPY --from=build /app/target/app.jar /app.jar
ENTRYPOINT ["java", "-jar", "/app.jar"]

Two nuances to keep in mind:

• Only the stages your target needs are built. Building with --target build (say, for a test stage in CI) skips the runtime stage entirely — and its layers never enter the cache for that run.
• This is exactly why mode=max exists. With the default mode=min, a registry cache records only the layers of the final stage — your expensive build-stage layers (the JDK, the dependency downloads) are not exported, so a fresh CI runner misses on precisely the layers that cost the most. mode=max exports every stage's layers. For multi-stage builds in CI, it's rarely the wrong choice.

Debugging cache misses

When a build is slower than it should be, don't guess — make BuildKit show its work:

$ docker build --progress=plain .
  #7 [3/6] COPY mvnw pom.xml ./
  #7 CACHED
  #8 [4/6] RUN ./mvnw -q dependency:go-offline
  #8 CACHED
  #9 [5/6] COPY src src        <-- first step with no CACHED line
  #10 [6/6] RUN ./mvnw -q package -DskipTests

Read it top to bottom and find the first step that isn't marked CACHED — that's your miss; everything after it is collateral damage from the chain rule. Then ask which of that step's inputs changed: the instruction text, the checksums of copied files (see the build-context section above), a build ARG, or the base image tag resolving to a new digest.

For the cache itself, docker buildx du shows what's in it and how big each entry is, and docker builder prune clears it — --keep-storage 10GB keeps the working set while trimming the tail. If a build behaves as if the cache doesn't exist at all, remember both sides must be BuildKit: a legacy builder and BuildKit don't share cache.

Takeaways

• Order Dockerfile instructions so volatile things (your source) come after stable things (your dependencies) — the chain rule makes everything downstream of a miss rebuild.
• Keep the build context clean with a deliberate .dockerignore — COPY . . checksums everything in it.
• Consume volatile ARGs (git SHAs, timestamps) as late in the Dockerfile as possible — using one taints every instruction after it.
• On one machine, the built-in layer cache is automatic; add cache mounts so re-run steps don't re-download the world.
• Across machines, use the registry-based cache with --cache-from/--cache-to and mode=max — one standardized cache for the whole fleet, including your multi-stage build layers.
• Expect the network cost: central cache trades a little speed for consistency everywhere.
• When builds slow down, --progress=plain and the first missing CACHED line tell you exactly where the cache broke.

This is exactly how ModStack's managed build infrastructure is wired — registry-backed layer cache plus cache mounts — which is why the deploy pipeline only rebuilds what your commit actually touched.