The build that ran twice
Our dashboard took 12 minutes to deploy and 3 to build on a laptop. Reading the pipeline line by line found the app being built twice.
Every deploy of our merchant dashboard took about twelve minutes. The same production build took three on my laptop. Slow deploys tax everything: hotfixes wait, people batch changes to avoid the queue, and small, frequent releases stop feeling cheap.
The easy explanation was “the build machine is slower”. It was, a little. But not four times slower.
Reading the pipeline
The deploy ran on AWS CodeBuild using a buildspec.yml. I read it top to bottom, then the Dockerfile it used. The pattern was hiding in plain sight:
- The buildspec ran
npm installandnpm run buildon the build machine. - Then it ran
docker build. - The Dockerfile ran
npm ciandnpm run buildagain, inside the image.
The output of the first build was never copied into the image. It was built, then thrown away. Every deploy paid for the whole build twice.
The fix
- Removed the outer build. Its output wasn’t used, so nothing depended on it. That alone saved four to five minutes.
- Skipped lint in the production build only. Lint already runs on every push, so the production image was re-checking code that had already passed. Lint stays on for development and UAT builds, where it still catches problems early.
The whole pipeline went from 12 minutes 3 seconds to 3 minutes 15 seconds, about 73% faster.
12:03
Deploy time, before
- Fetch source
- npm run build (outside Docker, output never used)
- docker build → npm run build
- lint inside the production build
- Push image and deploy
Stage widths are illustrative; the start and end totals are the measured pipeline times. Lint still runs on every push, and stays on in dev and UAT builds.
The gap that’s left
After the fix, the cloud build was still a little slower than local. That one really was hardware: a recent laptop against a 4 vCPU, 8 GB build machine. Locally, the Docker build took about 3 minutes, and 93 seconds without lint.
More vCPUs would close the gap, but every build would cost more. I wrote the trade-off down rather than making the change, because it depends on how often the team deploys, and that’s a team decision.
What I took from it
- Read the config, not just the logs. The logs said “building”. Only the config said “building twice”.
- Every step should have a consumer. If nothing uses a step’s output, the step is probably waste.
- Keep safety nets where they catch things. Removing lint everywhere would have been faster still, and wrong.