Writing a migration playbook for an AI agent
Porting a legacy tool screen by screen is repetitive and easy to get subtly wrong. Writing the rules down as an agent skill made every port consistent.
When we moved our operations console to a new app, the work was mostly the same task repeated: take a screen from the old React 17 tool, rebuild it in the new Next.js app, and make sure it behaves the same. Repetitive work is where an AI coding agent helps, and also where small inconsistencies multiply fastest.
So before porting anything at scale, I wrote the rules down as a skill: a folder of instructions and reference notes the agent loads when it’s asked to migrate a feature.
What went into the playbook
- A route map. Which old route becomes which new route, so links and bookmarks keep working.
- Permissions. How the old tool checked access and how the new app expresses the same checks. This is the part you most want to get right and least want to improvise.
- Where the old code lives. Which repositories and folders to read for a given feature, so the agent works from the real source rather than guessing.
- What parity means. The new screen should match the old one’s data and behaviour, but use the new design system’s components rather than copying old markup.
- House conventions. API calls through shared hooks, services per feature, the shared table and filter components.
How it changed the work
Each port started from the same checklist. Reviews got shorter, because the predictable parts were predictable, and the conversation could focus on what was actually different about a screen. About sixteen merchant configuration screens moved across this way, along with reports and compliance tools.
Checking the result
Behaviour and style changes are easy to miss in code review. For a batch of changes, I had the agent produce a smoke-test sheet: every affected page, what changed and what to check. I went through it page by page, marked each test pass or fail with notes, and fed the failures back. It’s low-tech and it caught real issues, like a table whose fixed columns had the wrong background and a filter handler that silently did nothing.
What I learned
- Write the playbook before the hundredth port, not after. The rules are cheapest to set when there’s nothing to undo.
- Put judgement in the rules, not in each prompt. If you find yourself repeating a correction, it belongs in the playbook.
- Verify like a person. The agent writes faster than anyone can review, so give the review a structure.