Introduce AI tools without synthetic velocity
Adopt AI by measuring whether it improves durable outcomes like clarity, cycle time, quality, and system understanding - not just output volume, ticket count, or code throughput.
- Situation
- A team is adopting AI tools and wants real leverage, not just more apparent output.
- Goal
- Prevent AI adoption from inflating visible activity while leaving the team less certain, less coherent, or more brittle.
- Do not use when
- AI usage is still too limited to evaluate meaningfully
- Primary owner
- engineering manager
- Roles involved
engineering managertech leadcontributorsstaff engineer or architectdelivery lead where throughput claims matter
Context
The situation
Deciding whether to reach for this playbook: when it fits, and when it doesn't.
Use when
Conditions where this playbook is the right tool.
- Leadership or the team wants to scale AI use quickly
- Tool output is rising but confidence is not
- There is pressure to prove productivity gains
- The team senses that AI is changing behavior faster than it is changing results
Stakes
Why this matters
What this playbook protects against, and why skipping or half-running it tends to be expensive.
Synthetic velocity is one of the most dangerous AI-era traps. It feels like acceleration because artifacts multiply. But if understanding, maintainability, and decision quality do not keep up, the team is building future drag while celebrating present speed.
Quality bar
What good looks like
The observable qualities of a team or system that is actually doing this well. Not just going through the motions.
Signs of the playbook done well
- AI usage is tied to specific workflow improvements, not vague productivity claims
- The team measures confidence and quality alongside throughput
- Adoption starts in areas where value can be seen and damage is limited
- Teams can explain where AI saved time and where it added review cost
- Leaders hear a balanced story, not a hype narrative
Preparation
Before you start
What you need available and true before running the procedure. Skipping this is the most common reason playbooks fail.
Inputs
Material you'll want to gather first.
- Current workflow pain points
- Baseline cycle time and quality signals
- Candidate AI tools and target use cases
- Review and release expectations
- Team AI skill variance
Prerequisites
Conditions that should be true for this to work.
- The team has named workflows where AI might help
- There is at least a minimal baseline of current performance and quality
- The organization is willing to measure more than output
Procedure
The procedure
Each step carries its purpose (why it exists), its actions (what you do), and its outputs (what you produce). Read the purpose. It's what keeps the step from degenerating into checklist theatre.
Start from workflow pain, not tool excitement
Tie adoption to real leverage opportunities.
Actions
- Identify slow, repetitive, ambiguous, or cognitively expensive tasks
- Choose a few high-friction workflows as pilot targets
- State what better would look like for each
Outputs
- AI pilot target list
Define success beyond volume
Prevent throughput-only self-deception.
Actions
- Choose indicators like time saved, review cost, defect rate, handoff clarity, or onboarding help
- Pair every output metric with a confidence or quality metric
- State what failure would look like even if output rises
Outputs
- AI adoption scorecard
Run a bounded pilot
Learn before scaling.
Actions
- Apply the tool to a small number of workflows or people first
- Keep the use visible enough to inspect
- Compare before and after without turning the pilot into theater
Outputs
- Bounded pilot plan
- Pilot observations
Audit second-order effects
See whether speed created hidden costs.
Actions
- Check review burden, maintainability, rework, and confusion after AI-assisted output enters the flow
- Look for places where the team is shipping more but understanding less
- Identify whether the tool improves the workflow or only the artifact count
Outputs
- Second-order effects review
Scale selectively and narrate honestly
Expand only where real leverage exists.
Actions
- Scale the tool where gains are durable and visible
- Tighten or restrict use where the signal is weak or harmful
- Report adoption using balanced evidence rather than enthusiasm
Outputs
- AI adoption decision report
Judgment
Judgment calls and pitfalls
The places where execution actually diverges: decisions that need thought, questions worth asking, and mistakes that recur regardless of good intent.
Decision points
Moments where judgment and trade-offs matter more than procedure.
- Which workflows show real leverage potential?
- What indicators prove improvement rather than busyness?
- Where is AI helping, neutral, or actively harmful?
- What should scale now versus stay experimental?
Questions worth asking
Prompts to use on yourself, the team, or an AI assistant while running the procedure.
- What workflow pain are we actually trying to reduce with this tool?
- What would count as failure even if output goes up?
- Where is AI improving the work versus only increasing visible activity?
Common mistakes
Patterns that surface across teams running this playbook.
- Starting with tool capabilities instead of workflow pain
- Measuring only output volume
- Rolling out broadly before second-order effects are visible
- Presenting adoption success as a narrative rather than a comparison
Warning signs you are doing it wrong
Signals that the playbook is being executed but not landing.
- Code and doc output rise while review cost rises too
- Contributors seem faster but the team feels less certain
- Leaders ask for more AI use without understanding where it helps
- The team uses AI everywhere because it is there, not because it fits
Outcomes
Outcomes and signals
What should exist after the playbook runs, how you'll know it worked, and what to watch for over time.
Artifacts to produce
Durable outputs the playbook should leave behind.
- AI pilot target list
- AI adoption scorecard
- Bounded pilot plan
- Pilot observations
- Second-order effects review
- AI adoption decision report
Success signals
Observable changes that mean the playbook landed.
- Specific workflows become measurably easier or faster with no major quality loss
- The team can say where AI helped and where it did not
- Review and maintenance burdens remain visible in adoption decisions
- Adoption expands selectively instead of ideologically
Follow-up actions
Moves that keep the playbook's effects compounding after it finishes.
- Repeat the evaluation on new workflows as the team matures
- Connect synthetic-velocity signals to review, ownership, and documentation work
- Refresh the scorecard as tools and baseline performance change
Metrics or signals to watch
Longer-horizon indicators that the underlying problem is receding.
- Time saved per targeted workflow
- Review cost or review depth change
- Post-merge defect rate in AI-assisted work
- Team confidence in generated artifacts
- Rework rate after AI-assisted output
AI impact
AI effects on this playbook
How AI-assisted and AI-driven workflows help execution, and the ways they can make it worse.
AI can help with
Where AI tooling genuinely reduces the cost of running this playbook well.
- Summarizing pilot observations and before-after comparisons
- Drafting balanced adoption reports
- Clustering recurring AI win and loss patterns
AI can make worse by
Distortions AI introduces that make the underlying problem harder to see.
- Creating more polished but misleading productivity narratives
- Inflating artifact volume so measurement looks better than reality
- Reducing friction around bad use cases, which makes misuse scale faster
Relationships
Connected playbooks
Failure modes this playbook tends to address, decisions behind the situation, red flags that motivate running it, and neighboring playbooks.