Skip to main content
The Hard Parts.dev
RF-36 Ai · Ai Quality RF Red Flags
Severity high Freq increasing

More output, less certainty

The volume of produced artifacts increases, but confidence in correctness, design quality, or shared understanding does not keep pace.

Severity
high
Frequency
increasing
trend
First noticed by
reviewers · staff engineers · engineering manager
Detectability
visible-if-you-look
Confidence
high
At a glanceRF-36
Where you see this

AI-assisted codingAI-generated docsAI-assisted planning and analysis

Not necessarily a problem when
the team is in a bounded experimentation phase and is explicitly studying this gap
Often mistaken for
we are more productive because we produced more
Time horizon
near-term
Best placed to act

engineering leadAI tooling owner

The signal

What you would actually notice

This is one of the clearest signals of synthetic velocity.

Field observation

More code, docs, tickets, or analyses appear quickly, but reviewers, owners, or users feel less sure what is actually trustworthy.

Also observed

  • We shipped much more, but I trust it less.
  • The deck looks great, but I do not know if the system is healthier.

Primary reading

What it usually indicates

Most likely underlying patterns when this signal shows up. Not a diagnosis, a starting hypothesis.

Usually indicates

Most likely underlying patterns when this signal shows up.

  • AI leverage outrunning verification
  • approval-shaped review
  • weak evaluation discipline

Stakes

Why it matters

This is one of the clearest signals of synthetic velocity.

Inspection

What to check next

Deliberate steps to confirm or disconfirm the primary reading above. Not a checklist. An order of inspection.

  1. review depth
  2. reopen rates
  3. bug and incident patterns
  4. evaluation quality

Diagnostic questions

Questions to ask the team, or yourself, before concluding anything.

  1. What confidence mechanism kept pace with the output increase?
  2. Who actually trusts the new artifacts?
  3. What got faster without getting safer?

Progression

Under the signal

Where this pattern tends to come from, what's holding it up, and where it goes if nothing changes.

Leading indicators

What tends to show up first.

  • authors hedge more despite faster production
  • reviewers rely on green checks over understanding
  • teams reopen work more often after 'faster delivery'

Common root causes

What is usually sitting under the signal.

  • tool enthusiasm
  • weak verification design
  • output-biased incentives

Likely consequences

What happens if nothing changes.

  • synthetic velocity
  • review fatigue
  • hard-to-maintain systems

Look-alikes

Not what it looks like

Patterns that can be mistaken for this signal, and 'fix' attempts that make it worse.

False friends Things the signal is often confused with, but isn't.
  • we are more productive because we produced more

Anti-patterns when responding

Responses that feel sensible and usually make the underlying pattern worse.

  • celebrating throughput without confidence evidence
  • assuming more generated artifacts equals more solved work

Context

Context and ownership

Where this signal surfaces, who sees it first, who can actually act, and how much runway there usually is before escalation.

Common contexts

Where it shows up

  • AI-assisted coding
  • AI-generated docs
  • AI-assisted planning and analysis
Most likely to notice

Who sees it first

Before it escalates.

  • reviewers
  • staff engineers
  • engineering manager
Best placed to act

Who can move on it

Not always the same as who notices it.

  • engineering lead
  • AI tooling owner
Time horizon

near-term

How much runway there usually is before the signal hardens into the underlying pattern.

AI impact

AI effects on this signal

How AI-assisted and AI-driven workflows tend to amplify or hide this signal.

AI amplifies

Ways AI tooling tends to make this signal louder or more common.

  • This is the core AI-amplified pattern itself.

AI masks

Ways AI tooling tends to hide this signal, so it keeps growing under the surface.

  • High polish and volume obscure falling confidence.

Relationships

Connected signals

Related failure modes, decisions behind the signal, response playbooks, and neighboring red flags.