Chapter 10: Measuring Success: Solo + Team Metrics Without Fake Precision

Series: LLM Development Guide

Chapter 10 of 15

Previous: Chapter 9: Stop Rules + Pitfalls: When to Upgrade, Bail, or Go Manual

Next: Chapter 11: Team Collaboration: Handoffs, Shared Prompts, and Review

What you’ll be able to do

You’ll be able to tell, with reasonable honesty, whether the workflow is helping:

• Pick a small set of metrics you can actually measure.
• Separate leading indicators (process) from lagging indicators (outcomes).
• Avoid fake precision and vanity metrics.

TL;DR

• If you can’t measure reliably, don’t invent numbers.
• Track a baseline (a few representative tasks) before you claim improvement.
• Favor cheap metrics: time to first commit, PR revision rounds, post-merge bugs.
• Use leading indicators daily; use lagging indicators in retros.

Table of contents

• What to measure
• Solo baseline
• Leading vs lagging indicators
• Lightweight reporting template
• Verification

What to measure

Pick a small set that maps to real outcomes.

Velocity indicators:

• Time to first commit.
• Phase completion time.
• PR cycle time.

Quality indicators:

• PR revision rounds.
• Bugs caught in review.
• Post-merge bugs.

Efficiency indicators:

• Rework rate (time fixing output vs total time).
• Session count per task.
• Handoff success (can someone else continue without re-explaining).

Solo baseline

If you’re working solo, you can still create a baseline.

Track per task:

• Start time.
• First commit time.
• Total time to done.
• Number of “LLM retries” (how many prompt iterations for the same logical unit).
• Bugs you found after “done”.

The point is not perfect measurement. The point is noticing patterns.

Leading vs lagging indicators

Leading indicators predict success:

• Work notes are updated.
• Prompts contain verification.
• Commits are atomic.
• References are provided.

Lagging indicators confirm success:

• PR merged with low rework.
• Low post-merge bug rate.
• Handoffs succeed.

Lightweight reporting template

## LLM-Assisted Development Summary (Month)

### Adoption
- Tasks completed with workflow: <N>

### Velocity
- Median time to first commit: <X>
- Median PR cycle time: <Y>

### Quality
- Median PR revision rounds: <Z>
- Post-merge bugs: <N>

### Costs
- LLM cost estimate: <X>

### Notes
- What worked:
- What failed:
- Changes for next month:

Verification

Keep a simple CSV so you can graph later if you want.

mkdir -p work-notes

cat > work-notes/metrics.csv <<'CSV'
date,task,time_to_first_commit_minutes,total_time_minutes,llm_retries,pr_revision_rounds,post_merge_bugs,notes
CSV

Expected result:

• You can append one row per task in under a minute.

Continue -> Chapter 11: Team Collaboration: Handoffs, Shared Prompts, and Review