A list of characteristics and criteria to judge the quality of an AI-augmented workflow, and compare different workflows.
Whether you use a popular third-party workflow, pursue the vision of an AI software factory, or rely on a homegrown approach, use these criteria presented below to audit your current workflow and harness, compare alternatives, and identify opportunities for improvement.
AI is a multiplier: what you feed in plays a role in what you get out
AI is a multiplier; it means that your environment influences the effectiveness of your workflow and its harness. Factors that play a role include:
- the AI readiness of your existing codebase and the infrastructure
- your org and teams’ ways of working and how well they fit AI-augmented development
- your org and teams’ existing skills and experience in using, adapting and evolving the workflow and its harness
- how tightly the workflow and its harness fit the context and circumstances 1
In general terms
Individual components such as the foundation models used, tooling, guardrails, deterministic verification, self-correcting loops, MCPs, Skills, Hooks, all need to be good. Nonetheless, the evaluation requires looking at the outcomes and impacts they create together when used with the given codebase by the people doing the work.
Characteristics and criteria
The characteristics and criteria visible in a good workflow and its harness are listed below, organised in six groups.
A workflow with its harness is good when it …
Quality
- Improves or preserves the quality of the code, and doesn’t induce tech debt under the radar
- Reduces or doesn’t increase the number of defects and production incidents
- Maintains an overall architectural and design coherence without causing a drift
ROI
- Is efficient in terms of tokens spent and LLM costs, with bounded, predictable costs
- Ships code faster, driving positive business impact and increasing business responsiveness
People
- Does not make cognitive fatigue spike
- Limits systems and domain knowledge erosion
- Runs autonomously for long stints without causing frequent context-switching and multi-tasking
- Assists code reviewers with automated pre-reviews and guides that make sense of complex changes
- Supports the collaboration between Product Managers and Developers without causing exhaustion
- Delivers valuable change at a sustainable pace without causing users’ exhaustion
Workflow
- Keeps the planning (the P of the PEV phases) time and effort proportional to the actual complexity of the task
- Supports Business-Product-Tech collaboration leading to global (not local) improvements
- Augments employees effectively rather than replacing them, without causing overall deterioration
- Keeps tooling and CI/CD pipelines running without being overloaded and becoming a bottleneck
Model
- Performs consistently and predictably, creating a sense of trust and giving a sense of control
- Paces time and effort proportionally to the complexity of the task at hand
- Communicates and offers solutions transparently, in a way that feels natural to the human users
Workflow and Harness itself
- Is fit for purpose, for the specific needs, context and circumstances
- Is simple 2:
– easy to understand, evolve, adapt to changing needs and circumstances
– easy to recreate with AI, whenever new models and tools become available
How some of these criteria come together with their tradeoffs, read this 3.
What other characteristics and criteria would you suggest?
Other suggested reading
Ship Production-grade Software Faster with AI. No AI slop.
Drive Value Faster with AI. No AI bloat.
See how we can help across the full AI adoption lifecycle.