Evaluation

What question does this solve?

An output exists. Is it good enough for its purpose? Answering that requires knowing which output is being examined, under which conditions it was produced, which criteria still apply, and what information would support a conclusion. It also requires keeping the examination apart from the decision that follows it.

Definition and role

Evaluation examines outputs and evidence against the relevant criteria to support the Quality Gate decision. It relates criteria, output and Evidence to conclude whether there are grounds for acceptance or whether some other routing is needed. Its purpose depends on the Stage Contract and on the decision that has to be made.

Evaluating a feasibility analysis is not the same as evaluating an implementation. In the first case, a conclusion that something is not viable can meet the purpose. In the second, an output that does not implement the agreed behaviour may need correction. Applying generic checks unrelated to the purpose produces records without supporting the relevant decision, and examining out-of-date material can lead to a wrong conclusion about the current situation.

Evaluation is not acceptance. The criterion describes the condition of adequacy; evidence provides relevant information; evaluation interprets it; the Quality Gate establishes acceptance within the available authority. These can happen in one interaction, but a completed check is not automatically an acceptance, and a favourable recommendation does not grant authority to deploy a change.

Why it matters for the hypothesis

A small advance only becomes something that can be bought again and again if its result can also be examined at a compatible cost. AI can take part in preparing checks, interpreting results and formulating usable feedback, and engineering of criteria and environments lets evaluation capacity be reused across demands while keeping each demand’s context. The cost includes preparation, running the procedures, interpretation and re-evaluation, plus the effect of wrong acceptances and wrong rejections.

How it works

Preparing the evaluation

Before evaluating, the reference of the output, the availability of information and the conditions of the environment have to be checked. A missing precondition can prevent evaluation without showing that the output failed. The procedure must fit the question: some criteria need observable behaviour, others clarity of a decision or compatibility with a constraint. Who may evaluate and who may accept can coincide, but that should not be assumed; the Execution Policy and Governance set competence, supervision and authority. Preparation consumes resources, and producing an output with no means of evaluating it only moves the waiting elsewhere.

These dimensions help choose relevant criteria. They are questions for evaluation, not five extra gates, and not an obligation to examine everything to the same depth.

Dimension	Question	Possible application
Conformity	Does the output contain and do what the contract required?	Check the agreed behaviours and the planned evidence.
Coherence	Does it respect the requirements, constraints and decisions in force?	Check that the booking rule implemented matches the accepted version.
Adequacy	Does it serve its purpose in the stated context?	Check that discovery clarifies the alternatives needed to decide, including when it shows something is not viable.
Risk	Are limitations, uncertainties and relevant consequences explicit?	Relate an unverified dependency to the scope of the conclusion and the conditions for continuing.
Operability	When software will be used, can it be deployed, observed, maintained and recovered?	Examine deployment and recovery in the agreed environment; a discovery need not be deployable.

The Stage Contract bounds which dimensions matter, to what extent and with which evidence. Adequacy to the stage’s purpose does not prove a business effect.

Forms of evaluation

People, agents with relevant competence and context, automated checks, or a combination can carry out evaluation. An automated check can examine bounded conditions; a human review can interpret context and adequacy; an agent can identify divergences, examine alternatives and form a conclusion, and with delegated authority a system can also decide acceptance within the applicable scope. No origin guarantees sufficiency on its own. When production and evaluation share the same mistaken interpretation, agreement can leave the problem unexamined; depending on risk, an additional reference or separated responsibilities may be needed, without requiring full separation everywhere. Human evaluation needs real working conditions: assigning acceptance to someone without context, time or access does not demonstrate effective control.

Results and what follows

Situation found	Possible next step
Criteria met with sufficient support	Acceptance decision by a competent authority.
Divergence between output and the criterion in force	Correction or a bounded new execution.
Insufficient evidence	Producing or confirming relevant information.
Ambiguous or contradictory criterion	Clarifying the contract with whoever can decide.
Missing precondition for evaluation	Waiting, or resolving the dependency.
Relevant change of context	Revising the scope of the evaluation and the conditions in force.

A partly adequate output should not be treated as fully accepted for convenience; if the contract allows partial acceptance, its scope and pending items stay visible. An inconclusive evaluation is not a final rejection, and its reason must be kept so that coordination routes the right action rather than repeating production when the problem lies in the available information.

Usable feedback for correction

When an output misses a criterion, the result should make the divergence understandable: relate the behaviour or content examined to the criterion and to the information supporting the conclusion. It need not prescribe the whole solution. It should distinguish a requirement in force from a new preference; treating every suggestion as a defect changes the commitment without making the change explicit. The next evaluation should cover the correction and its relevant effects; fixing one divergence does not show that the other conditions still hold, and re-evaluation should reflect what changed.

Changing criteria and diverging evaluations

Criteria may need revising, with justification and authority, while preserving which conditions guided the earlier production. Changing the reference only to declare success weakens the whole function of evaluation. Evaluators can disagree; before deciding, check whether they examined the same output, criteria and context. Divergence is not settled by majority or by the most convenient result. If the conflict persists, Escalation takes it to the right authority or competence, presenting the point of divergence and its consequences.

In the running example

At the barbershop, an implementation lets customers book times but allows a conflict when services have different durations. Evaluation compares the observed behaviour with the accepted rules and returns a usable divergence for correction. An authorized executor can find the cause, change the implementation and produce new evidence; an evaluation mechanism can examine the behaviour again and support acceptance with no human review on that path, if that is the applicable policy. Another configuration may require a person in the evaluation; the technical function stays the same.

The executor’s statement that it fixed the problem does not replace the relevant checks, and two agents agreeing does not, alone, show that the rule is met: they may share an incomplete interpretation. Observing the environment, the relevant cases and the reference in force helps support the conclusion. If the duration rule was never clarified, the problem is different from breaking it, and the operation seeks the decision needed. Changing a criterion only to accept a defective output is not a correction.

Boundaries and failure modes

Evaluation can set the pace. If outputs arrive faster than they can be examined, the operation has to deal with that relation instead of measuring production alone.

More is not always better. Repetition without purpose increases waiting; insufficient evaluation lets work continue without support. Usefulness depends on the criteria and the commitment. How to demonstrate sufficiency, organize independence and reuse results remains open; the model offers no universal guarantee of quality or mandatory protocol. Technical acceptance does not show a business benefit or authorize continuing.

Connected concepts

• Engine overview
• Next: Quality Gate, the acceptance decision evaluation supports
• Evidence, the information evaluation interprets
• Retry and Escalation, what can follow an unfavourable result
• Glossary