How often should the policy be reviewed?
“Every 12 months.”
The first sentence states this directly. The answer addresses the question without adding another requirement.
Worked example · illustrative
A plausible answer is not enough.
Follow a small answer review from source material to judgment. Every statement below is visible, so you can examine the reasoning yourself.
The material and the task.
Can the answer stay within what the source actually says?
This is an authored teaching example with an invented policy and deliberately constructed answers. No model was run to produce these outputs. It is not a client project, a product demonstration or a benchmark result.
Review this policy every 12 months. The policy owner records the review date. Changes to the policy require approval from the operations lead before publication.
Invented source · Policy A, version 1
The reviewer may use only this source. A correct answer should preserve the distinction between reviewing a policy and approving a change. If the source does not answer a question, the response should say so.
Inspect the judgments
What each answer gets right or wrong.
Who approves changes to the policy?
“The operations lead and finance director approve changes.”
The source names the operations lead. It says nothing about a finance director. The added approver is unsupported even though part of the answer is correct.
When and by whom must changes be approved?
“The operations lead approves changes.”
The approver is correct, but the answer omits the requirement for approval before publication. Completeness must be judged against the whole question.
How long should the review records be retained?
“The source does not state a retention period.”
No retention period appears in the permitted material. Inventing one would be an error; identifying the missing information is the useful response.
A scoring guide you can adapt.
Review important claims against the permitted evidence. Record support, completeness and whether the response addresses the question. Keep material error flags separate: a correct approver does not cancel an invented second approver.
The labels here are manual judgments about authored examples. Four examples cannot estimate a model's accuracy or establish readiness for real use. A proper evaluation needs representative material, domain review, recorded conditions and an assessment sample separated from development.
The downloadable worksheet contains the same source, questions, answers, labels and reasons shown on this page. It is a starting format, not a universal scoring standard.
Read the LLM evaluation guide for the wider method, or see Model Evaluation & Benchmarking for a scoped investigation.