VENN grading
The evidence-based training competency grading method (VENN) is the four-dimension method that turns observed crew behaviour, already classified against a competency, into a single grade for that competency. The dimensions are HOW WELL, HOW OFTEN, HOW MANY and OUTCOME. Together they stop a grade from being a gut feeling or a count of ticks: every cell in the word-picture rubric encodes all four.
VENN is a common label for the assessment lens. ICAO Doc 9995 describes the same idea without always calling it "VENN": assess how many observable behaviours (OBs) were demonstrated when required, how often they were demonstrated, and TEM outcome related to that competency; then map the combined picture onto a quality scale and a numeric grade.
The four dimensions
| Dimension | Question | What it measures | Typical word-picture scale |
|---|---|---|---|
| HOW WELL | How effectively was the competency demonstrated? | Quality of performance | ineffectively → minimally acceptable → adequately → effectively → in an exemplary manner |
| HOW OFTEN | How frequently were the indicators shown when required? | Robustness | rarely → occasionally → regularly → routinely → always / almost always |
| HOW MANY | How broad a set of indicators appeared when required? | Acquisition / breadth | few / hardly any → some → many → most → all / almost all |
| OUTCOME | What did TEM look like for this competency? | Effectiveness as a countermeasure | safety compromised → momentarily reduced → maintained → improved → optimized |
ICAO Appendix 4 word pictures supply the how-many, how-often, TEM-outcome and competency-assessment (quality) scales. Five-grade tables for each of the nine competencies weave those scales into one sentence per grade, for example level 3 PRO (Application of Procedures): applied procedures adequately, by regularly demonstrating most of the performance indicators when required, which resulted in a safe operation.
Read each grade cell as one integrated pattern, not four independent votes. If three dimensions sit at level 3 and the fourth is clearly level 1 (unsafe outcome), the outcome dimension carries operational weight: an unsafe result is not rescued by frequent partial indicator use.
The five-point scale
| Grade | Competency assessment (ICAO wording) | Pattern (all nine competencies) |
|---|---|---|
| 5 | in an exemplary manner | Exemplary; always all indicators when required; significantly enhanced safety, effectiveness and efficiency |
| 4 | effectively | Effective; regularly all indicators when required; enhanced safety |
| 3 | adequately | Adequate; regularly most indicators when required; safe operation |
| 2 | minimally acceptable | Minimum acceptable; occasionally some indicators when required; did not result in an unsafe situation |
| 1 | ineffectively | Not adequate / not effective; rarely any indicators when required; resulted in an unsafe situation |
Parallel wording across competencies is deliberate: once a rater internalises the level-2 row for one competency, they hold the level-2 standard for the others. The only competency-specific element is the opening verb phrase ("communicated", "managed the automation", "had … knowledge").
Where VENN sits in the grading cycle
- Observe crew behaviour in the FSTD session.
- Record significant moments (timeline; effective and ineffective).
- Classify against behavioural indicators and allocate to competencies.
- Evaluate with VENN → one grade 1–5 per competency for the phase or overall rules.
Skipping Record leaves the grade undefended. Skipping Classify makes it free-floating opinion. Skipping Evaluate (no rubric) leaves it as intuition.
Operational thresholds (illustrative vs ICAO example)
Illustrative recurrent EBT module rules:
- Any competency at level 1 → session outcome ADDITIONAL TRAINING REQUIRED.
- All competencies at level 2 or above → COMPETENT.
- Consecutive level-2s in the same competency across recurrent cycles trigger required training within six months even if each session alone was COMPETENT (catch slow drift).
- Record applicable indicators for grades 1, 2 and 5; electronic forms may default unentered cells to level 3.
- Day 2 scenario-based training: weight process (how competencies were applied) more than raw outcome; the phase is deliberately demanding and may need instructor assistance.
- In-seat instruction: not graded.
ICAO Doc 9995 example policy language often illustrates "required level" as grade 3 or above on a five-point scale, with below that treated as not fully achieving module objectives. The illustrative rules above set the hard floor at level 2 (minimum acceptable, no unsafe situation). On the floor, bind to the approved local programme; know which rule set you are under when you debrief and file.
Level 1 under high demand, once, may be a reactive failure rather than a settled competency gap. ADDITIONAL TRAINING REQUIRED is strongest when low demand, repetition, and stalled learning coincide.
Instructor use
- Decompose a disputed grade along the four axes in calibration or debrief: which dimension is the disagreement on?
- Let unsafe outcome force a low grade even when some indicators looked good.
- Do not invent a hybrid score ("3.5"); pick the word picture that best matches the whole pattern.
- On Day 2, resist grading process as if it were a checking outcome under light demand.
- Protect inter-rater reliability by quoting the cell language, not personal synonyms ("pretty good", "rough").
Connections
- Core competencies. The nine cells graded with VENN each session phase.
- Behavioural indicators. Supply how-many and how-often; evidence for how well.
- Word pictures. The verbal anchors of each grade cell; VENN is how you read them as four dimensions.
- Inter-rater reliability. Word pictures and VENN exist largely to stabilise grades across instructors.
- From competency to observable behaviour. VENN is the evaluate step in that chain.
- Threat and error management. Outcome dimension is TEM-specific to the competency assessed.
- Evidence-based training. Grading produces the data EBT programme analysis depends on.
- Fault analysis. Low grades on specific dimensions point remediation.
- Evaluation cycle. Design philosophy (criterion-referenced testing (CRT), fidelity, profile reporting) that VENN implements on the floor.
- Day 1 versus Day 2 grades. How Day 2 process-over-outcome grading and high-demand level-1 caution prevent misreading stretch grades as Day-1 check failure.
Sources
- A4.2.4 Grading Methodology for Recurrent Training and Checking. Observe–Record–Classify–Evaluate; VENN four dimensions; phase timing; level-1 and consecutive-2s rules.
- A4.E Performance Grades. Verbatim 1–5 word pictures for nine competencies; four axes encoded in every cell.
- Doc 9995, Part I Ch 7 (Conduct of evidence-based training). How many OBs, how often, TEM outcome; grading table 1–5; word pictures support inter-rater reliability (IRR) and validity.
- Doc 9995, App 4 (Word pictures for competency assessment). Official scales for how many, how often, TEM outcome, and competency assessment quality.