Word pictures
Word pictures are the approved verbal anchors for each competency grade. They are short, fixed sentences (or dimension-scale phrases) that say what performance at that level looks like, so different instructors apply the same standard instead of a personal thesaurus.
They are not figurative teaching language, not training aids, and not the behavioural indicators themselves. Indicators are the evidence menu; word pictures are the grade language you match the observed pattern against.
What they look like
Two layers appear in practice:
| Layer | What it supplies | Where |
|---|---|---|
| Dimension scales | Shared vocabulary for how many, how often, TEM outcome, and quality of competency demonstration | ICAO Doc 9995 Appendix 4 |
| Full grade cells | One integrated sentence per grade (1–5) per competency, weaving those scales together | Course performance-grade tables (e.g. Train-the-Trainer Reference E); approved local programme if it differs |
ICAO Appendix 4 scales (paraphrased):
| Dimension | Scale language (low → high) |
|---|---|
| HOW MANY | few / hardly any → some → many → most → all / almost all |
| HOW OFTEN | rarely → occasionally → regularly → routinely → always / almost always |
| OUTCOME (TEM) | safety compromised → momentarily reduced → maintained → improved → optimized |
| Competency assessment | ineffectively → minimally acceptable → adequately → effectively → in an exemplary manner |
A full cell binds those axes into one pattern. The course performance-grade tables do that for each competency at each grade; the section below shows how those sentences are assembled.
How a full word picture is built
Course cells are not free prose. They follow a fixed sentence template that slots one scale term from each VENN axis next to a competency-specific verb.
Template (course performance grades):
The pilot [competency verb + HOW WELL], by [HOW OFTEN] demonstrating [HOW MANY] of the performance indicators when required, which [OUTCOME].
| Slot | What goes in | Source of vocabulary |
|---|---|---|
| Competency verb | Only element that changes by competency | Adapted model (e.g. applied procedures, communicated, managed the automation) |
| HOW WELL | Quality adverb or phrase | ICAO competency-assessment scale / course grade language |
| HOW OFTEN | Frequency when required | ICAO how-often scale |
| HOW MANY | Breadth of indicators when required | ICAO how-many scale |
| OUTCOME | TEM / safety result for that competency | ICAO TEM-outcome scale (course wording may compress "safety maintained" into "safe operation") |
The five grade patterns (shared across competencies)
Once the template is fixed, each grade is a package of four scale terms. Parallelism is the design: level 3 for PRO and level 3 for COM use the same four-term package; only the opening verb changes.
| Grade | HOW WELL | HOW OFTEN | HOW MANY | OUTCOME (course wording) |
|---|---|---|---|---|
| 5 | in an exemplary manner | always | all | significantly enhanced safety, effectiveness and efficiency |
| 4 | effectively | regularly | all | enhanced safety |
| 3 | adequately | regularly | most | resulted in a safe operation |
| 2 | at the minimum acceptable level | only occasionally | some | did not result in an unsafe situation |
| 1 | did not … correctly / effectively | rarely | any | resulted in an unsafe situation |
Worked example: assemble PRO level 3
| Slot | Choice for grade 3 |
|---|---|
| Competency verb | applied procedures |
| HOW WELL | adequately |
| HOW OFTEN | regularly |
| HOW MANY | most |
| OUTCOME | resulted in a safe operation |
Assembled cell (verbatim course rubric):
The pilot applied procedures adequately, by regularly demonstrating most of the performance indicators when required, which resulted in a safe operation.
Decomposed back to axes (same cell, read for debrief or standardisation):
| Axis | Term in the sentence |
|---|---|
| HOW WELL | adequately |
| HOW OFTEN | regularly |
| HOW MANY | most of the performance indicators when required |
| OUTCOME | safe operation |
Parallel build: same grade, different competency
Swap only the competency verb. The four scale terms stay grade 3.
| Competency | Grade 3 cell (course wording) |
|---|---|
| PRO | The pilot applied procedures adequately, by regularly demonstrating most of the performance indicators when required, which resulted in a safe operation. |
| COM | The pilot communicated adequately, by regularly demonstrating most of the performance indicators when required, which resulted in a safe operation. |
Level 1 uses the same swap rule on the bottom package:
| Competency | Grade 1 cell (course wording) |
|---|---|
| PRO | The pilot did not apply procedures correctly, by rarely demonstrating any of the performance indicators when required, which resulted in an unsafe situation. |
| COM | The pilot did not communicate effectively, by rarely demonstrating any of the performance indicators when required, which resulted in an unsafe situation. |
Full PRO ladder (all five packages)
| Grade | Full word picture (course PRO cells) |
|---|---|
| 5 | The pilot applied procedures in an exemplary manner, by always demonstrating all of the performance indicators when required, which significantly enhanced safety effectiveness and efficiency |
| 4 | The pilot applied procedures effectively, by regularly demonstrating all of the performance indicators when required, which enhanced safety |
| 3 | The pilot applied procedures adequately, by regularly demonstrating most of the performance indicators when required, which resulted in a safe operation |
| 2 | The pilot applied procedures at the minimum acceptable level, by only occasionally demonstrating some of the performance indicators when required, but which overall did not result in an unsafe situation |
| 1 | The pilot did not apply procedures correctly, by rarely demonstrating any of the performance indicators when required, which resulted in an unsafe situation |
Parallel wording is deliberate. A rater who internalises the level-2 package holds the level-2 standard for every competency. An instructor who narrates level 3 with level 4 vocabulary ("communicated effectively, … all of the indicators") has by definition graded 4, whatever their impression of the session.
How they relate to VENN and indicators
VENN grading is the method for reading a cell: HOW WELL, HOW OFTEN, HOW MANY, and OUTCOME at once. Word pictures are the language of those cells. Behavioural indicators (observable behaviours) tell you what you saw; the word picture tells you which single grade that session pattern is.
Read each grade as one integrated pattern, not four independent votes. An unsafe outcome is not rescued by frequent partial indicator use. Do not invent hybrids ("3.5"); pick the word picture that best matches the whole pattern.
evidence menu] --> Obs[Observed pattern] Obs --> WP[Word picture cell
1 to 5] WP --> Grade[Competency grade]
Why they exist
Competency programmes replace simple manoeuvre pass/fail with multi-competency grades that drive remediation, licence trust, and programme data. That only works if Instructor A and Instructor B would score the same evidence the same way.
Word pictures exist largely for Inter-rater reliability: shared verbal anchors for levels 1–5 so raters converge. They also support assessment validity: grades must align with the approved terms of reference (adapted competency model, word pictures, and policy thresholds), not only with peer consensus.
Authority layers
| Question | Reach for |
|---|---|
| What dimension language is international method? | Doc 9995 Appendix 4 and Part I Chapter 7 |
| What exact 1–5 cells does this course teach? | A4.E Performance Grades |
| What is binding on the shop floor for a given operator? | The authority-approved adapted model and grade forms |
Trace upward when challenged: local word picture → approved adapted model → ICAO framework and Doc 9995 scales.
Instructor use
- Quote the cell language when you grade and when you debrief extremes (typically levels 1, 2, and 5).
- Decompose a disputed grade on the four VENN axes; find which dimension is the real disagreement.
- Do not count indicators as a checklist and call that a grade; match the pattern to one word picture.
- On Day 2 scenario training, keep the same word pictures under process-weighted phase rules; do not invent a tougher personal scale.
- In standardisation, defend grades with the cell text and recorded indicators, not narrative impressions.
Connections
- VENN grading. Method that turns the four dimensions into one grade; every cell is a VENN-encoded word picture.
- Behavioural indicators. Evidence menu that how-many and how-often dimensions quantify.
- Inter-rater reliability. Primary reason word pictures exist: shared anchors across the instructor roster.
- Core competencies. The nine (or local adapted) cells graded with word pictures each phase.
- From competency to observable behaviour. Word pictures are the evaluate-step standard in the assessment chain.
- Evaluation cycle. Profile reporting uses competency × grade word pictures, not a single collapsed mark.
- Day 1 versus Day 2 grades. Same word pictures under different phase conditions (criterion-referenced testing).
- ICAO Doc 9995. Appendix 4 and Part I Chapter 7 carry the ICAO word-picture scales and assessment method.
- Evidence-based training. Programme data quality depends on grades that match shared word pictures.
Sources
- Doc 9995, App 4 (Word pictures for competency assessment). Official scales for how many, how often, TEM outcome, and competency assessment quality.
- Doc 9995, Part I Ch 7 (Conduct of evidence-based training). Competency assessment method; word pictures support inter-rater reliability and assessment validity.
- A4.E Performance Grades. Verbatim 1–5 word pictures for nine competencies; parallel cell design.
- A4.2.4 Grading Methodology for Recurrent Training and Checking. Grades based on word pictures as observations of fact; observe–record–classify–evaluate and VENN.
- 11.2 Functions of Evaluation. Word-picture rubric as the profile-reporting tool for EBT.