From competency to observable behaviour

How does an abstract pilot capability become a defensible grade and an operational training consequence? One chain runs from competency framework through indicators, observation, grade, and training consequence, with different labels at each layer. Join those layers and the instructor can walk the whole path without flipping between manuals mid-debrief.

#Position

A competency is only assessable when it is reduced to observable behaviour, graded against a shared word picture, and filed under a policy threshold. Skip any link and the result is opinion, checklist theatre, or unreliable data.

The numbered steps below walk the same chain in more detail than the diagram above.

#1. Internal capability: KSA

Knowledge, skills and attitudes are what the pilot draws on. Training plans list them; they are not usually the primary scored unit in the FSTD. A knowledge gap, a motor-skill deficit and an attitude that blocks use of existing knowledge can produce the same failed manoeuvre. Diagnosis later in the chain must be able to reach back to this layer.

#2. Integrated unit: competency

The adapted model grades nine core competencies (eight ICAO plus KNO). ICAO aeroplane-pilot baseline is eight. Competencies are transferable across scenarios; tasks and manoeuvres are vehicles. The point of the frame is root cause and resilience to the unforeseen, not coverage of a fixed event list.

#3. Evidence menu: behavioural indicators

Each competency carries short, observable indicators (ICAO: OBs; often called performance indicators in grading tables). They define the competency for assessment purposes. They are not a checklist that sums to a grade. A below-standard call should name an indicator that could lead to an unsafe situation.

#4. Process: Observe → Record → Classify → Evaluate

Doc 9995 and the grading methodology agree on the sequence:

1. Observe performance in the session.
2. Record effective and ineffective behaviours (notes / timeline).
3. Classify against indicators and allocate to competencies.
4. Assess / evaluate against the standard (word pictures + dimensions).

Low performance on a competency normally names the area to remediate next.

#5. Lens: evidence-based training competency grading method (VENN) four dimensions

For each competency, integrate:

• HOW WELL the competency was demonstrated (quality)
• HOW OFTEN indicators appeared when required (robustness)
• HOW MANY indicators appeared when required (breadth)
• OUTCOME of TEM related to that competency (effectiveness as countermeasure)

ICAO Appendix 4 supplies the word-picture scales; Train-the-Trainer Reference E encodes them in one sentence per grade per competency.

#6. Number: grade 1–5

One grade per competency for the phase/session rules, not one grade per indicator. Parallel word pictures across competencies support standardisation. Record supporting indicators for extreme and threshold grades (typically 1, 2 and 5).

#7. Consequence: policy thresholds

Under illustrative recurrent rules, any level 1 → ADDITIONAL TRAINING REQUIRED. All competencies ≥2 → COMPETENT. Consecutive 2s in the same competency across cycles trigger follow-up training. ICAO example text often illustrates "required level" as ≥3. On the floor, bind to the approved local programme; know which policy you are under. Day-2 scenario training weights process over raw outcome. In-seat instruction is not graded.

#8. Social condition: inter-rater reliability

The chain only produces comparable data if different instructors walk it the same way. Word pictures, observe, record, classify, evaluate (ORCE) discipline, instructor training and concordance (ICAP) exist so a grade is about the pilot, not the rater. Validity (alignment with the approved terms of reference) must sit beside reliability.

#Where instructors break the chain

#Instructor use

• Before the session: know which competencies the lesson plan must exercise and which indicators you expect to see.
• During: observe and record; do not freeze the session to complete a mental checklist of all indicators.
• After classify and evaluate: state session outcome first in debrief, then open the facilitated discussion on the critical competencies.
• Link remediation to the weak competency and the KSA layer beneath it, not only to "try the approach again."
• When challenged, walk the chain aloud: what I saw → which indicator → which VENN dimensions → which word picture → which policy rule.

#Connections

• Evaluation cycle. Design philosophy (criterion-referenced testing (CRT), fidelity, diagnostic vs achievement) that ORCE and VENN implement per session.
• Core competencies. Layer 2: the nine graded capabilities.
• Behavioural indicators. Layer 3: observable evidence menu.
• Knowledge, skills and attitudes. Layer 1: internal capability behind behaviour.
• VENN grading. Layers 5–6: dimensions and grade cells.
• Word pictures. The shared evaluate-step language for each grade level.
• Inter-rater reliability. Layer 8: consistency condition for the whole chain.
• Instructor competencies. Framework that qualifies people to run the chain.
• Fault analysis. Tool for moving from weak grade back to root cause and KSA.
• Evidence-based training. Programme that depends on this chain for individual and fleet decisions.
• Threat and error management. Outcome dimension and operational context for competent performance.
• Facilitation. Debrief technique that turns the grade into trainee insight without abandoning the evidence chain.

#Sources

• Doc 9868, Part I Ch 2. Competency framework structure; adapted models; KSA Attachment B; assessment reliability principles; tasks as vehicles for competencies.
• Doc 9868, Part II §1. Aeroplane-pilot competencies and OBs; instructor/evaluator competencies.
• Doc 9995, Part I Ch 7 (Conduct of evidence-based training). Observe–record–classify–assess; how many / how often / TEM outcome; grading 1–5; KSA; outcome management.
• Doc 9995, App 4 (Word pictures for competency assessment). Dimension scales for assessment.
• Doc 9995, Part I Ch 6. Instructor competence and ICAP.
• A4.D Core Competencies. Nine competencies and KNO; root-cause purpose of the framework.
• A4.E Performance Grades. Word-picture rubric.
• A4.2.4 Grading Methodology for Recurrent Training and Checking. ORCE, VENN, phase rules, COMPETENT vs ADDITIONAL TRAINING REQUIRED.
• A4.1.6 Competencies. Competencies over event lists; observable behaviour required for below-standard.