A4.2.4 Grading Methodology for Recurrent Training and Checking

#How to Grade

The grading system has been designed to follow what is believed to be natural behavior in the training role. It is set out as a four-step sequence:

• Observe everything the crew do, including SOP actions, verbal and non-verbal communication, use of OM information, checklists, management of the flight path, etc.
• Record anything of interest, i.e. when you see the crew doing something well or doing something that may need some improvement. Keep a timeline and highlight significant parts of the session.
• Classify what you have recorded against competencies, using performance indicators as a guide.
• Evaluate the appropriate rating for performance in each competency using the VENN system.

After a training event or phase, pilot performance for each Core Competency is then rated with reference to a 5-point scale. Each grade point for all 9 Core Competencies is described in Pilot Performance Grades, the 9-competency × 5-grade word-picture rubric. When determining the grade, reference is made to the 9 core competencies. The grades are intended to reflect an overall assessment of the trainee's performance during the event or phase.

Use the VENN methodology to assign the most appropriate grade for each competency, based on:

• HOW WELL (e.g. ... the pilot did not communicate effectively...)
• HOW OFTEN (e.g. ... by rarely demonstrating...)
• HOW MANY (e.g. ... any of the performance indicators when required...)
• OUTCOME (e.g. ... that resulted in an unsafe situation.)

Grades based on the word pictures are based on observations of fact. Word pictures for each of the 5 grades of all 9 competencies are specified in Pilot Performance Grades.

#When to Grade

There are 4 phases in the Recurrent Program:

1. The Evaluation Phase;
2. Maneuvers Validation;
3. Scenario Based Training; and
4. In Seat Instruction.

Grading is done at different times according to the phase.

#1. Evaluation Phase

You should rate and record performance in all 9 competencies at the end of the Evaluation. This is very important, because it is the "first look" at each pilot and crew, demonstrating as far as possible natural performance in a simple line orientated evaluation.

You may be asked to rate and record performance in certain competencies, at selected points during the phase. If required, the specific competencies to be rated will be determined in the curriculum and clearly indicated in the lesson plan. The purpose of this is to look at the fleet and airline performance. Grades assigned at these intermediate points should not directly influence the grades given to pilots at the end of the session.

For any grades of 1, 2 or 5, the applicable performance indicators should be recorded. When using the electronic grading form, if you do nothing, it will be assumed that performance was observed at level 3 and recorded accordingly.

#2. Maneuvers Validation Phase

You should rate and record performance at the end of each manoeuver, where any competencies are observed and rated at level 1, 2 or 5. For grades of 1, 2 or 5, the applicable performance indicators should be recorded.

When using the electronic grading form, if you do nothing, it will be assumed that performance was observed at level 3.

You should rate and record performance at the end of Day 1. This should be based primarily on your assessment of the Evaluation Phase, especially where all the Maneuvers Validation Phase has been rated at level 2 or above.

An overall completion grade for all competencies shall be assigned. For grades of 1, 2 or 5, the applicable PI's are to be recorded.

Repeats and retests may be applied during the maneuvers validation. For guidance on Repeats and Re-Tests, refer to 12.8 Operations Manual Part D.

#3. Scenario-based Training Phase

You may rate and record performance in all or (where indicated) preselected competencies at selected points during this phase. This will be clearly indicated in the lesson plan, and is for the purpose of looking at the fleet and airline performance. Grades assigned at these intermediate points should not directly influence the grades given to pilots at the end of the session.

#4. In-Seat Instruction Phase

Grading is not applicable to this phase, nor should any performance observed be considered for the overall rating for the session.

#Overall Grading

You should rate and record performance at the end of Day 2. This should be based on your overall assessment of the phase. As a reminder, your trainees have already completed the check elements of Day 1 successfully. The SBT phase is designed, deliberately, to be more demanding and to require your assistance at times to achieve the learning objectives.

Finally, you will need to select an overall outcome based on your rating of competencies for the session. This is the load-bearing summary categorisation:

• Where any competency is rated as level 1, an outcome of ADDITIONAL TRAINING REQUIRED should be recorded.
• Where all competencies are rated as 2 or above, the outcome should be COMPETENT.

In cases where there are consecutive grades of 2 in the same competency, training is required at a suitable time during the following 6 months. For precise details of the process required where you observe competencies at or below level 2, see the 12.8 Operations Manual Part D and training disruption procedure.

An outcome of ADDITIONAL TRAINING REQUIRED may only be considered when you observe performance at the level of 1 under conditions which do not place a high demand on the crew, where this performance is repeated at the same level on several occasions and you consider that learning is not being achieved. The combination of low demand, repetition, and stalled learning is what distinguishes ADDITIONAL TRAINING REQUIRED from a single bad day; an isolated level-1 under high demand may be a reactive failure, not a competency gap.

#The grading discipline as one continuous chain

The four sub-sections above (Observe-Record-Classify-Evaluate, VENN, per-phase timing, overall outcome) are stages of one continuous chain that turns observed crew behaviour into an operational consequence. The chain at a glance:

Each row consumes the previous row's output. Skipping a row breaks the chain: the operational consequence at the bottom only carries authority if every row above it was executed in order.

#Connections

• A4.2.1 Guidance for Examiners. The 14-step examiner sequence applies this methodology at Steps 3 and 11.
• A4.2.2 Guidance for Instructors. The 9-step instructor sequence applies this methodology at Step 6.
• A4.2.5 Conduct of Debriefing. The 9-step debrief sequence opens with the result this methodology produces (Step 2 of the debrief: state the COMPETENT vs ADDITIONAL TRAINING REQUIRED outcome).
• A4.D Core Competencies. The 9 competencies and their performance indicators that the Classify step maps observed behaviour against.
• A4.E Performance Grades. The 5-grade word-picture rubric the Evaluate step references for every competency cell.
• VENN grading. The HOW WELL / HOW OFTEN / HOW MANY / OUTCOME four-dimension lens used to assign the most appropriate grade.
• Core competencies. The 9-competency framework (8 ICAO core competencies + Appendix 4's KNO) the methodology rates against.
• 12.8 Operations Manual Part D. The Operations Manual Part D containing the repeats-and-retests guidance and the training disruption procedure.
• Behavioural indicators. The performance-indicator construct the Classify step uses as a basis for the competency mapping.
• 11.1 Purpose of Evaluation. The general evaluation-and-testing framework the EBT-specific grading methodology here operationalises.