A3.5 Category 4 Focus on Crew Analysis and Evaluation
Category purpose. The goal of the debriefing session is to get the crew to evaluate and analyze their own CRM performance so they will learn more deeply and can gain practice in debriefing themselves, a skill they can then begin to use on the line.
This category is the analytical core of the rubric. Categories 1 to 3 measure facilitation discipline (framing, questioning, encouragement); this category measures the analytical product the discipline is supposed to enable. The four markers map directly onto the C-A-L discussion frame reproduced in A1.3 The C-A-L Debriefing Model: Markers 1 and 2 cover the "C" (CRM) and "A" (Analysis and evaluation) components, Marker 3 covers the "L" (Line operations) component, and Marker 4 measures the depth of analysis across all three.
Behavioural markers
The four markers for Category 4, scored on the 1-to-5 scale defined in A3.1 Purpose and Directions:
- CRM-dimension analysis. Encourages crew to analyze along CRM dimensions the situation that confronted them, what they did to manage the situation, and why they did it that way.
- Performance evaluation. Encourages crew to evaluate their CRM performance and/or ways they might improve.
- Line-operations transfer. Encourages crew to explore relevant CRM issues, how they specifically affected their performance in the LOFT session, and could affect their performance during line operations.
- Depth of analysis. Encourages crew to analyze issues, factors, and outcomes in depth, going beyond simply describing what happened and what they did.
The category also carries an "Overall rating of Focus on Crew Analysis and Evaluation" line, which is the total of the four marker scores.
Rating standards
The 20 rating-anchor descriptors for Category 4, reproduced verbatim. Rows are the four markers; columns are the five rating levels (Poor 1, Marginal 2, Adequate 3, Good 4, Very Good 5).
| Marker | Poor (1) | Marginal (2) | Adequate (3) | Good (4) | Very Good (5) |
|---|---|---|---|---|---|
| 1. CRM-dimension analysis | Does not encourages crew to analyze along CRM dimensions the situation that confronted them, what they did to manage the situation, or why they did it. | Only minimally encourages crew to analyze along CRM dimensions the situation that confronted them and/or what they did to manage it. Does not push crew to discuss why they did what they did. | On average encourages crew to analyze along CRM dimensions the situation that confronted them and what they did to manage the situation. Encourages but does not push crew to discuss why they did what they did. | Frequently encourages and pushes crew to analyze along CRM dimensions the situation that confronted them, what they did to manage the situation, and why they did it. | Continually encourages and pushes crew to analyze along CRM dimensions the situation that confronted them, what they did to manage the situation, and why they did it. |
| 2. Performance evaluation | Rarely encourages crew to evaluate their performance or ways they might improve. | Only occasionally encourages crew to evaluate their performance and/or ways they might improve. | Tends to encourage crew to evaluate their performance and/or ways they might improve, but may not pursue thoroughly. | Frequently encourages crew to evaluate their performance and/or ways they might improve. | Consistently encourages and pushes crew to evaluate their performance and/or ways they might improve. |
| 3. Line-operations transfer | Rarely encourages crew to explore CRM issues. | Occasionally encourages crew to explore CRM issues, and does not encourage crew to discuss how they affect LOFT performance or line operations. | On average encourages crew to explore CRM issues but tends not to get crew to discuss how they specifically affect both LOFT performance and line operations. | Frequently encourages crew to explore CRM issues and how they specifically affect LOFT performance and line operations. | Consistently encourages crew to explore CRM issues and how they specifically affect LOFT performance and line operations. |
| 4. Depth of analysis | Rarely encourages crew to analyze issues, factors, and outcomes in depth. | Only occasionally encourages crew to analyze issues, factors, and outcomes in depth. Content for crew to describe and what they did. | Generally encourages crew to analyze issues, factors, and outcomes, but settles for moderate depth, sometimes letting crew simply describe what happened and what they did. | Frequently encourages crew to analyze issues, factors, and outcomes in depth, going beyond simply describing what happened and what they did. | Continually encourages crew to analyze issues, factors, and outcomes in depth, going beyond simply describing what happened and what they did. |
How to read the marker progression
Markers 1 and 4 use "continually" rather than "consistently" at Very Good; the lexical shift is rubric-internal but appears to escalate above "consistently" toward unbroken sustained behaviour throughout the debrief. Markers 2 and 3 use "consistently" at Very Good. The distinction is operational rather than purely stylistic: an evaluator scoring Marker 1 or Marker 4 at Very Good is judging whether the instructor sustains the analytical push across the entire debrief without letting up.
The progression in Marker 1 introduces a "push" verb at Good and Very Good ("encourages and pushes"); at Adequate the descriptor reads "Encourages but does not push crew to discuss why they did what they did." The why-they-did-it dimension is the FSF / NASA discipline reproduced in A1.2 Instruction vs Facilitation ("crew members analyse in depth their performance, discussing the situations they confronted, what they did, and why they made the decisions and performed the actions they did"). The rubric scores the operationalisation of that criterion: at Adequate the instructor opens the why question but does not insist; at Good and Very Good the instructor ensures the why is worked.
Marker 4 (depth of analysis) operationalises the FSF / NASA "go beyond simply describing what happened" criterion. The Marginal descriptor's truncated phrase "Content for crew to describe and what they did" (sic, reproduced verbatim) is grammatically incomplete in the source; an evaluator should read it as "Content for crew to describe what happened and what they did" by parallel with the Adequate and Good descriptors.
Cross-reference to the C-A-L model
The four markers map to the C-A-L frame reproduced as Table 1 of A1.3 The C-A-L Debriefing Model:
| Rubric marker | C-A-L component | What it measures |
|---|---|---|
| Marker 1 (CRM-dimension analysis) | C (CRM, applying the company model) and A (Analysis of LOS performance) | Whether the instructor anchors the analysis in CRM dimensions and probes the why |
| Marker 2 (Performance evaluation) | A (Evaluation of LOS performance) | Whether the instructor pushes the crew through the evaluation half of the A-component |
| Marker 3 (Line-operations transfer) | L (Line operations, applying lessons from the LOS) | Whether the instructor extends the analysis from LOFT to line |
| Marker 4 (Depth of analysis) | All three components, applied to each topic | Whether the instructor sustains analytical depth rather than settling for description |
The mapping is exact for Markers 2 and 3; Markers 1 and 4 are cross-cutting and apply across all three C-A-L components.
Connections
- A3.1 Purpose and Directions. The 1-to-5 scale and the rubric's overall framing.
- A3.4 Category 3 Encouragement. The encouragement category, whose listening discipline is a precondition for the analytical product this category measures.
- A3.6 Category 5 Use of Videos. The video category, whose discipline supports the analytical product through evidence-based discussion of what actually happened.
- A1.3 The C-A-L Debriefing Model. The C-A-L frame the four markers map to.
- A1.2 Instruction vs Facilitation. The "Criteria for Effective Instructor Facilitation" upstream of these markers; the "what they did and why" criterion that Marker 1's why-dimension operationalises.
- 7.5 LOFT Debriefing - Introduction. The LOFT-debriefing treatment that this category's markers most directly score.
- C-A-L model. The discussion frame the four markers operationalise.
- CRM. The performance dimensions Markers 1, 2, and 3 explicitly invoke.
- LOFT. The training format the rubric is designed to score the debrief of.