A1.5 Five-Point Facilitator Rating Scale

Two jobs run together here. First, The C-A-L Model in Action is reproduced faithfully, including every worked dialogue contrast that shows how the C, A and L components actually run in a session. Second, the lineage from these qualitative criteria to the quantitative 5-level rubric in A3.1 Purpose and Directions is made explicit (the qualitative anchors are upstream FSF / NASA work; the Facilitation Assessment Tool quantifies them).

#The C-A-L Model in Action

The C-A-L model provides a way to structure the debriefing. The first section, CRM, suggests strategies for helping the crew focus on CRM techniques that played a role in the LOS. The second section, Analysis and Evaluation, shows how to guide crews to identify and evaluate aspects of their performance that went well, or could use improvement, including analysis of why the crew did what they did and why things turned out the way they did. The third section, Line Operations, provides a structure for helping the crew explore how they can apply what they learned from their analysis of the LOS to line operations.

A good way to organise debriefing each segment of the LOS is to show the appropriate video segment and then use the components of the C-A-L model to guide the discussion.

#C: CRM. Applying the company model

The major purpose of LOS is to give crews a chance to practise using CRM concepts and techniques in realistic flight scenarios. Typically, crew members are much better prepared to talk about the purely technical aspects of a flight situation than the CRM aspects, so they may need to be guided toward discussion of CRM issues pertinent to their performance in the LOS. Rather than discussing CRM as a set of abstractions, you can help crews the most by guiding them to consider how specific CRM techniques can be used to manage various flight situations.

#Focusing on CRM

You can use several techniques to bring CRM into the crew's discussion in a concrete, relevant way.

Refer to the posted CRM concepts

Each company teaches a specific framework for CRM concepts that reflects the company's philosophy. Posting this conceptual framework on wallboards and referring to it during the discussion can help the crew relate CRM concepts and techniques to specific operational situations.

Use CRM-specific questions

Get the crew to explore specific CRM issues and techniques that presented themselves during the LOS.

Use guiding questions

Guiding questions can be used to lead the crew to more specific and in-depth aspects of their LOS performance. In the following example the instructor encourages the crew to discuss specific aspects of teamwork that occurred during the LOS.

In the next example, the instructor facilitates crew discussion of specific aspects of workload management that occurred during the LOS.

#Reinforcing the utilisation of CRM through crew interaction

Instructors and crews too often fall into a pattern of discussion that centres entirely on the instructor: the instructor asks a question, a crew member responds, the instructor comments, and the cycle repeats. The crew will benefit in several ways, however, if you can get them to discuss their performance in the LOS directly with each other. Interactive discussion between crew members during the debriefing allows them to practise CRM skills such as communication and problem solving. It also leads them toward the ultimate goal of being able to debrief themselves in line operations. You can counter the crew members' natural tendency to direct their comments to you by using the following techniques.

Ask crew members to discuss how they were affected by each other's actions

It is important that crew members understand how their actions effect each other. By openly discussing these issues, crew members may become more aware of the impact of their actions and the importance of communicating what they are doing and why.

Ask crew members to discuss what they were each thinking

Encouraging crew members to openly articulate and discuss what they were thinking may help them understand each other's point of view and thereby enhance communication.

#A: Analysis and Evaluation of LOS Performance

For crew members to learn from their LOS experience, it is essential that they analyse and evaluate what happened. Crews should analyse both what went well during their LOS and what did not work as well. The analysis must go beyond simply naming the strong and weak points of their performance. Crews can gain powerful insight by analysing why things turned out the way they did, including factors that either enabled or hindered their success. Remember to refrain from giving your analysis until the crew has completed theirs.

#Getting crews to evaluate their performance

It is important for crews to learn to critically evaluate their own performance in the LOS so they can carry this skill over into line operations. As the crew discusses the LOS, you may find it helpful to use a wallboard to list "strengths" and "areas for improvement" as they are identified by the crew. The following techniques may be useful in encouraging crews to evaluate their performance in depth.

Get the crew to talk about what went well

Discussing what they did well helps the crew to recognise what strategies were effective in managing the LOS challenges and how these strategies might be used in line situations. This is also a good strategy to use when crews say everything went great during the LOS, so they can actively identify exactly what went well and why.

Get the crew to talk about what could be improved, and how

Discussing what did not go well helps the crew identify problems that occurred, examine why they occurred, and determine how to resolve or avoid similar problems in the future.

Troubleshooting: when the crew says everything went great

If, when asked to evaluate their performance, the crew members say everything went great, facilitation may be required to encourage the crew to analyse and evaluate in more depth. If everything did go well in the LOS, encourage the crew members to discuss specific instances of good performance and analyse why they went as well as they did. Also, encourage them to discuss how they could have handled situations if they had not gone so well. If there were, in fact, situations that were not handled effectively, draw the crew's attention to a specific situation and ask if there is another way it could have been handled. Regardless of whether or not everything went smoothly in the LOS, it is important that the crew members understand both the factors that led to their successes and the factors that led to weaker aspects of their performance.

#Eliciting deep analysis

To learn deeply from the LOS experience and take the lessons learned back to the line, crew members need insight into why events in the LOS turned out the way they did. Crew members can gain this insight through in-depth analysis of their LOS performance. You can help the crew analyse in depth by asking questions that require careful thought and detailed responses.

Ask questions that require description and analysis of LOS events

Open questions that require descriptive or analytical responses lead crew members to explore issues more thoroughly, which opens the door for deeper learning.

Get the crew to analyse why they made the decisions they made

Articulating why they did what they did helps crew members gain insight into their decision-making processes, as well as the factors that influenced, or should have influenced, their decision-making.

When you analyse for the crew it leaves little for them to say. It also gives them the message that you are teaching them so they are not expected to analyse for themselves.

Get crew members to discuss what they were thinking

Getting the crew to discuss what they were thinking during the LOS can help them discover what information and events influenced their actions.

Identifying underlying factors that enabled or impeded their success in the LOS can help the crew members recognise similar factors when they occur on the line.

#L: Line Operations. Applying Lessons from LOS

To help crew members transfer the lessons they learn in the LOS to the line, encourage them to discuss how LOS performance and associated CRM issues relate to effective line operations.

#Get the crew to discuss related line incidents

Getting the crew members to discuss actual line incidents and accidents related to CRM issues that arose during the LOS can help them to appreciate the role of CRM in line operations. For example, if a crew communicated well during the LOS which resulted in a positive outcome, the importance of their effective communication can be reinforced by having them discuss a line accident that resulted from poor communication.

#Get the crew to discuss how to apply their success to line operations

The next step in getting the crew members to transfer what they learn in the LOS to line operations is to get them to explore how they can apply techniques they utilised successfully in the LOS to overcome obstacles on the line. Getting crew members to talk about how they would handle difficult situations enables them to develop effective, pre-planned strategies for dealing with real situations when they occur on the line. For example, one crew's success in a particular LOS might have been aided by the fact that both crew members were good at explicit sharing of appropriate information. In the debriefing, you might find it useful to facilitate a discussion of how to deal with crew members on the line who are not as good at sharing information.

#Get the crew to discuss what they would do differently

Discussing what they would do differently enables crew members to develop strategies they can use to make more effective decisions and to avoid similar incidents in line operations. Ideally, crew members should discuss the strategies they can use to turn each item in their areas for improvement list into strengths. It is often useful to frame this discussion in terms of how the crew members can handle similar situations if they occur on the line.

#Get the crew to discuss how they will do things differently on the line based on their experience in the LOS

This final step requires crews to specifically tie what they have learned in the debriefing to the line. Having crews discuss how they will perform differently based on what they have learned can help them make the connection necessary to transfer thoughts into actions.

#From qualitative criteria to quantitative rubric

The Facilitation Assessment Tool turns the qualitative criteria above (and the Criteria for Effective Instructor Facilitation set reproduced in A1.2 Instruction vs Facilitation) into a 5-point quantitative rubric an instructor-evaluator can score with. The lineage is direct and worth tracing once here: upstream FSF / NASA qualitative anchors, quantitative scale in A3.1 Purpose and Directions.

#How the criteria became the 5-level rubric

Each Facilitation Assessment Tool rubric category has a clear antecedent in the qualitative criteria reproduced across this appendix. The mapping below is the lineage; the 100-cell rubric itself (5 categories x 4 markers x 5 levels) is reproduced verbatim in A3.1 Purpose and Directions.

#How the 5-level scale anchors what "good" looks like

The five levels of the Facilitation Assessment Tool rubric (typically 1 = ineffective, 5 = highly effective) are calibrated against the Criteria for Effective Instructor Facilitation reproduced in A1.2 Instruction vs Facilitation:

• Highest level (5): facilitation occurs at the highest level possible (the first criterion). The instructor refrains from interfering when the crew does not need guidance, ensures all critical topics are covered, and the crew analyses in depth, identifies CRM techniques and discusses with each other interactively.
• Intermediate (3-4): facilitation level is modified to match the crew's needs throughout the debriefing (the second criterion). The instructor uses intermediate strategies for guidance and reverts to low-level only if necessary.
• Lowest level (1): the instructor lectures or interrogates, does not draw the crew out, fails to ensure all critical topics are covered. The five "Avoid" patterns reproduced in A1.2 Instruction vs Facilitation are the failure-mode anchors.

The full descriptor text for each cell (5 categories x 4 markers x 5 levels = 100 cells) lives only in A3.1 Purpose and Directions. The role here is the lineage, not the rubric reproduction.

#Connections

• A1.4 Facilitation Techniques. The toolkit applied here.
• A1.6 Sample Question Forms and Closing Notes. The closing crib sheet that condenses the techniques across the appendix.
• A3.1 Purpose and Directions. The downstream quantitative rubric whose qualitative anchors are reproduced in this appendix.
• 7.3 General Debrief Techniques. Downstream chapter that summarises the C-A-L organising frame and the toolkit.
• C-A-L model. Synthesised concept; the walkthrough above is the appendix's reproduction of the model in action.
• CRM. The C component of the model; the focus of the Focusing on CRM subsection.
• Facilitation. The instructional technique whose qualitative anchors are reproduced here.