7/24/2019 220 Cabin ESBs

1/15

Human actors and erospace Safety

4 3) 153-167

2004, Ashgate Publishing

FORMAL PAPERS

bin crew expected s fety

beh viours

Peter Simpson, Christina Owens* and Graham Edkins**

Cathay Pacific Airways Ltd, Hong Kong

* Qantas Airways Ltd, Australia

Victorian Department of Infrastructure, Australia

Abstract

Expected Safety Behaviours ESBs) or non-technical skills are becoming an

accepted aspect of pilot training, however, the concept has not been widely

applied to cabin crew. This situation is changing, as regulatory authorities start to

require the assessment of cabin crew CRM and safety behaviours. Qantas has

developed a set of expected safety behaviours for the training and assessment of

cabin crew. Unlike previous sets of behavioural markers, the cabin markers were

developed using a valid scientific methodology. The ESBs were derived from 80

interviews conducted with experienced cabin crew using the critical decision

method protocol, which uses probing questions to explore critical decision events.

Seven ESB categories were derived; situation awareness, passenger management,

crew management, workplace safety, operational understanding, negotiation and

influencing skills, and information and resource management.

Introduction

Crew Resource Management CRM) has been considered an integral aspect of

pilot training since the early 1980 s, and of cabin crew training since the early

1990 s. It is only in the last few years that CRM skills and competencies have

begun to be assessed as a regular part of pilot check and training. The concept of

CRM competency assessment may be relatively new to the cockpit, but it is even

7/24/2019 220 Cabin ESBs

2/15

154 Peter Simpson. Christina Owens and Graham Edkins

newer in the cabin. This paper describes the method that Qantas has used to

develop CRM competencies also known as non-technical skills or expected

safety behaviours - ESBs for cabin crew, and the process Qantas is developing

for their use and assessment.

Expected safety behaviours ESBs and CRM competencies

CRM and human factors training have been required by regulatory authorities

e.g., CASA, FAA, JAA as part of pilot licensing for several years. Cabin crew in

Australia do not require licensing, but most have been exposed to CRM and

human factors training via company training programs. However, until now, the

Australian Civil Aviation Safety Authority CASA had not required the

development or assessment of CRM competencies ESBs for cockpit or cabin

crew; Civil Aviation Safety Regulation 121A will require this. CRM is finally

starting to be recognised as being as important as technical skills for the safe and

efficient management of a flight.

Expected safety behaviours ESBs are generally thought of as observable, non

technical behaviours that contribute to effective performance within a specific

work environment. They are usually structured into categories, and the categories

are constructed of several elements or behaviours. A behavioural marker should

describe a specific, observable behaviour, not an attitude or personality trait.

These elements or behavioural markers can enable CRM performance

measurement and assessment, and can also be used to build performance

databases to identify norms and prioritise training needs, including the evaluation

of training Klampfer et aI., 2001 .

In 2001, Qantas started a two-year research project to develop ESBs for cabin

crew. Like other airlines, Qantas already had some experience in observing and

scoring pilot CRM for check and training purposes, although this was not jeopardy

assessment. Methodological guidelines for the development of behavioural

markers and CRM competencies are lacking. Programs such as NOTECHS van

Avermeate and Krujisen, 1998; Flin et aI., 2003 , LOSA HeImreich, Klinect and

Wilhelm, in press , and GIHRE Klampfer et aI., 2001 have developed their lists

of pilot CRM skills and behaviours from training workshops and meetings, and

they are based on opinion and experience, using a variety of informal methods and

techniques, such as amalgamating existing marker systems. A number of airlines

have developed their own behavioural markers systems based on these programs

Flin and Martin, 2001 , or using their methods and processes e.g., Qantas .

Furthermore, there does not appear to be any published research into the

development of behavioural markers for cabin crew, despite cabin crew CRM

training being mandated in many countries. Numerous investigation reports

reinforce the need for cabin crew to take appropriate action to manage threats and

errors for example, Air Ontario at Dryden, American Airlines AA132 at

Nashville, British Air Tours runway fire at Manchester, Qantas QFl at Bangkok,

7/24/2019 220 Cabin ESBs

3/15

Cabin crew expected safety behaviours

Singapore Airlines SQ6 at Taipei). Handling many of these threats and errors calls

for knowledge, skills and abilities quite different from those associated with the

provision of normal service duties, yet there is no consensus on which skills are

needed for effective cabin crew CRM (lCAO, 2002).

The cabin crew ESB project aimed to develop a valid and reliable method of

identifying and developing ESBs, based on established human factors and

behavioural science methods, the primary method being the Critical Decision

Method (CDM) (Klein, Calderwood and McGregor, 1989). The CDM is a

retrospective interview strategy that applies a set of cognitive probes (see table 1)

to non-routine incidents that requires expert judgement or decision making (Klein

et aI., 1989). A semi-structured interview format is used to probe different aspects

of the decision process and situation assessment. The CDM is a variant of

Flanagan s (1954) critical incident technique. The critical incident technique can

be applied for a variety of situations and uses in aviation, including: developing

critical requirements for evaluating the typical performance and safety behaviour

of an operator, evaluating operator proficiency in a check and training situation,

and providing an indication of the effectiveness of training programs (Flanagan,

1954). These are three areas that the ESB project will cover at Qantas, and all

three areas are useful in meeting the upcoming Australian regulatory

requirements.

The CDM is effective in revealing experts knowledge, especially tacit

knowledge, reasoning and decision strategies. Compared to other methods of

knowledge elicitation, the CDM yields more information, including a wider variety

of specific cognitive details, more information about underlying causal linkages

among core subjects, and the revelation of tacit knowledge (Hoffman, Shadbolt,

Burton and Klein, 1995). The reliability of the procedure is based on the idea that

experts have clear memories of salient or unusual safety-related incidents (Hoffman

et aI., 1995). This method of knowledge elicitation has been used successfully in

naturalistic environments such as fire fighting, para-medicine, nursing, helicopter

flying, and military command and control (see Simpson, 2001 for a review). The

CDM is now established as a valid and reliable method of cognitive task analysis

and knowledge elicitation (Hoffman, Crandall and Shadbolt, 1998; Taynor,

Crandall and Wiggins, 1987). However, the limitations of this method are discussed

by O Hare, Wiggins, Batt and Morrison (1994), and Hamm (2004).

etho for eveloping the ESBs

Participants

The participants for the critical decision interviews consisted of eighty Customer

Service Managers (CSMs) (known as Pursers or Cabin Supervisors in some

airlines). The CSMs were a mixture of short haul (54 ) and long haul (46 ) from

7/24/2019 220 Cabin ESBs

4/15

156 Peter Simpsoll. Christilla Owells alld Graham Edkills

various bases across Australia. They had spent an average of 7.5 years operating

as a CSM range = 1-25yrs, sd = 6.2yrs), and their mean age was 42 years sd =

7.0yrs).

The criteria for participation were that the person must be a CSM, and that they

could discuss, in detail, a recent within 18 months) safety-related event that was

challenging. CSMs were paid for the ground duty time they spent being interviewed

usually 1.5 - 2 hours). Six people, who were also CSMs, conducted the interviews.

Although their main role was interviewing, these six team members also helped

with data analysis and development of the safety behaviours because they are

subject matter experts, with the domain knowledge area of cabin safety.

Desigll

The variables of home base location Sydney, Melbourne, Brisbane, Perth) and

operation type short or long haul) were controlled to ensure the CSMs

proportionally represented the Qantas operation. The interview process was based

on the Critical Decision Method CDM) described in the Introduction.

Procedure stage

critical decisioll method CDM The procedure for

employing CDM is well documented Klein et aI., 1989; Hoffman et aI., 1998),

but the basic steps used in the interviewing process included:

1. Incident selection - CS Ms select a recent, non-routine incident that was

challenging;

2. The interviewer obtains an unstructured free recall of the event;

3. Both participant and the interviewer establishes the sequence of decision

events and constructs a time line;

4. Decision point identification - the interviewer identifies specific decisions

that were made;

5. Decision point probing - probing/questioning techniques are used to identify

effective decisions and resultant behaviours see Table 1 for examples of the

probes);

6. Hypotheticals and what ifs? The interviewer chooses several decision

points, and asks hypothetical questions based upon different event outcomes

eg., what would you do if the Captain didn t take your request seriously?).

These queries serve to identify potential errors, alternative decision-action

paths, and expert/novice differences Hoffman et aI., 1998); and

7. Standard case study - CSMs are provided with a standard case studyl and

repeat steps 3 to 6 as if they were the CSM onboard that aircraft.

The standard case study was a real Qantas incident on-board a B737. It involved smpke and

fumes in the cabin. with many CRM issues and problems.

7/24/2019 220 Cabin ESBs

5/15

Cabin crew expected safety behaviours 157

Table 1 A sample of CDM probe questions adapted from Hoffman et aI.,

1998

Probe type

Cues

Knowledge

Analogues

Standard

scenarios

Goals

Options

Mental

models

Experience

Decision

making

Aiding

Errors

Probe content

What were you seeing, hearing, smelling?

What info did you use in decision making and how was it

obtained?

Were you reminded of any previous experience?

Does this case reflect a typical scenario or a scenario you

were trained in?

What were your specific goals and objectives at the time?

What other courses of action were considered or were

available?

Did you imagine the possible consequences of your action?

What specific training, experience or knowledge was

necessary?

Was there time pressure? How long did it take to make the

decision?

What training, knowledge or experience could have helped

you?

What mistakes are common at this point? How might a

novice act?

Participants were informed that the interviews were anonymous. Interviews

were conducted in a quiet office and tape-recorded if permission was granted .

Most interviews lasted for 1.5-2 hours; requiring at least one hour for the first

critical incident, and half to one hour for the repeated case-study incident.

Verbatim transcripts were made from the tapes.

Procedure stage

2 -

development of expected safety behaviours ESBs

The

second stage of the study involved the coding of the interviews and development

of the ESBs. The procedure consisted of a number of steps:

1. Initial review and coding of the transcripts to develop behavioural markers.

This was conducted by the research team in a workshop situation. Ten

interviews were used to develop the initial ESB list. This critical step is

7/24/2019 220 Cabin ESBs

6/15

58 Peter Simpson Christina Owens and Graham Edkins

2. Improve code structure and markers with feedback from cabin crew subject

matter experts the six CSMs within the project team).

3. Test the improved ESB list by coding a further 10 interviews for refinement.

The aim of this step was to determine whether the list of ESBs was able to

cover a multitude of cabin situations that required both technical and non

technical skills.

4. Analysis of behavioural markers by general CSM population, including

written feedback requested from the entire CSM population to gain general

comment and feedback on the list of ESBs, and brief interviews 20 mins)

with random CSMs n=20) in crew lounges to attain more detailed

responses. Both methods included rating the relative importance of each

behavioural marker to the safe, efficient conduct of the flight.

5. Produce a master list of ESBs.

6. Code remainder of interviews approximately 60).

7. Attain a frequency count of behavioural marker elements occurring in each

critical incident.

8. Construct a training focus matrix frequency ESB occurring versus

perceived importance ESB) for the training of expected safety behaviours

see figure I). This is based upon the results from steps 4 and 7 and allows

objective prioritisation of training.

High

I

M

P

o

R

T

N

C

E

Low

Low

Priority

High FREQUEN Y Low

Figure 1 Expected safety behaviour training focus matrix

Achieving a high degree of inter-rater reliability is one of the challenges of

coding transcripts. To avoid this problem completely, once this final ESB coding

structure was developed, a single person coded all the remaining interviews.

7/24/2019 220 Cabin ESBs

7/15

7/24/2019 220 Cabin ESBs

8/15

16 Peter Simpson Christina Dwens and Graham Edkins

Table Expected safety behaviour categories and elements

Expected safety

behaviour category

Situation

awareness

Passenger

management

Workplace safety

Crew

management

Operational

understanding

Negotiation and

influencing skills

Information and

resource

management

Expected safety behaviour element behaviour

Demonstrates awareness of flight phase or situation

Considers political and cultural context

Considers time constraint

Recognises higher safety goals and priorities

Anticipates decision consequences

Develops contingency plans

Assesses passengers boarding or in-flight

Monitors potentially threatening pax behaviour/condition

Acts decisively to modifY passenger

behaviours/condition

Considers passengers well-being

Presents a calm, controlled image to passengers

Diffuses situation in a non-confronting manner

Minimises cabin disruption

Pro actively manages OHandS situations

Reactively manages OHandS situations

Follows-up OHandS situations

Communicates OH andS importance

Displays role model OHandS behaviours

Assesses crew

Provides onboard coaching and training to modifY behaviour

Considers crew well-being

Considers impact of non-routine events on crew performance

Allows and provides crew debrief

Demonstrates Basic Aeronautical Knowledge BAK

Understands authority/duty of CSM

Understands authority/duty of others

Consults with others to develop a common strategy

Manages upwards 1 - identifies problem

Manages upwards 2 - expresses concern

Manages upwards 3 - provides options

Manages upwards 4 - uses emergency language

Identifies and utilises all resources

Gathers information

Confirms common understanding of information

Critically analyses information

Provides timely feedback to those who need to know

Prioritises tasks

7/24/2019 220 Cabin ESBs

9/15

Cabin crew expected safety behaviours

As a learning point from the study, other airlines would be advised to carefully

select their interviewers. Although senior cabin supervisors are experienced in

cabin safety, they are not necessarily experienced in interviewing. Several

interviews were not as detailed as they could have been, due to a focus on physical

events rather than decisions, and many were war stories rather than cognitive

analysis. However, all the interviews were useful and have proven highly valuable

as examples and support for training development.

The frequency of ESB elements was summed for the 80 critical incidents,

yielding over 1,500 discrete occurrences of the elements. This allowed the

determination of the safety behaviours most frequently used by CSMs in normal

operational events and incidents. Figure 3 shows a selection of the frequency

results. The category of Information and Resource Management was the most

frequently occurring safety behaviour. It is thought that the high frequency of this

behaviour is unique to CSMs, because they are the cabin manager, and are

responsible for the control of information and resources within the cabin, and

between the cockpit and cabin. The CSM is also the cabin member most often

making decisions and solving problems for all the cabin crew.

The second most frequently displayed safety behaviour was the category of

Situation Awareness. Again, as the cabin manager, it is not surprising that the

CSMs behaviour and decisions were very closely related with aspects of being

situationally aware. All three levels or stages of situation awareness - perception,

comprehension, and projection Endsley, 1995) - were represented in the safety

behaviour elements. Projection was evident when the CSMs determined the

consequences of their decisions before implementing them, and their development

of contingency plans in case situations and circumstances changed.

With the frequency count of the ESBs, and the rating of perceived importance by

the CSM population, Qantas now has information on what the most commonly

displayed cabin safety behaviours were, as well as which were perceived to be the

most important. It is interesting to note that the most frequent category, Information

and Resource Management, was considered to be the least important category.

Situation Awareness was ranked highly both in frequency and perceived importance.

This indicates that what crew perceive to be important safety behaviours, may not

reflect reality. This may have an impact on the validity of behavioural marker lists

derived from workshop and round-table discussions that are based on experience

and perception, rather than valid and reliable scientific methodology.

pplication of the ES s

Use in training

Qantas now has 80 real, recent, and highly detailed safety events that can be used

as case studies and examples. They all detail actual threats encountered and errors

7/24/2019 220 Cabin ESBs

10/15

162 Peter SimpSOll Clzristilla QwellS alld Gralzam Edkins

made on Qantas aircraft, and most are very good examples of threat and error

management. This is an excellent resource the company can use for many years.

CREW debrief

CREW

impact

CREW wellbe.;ng

F: ~ ;'

REW coaching ,

CREW

assess ,- . ,

PAXdisruption

PAXdiffuse

f.:.:.:~

PAXcalm f .-~

PAX wellbe;ng ~

::~~~:~:+~--::

:~t~:~::

,:::t;:;:::.J.

~~~i ~;:~:~~

l::: E:d

~=:~

SA time, ~:=-~ ~:.',m'-~.

A political ~ .,~:. . ~_

I

i~~~~~el

r--'''' ,. .,..-=,~=. .~-. .....,. ,..=~.

_.= ' '~'-~'=====~

:~~~~~o~~:~k

~::~':

:,:~.:.,:::::.~.-~,.._-=.J

INFO

analyse

INFO common

INFO gather

INFO

resources

I

;::

I

.~

.,.~ '. '

o

20 40

8

100 120 140

frequency

Figure 3 Frequency count of selected expected safety behaviours

The ESBs are currently being used in cabin crew training as part of the annual

CRM Emergency Procedures refresher training (a two-day course that all cabin

7/24/2019 220 Cabin ESBs

11/15

Cabin crew expected safety behaviours 6

crew undergo annually). During a classroom session, cabin crew listened to three

of the original incident summaries that had been recorded onto audio file. Each

was a few minutes long, and was recorded in the first person, as if the voice on the

recording was that of the CSM recalling his/her own safety event. At the end of

each incident summary, the cabin crew were asked to assess a specific category of

CRM from the ESB list e.g., assess the passenger management competencies ).

A mock-up CRM assessment form was used along with the ESBs in table I. The

purpose of this exercise was to introduce cabin crew to the concept of CRM

assessment, and to give them some experience of the forms and categories that

may be used to train and assess them in the future. Unlike pilots, CRM assessment

and CRM competencies are a new concept for cabin crew.

Currently the ESBs are not being used for assessment purposes in the cabin.

Although, this situation will change in the next few years. It is expected that

initially, the ESBs will be used for training and checking, but not necessarily for

jeopardy assessment. This type of assessment will not occur until a level of

acceptance amongst crew and checker/trainer skill has been attained in assessing

CRM competencies.

Development of ESB application guidelines

Broad guidelines for the appropriate application and use of ESBs and their

assessment are being developed across the Qantas group i.e., international,

domestic and regional operations) for both cockpit and cabin crew. These

guidelines are based on the NOTECHS guidelines van Avermaete and Krujisen,

1998) and have been adapted to suit the operational and regulatory requirements

faced by Qantas. The guidelines have been developed in conjunction with

stakeholders mainly Human Factors representatives) from the various flight and

cabin operations areas of the Qantas group. The guidelines describe such issues as;

1. Introduction and definitions of ESB e.g., categories, elements, behaviours).

2. Philosophy behind the use of the ESBs e.g., to train, reinforce and support

the skills and behaviours considered necessary to provide safe, effective

outcomes ).

3. Preferred methods of assessment e.g., Ratings should be done at the

category level. An overall rating is given for the category, rather than for

individual components, but that category rating is based upon the

performance outcomes of the elementslbehaviours. ).

4. Process for failed competencies e.g., Ineffective behaviour should be

observed repeatedly to conclude that it is ineffective. It is not the goal to fail

someone for failing to discuss a decision on a single occasion ... ).

5. Recommended evaluator training and qualifications e.g., Complete initial

training on ESB system, Formal assessment as competent and calibrated

following ESB training in classroom, Calibration in operational

environment, and Periodic re-calibration ).

7/24/2019 220 Cabin ESBs

12/15

64 Peter Simpson Christina Owens and Graham Edkins

Cabin crew LOSA

While behavioural marker systems can provide a focus for training goals and

needs, they are limited in that they cannot capture every aspect of normal

performance because of the infrequent occurrence of some behaviours e.g.,

sporadic behaviours such as aggression , especially during an assessment

situation, such as a check flight. Hence, the University of Texas has developed a

method of normal operations, non-jeopardy, Line Operations Safety Audits

LOSA in the cockpit. LOSA enable the collection of data on threats and errors

and their management Helmreich, Klinect and Wilhelm, in press .

LOSA utilises trained observers to collect data about flight crew behaviour on

normal flights under non-jeopardy conditions. Observers record information based

on three worksheets. The first worksheet records a description of external threats

e.g., weather or ATC that may influence crew performance and how these

threats are managed. The second worksheet describes the errors made by the flight

crew and what strategies were used to detect and recover from these errors. The

final worksheet records the behavioural markers based on three categories;

Planning, Execution and Review/Modify Plans. The behavioural markers are used

repeatedly for every flight phase and rated on a four-point scale poor

outstanding Helmreich, Klinect and Wilhelm, in press .

LOSA is currently used only in cockpit situations. In the longer term, the cabin

crew ESBs can be used in a similar manner to the way in which they are used for

cockpit crew; the evaluation of CRM and non-technical skills in training and

normal operations. This could feedback into the training system, allowing the

refinement and development of ESBs and the training program. The greatest

challenge is moving the evaluation and observation of safety behaviours out of the

training environment and into normal line operations, in the form of a cabin crew

LOSA program. There are many issues and problems to overcome before in-flight

cabin observations can take place. Such problems include:

Cabin environment is not as contained as a cockpit.

Double deck aircraft B747 and A380 .

All information goes through CSM.

Observers are more obtrusive in cabin.

Errors tend to be less consequential in the cabin.

Impact on customers and service.

Multiple crew to observe, who cannot all be observed at once.

Less external threats to inflight safety in the cabin.

Check and training for technical skills has been an accepted practice for

decades. Further, CRM and non-technical skills audit and evaluation is gaining

acceptance in the cockpit, and a cockpit LOSA program is running at Qantas. As

yet, no airline has committed to a LOSA-style program for cabin crew. Qantas is

considering incorporating LOSA style threat and error management within its

7/24/2019 220 Cabin ESBs

13/15

Cabin crew expected safety behaviours 165

inflight cabin observation program, which is currently being developed and due

for implementation mid-2004. The observation protocol has been developed in

line with general LOSA principles, as far as possible taking into account the issues

above. The protocol is currently under review, and the final version should be

available January 2004. To our knowledge, the proposed Qantas project is the first

attempt to apply this program within the cabin environment.

ummary

As far as is known, the Qantas Cabin Crew ESB study is the first attempt to

develop behavioural markers in aviation using a robust, scientific method. The

CDM was a practical method for this purpose, as has been demonstrated in other

naturalistic environments. It is hoped that other authors will use similar, formal

methods for developing behavioural markers, rather than using the traditional,

informal process of workshops and round-table discussions. The behavioural

markers developed from these informal processes are based on experience,

opinion and perception, and as this current study indicated, the behaviours that

experienced operators perceive to be important to safety may not reflect

operational reality. Hence, any of the current lists of safety behaviours and

behavioural markers being used in aviation and other industries require scientific

validation.

The study has also provided a useful and extensive database of positive

outcome safety scenarios for use in cabin crew training. The case studies are

specific to the operating environments of Qantas, and are a good resource for the

current CRM focus of threat and error management. For these reasons, it is highly

recommended that other airlines implement a similar program.

The LOSA program has proven successful in the cockpit, and its use in the

cabin and other environments e.g., ATC appears to be a good concept in theory,

yet logistical issues and other challenges continue to impede its implementation.

No conclusion can be drawn from this current study regarding the use of the cabin

crew LOSA. However, some form of non-jeopardy, normal operations observation

is required to further understand the nature of threat and error management, and

effective safety behaviours in the cabin. Implementing such a program is now a

major challenge facing the aviation industry.

eferences

van Amermaete, J. and Krujisen, E. Eds 1998 . NOTECHS: The evaluation of

non technical skills ofmulti pilot aircrew in relation to JAR FCL requirements

National Aerospace Laboratory NLR.

7/24/2019 220 Cabin ESBs

14/15

166 Peter Simpson. Christina Owens and Graham Edkins

Endsley, M. 1995). Towards a theory of SA in dynamic systems.

Human Factors,

37, 32-64.

Flanagan, J. 1954). The critical incident technique.

Psychological Bulletin, 51,

327-358.

Flin, R. and Martin, L. 2001). Behavioural markers for Crew Resource

Management: A survey of current practice.

International Journal of Aviation

Psychology, 11,95-118.

Flin, R., Martin, L., Goeters, K., Hormann, H., Amalberti, R., Valot, C. and

Nijhuis, H. 2003). Development of the NOTECHS non-technical skills) for

assessing pilots CRM skills.

Human Factors and Aerospace Safety,

3,97-119.

Hamm, R. 2004). Protocol analysis: The emperor needs a shave.

Medical

Decision Making,

24, 681-686.

Helmreich, R., Klinect, J. and Wilhelm, J. in press). System safety and threat and

error management: The Line Operations Safety Audit LOSA). In,

Proceedings

of the Eleventh International Symposium in Aviation Psychology.

Columbus,

OH: The Ohio State University.

Hoffman, R., Crandall, B. and Shadbolt, N. 1998). Use of the CDM to elicit

expert knowledge: A case study in the methodology of cognitive task analysis.

Human Factors, 40,254-276.

Hoffman, R., Shadbolt, N., Burton, A. and Klein, G. 1995). Eliciting knowledge

from experts: A methodological analysis. Organisational Behaviour and

Human Decision Processes.

62,129-158.

International Civil Aviation Organisation ICAO). 2002).

Draft Human factors

digest: Humanfactors in cabin safety.

Montreal,9anada.

Klampfer, et al. 2001). Behavioural markers workshop. Swissair Training Centre,

Zurich, 5-6 July.

Klein, G., Calderwood, R. and McGregor, D. 1989). Critical decision method for

eliciting knowledge.

IEEE Transactions on Systems, Man and Cybernetics, 19,

462-467.

O Hare, D., Wiggins, M., Batt, R. and Morrison, D. 1994) Cognitive failure

analysis for aircraft accident investigation. Ergonomics. 37, 1855-1869.

Qantas 2002).

Safety analysis paper: Cabin safety incidents

causal factors SAP

06/1 .

Sydney, Australia: Qantas, Corporate Safety Dept.

Simpson, P. 2001).

Naturalistic decision making in aviation environments.

DSTO-GD-0279. Melbourne, Australia: DSTO, AMRL.

Taynor, J., Crandall, B. and Wiggins, S. 1987). The reliability of the critical

decision method. KA-TR-863 b -8707F.

Yellow Springs, OH: Klein Associates.

7/24/2019 220 Cabin ESBs

15/15

Cabin crew expected safety behaviours 7

ppen ix

A list of safety behaviours and non-technical skills could be very lengthy.

Developing the list is a compromise between having a practical, useable list for

operational purposes training, assessing, checking, auditing, etc) and having a

long, exhaustive list to cover every behaviour and situation. The goal was to

account for the majority of safety-related behaviours, not every individual

behaviour.

Despite the body of literature on verbal protocol analysis and CDM, there is

very little information explaining the practical aspects of coding the transcripts.

The procedure developed for this project used a team of two psychologists, a

human factors specialist and a cabin safety specialist to develop the ESB codes.

The process was:

1. Ten interviews were selected that covered ten different situations.

2. All team members read each transcript, focussing on the decision points.

The actions and behaviours surrounding the decisions were noted, and an

attempt made to attach a recognised human factors / psychological term to

describe the behaviour.

3. After each transcript was worked on in isolation, the team met to discuss

their ideas and findings. As a group, they developed relevant descriptors for

the behavioural elements.

4. This process of coding in isolation then meeting together as a team occurred

for each interview transcript until the initial 10 transcripts were complete.

5. Diminishing returns occurred, where the initial interviews yielded the

largest array of behaviours and elements, and by the tenth transcript, the

codes were being refined rather than new behaviours being uncovered.