|

Calibration Process

Lou CockerhamEducation Specialist

Call Calibration

• A process designed to ensure consistent evaluation of same call across two or more people

• A tool for reducing the number of quality evaluation challenges/disputes within your organization

• Maintains the internal integrity of the quality evaluation process

Benefits

• Agent– Fairness, regardless of who completes evaluation– Consistency in scoring, especially when QA assignments rotate– Developmental tool

• Supervisor– Ease of coaching scored evaluations– Shows consistency in scoring– Measures everyone in same fashion

• Manager– Maintains integrity of QA department– Reinforces quality culture– Identification of knowledge gaps

Before You Calibrate

• Before starting calibration process, check – Is your Quality Evaluation Form in place?– QSDD (Quality Standards Definition

Document)?– Training on QA form and QSDD for anyone

involved in evaluation process?

Calibration Process

• Decision Points– How often should you calibrate?– Who should facilitate sessions?– Who owns calibration process?– How should calls be selected for calibration?– Who owns change process for materials needing

updates due to calibration – QSDD and QA Form?– How measure effectiveness of calibrating?

How Often Calibrate?

• Recommendations– Weekly at initial rollout– Bi-weekly once Standard Deviation goal is met

• Factors that influence change in calibration frequency– Changes in business– Updates to QA form– Updates to QSDD– New staff– PTO

Who Facilitates Sessions?

• Facilitators are not responsible for decisions• Facilitators are responsible for enabling process• Facilitators handle

– Logistics (where, when, and for how long you are meeting)– Booking rooms– Setting up technology– Sending out the e-mail with the record IDs to be calibrated on

• Facilitators keep group on track and on target• Facilitators have leadership and facilitation skills• Facilitator role may be filled on a rotating basis

Who Owns Calibration Process?

• Calibration process owner needs– Project management skill– Delegation skill– Authority to enforce session attendance– Communication skill to report results and

issues

How Select Calls to Calibrate?

• Calls selected for calibration may be– Nominated– Randomly selected

• Nominated calls must have Record ID and justification e-mailed to facilitator prior to session

• Calls randomly selected should use multi-step filter, randomizer option, with other selection criteria, to generate batch of calls as basis for calibration

Who Owns Change Process?

• Will you need management approval to make changes?• How will updates be communicated to team?

– Agents– Supervisors– Managers

• Who will update QSDD and QA forms?• Where does the ‘buck’ stop?

How Measure Effectiveness?

• Set your standard deviation goal• Compile results of sessions• Track progress toward goal• Present results during sessions and in

management meetings• Adjust standard deviation goal as appropriate

Calibration Participants

• Who should be included?– QA staff– Supervisors (staff representative if supervisors not scoring)– Management– Agents

• Pros of agents in sessions– Inputs from front line– Reduces “Us” versus “Them” feeling– Personal development

• Cons of agents in sessions– Potential confidentiality issues– Takes agents away from working calls

Calibration Recommendations

• Session length should be one hour• Group size: four minimum and 15 maximum

participants• If agents participating, get their permission share their

calls• Calibrate departments separately• Calibrate different locations separately frequently and

collectively quarterly

Calculating Standard Deviation

• Steps1. Calculate average of all evaluation scores2. Subtract average from each score – some will be negative3. Square each number and add them together4. Divide total by number of evaluations5. Take square root of that number, which is your Standard

Deviation

15

• Let Excel help– Enter evaluation

score– Enter formula =

STDEVP

Calculating Standard Deviation

Recommended Standard Deviation

• SD = 5– Great starting point if center is new to QA

• SD = 3– Most contact centers

• SD = 2– Contact centers considered very well calibrated

Before Conducting Sessions

• Create duplicate of your live scoring form in Discover, name it CALIBRATION – use version numbering scheme

• Select calls for calibration and send ID numbers to participants two days prior to session

• Participants should score selected calls on CALIBRATION forms prior to session

Conducting Calibration Sessions

• Establish ground rules• Maintain confidentiality – especially important

to enforce if agents are included in the process• Listen without reacting• Focus on how it should be, not on being right• Allow others a chance to have their say

Conducting Calibration Sessions

• Review calibration goals– Review results from prior session– Review progress toward SD goal

Conducting Calibration Sessions

• Replay first call– Share no reactions—physical or verbal– Score on paper in actual session

• Ok to have differences in paper and digital forms

– Identify differences between paper and digital versions

Conducting Calibration Sessions

• Gather feedback from group– Positive– Areas for improvement– Ideal situation

• Facilitator must keep feedback moving and focused– Ensure everyone expresses their views

Conducting Calibration Sessions

• Review QSDD–Make sure definitions are accurate and

describe how to best evaluate–Make note of any potential changes or

additions• Who is responsible for making these

changes?

Conducting Calibration Sessions

• Repeat for the next call(s)– Replay call– Review scores– Provide feedback– Validate definitions

• Process drives group’s calibration and success in reaching SD goal

Conducting Calibration Sessions

• Leader uses Discover’s reporting tools to explore and validate calibration group’s experiences

Conducting Calibration Sessions

Conducting Calibration Sessions

• Recap and close– Identify action items and their owners– Communicate and apply updates to

process– Schedule next session– Assign dedicated facilitator as necessary