results from the performance management survey
Post on 06-Jan-2016
26 Views
Preview:
DESCRIPTION
TRANSCRIPT
1
Results from the Performance Management Survey
Prepared by Janice Rouiller and Laura Stouffer, SAICand
Joe Wolski, OQM
9 May 2005
2
Table of Contents• Survey Background……….…………………………………………………………...3
• Respondent Information…………………………………………………………….…7
• Perceptions of Cycle 3 PM Implementation………………………………………..15
• Perceptions of Needs for Cycle 4…………………………………………………...52
• Organizational Climate Perceptions………………………………………………...68
• Organizational Climate Perceptions by Program Area……………………………78
• Organizational Climate Perceptions by Role………………………………………84
• Customer Scorecard Ratings………………………………………………………..88
• Summary of Comments..…………………………………………………………...101
• Summary………..……………………………..………..…………………………...105
• Appendix - Individual Comments from Survey…………………………….……..111
3
Survey Background
4
Purpose
• Administer a customized version of the ORS Customer Scorecard to:• Gather feedback on past Performance
Management (PM) implementation efforts• Collect needs assessment information to tailor the
on-going implementation of PM• Gather data on the required Customer Scorecard
dimensions• Collect organizational culture change information
for the Evaluation Study
5
Methodology
• Administer the survey at the completion of the third cycle of Performance Management implementation on an organization-wide basis
• Which occurred in February 2005
• Designed and tested web survey during February 2005
• Survey went “live” on 3 February, 2005• Email requests sent to all PM Team Leaders, Members, and
additional senior ORS/ORF managers• First reminder email sent 17 February 2005• Closed survey field period 24 February 2005
• Data file sent to contractor for analysis
6
Survey Distribution
Number of email recipients 186
Number of respondents 41
Response Rate 22%
7
Respondent Information
8
Respondents by Role
311
28
75
0
5
10
15
20
25
30
35
40
45
50
Office/Associate/Division Director
PM Team Leader PM Team Member PM Consultant ManagementCouncil
Other
N = 45
Note: Multiple responses allowed.
Nu
mb
er
of
Resp
on
den
ts
9
Respondents by Organization
30
47
0
5
10
15
20
25
30
35
40
45
50
55
60
ORS ORF Other
N = 41
Nu
mb
er
of
Resp
on
den
ts
10
Respondents by Program Area
7
4
88
10
4
0
3
6
9
12
15
Mgmt Services Prgm & Empl Svcs Security & EmergResponse
Scientific Resources Real Estate &Facilities
Not Part of theServices Hierarchy
N =41
Nu
mb
er
of
Resp
on
den
ts
11
Experience with the PM Process
49%
69%
58%
63%
100%
0% 20% 40% 60% 80% 100%
Part of ASA Team in FY02
Part of ASA Team in FY03
Part of a PMP Team in FY04
Attended FY04 Managing with MeasuresTraining
Attended FY04 PM Using BSC ApproachTraining
Percent “Yes”
N = 41
N = 40
N = 40
N = 36
N = 35
12
Approximately what percent of time did you spend on PM-related activities during FY04?
2
01
0 0
2
55
8
18
0
2
4
6
8
10
12
14
16
18
20
0-10% 11-20% 21-30% 31-40% 41-50% 51-60% 61-70% 71-80% 81-90% 91-100%
N = 41
Percent of time Spent
Nu
mb
er
of
Resp
on
den
ts
13
On average, how often did your PM team meet during FY04?
4
8
17
9
3
00
5
10
15
20
25
30
Several Timesa Week
Once a Week Biweekly Once a Month Once a Quarter Less thanOnce a Quarter
N = 41
How Often PM Team Met
Nu
mb
er
of
Resp
on
den
ts
14
0
1 1
0
7
8
7
3 3
4
5
2
0
3
6
9
12
15
1 2 3 4 5 6 7 8 9 10 DK NA
Fre
qu
ency
of
Res
po
nse
To what extent did your team members work between meetings to implement PM?
To No Extent
To a Great Extent
N = 41
Mean = 6.71
Median = 6.5
15
Perceptions of Cycle 3 PM Implementation(January 2004 - January 2005)
16
How helpful did you find the following in implementing PM?
4.78
4.96
5.38
5.44
5.46
5.74
5.86
5.88
6
6.06
6.18
6.25
6.29
6.38
6.45
6.52
7.15
7.82
7.97
1 2 3 4 5 6 7 8 9 10
PM Website
Other PM Teams
Design of Data Collection
PM Conference
Services Hierarchy
PM Related Emails
Process Behavior
Measures Reporting Form
Data Collection
Graphing Data
Measures Roadmap
Initiative Roadmap Template
Strategy Mapping
OQM Staff
ORS Customer Scorecards
PM Presentation Template
PM Template
Your Supervisor
PM Consultants
Not at all Helpful
Extremely
HelpfulMean Ratings
17
How helpful did you find the following tools in implementing PM?
6.52
5.88
6.25
6.18
6.29
7.15
5.46
1 2 3 4 5 6 7 8 9 10
PM PresentationTemplate
Measures ReportingForm
Initiative RoadmapTemplate
Measures Roadmap
Strategy Mapping
PMP Template
Listings of theServices Hierarchy
N = 41
Not at all Helpful
Extremely
HelpfulMean Ratings
N = 41
N = 41
N = 41
N = 38
N = 40
N = 41
18
5
2 2 2
5
6
4
5
1
3
4
2
0
3
6
9
12
15
1 2 3 4 5 6 7 8 9 10 DK NA
Fre
qu
ency
of
Res
po
nse
How helpful did you find the following tools in implementing PM?
Listings of the Services Hierarchy
Not at all Helpful
Extremely
Helpful
N = 41
Mean = 5.46
Median = 6
19
2
0
3
0
2
3
6
15
2
6
2
00
3
6
9
12
15
1 2 3 4 5 6 7 8 9 10 DK NA
Fre
qu
ency
of
Res
po
nse
How helpful did you find the following tools in implementing PM?
Performance Management Plan (PMP) Template
Not at all Helpful
Extremely
Helpful
N = 41
Mean = 7.15
Median = 8
20
4
1
2
0
3
4
6
11
1
3
5
00
3
6
9
12
15
1 2 3 4 5 6 7 8 9 10 DK NA
Fre
qu
ency
of
Res
po
nse
How helpful did you find the following tools in implementing PM?
Strategy Mapping
Not at all Helpful
Extremely
Helpful
N =40
Mean = 6.29
Median = 7
21
4
2 2
0
2
6
4
8
2
4
6
1
0
3
6
9
12
15
1 2 3 4 5 6 7 8 9 10 DK NA
Fre
qu
ency
of
Res
po
nse
How helpful did you find the following tools in implementing PM?
Measures Roadmap Template
Not at all Helpful
Extremely
Helpful
N = 41
Mean = 6.18
Median = 7
22
4
3 3
0 0
5
4
9
4 4 4
1
0
3
6
9
12
15
1 2 3 4 5 6 7 8 9 10 DK NA
Fre
qu
ency
of
Res
po
nse
How helpful did you find the following tools in implementing PM?
Initiative Roadmap Template
Not at all Helpful
Extremely
Helpful
N = 41
Mean = 6.25
Median = 7
23
5
2
1 1
3
4
5
6
4
2
7
1
0
3
6
9
12
15
1 2 3 4 5 6 7 8 9 10 DK NA
Fre
qu
ency
of
Res
po
nse
How helpful did you find the following tools in implementing PM?
Measures Reporting Form
Not at all Helpful
Extremely
Helpful
N = 41
Mean = 5.88
Median = 7
24
3 3
1
0
3
0
5
9
2
5
6
1
0
3
6
9
12
15
1 2 3 4 5 6 7 8 9 10 DK NA
Fre
qu
ency
of
Res
po
nse
How helpful did you find the following tools in implementing PM?
PM Presentation Template for Conference
Not at all Helpful
Extremely
Helpful
N = 38
Mean = 6.52
Median = 8
25
How helpful did you find the following data analysis services in implementing PM?
6.45
5.86
6.06
6.00
5.38
1 2 3 4 5 6 7 8 9 10
ORS CustomerScorecards
Process Behavior
Graphing Data
Data Collection
Design of DataCollection
Not at all Helpful
Extremely
HelpfulMean Ratings
N = 41
N = 41
N = 40
N = 41
N = 41
26
5
4
2
1
5
3
2
8
1
3 3
4
0
3
6
9
12
15
1 2 3 4 5 6 7 8 9 10 DK NA
Fre
qu
ency
of
Res
po
nse
How helpful did you find the following data analysis services in implementing PM:
Design of Data Collection Forms
Not at all Helpful
Extremely
Helpful
N = 41
Mean = 5.38
Median = 5.5
27
4 4
2
0
3 3
5
9
2
4
2
3
0
3
6
9
12
15
1 2 3 4 5 6 7 8 9 10 DK NA
Fre
qu
ency
of
Res
po
nse
How helpful did you find the following data analysis services in implementing PM:
Establishing Data Collection Methodologies
Not at all Helpful
Extremely
Helpful
N = 41
Mean = 6.00
Median = 7
28
4 4
01
2 2
8 8
23
2
4
0
5
10
15
20
1 2 3 4 5 6 7 8 9 10 DK NA
Fre
qu
en
cy
of
Re
sp
on
se
How helpful did you find the following data analysis services in implementing PM:
Graphing Data
Not at all Helpful
Extremely
Helpful
N = 40
Mean = 6.06
Median = 7
29
4 4
0
1 1
3 3
10
0
3
5
7
0
3
6
9
12
15
1 2 3 4 5 6 7 8 9 10 DK NA
Fre
qu
ency
of
Res
po
nse
How helpful did you find the following data analysis services in implementing PM:
Process Behavior Chart Analysis
Not at all Helpful
Extremely
Helpful
N = 41
Mean = 5.86
Median = 7
30
2
3
1 1
4
2
5
9
1
5
4 4
0
3
6
9
12
15
1 2 3 4 5 6 7 8 9 10 DK NA
Fre
qu
ency
of
Res
po
nse
How helpful did you find the following data analysis services in implementing PM:
ORS Customer Scorecards
Not at all Helpful
Extremely
Helpful
N = 41
Mean = 6.45
Median = 7
31
How helpful did you find the following communication vehicles in implementing PM?
5.44
5.74
4.78
1 2 3 4 5 6 7 8 9 10
PM Conference
PM Related Emails
PM Website N = 40
Not at all Helpful
Extremely
HelpfulMean Ratings
N = 40
N = 40
32
6
2
5
4
3
2
1
5
2 2
5
3
0
3
6
9
12
15
1 2 3 4 5 6 7 8 9 10 DK NA
Fre
qu
ency
of
Res
po
nse
How helpful did you find the following communication vehicles in implementing PM:
Performance Management Website
Not at all Helpful
Extremely
Helpful
N = 40
Mean = 4.78
Median = 4
33
2
3
2
3
7
2
6 6
2 2
3
2
0
3
6
9
12
15
1 2 3 4 5 6 7 8 9 10 DK NA
Fre
qu
ency
of
Res
po
nse
How helpful did you find the following communication vehicles in implementing PM:
PM Related Emails from OQM
Not at all Helpful
Extremely
Helpful
N = 40
Mean = 5.74
Median = 6
34
6
1
4
0
4
2
5
6
1
3
4 4
0
3
6
9
12
15
1 2 3 4 5 6 7 8 9 10 DK NA
Fre
qu
ency
of
Res
po
nse
How helpful did you find the following communication vehicles in implementing PM:
PM Conference
Not at all Helpful
Extremely
Helpful
N = 40
Mean = 5.44
Median = 6
35
How helpful did you find assistance from the following groups in implementing PM?
7.82
4.96
7.97
6.38
1 2 3 4 5 6 7 8 9 10
Your Supervisor
Other PM Teams
PM Consultants
OQM Staff N = 39
Not at all Helpful
Extremely
HelpfulMean Ratings
N = 40
N = 38
N = 38
36
3
0
4 4
1
3
2
9
3
5
4
1
0
3
6
9
12
15
1 2 3 4 5 6 7 8 9 10 DK NA
Fre
qu
ency
of
Res
po
nse
How helpful did you find assistance from the following groups in implementing PM?
OQM Staff
Not at all Helpful
Extremely
Helpful
N = 39
Mean = 6.38
Median = 8
37
0
21
01 1
8
5
7
11
2 2
0
5
10
15
20
25
30
1 2 3 4 5 6 7 8 9 10 DK NA
Fre
qu
ency
of
Res
po
nse
How helpful did you find assistance from the following groups in implementing PM?
PM Consultants
Not at all Helpful
Extremely
Helpful
N = 40
Mean = 7.97
Median = 9
38
7
3
1 10 0
2
7
12
3
11
0
5
10
15
20
1 2 3 4 5 6 7 8 9 10 DK NA
Fre
qu
ency
of
Res
po
nse
How helpful did you find assistance from the following groups in implementing PM?
Other PM Teams
Not at all Helpful
Extremely
Helpful
N = 38
Mean = 4.96
Median = 6
39
1
0
1 1
3
1
3
8
6
9
2
3
0
3
6
9
12
15
1 2 3 4 5 6 7 8 9 10 DK NA
Fre
qu
ency
of
Res
po
nse
How helpful did you find assistance from the following groups in implementing PM?
Your Supervisor
Not at all Helpful
Extremely
Helpful
N = 38
Mean = 7.82
Median = 8
40
Immediate Supervisor Involvement in PM
7.53
1 2 3 4 5 6 7 8 9 10
To what extent wasyour immediate
supervisorinvolved in your
PM process duringthis PM cycle?
To no Extent
To a Great Extent
7.60
1 2 3 4 5 6 7 8 9 10
How helpful wasthe involvement of
your immediatesupervisor in PMrelated matters
during this cycle?
Not at all Helpful
Extremely Helpful
Mean Ratings
N = 38
N = 39
41
1
3
1
0
2
4
2
7
3
13
1 1
0
3
6
9
12
15
1 2 3 4 5 6 7 8 9 10 DK NA
Fre
qu
ency
of
Res
po
nse
What extent was your immediate supervisor involved in your PM process during this PM cycle?
To No Extent
To a Great Extent
N = 38
Mean = 7.53
Median = 8
42
1
2
1
0
1
4
5
6
4
11
2 2
0
3
6
9
12
15
1 2 3 4 5 6 7 8 9 10 DK NA
Fre
qu
ency
of
Res
po
nse
How helpful was the involvement of your immediate supervisor in PM related matters during this cycle?
Not at all Helpful
Extremely
Helpful
N = 39
Mean = 7.60
Median = 8
43
To what extent have your knowledge/skills/abilities (KSAs) improved in the following areas during this PM cycle?
4.19
4.79
5.03
5.54
5.54
5.68
6.18
6.03
1 2 3 4 5 6 7 8 9 10
Using Visio
Using PowerPoint
Using Excel
Working in a Team
Presentation, Preparationand Delivery
Methods to Analyze Data
Methods to Gather Data
BSC approach to PM N = 38
To No Extent
To a Great
ExtentMean Ratings
N = 40
N = 40
N = 40
N = 39
N = 38
N = 39
N = 38
44
3
2
4
1
3
5 5
7
2
4
2
00
3
6
9
12
15
1 2 3 4 5 6 7 8 9 10 DK NA
Fre
qu
ency
of
Res
po
nse
To what extent have your knowledge/skills/abilities (KSAs) improved in the following areas during this PM cycle?
Balanced Scorecard Approach to PM
To No Extent
To a Great Extent
N = 38
Mean = 6.03
Median = 7
45
4
2
4
1
3
2
7
8
3
5
1
00
3
6
9
12
15
1 2 3 4 5 6 7 8 9 10 DK NA
Fre
qu
ency
of
Res
po
nse
To what extent have your knowledge/skills/abilities (KSAs) improved in the following areas during this PM cycle?
Methods to Gather Data
To No Extent
To a Great Extent
N = 40
Mean = 6.18
Median = 7
46
5
3 3
2
3
2
7
6
2
4
1
2
0
2
4
6
8
10
1 2 3 4 5 6 7 8 9 10 DK NA
Fre
qu
ency
of
Res
po
nse
To what extent have your knowledge/skills/abilities (KSAs) improved in the following areas during this PM cycle?
Methods to Analyze Data
To No Extent
To a Great Extent
N = 40
Mean = 5.68
Median = 7
47
6
1
2
4
3
1
8
5
3
2 2
3
0
3
6
9
12
15
1 2 3 4 5 6 7 8 9 10 DK NA
Fre
qu
ency
of
Res
po
nse
To what extent have your knowledge/skills/abilities (KSAs) improved in the following areas during this PM cycle?
Presentation Preparation and Delivery
To No Extent
To a Great Extent
N = 40
Mean = 5.54
Median = 7
48
5
0
45
23
56
4
1 1
3
0
5
10
15
20
1 2 3 4 5 6 7 8 9 10 DK NA
Fre
qu
ency
of
Res
po
nse
To what extent have your knowledge/skills/abilities (KSAs) improved in the following areas during this PM cycle?
Working in a Team
To No Extent
To a Great Extent
N = 39
Mean = 5.54
Median = 6
49
6
2
1
3
4
7
3
5
2
0
1
4
0
3
6
9
12
15
1 2 3 4 5 6 7 8 9 10 DK NA
Fre
qu
ency
of
Res
po
nse
To what extent have your knowledge/skills/abilities (KSAs) improved in the following areas during this PM cycle?
Using Excel
To No Extent
To a Great Extent
N = 38
Mean = 5.03
Median = 6
50
8
2
0
3
5 5
3
5
2
0
1
5
0
3
6
9
12
15
1 2 3 4 5 6 7 8 9 10 DK NA
Fre
qu
ency
of
Res
po
nse
To what extent have your knowledge/skills/abilities (KSAs) improved in the following areas during this PM cycle?
Using PowerPoint
To No Extent
To a Great Extent
N = 39
Mean = 4.79
Median = 5
51
10
01
2
43
4
2
01 1
10
0
5
10
15
20
1 2 3 4 5 6 7 8 9 10 DK NA
Fre
qu
ency
of
Res
po
nse
To what extent have your knowledge/skills/abilities (KSAs) improved in the following areas during this PM cycle?
Using Visio
To No Extent
To a Great Extent
N = 38
Mean = 4.19
Median = 5
52
Perceptions of Needs for Cycle 4
53
In planning for the next PM cycle, how helpful would each of the following be to your PM Team?
5.59
5.94
5.97
6.45
6.54
6.56
6.57
6.58
6.86
7.03
7.47
7.84
1 2 3 4 5 6 7 8 9 10
Yearly Performance Mgmt Conf
Rotating Quarterly Presentation Meetings
Training on the BSC
Training on data collection systems and forms
Training on Process Behavior Charts
Training on ORS ABC Cost Accounting Model
OQM Staff
Training on Analyzing Data through Graphs
Training on Customer Assessment
Regular Meetings with Mgmt to discuss PM data
IT Support to Establish Data Collection Systems
PM Consultants provided by OQM
Not at all Helpful
Extremely
HelpfulMean Ratings
54
In planning for the next PM cycle, how helpful would each of the following tools/resources be to your PM Team?
7.47
5.59
5.94
7.03
6.57
7.84
1 2 3 4 5 6 7 8 9 10
IT Support to Establish Data CollectionSystems
Yearly PM Conference
Rotating Quarterly Presentation Meetings
Regular Meetings with your Mgmt todiscuss PM Data
OQM Staff
PM Consultants provided by OQM N = 40
Not at all Helpful
Extremely Helpful
Mean Ratings
N = 40
N = 37
N = 39
N = 38
N = 37
55
21
21
01
4
6
8
13
2
00
5
10
15
20
25
30
1 2 3 4 5 6 7 8 9 10 DK NA
Fre
qu
ency
of
Res
po
nse
In planning for the next PM cycle, how helpful would each of the following tools/resources be to your PM Team:
PM Consultants Provided by OQM
Not at all Helpful
Extremely Helpful
N = 40
Mean = 7.84
Median = 9
56
2 2
4
1
3
2
4
7
3
7
5
00
3
6
9
12
15
1 2 3 4 5 6 7 8 9 10 DK NA
Fre
qu
ency
of
Res
po
nse
In planning for the next PM cycle, how helpful would each of the following tools/resources be to your PM Team:
OQM Staff
Not at all Helpful
Extremely Helpful
N = 40
Mean = 6.57
Median = 7
57
3
2
1 1
3
2
3
9
1
11
0
1
0
3
6
9
12
15
1 2 3 4 5 6 7 8 9 10 DK NA
Fre
qu
ency
of
Res
po
nse
In planning for the next PM cycle, how helpful would each of the following tools/resources be to your PM Team:
Regular Meetings with your Management
Not at all Helpful
Extremely Helpful
N = 37
Mean = 7.03
Median = 8
58
7
2
0 0
3 3
5
6
1
6 6
00
3
6
9
12
15
1 2 3 4 5 6 7 8 9 10 DK NA
Fre
qu
ency
of
Res
po
nse
In planning for the next PM cycle, how helpful would each of the following tools/resources be to your PM Team:
Rotating Quarterly Presentation Meetings
Not at all Helpful
Extremely Helpful
N = 39
Mean = 5.94
Median = 7
59
7
2
1 1
4
1
5 5
0
6
5
1
0
3
6
9
12
15
1 2 3 4 5 6 7 8 9 10 DK NA
Fre
qu
ency
of
Res
po
nse
In planning for the next PM cycle, how helpful would each of the following tools/resources be to your PM Team:
Yearly Performance Management Conference
Not at all Helpful
Extremely Helpful
N = 38
Mean = 5.59
Median = 7
60
21
21
21
3
65
11
21
0
5
10
15
20
1 2 3 4 5 6 7 8 9 10 DK NA
Fre
qu
ency
of
Res
po
nse
In planning for the next PM cycle, how helpful would each of the following tools/resources be to your PM Team:
IT Support to Establish Data Collection Systems
Not at all Helpful
Extremely Helpful
N = 37
Mean = 7.47
Median = 8
61
In planning for the next PM cycle, how helpful would each of the following training sessions be to your PM Team?
6.56
6.86
6.54
6.58
6.45
5.97
1 2 3 4 5 6 7 8 9 10
ORS ABS Cost Accounting Model
Customer Assessment (interviews, focusgroups, surveys)
Analyzing Processes with Process BehaviorCharts
Analyzing Data through Graphs
Setting up Data Collection Systems andForms
Balanced Scorecard Methodology N = 38
Not at all Helpful
Extremely Helpful
Mean Ratings
N = 39
N = 40
N = 39
N = 40
N = 40
62
6
1 1
5
1
4
2
7
1
7
3
00
3
6
9
12
15
1 2 3 4 5 6 7 8 9 10 DK NA
Fre
qu
ency
of
Res
po
nse
In planning for the next PM cycle, how helpful would each of the following training sessions be to your PM Team?
Balanced Scorecard Methodology
Not at all Helpful
Extremely Helpful
N = 38
Mean = 5.97
Median = 6
63
6
2
0 0
4
2
6
8
3
7
1
00
3
6
9
12
15
1 2 3 4 5 6 7 8 9 10 DK NA
Fre
qu
ency
of
Res
po
nse
In planning for the next PM cycle, how helpful would each of the following training sessions be to your PM Team?
Setting up Data Collection Systems and Forms
Not at all Helpful
Extremely Helpful
N = 39
Mean = 6.45
Median = 7
64
6
1
0
1
3 3
5
9
2
8
2
00
3
6
9
12
15
1 2 3 4 5 6 7 8 9 10 DK NA
Fre
qu
ency
of
Res
po
nse
In planning for the next PM cycle, how helpful would each of the following training sessions be to your PM Team?
Analyzing Data Through Graphs
Not at all Helpful
Extremely Helpful
N = 40
Mean = 6.58
Median = 8
65
5
2
1
0
3
1
5
8
3
7
4
00
3
6
9
12
15
1 2 3 4 5 6 7 8 9 10 DK NA
Fre
qu
ency
of
Res
po
nse
In planning for the next PM cycle, how helpful would each of the following training sessions be to your PM Team?
Analyzing Processes with Process Behavior Charts
Not at all Helpful
Extremely Helpful
N = 39
Mean = 6.54
Median = 8
66
3
2 2
0
4
1
3
10
3
8
4
00
3
6
9
12
15
1 2 3 4 5 6 7 8 9 10 DK NA
Fre
qu
ency
of
Res
po
nse
In planning for the next PM cycle, how helpful would each of the following training sessions be to your PM Team?
Customer Assessment through Interviews, Focus Groups and Surveys
Not at all Helpful
Extremely Helpful
N = 40
Mean = 6.86
Median = 8
67
4
2
0 0
6
1
2
8
2
7
8
00
3
6
9
12
15
1 2 3 4 5 6 7 8 9 10 DK NA
Fre
qu
ency
of
Res
po
nse
In planning for the next PM cycle, how helpful would each of the following training sessions be to your PM Team?
ORS ABC Cost Accounting Model
Not at all Helpful
Extremely Helpful
N = 40
Mean = 6.56
Median = 8
68
Organizational Climate Perceptions
69
4.15
3.56
3.21
3.84
3.70
3.50
4.03
3.59
1 2 3 4 5
Actively Involved in Data Collection
PM Assist with A-76
PM Contributed to Improvements
Accountability is an Important Value
Results-Oriented Culture
I Understand Services Hierarchy
Committed to PM
See Value of PM
Organizational Climate PerceptionsSummary
Strongly Disagree
Strongly Agree
N = 40
N = 39
N = 40
N = 40
N = 39
N = 40
N = 40
N = 40
Neither Agree nor Disagree
70
4 4
7
13
11
10
0
5
10
15
20
25
1 2 3 4 5 DK NA
Fre
qu
en
cy
of
Re
sp
on
se
I see the value of performance management for improving business operations in my area.
Strongly Disagree
Strongly Agree
N = 40
Mean = 3.59
Median = 4
Neither Agree nor Disagree
71
2 2
5
14
16
0 00
5
10
15
20
25
1 2 3 4 5 DK NA
Fre
qu
en
cy
of
Re
sp
on
se
My organization (ORS or ORF) is committed to the performance management effort.
Strongly Disagree
Strongly Agree
N = 39
Mean = 4.03
Median = 4
Neither Agree nor Disagree
72
7
3 3
1112
4
00
5
10
15
20
25
1 2 3 4 5 DK NA
Fre
qu
en
cy
of
Re
sp
on
se
I understand what the Services Hierarchy is and its purpose.
Strongly Disagree
Strongly Agree
N = 40
Mean = 3.50
Median = 4
Neither Agree nor Disagree
73
4
2
8
10
13
3
00
5
10
15
20
25
1 2 3 4 5 DK NA
Fre
qu
en
cy
of
Re
sp
on
se
The culture of my organization (ORS or ORF) is changing to be more results-oriented.
Strongly Disagree
Strongly Agree
N = 40
Mean = 3.70
Median = 4
Neither Agree nor Disagree
74
3 3
6
10
15
2
00
5
10
15
20
25
1 2 3 4 5 DK NA
Fre
qu
en
cy
of
Re
sp
on
se
Managers in my organization (ORS or ORF) believe accountability is an important organizational value.
Strongly Disagree
Strongly Agree
N = 39
Mean = 3.84
Median = 4
Neither Agree nor Disagree
75
7
2
8
18
32
00
5
10
15
20
25
1 2 3 4 5 DK NA
Fre
qu
ency
of
Res
po
nse
Performance management has contributed to improvements in my area.
Strongly Disagree
Strongly Agree
N = 40
Mean = 3.21
Median = 4
Neither Agree nor Disagree
76
7
1
5
8
13
4
2
0
5
10
15
20
25
1 2 3 4 5 DK NA
Fre
qu
en
cy
of
Re
sp
on
se
Performance management will assist my organization if we have to go through an A-76 competition.
Strongly Disagree
Strongly Agree
N = 40
Mean = 3.56
Median = 4
Neither Agree nor Disagree
77
32
1
14
20
0 00
5
10
15
20
25
1 2 3 4 5 DK NA
Fre
qu
en
cy
of
Re
sp
on
se
My PM Team (or the groups I am responsible for) are actively involved in data collection and analysis.
Strongly Disagree
Strongly Agree
N = 40
Mean = 4.15
Median = 5
Neither Agree nor Disagree
78
Organizational Climate Perceptions by Program Area
79
Program and Employee Services Climate PerceptionsPositive Responses
57%
71%
57%
71%
71%
100%
57%
100%
0% 20% 40% 60% 80% 100%
Actively Involved in Data Collection
PM Assist with A-76
PM Contributed to Improvements
Accountability is an Important Value
Results-Oriented Culture
I Understand Services Hierarchy
Committed to PM
See Value of PM
Percent Agree or Strongly Agree
N = 7
N = 7
N = 7
N = 7
N = 7
N = 7
N = 7
N = 7
80
Scientific Resources Climate PerceptionsPositive Responses
100%
50%
50%
83%
17%
33%
67%
50%
0% 20% 40% 60% 80% 100%
Actively Involved in Data Collection
PM Assist with A-76
PM Contributed to Improvements
Accountability is an Important Value
Results-Oriented Culture
I Understand Services Hierarchy
Committed to PM
See Value of PM
Percent Agree or Strongly Agree
N = 6
N = 6
N = 6
N = 6
N = 6
N = 6
N = 6
N = 6
81
Security and Emergency Response Services Climate PerceptionsPositive Responses
75%
50%
38%
38%
75%
63%
75%
50%
0% 20% 40% 60% 80% 100%
Actively Involved in Data Collection
PM Assist with A-76
PM Contributed to Improvements
Accountability is an Important Value
Results-Oriented Culture
I Understand Services Hierarchy
Committed to PM
See Value of PM
Percent Agree or Strongly Agree
N = 8
N = 8
N = 8
N = 8
N = 8
N = 8
N = 8
N = 8
82
Management Services Climate PerceptionsPositive Responses
67%
33%
67%
67%
67%
67%
100%
67%
0% 20% 40% 60% 80% 100%
Actively Involved in Data Collection
PM Assist with A-76
PM Contributed to Improvements
Accountability is an Important Value
Results-Oriented Culture
I Understand Services Hierarchy
Committed to PM
See Value of PM
Percent Agree or Strongly Agree
N = 3
N = 3
N = 3
N = 3
N = 3
N = 3
N = 3
N = 3
83
Real Estate and Facilities Climate PerceptionsPositive Responses
75%
50%
25%
75%
50%
75%
50%
75%
0% 20% 40% 60% 80% 100%
Actively Involved in Data Collection
PM Assist with A-76
PM Contributed to Improvements
Accountability is an Important Value
Results-Oriented Culture
I Understand Services Hierarchy
Committed to PM
See Value of PM
Percent Agree or Strongly Agree
N = 4
N = 4
N = 4
N = 4
N = 4
N = 4
N = 4
N = 4
84
Organizational Climate Perceptions by Role
85
Division Directors’/Office Directors’ Climate Perceptions
80%
80%
80%
40%
80%
100%
100%
80%
0% 20% 40% 60% 80% 100%
Actively Involved in Data Collection
PM Assist with A-76
PM Contributed to Improvements
Accountability is an Important Value
Results-Oriented Culture
I Understand Services Hierarchy
Committed to PM
See Value of PM
Percent Agree or Strongly Agree
N = 5
N = 5
N = 5
N = 5
N = 5
N = 5
N =5
N = 5
86
PM Team Leaders’ Climate Perceptions
100%
43%
57%
86%
57%
57%
71%
71%
0% 20% 40% 60% 80% 100%
Actively Involved in Data Collection
PM Assist with A-76
PM Contributed to Improvements
Accountability is an Important Value
Results-Oriented Culture
I Understand Services Hierarchy
Committed to PM
See Value of PM
Percent Agree or Strongly Agree
N = 7
N = 7
N = 7
N = 7
N = 7
N = 7
N = 7
N = 7
87
PM Team Members’ Perceptions
85%
52%
52%
65%
56%
52%
77%
56%
0% 20% 40% 60% 80% 100%
Actively Involved in Data Collection
PM Assist with A-76
PM Contributed to Improvements
Accountability is an Important Value
Results-Oriented Culture
I Understand Services Hierarchy
Committed to PM
See Value of PM
Percent Agree or Strongly Agree
N = 27
N = 27
N = 27
N = 27
N = 26
N = 27
N = 27
N = 27
88
Customer Scorecard Ratings
89
Customer Scorecard Ratings for PMOverview for Cycle 3
7.18
7.71
7.23
7.16
7.16
6.82
6.67
6.77
5.92
1 2 3 4 5 6 7 8 9 10
Handling of Problems
Competence
Convenience
Responsiveness
Availability
Reliability
Timeliness
Quality
Cost N = 40
Unsatisfactory Outstanding
Mean Ratings
N = 40
N = 39
N = 38
N = 40
N = 40
N = 39
N = 40
N = 40
90
2
01 1 1 1
3
0
4
0
25
2
0
5
10
15
20
25
30
1 2 3 4 5 6 7 8 9 10 DK NA
Fre
qu
en
cy
of
Re
sp
on
se
Cost Satisfaction Ratings
Unsatisfactory Outstanding
N = 40
Mean = 5.92
Median = 7
91
2
0
1 1
3 3
5
9
6
0
10
00
3
6
9
12
15
1 2 3 4 5 6 7 8 9 10 DK NA
Fre
qu
en
cy
of
Re
sp
on
se
Quality Satisfaction Ratings
Unsatisfactory
Outstanding
N = 40
Mean = 6.77
Median = 8
92
3
0
1 1
3
2
6 6
7
1
9
00
3
6
9
12
15
1 2 3 4 5 6 7 8 9 10 DK NA
Fre
qu
en
cy
of
Re
sp
on
se
Timeliness Satisfaction Ratings
Unsatisfactory
Outstanding
N = 39
Mean = 6.67
Median = 7
93
2
0
2
1
0
2
7
8
6
0
10
00
3
6
9
12
15
1 2 3 4 5 6 7 8 9 10 DK NA
Fre
qu
en
cy
of
Re
sp
on
se
Reliability Satisfaction Ratings
Unsatisfactory
Outstanding
N = 38
Mean = 6.82
Median = 8
94
2
0
1
3
1 1
6
7 7
4
8
00
3
6
9
12
15
1 2 3 4 5 6 7 8 9 10 DK NA
Fre
qu
en
cy
of
Re
sp
on
se
Availability Satisfaction Ratings
Unsatisfactory Outstanding
N = 40
Mean = 7.16
Median = 8
95
2
0
2
3
0
2
4
6
9
4
8
00
3
6
9
12
15
1 2 3 4 5 6 7 8 9 10 DK NA
Fre
qu
en
cy
of
Re
sp
on
se
Responsiveness Satisfaction Ratings
Unsatisfactory
Outstanding
N = 40
Mean = 7.16
Median = 8
96
2
0
2 2
0 0
7 7 7
4
8
00
3
6
9
12
15
1 2 3 4 5 6 7 8 9 10 DK NA
Fre
qu
en
cy
of
Re
sp
on
se
Convenience Satisfaction Ratings
Unsatisfactory Outstanding
N = 39
Mean = 7.23
Median = 8
97
1
0
1 1
0
2
6
7
9
4
9
00
3
6
9
12
15
1 2 3 4 5 6 7 8 9 10 DK NA
Fre
qu
en
cy
of
Re
sp
on
se
Competence Satisfaction Ratings
Unsatisfactory
Outstanding
N = 40
Mean = 7.71
Median = 8
98
2
0
2
1 1 1
5
4
9
3
10
2
0
3
6
9
12
15
1 2 3 4 5 6 7 8 9 10 DK NA
Fre
qu
en
cy
of
Re
sp
on
se
Handling of Problems Satisfaction Ratings
Unsatisfactory Outstanding
N = 40
Mean = 7.18
Median = 8
99
Customer Scorecard Ratings for PMTrend Analysis
7.03
7.06
7.36
6.57
6.83
6.58
6.82
6.12
6.82
6.67
6.77
5.92
1 2 3 4 5 6 7 8 9 10
Reliability
Timeliness
Quality
Cost
Cycle 1 Cycle 2 Cycle 3Unsatisfactory Outstanding
Mean Ratings
N = 37
N = 70
N = 67
N = 67
N = 40
N = 40
N = 39
N = 38
N = 68
N = 68
N = 67
N = 68
100
Customer Scorecard Ratings for PMTrend Analysis (cont.)
6.33
7.33
7.50
7.50
7.17
6.33
7.06
6.78
7.14
7.03
7.18
7.71
7.23
7.16
7.16
1 2 3 4 5 6 7 8 9 10
Handling ofProblems
Competence
Convenience
Responsiveness
Availability
Cycle 1 Cycle 2 Cycle 3Unsatisfactory Outstanding
Mean Ratings
N = 68
N = 66
N = 69
N = 65
N = 40
N = 40
N = 39
N = 40
N = 40
N = 68N = 69
N = 68
N = 68
N = 68
N = 65
101
Summary of Comments
102
What did you value most about this cycle of performance management?
• Knowledge and Experience Gained (4)• Results show that we have improved since last year.• Surveys have provided an opportunity to find out how
customers perceive quality of service.• Data gathering has helped quantify quality of service.
• Assistance provided by consultants and OQM staff (3)• Interaction with OQM staff & consultants in analyzing
performance.• Knowledgeable and helpful consultants.
• Measurement Tools (3)• The use of data reporting template and the initiatives template.• The Measures Reporting Form allowed us to look at the
objectives and measures in a more relevant perspective.
Note: Comments were sorted into categories. Most frequently mentioned categories listed first.
103
What obstacles were most challenging to you during this cycle of performance management?
Note: Comments were sorted into categories. Most frequently mentioned categories listed first.
• Lack of Time and/or Resources (6)• Lack of time to devote to PMP and keep up with workload• Short notice• OQM website is out of date
• Miscellaneous (4)• Learning to use Visio and Excel• Preparing the report
• Lack of Management Involvement (3)• Not having discussions about PM in staff meetings• Managers need to use data in decision making
• Data Collection and Measurement (2)• Obtaining data
• Lack of commitment from team members (2)• Getting all team members to contribute
104
How can we help integrate PM into your daily business activities?
• Increase support currently provided (6)• Need help in learning how to distribute data collection
throughout the division.• Need help collecting and interpreting data.
• Miscellaneous (3)• Work to establish timelines so that we see the year at a glance.• Continue emphasis on the importance of performance
management.
105
Summary
106
SummarySurvey Respondents
• Response rate of 22%• Respondents were primarily from ORS (73%)• The majority of respondents were team members
(68%)• About 75% of respondents were experienced with
PM (i.e., participated in FY03)• About 60% of respondents attended FY04 training• About 63% of respondents spent 20% or less of their
time during the past year on PM• Team meetings typically occurred once a month
(41%), biweekly (22%), and quarterly (20%)
107
Summary (cont.)
Feedback on Past Cycle
• Most Helpful Tools/Resources• PM consultants• Service group supervisors• PM template• PM presentation template• ORS Customer Scorecards• OQM Staff. • These are the same tools/resources receiving the highest ratings in
FY03.
• KSA Improvements• Just above mid-scale improvements noted on all topics with the
following exceptions: PowerPoint and Visio. • Perhaps respondent familiarity with these tools has leveled off.
108
Summary (cont.)
Planning for Next PM Cycle
• Future needs• Most helpful to teams for the future would be
• PM consultants• IT support for data collection systems• Regular meetings with management to discuss PM data• Training on customer assessment • Training on analyzing data
• These requests are similar to previous years
109
Summary (cont.)
Organizational Climate
• Majority of respondents either agree or strongly agree that:
• PM Teams are actively involved in data collection (85%)• Their organization is committed to PM (77%)• Managers in their organization believe accountability is
important (64%)• They see the value of PM for improving business operations
(60%)• The culture of their organization is changing to be more
results-oriented (58%)
110
Summary (cont.)
Customer Scorecard Trend Analysis
• Ratings on dimensions show minor variance from cycle to cycle but none of the differences are statistically significant.
• Current ratings:• Highest on competence, convenience, and handling of
problems, • Lowest on cost
111
Appendix
Individual Comments From Survey
112
What did you value most about this cycle of performance management? Individual Comments
• The opportunity to find out how my customers perceive the quality of the job I'm doing.
• Data gathering enabled us to quantify that construction inspection was decreasing our submittal review response time.
• Our consultant Janice Rouiller.• The survey provided verification of the quality of service DMCS is
providing the NIH community.• The numbers show that last year's improvements have not only
been sustained, but improved upon.• Honestly, I must state NOTHING! This ploy that the PMP process is
management tool is hog-wash, the true intent of the PMP is to tie it to how much time staff spend in each function and to cut staff.
• Interaction with OQM staff & consultants in analyzing performance.
113
What did you value most about this cycle of performance management? (cont.) Individual Comments
• Patrick Patterson, is the consultant with whom our team works on PMP. His skills & knowledge of all aspect of the program are excellent. He ensures that we ‘see’ the process & purpose rather than ‘just do it’.
• The willingness of the ORS Scientific resources to be first in the rotation of PM conferences.
• The use of data reporting template and the initiatives template.• Meeting as a group to plan ahead.• the introduction of the use of the Business Framework and the
concept of the Balanced Scorecard.• It was helpful to focus just on the high impact objectives.• The Measures Reporting Form allowed us to look at the objectives
and measures in a more relevant perspective.
114
What obstacles were most challenging to you during this cycle of performance management?Individual Comments
• Preparing the report is painful.• Being taken away from my primary duties created undue stress and
hardship when OQM wanted their deadlines met, yet they couldn't seem to get their parts done in a timely fashion.
• We did not have the time to do the customer survey focus groups we had planned on doing.
• OQM website is out of date and needs updating.• The challenge of continuing performance gains without losing any
quality.• learning to use Excel and Visio.• Lack of time to devote to PMP AND perform ones own work. PMP
is almost a full time job. This is an OQM function and should be performed by that staff.
• Supervisors and managers should use data more often in their decision-making.
115
What obstacles were most challenging to you during this cycle of performance management? (cont.) Individual Comments
• Commitment from all the team members.• obtaining data.• Getting all team members to contribute to the tasks of identifying
relevant data and analyzing it.• The way the PMP was created in Excel with an entire page in one
row of cells made it extremely time consuming & tedious to update.• Switching the rules in the middle of the game! We went from PMP,
to score card, then to the dash board. These aberrations made staff wonder what value all of this has to our the ultimate duty to provide the service to the Institutes.
• Not having discussions about the performance of service groups/discrete services in staff meetings.
• Lack of a program area manager who accepts PM plans and manage their implementation.
116
What obstacles were most challenging to you during this cycle of performance management? (cont.) Individual Comments
• Concentration on the Measures Reporting Form throughout the year caused the PMP to be neglected. At the end of the cycle, it took a lot of time to update the PMP to match the Measures Reporting Form.
• The short notice and scheduling a report due for the week of January 17 was inane!
117
How can we help integrate PM into your daily business activities? Individual Comments
• We have always been collecting and analyzing data to improve our program--no value added from this formal process.
• In my opinion performance management might be applicable for some businesses in private industry but not for the US Gov,t.
• It already is, and was before I was tasked with being a part of this initiative.
• Work to establish timelines so that we see the year at a glance. Several times my consultant worked with me to make deadlines at the last minute.
• Implement the ORF design review and inspection processes• Already done.• Have the ORS Office of Quality Management (OQM) staff do the work.
In many agencies and in private sector companies this function is not put on the backs of the line staff but is performed by either consultants or an office such as OQM.
118
How can we help integrate PM into your daily business activities? (cont.) Individual Comments
• Need help in learning how to distribute data collection throughout the division.
• Follow up with Quarterly team meetings.• Continue emphasis on the importance of performance management to
efficient 7 effective business operations.• Need help collecting observation type data.• Need help interpreting data.
top related