discovering the right key performance indicators
TRANSCRIPT
1
DISCOVERING THE RIGHT KEY PERFORMANCE INDICATORS IN
LIBRARIES: A REVIEW OF LITERATURES
Amir Hussain Md Ishak Perpustakaan Sultan Abdul Samad, Universiti Putra Malaysia
43400 UPM Serdang, Selangor, Malaysia Tel : +603-89468601 / Fax : +603-89483745
Mohd Dasuki Sahak Perpustakaan Sultan Abdul Samad, Universiti Putra Malaysia
43400 UPM Serdang, Selangor, Malaysia Tel : +603-89467040 / Fax : +603-89483745
ABSTRACT
This paper discusses the discovery of the right Key Performance Indicators (KPIs) in Malaysian libraries. As one of the measurement performance methodologies, KPIs are used to improve the overall organizational management and by applying the KPIs, it is considered an effort to transform libraries’ management towards excellence. The KPIs can be the key to organizations’ success, and for libraries to achieve their vision, mission, business strategy, objectives and targets. KPI Model was discussed by a few researchers in this paper to identify the right KPIs for libraries. The discussion covers the library performance indicator standards by The International Organization for Standardization (ISO). The performance indicators described in this International Standard can be used effectively in the evaluation of libraries such as resources, access, infrastructure, use, efficiency and potentials, and development. This paper also describes the flow of how to develop KPIs, as proposed by Malaysian Administrative Modernization and Management Planning Unit, Prime Minister's Department. Libraries need to identify their core businesses, core process and services before they identify the KPIs. The measures or indicators must be accepted, understood, and owned across the organization in order to ensure that the performance measurement becomes effective. The discussion in this paper will help libraries understand and identify the right KPIs for libraries.
Keywords: Key Performance Indicators, Libraries, Organizations success, The
International Organization for Standardization (ISO)
2
DISCOVERING THE RIGHT KEY PERFORMANCE INDICATOR (KPIs) IN
LIBRARIES
1. INTRODUCTION
Nowadays, performance measurement is a vital part for every organization, including
public sector organization for its success and long term survival in the current
competitive global market place. United States General Accounting Office (1998) defines
performance measurement as the ongoing monitoring and reporting of program
accomplishments, particularly progress towards pre-established goals. A “program” may
be any activity, project, function, or policy that has an identifiable purpose or set of
objectives.
An organization needs to understand how well its strategy is working and how to work
effectively to achieve the organization’s vision, mission and objectives. Bowen (2005)
states that vision statements provide a broad guideline of future organizational goals,
mission statements define the scope of the organization, differentiate it from competitors
and give a summary of why the organization exists. Objectives can be defined as tangible
milestones by which to get there. In order to determine the key performance indicators
for measuring the achievement, it is necessary to do an analysis starting from the vision
statement and then going through the mission statements and objectives of the
organization.
Performance measurement also helps organization to be consistent in making decision
with the intention to ensure the operational activities linked with the organization’s vision
and mission. The measurement of market share, customer demand and customer
satisfaction can be essential elements for an organization to understand its current
position and make necessary improvements to achieve its target. However, the process of
discovering the right measurement is very complex. Hauser and Katz (1998) have
3
mentioned that metrics are good if the actions and decisions which improve the metrics,
also improve the firm’s desired long-term outcomes.
Dato’ Sri Mohd Najib Tun Abdul Razak introduced Key Performance Indicator (KPIs) in
his administration when he became the sixth Prime Minister of Malaysia in 2009. The
purpose of KPIs is to evaluate and measure the performance of Cabinet Minister and
Senior Civil Servants and help government agencies to increase the quality of services
delivery for the citizens (Mohd Najib, 2009a). KPIs will be eventually expand to cover all
government matters, beginning with the monitoring exercise for all ministers. Minister in
the Prime Minister’s Department, Tan Sri Dr Koh Tsu Koon has been assigned to ensure
that KPI exercise is being carried out smoothly (Mohd Najib, 2009b).
KPIs is a general indicator of performance that focuses on critical aspects of outputs or
outcome and reflects the critical success factors of an organization (Chan & Chan, 2004;
John Reh, 1997). While, the focus of KPIs by Dato’ Sri Mohd Najib Tun Abdul Razak is
on the impact rather than input, on outcome rather than output (Mohd Najib, 2009b). The
measure or indicator must be accepted, understood, and owned across the ministry to
ensure performance measurement to become effective. KPIs will need to evolve and it is
likely that a set of KPIs will be subject to change and refinement (Chan & Chan, 2004).
The public universities in Malaysia and government agencies are also managed by the
Ministry of Higher Education. There are 20 public universities that act as higher
educational institutions, created by executive order of the provisions of the Universities
and University College Act 1971 (Ministry of Higher Education, 2011). The role of
universities is to produce a competent workforce equipped with the skills, knowledge,
attitude and behaviour to meet the demands of the high-technology environment through
the activities of learning, teaching, consultation and research and development.
These days, university libraries also include and consider the indicator performance when
measuring the university performance by the Malaysia Qualification Agency, Ministry of
4
Higher Education such as Self-accrediting Institution, Academic Performance Audit and
University Research Evaluation System (Ministry of Higher Education, 2011).
2. CONCEPT OF KEY PERFORMANCE INDICATOR
The origins of Key Performance Indicator can be traced to 1976 in an article published by
BusinessWeek. It described a key indicator system based on three concepts such as the
selection of key indicator represent organization, exception reporting and visual display
of that information (Sanchez & Robert, 2010). A performance indicator can be defined as
an item of information collected at regular interval to track the performance of the
system. According to Fitz-Gibbon (1990) performance indicators are collected in many
complex systems like education and deliver services.
Key Performance Indicator (KPIs) is defined as simple and an effective measurement
system that helps organization to explain and manage the services’ progress for
customers aligned with the vision and mission of the organization (MAMPU, 2005). KPIs
can be the key to organizations’ success to achieve the vision, mission, business strategy,
objectives and targets (Krishnapillai, 2009). KPIs use quantifiable measurements to
measure, report, and improve the organization performance (John Reh, 1997; Parmenter,
2009).
In other hand, Parmenter (2009) list seven characteristics of KPIs from an extensive
analysis and through discussions with over 3,000 participants in their KPIs workshops
that cover most organization types in the public and private sectors such as:
a. Are non-financial measures
b. Are measured frequently
c. Are acted on by the CEO and senior management team
d. Clearly indicate what action is required by staff
e. Are measures that tie responsibility down to a team
f. Have a significant impact
g. They encourage appropriate action
5
Performance Measurement Model by Boekhorst (1995) that is shown in Figure 1
indicates that performance measurement begins with determining what the organization’s
mission, goals, objectives and activities are. The performance will measure
organization’s activities to make sure that the activities were aligned and reachable with
the organization goals. The activities were measured by evaluating data produced during
organization operation. However, it is difficult to measure the impact of library services
on the user directly. Boekhorts (1995) also explained the criteria for library performance
indicator such as appropriate, reliable or accurate, reproducible, helpful and practical.
Source : Boekhorst (1995)
Figure 1 : Performance measurement
For example, circulation statistics were collected monthly by department of circulation in
the library. Through this method, library management was able to know the progress of
performance of library operation and achievement by comparing circulation data by
monthly, quarterly and yearly. This information is very valuable for future improvement
in the organization. In other hand, the comparing results of performance measurement
with goals and objectives will lead to re-formulation and specification of goals and
objectives.
6
Similarly with the previous model shown is the Performance Measurement Model;
adopted from MAMPU (2005) in the Figure 2. Malaysian government agencies use
Performance Measurement Model to measure performance of services’ delivery.
Libraries can refer this model to understand the flow on how to develop KPIs.
Commonly, the mission of the academic libraries is to deliver world-class library and
information services to meet the information needs of the students and academic staff and
to support university mission.
Source: MAMPU (2005)
Figure 2: Performance Measurement Model
Vision and Mission
Determine Core Services
Determine Core Process
Indentify Services
KPIS and Targets
Measure and Evaluate Performance
Supervise Performance
Added Value Performance
Customer Feedback
Reanalyze Agency
Performance
Fulfill
Customer Services Satisfaction
Continuous Evaluation
7
This model describes clearly about the setting of KPIs in the organization. Initially,
government agencies need to determine core business of the agencies based on vision and
mission. Later, KPIs will be created based on the core process and services.
This model clearly describes how to develop KPIs and link with the target to satisfy
customer needs. This model also indicates that KPIs is a continuous measurement
process to ensure services delivered for customers are always relevant with the agencies’
function and customer needs. The output of services is also an indicator of performance
measurement by monitoring customer feedback on services’ delivery.
The feedback can be collected through customer compliments, complaints and suggestion
through Customer Satisfaction Form, allocated at the service counter. Research on
customer satisfaction also needs to be conducted at least once a year to collect data about
the level of customer satisfaction. In the end, KPIs can be understood as one type of
dynamic performance management to drive the operational activities to the organization’s
strategy.
The traditional perception of successful of organization’s services and product usually
concerns with time, cost and quality of the product as listed in the objectives’ measures.
In order to sustain with the dynamic environment and to convince stakeholders, libraries
need to combine two types of measurements as explained by (Chan & Chan, 2004) to
clearly show the roles and impact of the libraries. (Toor & Ogunlana, 2010) also
proposed the other performance indicators that increasingly becoming important for
organization such as safety, efficient use of resources, effectiveness, satisfaction of
stakeholders, and reduced conflicts and disputes.
8
Source: (Chan & Chan, 2004)
Figure 3. KPIs for Project Success
Figure 3 (Chan & Chan, 2004) proposed KPIs’ calculation methods to suit the set of
criteria and standard for construction industry to measure the successful of construction
project. The calculation is divided into two groups which is objectives and subjective
indicator. Objectives measurement use mathematical formulae to calculate the respective
values. Formulae will be presented after the detail explanations of each KPIs, such as
time, cost, value, safety and environmental performance. While, subjective indicator uses
subjective opinions and personal judgment of the stakeholders such as quality,
functionality of building and the satisfaction level of various stakeholders.
Cameron Typology describes four models that focus by libraries to measure their
‘effectiveness’ as illustrate in the Figure 5 (cited in Cullen, 1998). Typology is systematic
classification of types that have characteristics or traits in common.
9
MODEL EVALUATION Goal Attainment Model Goals and Objectives
Bench-Marks Standards Output Measures Citizen’s Charter
Systems Resource Model Input Measures Library Statistics Bench-Marks Standards
Internal Systems Model Management Information Systems Decision Support Systems TQM- Total Quality Management ISO9000/9001 etc.
Multiple Constituencies Model
Service Quality Customer Satisfaction Total Quality Management ISO9000/9001 etc. Gap Reduction Marketing Key Performance Indicator (KPIs)
Source: Cameron (1986)
Figure 5: Systems of Evaluation used in Library Science on to Cameron’s typology
The description of four evolutions by Cameron Typology (cited in Cullen, 1998) as below:
a. In the goal attainment model, the organization assesses its effectiveness in terms
of the extent to which it achieves its goals and objectives.
b. In the systems resource (or external systems) model the organization measures its
effectiveness in terms of its ability to gain resources from its environment.
c. In the internal processes, or internal systems model, the emphasis is on the
organization’s internal communications systems, and the efficiency with which it
converts inputs into outputs.
d. In the multiple constituency, or constituency satisfaction model, the organization
looks outward to its different constituencies or stakeholder groups and measures
its effectiveness in terms of the extent to which the needs of these different
constituencies are met. Based on this criteria, KPIS were categories in this model.
10
3. IMPORTANCE OF KEY PERFORMANCE INDICATOR IN THE LIBRARIES
Teague and Eilon (1973) determined three broad purposes of traditional view
performance measurement which is to ensure the achievement of goals and objectives, to
evaluate, control and improve procedures and processes and to compare and assess the
performance of different organizations, departments and individual. Similarly, (Bourne &
Bourne, 2002) identify three main purposes of KPIs such as to explain directional
organization goals, to concentrate the core business and to reevaluate the performance
target.
John Reh (1997) and Parmenter (2009) define that the purpose of KPIs is to clarify what
should be done in order to increase performance dramatically. In library, the
implementation of KPIs does not only act by evaluating and monitoring performance, but
also as a recognition of sufficiency in governances and service system that exists in
library. Meanwhile, (Advanced Performance Institute, 2011) describe the reason to
measure the performance of organizations is to reduce the simple homilies.
Source: The Advanced Performance Institute (2011)
Figure 4. Key Performance Indicator
11
4. INTERNATIONAL STANDARD INFORMATION AND DOCUMENTATION-LIBRARY PERFORMANCE INDICATORS (BS ISO 11620:2008) ISO Standards are concerned with the evaluation of all types of libraries (International
Standard ISO 11620, 2005). The main purpose of this standard is to endorse the use of
performance indicators in libraries and to spread knowledge about how to conduct
performance measurement. Libraries in the developing and developed countries can use
the performance indicators to be advanced and will benefit from the knowledge and skills
associated with formal planning procedures and data collection processes. This standard
follows the Balanced Scorecard approach to create the four major performance
measurements such as resources, access, and infrastructure, use, efficiency, potentials and
development as being presented in the Figure 6 below:
Sources: The International Organization for Standardization (BS ISO 11620:2008)
Figure 6. Library Performance Indicator Standard
Resources, Access, and Infrastructure
Key Performance
Indicator
Use
Efficiency
Potentials and Development
12
This standard helps libraries to identify the major areas for performance by details
explaining the content and type of performance indicator. The performance indicators are
further categorized along with the five service/resource areas of collection, access,
facilities, staff and general.
13
Table 1 Organization KPIs for Academic Libraries
KEY PERFORMANCE
INDICATORS
OPERATIONAL DEFINITION
METHOD KPIS
TARGETS
1. Customer Satisfaction Index (CSI) (Academicians, Non-academicians, Post graduates students, Under graduates)
To assess the degree to which users are satisfied with the library services as a whole or with different services of the library.
The percentage of achievements of Customer Satisfaction Index at the level of satisfaction 4 of 5 Likert Scale.
4 of 5
(70%)
2. Percentage of required titles in the collection (RTC)
To assess to what extent titles in demand by the users are owned by the library. The performance indicator is used to assess the fit of the collection to the requirements of the users.
RTC= A / B x 100 A = the number of required titles in the sample owned by the library B = the total number of required titles in the sample.
70%
3. Cost per Content Unit Downloaded (CCUD) -six most expensive databases
To assess the contractual cost of an electronic resource related to the number of content units downloaded
CCUD = A / B x 100 A = cost of each electronic resource for a specified period B = number of content unit downloaded from each electronic resource during the same period
10% reduction of cost per download
14
The process to set the organization KPIs is the most crucial part by the top management
to determine which KPIs can show a high impact performance to the organization. Table
1 presents three KPIs examples from ISO Standard. This KPI was presented in the KPIs
Workshop among UPM’s Management in July 2011 at Pullaman, Putrajaya. Customer
satisfaction index is an area of general measurement, while percentage of required titles
in the collection and cost per content unit downloaded are areas of collection
measurement.
As shown in the table 1, each KPI was defined based on the area measurement,
operational definition and method while the KPI targets and baseline method were not
mentioned in this standard. It can be difficult for libraries to set the correct targets
initially. As shown in the Figure 7, Cullen (1998) proposed that evaluation of
organization includes inputs, processes, outputs and outcomes. Libraries should measure
the outputs of library for customers such as product and information services to indicate
the library performance for stakeholders.
Evaluation
feedback
inputs processes outputs outcomes
resources performance customers
(human/ financial etc)
measures service quality satisfaction
Source: Cullen (1998)
Figure 7. The Three Aspects of Evaluation
15
The outcome from KPI results is needed to be considered when setting organization
KPIs. The KPIs can help libraries to show multiple achievement outcomes if the libraries
are able to determine the right KPI from ISO Standard. For example, KPI for customer
satisfaction index as listed in the Table 1. The outcome of this KPIs is to help library to
improve any dissatisfaction faced by library customers.
The second objective is to measure the percentage of required titles in the collection, and
the outcome of this KPIs is to ensure that the library collection is relevant with the
university programs. Lastly, the third KPIs is to measure the cost per content unit
downloaded. The outcome of it is to ensure that the library is being utilized by the
customers, and they possess the information skills to make full use of the library online
database. Concisely, it is shown that the library online database is beneficial for the
library to continue with the subscription.
On the other hand, the input and processes such as resources, human, and financial can be
measured internally to determine the efficiency of the library operation. It is depicted that
the KPI results from each department in the libraries can be rolled up into and reflects the
total library performance and organization KPIs. In addition to that, libraries need to
consider the ability of organization when selecting KPIs from ISO Standard to ensure the
KPIs targets are achievable by the library team through study of the content and type of
KPIs. The decision to set the performance levels needs to be discussed clearly by the
management meeting or KPIs workshop among the librarians and supporting staff. The
cooperation and understanding about KPIs by all staffs are needed to ensure that the KPIs
can be implemented and be driven to what KPIs targets to be achieved by the top
management. To make KPI works effectively, the targets should be consulted by those
people who will need to achieve and benchmark with other libraries or previous
performances.
16
5. CONLUSION
The discussion in this paper will help libraries to understand and identify the right KPIs
in libraries and transform libraries towards excellence through the efficient and
effectiveness of core process in producing and delivering services, as well as ensuring
customers’ satisfaction with the services provided. The effectiveness of core process in
those services needs to be measured in terms of time and accuracy of the services. This
paper has provided the method to measure the effectiveness of the organization by the
number of output based on time requirement, while human resources productivity and
financial also need to be measured. KPIs also help libraries to reevaluate the whole
services to ensure the relevancy and effectiveness for customers parallel with the vision,
mission and objectives of government agencies (MAMPU, 2005). Measures, standard,
and KPIs are essential for the enhancement and library transformation towards
excellence. KPIs can also help libraries to distinguish the factors that will create value.
They can act as a catalyst for behavioural change and play an essential part in the short
and long-term planning of library (Murley, 1997). In order to increase the understanding
about KPIS among library staffs, more efforts are needed to ensure all of them will go
hand-in-hand in contributing to the process of achieving the KPIs.
17
REFERENCES
Advanced Performance Institute. (2011). Key performance indicators (KPIs) - explained:
Examples, reporting & case studies... Retrieved 5/1/2011, 2011, from
http://www.ap-institute.com/Key Performance Indicators.html
Bourne, M., & Bourne, P. (2002). Balanced scorecard in a week. London: Hodder &
Stoughton.
Chan, A. P. C., & Chan, A. P. L. (2004). Key performance indicators for measuring
construction success. Benchmarking: An International Journal, 11(2), 203-221.
Cullen, R. (1998). Measure for measure: A post modern critique of performance
measurement in libraries and information services. Iatul Conference Proceedings,
Fitz-Gibbon, C. T. (1990). Performance indicators. Clevedon, Avon, England;
Philadelphia: Multilingual Matters.
International standard ISO 11620. (2005). International standard. ISO 11620,
information and documentation = information et documentation. library
performance indicators = indicateurs de performance des bibliothèques. Geneva:
ISO Copyright Office.
John Reh, F. (1997). Key performance indicators – what are key performance indicators
or KPIS. Retrieved 3/19/2011, 2011, from
http://management.about.com/cs/generalmanagement/a/keyperfindic.htm
Krishnapillai, A. (2009). Understanding key performance indicators through driver
measures. (M.Sc., University of Ottawa (Canada)). , 115.
Pekeliling Kemajuan Pentadbiran Awam Bilangan 2 Tahun 2005.
Garis Panduan Bagi Mewujudkan Petunjuk-Petunjuk Prestasi Utama Atau Key
Performance Indicators (KPIS) Dan Melaksanakan Pengukuran Prestasi Di Agensi
18
Kerajaan, (2005). Retrieved from
http://docs.google.com/viewer?a=v&q=cache:0hbDCtU8Qw4J:e-maik.my
Ministry of Higher Education. (2011). University act. Retrieved 5/1/2011, 2011, from
http://www.portal.mohe.gov.my/portal/page/portal/ExtPortal/MOHE_MAIN_PAGE/
Information/University Act
Mohd Najib, T. R. (2009a). KPIS nasional diperkenal hujung bulan. Retrieved
3/19/2011, 2011, from
http://www.pmo.gov.my/?menu=newslist&news_id=239&news_cat=13&page=1731
&sort_year=&sort_month=
Mohd Najib, T. R. (2009b). Najib: KPIS for all government matters. Retrieved
4/30/2011, 2011, from
http://thestar.com.my/news/story.asp?file=/2009/6/20/nation/4162443&sec=nation
Murley, P. (1997). Gower handbook of customer service. Aldershot, Hampshire,
England; Brookfield, Vt.: Gower.
Parmenter, D. (2009). Key performance indicators (KPIS) developing, implementing, and
using winning KPIs.
Sanchez, H., & Robert, B. (2010). Measuring portfolio strategic performance using key
performance indicators. Project Management Journal, 41(5), 64-73.
Toor, S., & Ogunlana, S. O. (2010). Beyond the ‘iron triangle’: Stakeholder perception of
key performance indicators (KPIs) for large-scale public sector development
projects. International Journal of Project Management, 28(3), 228-236.