1 fy02 asa presentation provide library services presented by: susan whitmore chief, information...

1

FY02 ASA Presentation

Provide Library Services

Presented by:

Susan WhitmoreChief, Information & Education Services SectionNIH [email protected]

Office of Research ServicesNational Institutes of Health

18 November 2002

2

Table of Contents

Main PresentationASA Template ……………………………….…………………………….….4Customer Perspective……………………….…………………………….….5

Customer Segmentation …………………….…………………...…………..……5Customer Satisfaction……………………….…………………………………….12

Internal Business Process Perspective…………………………………….…….20Conclusions from Discrete Services Deployment Flowcharts……….…..……21Process Measures…………………………………………………………….…...22

Learning and Growth Perspective……………………………………………..….32Conclusions from Turnover, Sick Leave, Awards, EEO/ER/ADR Data….…..33Analysis of Readiness Conclusions…………………………………………..….34

Financial Perspective……………………………………………………..….36Unit Cost……………………………………………………………………….……36Asset Utilization……………………………………………………………….……39Unique Financial Measures..……………………………………………….……..43

Conclusions and Recommendations………………………………….…….50Conclusions from FY02 ASA..……………………………………………….……51Recommendations………………………………………………………….………52

3

Table of Contents

AppendixPage 2 of the ASA Template

Customer segmentation graphs

Customer satisfaction graphs

Block diagram

Process charts/Deployment flowcharts

Internal Business Performance Measures and Control Charts

Learning and Growth data table

Analysis of Readiness Information

Asset utilization data

Unit Cost of Information data

4

Rosalie Stroman, Lisa Wu, Ted Crump

Customer Intimacy Sustain

Susan Whitmore

Service Strategy

Team Members

DS1: Provide Custom Research Assistance

DS2: Provide Copies of Publications

DS3: Translate Documents

Product Leadership Harvest

DS5: Provide Self-Service Copiers

ASA Template - 2002

Customer Value Proposition

Team Leader

Provide Library Services

DS4: Provide Primary Library Services

Discrete Services

Service Group

Support the biomedical and behavioral research needs of the NIH community with a comprehensive range of scientific, medical, and administrative information and support services.

Operational Excellence GrowthX X

5

Customer Perspective

--Customer Segmentation

6

Telephone Survey • Telephone survey of 400 randomly chosen Library users

conducted in 2000 and 2002

• Included questions about primary library services, providing copies of publications, and self-service photocopy service

• Customer segmentation data reveals customer’s primary work and position

• Most respondents engaged in research

• Most respondents identified themselves as post doctoral fellow, staff scientist, administrator, senior investigator, or research fellow

7

How Telephone Survey Respondents Spend Their Time

0

10

20

30

40

50

60

Resea

rch

Admin

Clinica

l/Res

Admin/R

es

Clinica

l

Other

Clinica

l/Adm

in

All 3 ar

eas

Type of Work

% o

f 40

0 R

esp

on

den

ts

FY00

FY02

8

Positions Held by Telephone Respondents

0

5

10

15

20

25

Oth

er

Post d

oc fe

llow

Staff

scien

tist

Admins

trato

r

Senior

inve

stiga

tor

Resea

rch

fello

w

Resea

rch

assis

t

Staff

clini

cian

Teure

trac

k inv

estig

ator

Visitin

g sc

holar

Senior

staff

fello

w

Gra

d stud

ent

Clinica

l Fel

low

Senior

rese

arch

ass

ist

Science

writ

er

% o

f 40

0 R

esp

on

den

ts FY00

FY02

9

• Customer segmentation data by IC available for providing copies of publications, self-service copiers, translate documents and custom service

• Customers spread across all IC’s

• Heavier use of services by larger groups and IC’s engaged directly in biomedical research, ex., NCI, NIAD, NICHD, NINDS, NHLBI

Customer Segmentation

10

Customer Segmentation continued• Less use by smaller groups and IC’s whose work

is administrative or a support service, ex., OD, ORS, FIC, CIT

• The Library serves not only NIH but also FDA/CBER on the Bethesda campus and provides document delivery services to the National Institute of Standards and Technology (NIST), Centers for Disease Control (CDC), and NIEHS

• Customer segmentation data for self-service photocopy, translations, and custom services may be found in the Appendix

11

Customer Segmentation for Provide Copies of Publications

0

10,000

20,000

30,000

40,000

50,000

60,000

70,000

80,000

90,000

100,000

NCI

NIMH

CC

NICHD

NINDS

NIAID

NIDDK

NHLBI

CBERNID

ANIA

NIDCR

NIAAA

NHGRI

NIDCD

NEIO

D

NIAM

SNIN

R

NIGM

SNLM

ORS

NCCAMNCRR

FIC CITHHM

ICSR

NIEHS

NIBIB

NCMHD

Customers

Nu

mb

er o

f R

equ

ests

Fill

ed

FY01

FY02

12


--Customer Satisfaction

13

• The 2002 Telephone Survey revealed that 97% of NIH staff give a positive rating to Library services overall

• Customer satisfaction levels increased slightly in two years – in FY00 94% of surveyed staff gave a positive rating to Library services overall

• All services surveyed received a very positive rating

Customer Satisfaction

14

Percentage of NIH Staff Who Gave a Positive Rating to the Quality of Library Services in FY02

97 97 95 95 95 95 93 93 93 92 89 89 86 83

0

20

40

60

80

100

Ove

rall Q

uality

Datab

ase s

earc

h

Self-s

erv

copy

E-Doc

Req

uest

PDF Del

ivery

Assist

ance

Web

site

Web

of S

cienc

e

Oth

er d

atab

ases

Onli

ne C

atal

og

Resea

rch

updat

es

Web

deliv

ery

Inst

ructi

on

Books

15

Customer Satisfaction for Provide Translations

• Thirty-three percent response rate for evaluations

• Both product and customer service ratings exceeded the ORS average ratings

• Many customers commented that the translation service is excellent

• Timeliness was more important to the customers than cost

• Customers were given realistic expectations with regard to turnaround time but some would like documents more quickly

16

Customer Satisfaction for Translation Product

ORS Index = 8.27

8.40

8.28

8.48

7.42

1.00

4.00

7.00

10.00Cost

Quality

Timeliness

Reliability

Data based on 436 respondents

Service Group Index = 9.35

9.55

8.89

9.62

9.14

1.00

4.00

7.00

10.00Cost

Quality

Timeliness

Reliability


17

Customer Satisfaction for Translation Customer Service

ORS Index = 8.55

8.60

8.58 8.54

8.518.51

1.00

4.00

7.00

10.00Availability

Responsiveness

ConvenienceCompetence

Handling ofProblems


Service Group Index = 9.55

9.599.48

9.629.62

9.44

1.00

4.00

7.00

10.00Availability

Responsiveness

ConvenienceCompetence

Handling ofProblems


18

Unique Customer Measures – Maintain Customer Satisfaction for Seminars

• Seminars are provided to teach NIH staff how to effectively use electronic resources and bibliographic management software

• 77% of attendees (417/544) completed an evaluation form designed to assess satisfaction with the training classes

• Choices were 5=excellent, 4=very good, 3=good, 2=fair, 1=poor

• Control charts indicate no signals (Control charts may be found in the appendix)

19

Seminar Evaluation Results for FY02

0.00.51.01.52.02.53.03.54.04.55.0

Avera

ge R

esp

on

se (

5 p

t. s

cale

)

20

Internal Business Process Perspective

--Deployment Flowcharts

21

• Our Service Group completed deployment flowcharts for all five discrete services (which may be found in the appendix)

• Most discrete services are complex so very little detail could be indicated on the flowcharts

• Nothing new was learned from completing the flowcharts

Conclusions from Discrete Services Deployment Flowcharts

22

Internal Business Process

--Measures for each Discrete Service

23

DS1: Custom Services

• Objective: Increase number of custom service projects

• Measure: Percent increase in number of contracts established each year

• Results: Baseline data for FY02 – 17 projects

24

DS2: Provide Copies of Publications• Objective: Maintain or reduce turn around time for

filling document requests

• Measure: Percent of requests filled in 3 business days

• Goal: Fill 96% of requests in 3 business days

• Results: Average fill rate of 97% for FY02 is above the goal. Goal was not met in August due to two staff members on maternity leave and other staff on vacation

Control chart may be found in the Appendix

25

DS3: Translate Documents• Objective: Maintain or decrease the turn around time for translating

documents

• Measure: Percent of documents translated in 3 business days, protocols in 10 business days, articles in 30 business days

• Results: On average 55% of documents, 52% of protocols, and 47% of articles were translated in the required time period

• Priority is given to translating documents which include diplomas, transcripts, and personal documents which the FIC needs for appointments of research fellows. Protocols are primarily from English to another language and must be contracted out and reviewed in-house.

• The following control chart for document translations shows a signal for July when the translators were on vacation.

Other control charts are in the Appendix

26

Translation Requests Control Chart for Documents

Note: Percentage of documents translated within goal of 3 business days is depicted by month

FALSE

if 2/3 or more of the moving ranges are below mean moving rangeFALSE

Moving Range: FY02 Documents

0

20

40

60

80

Month

Mo

vin

g R

an

ge

(m

R)

Moving Range 7.00 25.00 19.00 20.00 21.00 7.00 14.00 8.00 57.00 36.00 24.00

Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep

UCL = .59.2

mR = 18.1

Individual Values: FY02 Documents

0

20

40

60

80

100

Month

Ind

ivid

ua

l Va

lue

s (

X)

% Meeting Goal 59.00 52.00 77.00 58.00 78.00 57.00 64.00 50.00 58.00 1.00 37.00 13.00

Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep

Mean = 54.8

LCL = 6.7

UCL = 100

Signal

27

However Translation Requests Are Increasing• The number of words translated

has increased 35% over the past year

• Translations for languages outside translator’s areas have always been contracted out. To keep up with demand, increasingly translations for in-house language capability are contracted out

• Contracted out translations take more time. Next year, the time it takes to complete a translation by in-house staff versus a contractor will be tracked separately

• If enough translation requests for one or several related languages are consistently received, this would justify hiring another translator

Translation Requests FY97-FY02

0

100200

300400

500600

700800

900

FY97FY98

FY99FY00

FY01FY02

Fiscal YearN

o. o

f R

equ

ests

Requests Filled In-House

Requests Filled by ContractorsTotal Requests Filled

28

DS4: Provide Primary Library Services• Objective: Provide 100% of

current journals in electronic format

• Measure: Percent of current journals subscribed to electronically

• Once almost all (93%) of the most used 350 journals were in electronic format, we changed out goal to have all journals in e-format. Currently, 75% of all journals are in electronic format

Percent of Journals in Electronic Format

0%

20%

40%

60%

80%

100%

FY01 FY02

Fiscal YearP

erc

en

tag

e

29

Additional Internal Business Measures for Provide Library Service located in the Appendix

--E-license agreements to include all of NIH

--Ready access to training

--Quick response to requests for tutorials

30


• Objective: Provide adequate number of photocopy machines for users

• Measure: Ratio of photocopy machines to visits

• Six copy machines were made available for customer throughout FY02. There is no waiting for a machine, therefore, the number of copiers is adequate

31

However Self-Service Use is Declining• Use of self-service

copying continues to decline as the Library obtains additional e-resources

• Hours of service were reduced in FY01 due to declining use

• The number of copiers and hours of service will be evaluated periodically and reduced as warranted

Visits to the Self-Service Copy Service in FY02

0

20

40

60

80

100

120

140

Oct

Dec

Feb

Apr Jun

Aug

Month

No

. of

visi

ts

32

Learning and Growth Perspective

33

• An average of 4.5 days sick leave per employee is above average for ORS and reflects the three staff who were on maternity leave in FY02

• Eight awards for every 10 employees is slightly below average for ORS. Two factors limit the number of awards given per person:

• The AFGE Union contract mandates that 70% of our award pool be given as performance awards

• As a team-based organization, we have chosen to use the other 30% of the award pool to reward teams.

• The turnover and EEO/ER/ADR data appear fairly average for ORS

Conclusions from Turnover, Sick Leave, Awards, EEO/ER/ADR Data

34

• The way in which information is being provided and accessed by customers is changing:

• Customers are increasingly using e-resources that they access from their desktop computers

• Customers are answering many of their own easier information questions

• Customers are increasingly accessing our services through our web site

• The services that aid customers in information identification, acquisition, and management are changing:

• Library staff need to provide services electronically such as the digital reference service

• Library staff need to reach out to customers in their offices and labs as the Informationists do

• Services that make it possible for users to search across a number of e-resources and link directly to online journals are needed such as MetaLib and SFX

• The Library’s web site continues to grow and be essential to service provision. The content needs to be dynamic and up-to-date

Analysis of Readiness - Conclusions

35

Analysis of Readiness – Conclusions -continued• Library staff need to be retrained to engage in new tasks

• Librarians need to learn in-depth searching techniques for animal alternatives and evidence-based medicine. They need biomedical subject expertise to offer customized services to researcher and clinicians in their offices and labs

• Library Technicians must learn new skills sets as the old work diminishes and new work is created

• The Library is identifying training programs and/or developing in-house programs to address the new skills that are needed by staff

• We are very aware of the changes in information resources and services and are well poised to take advantage of these changes

A more thorough analysis of the Library’s readiness may be found in the Appendix

36

Financial Perspective

--Unit Cost Measures

37

Unit Cost• The average unit cost has

decreased over the past two years for:

• Copies of publications (-2%/yr)

• Self-service copiers (-14%/yr)

• As many users are accessing information through the Library’s e-resources, these services have declined in use and we anticipate that they will continue to decline in use

• The cost of these services is primarily based on salaries, supplies and equipment. As the use of the service declines, the unit cost of the service will decline also

Unit Cost of Prividing Copies of Publications and

Self-Service Copiers

$0

$1

$2

$3

$4

$5

$6

FY01 FY02 FY03

Fiscal Year

Co

st

in D

olla

rs

Provide Copies of Publications

Provide Self-Service Copiers

38

Unit Cost continued• The average unit cost has

increased over the past two years for:

• Translate documents (+13%/yr) Primary library services (+7%/yr)

• Custom research assistance (+15%/yr)

• These services are growing. Costs are primarily based on salaries and purchase of print and electronic resources

• The cost of print and electronic resources rises 10-20% each year. The Library is increasing the amount spent on e-resources by 20% each year

Unit Cost of Primary Library, Translation, and Custom

Services

$0

$100

$200

$300

$400

$500

$600

FY01 FY02 FY03

Fiscal YearC

ost

in D

olla

rs

Provide Custom Research Assist

Translate Documents

Provide Primary Library Serv

39


--Asset Utilization Measures

40

Asset Utilization for Discrete Services• DS1: Provide Custom Research Assistance – no data

• DS2: Provide Copies of Publications – 98%

• DS3: Translate Documents – 74%

• DS4: Provide Primary Library Services – Instruction Program – 74%

• DS5: Provide Self-Service Copiers – 63%

Descriptions of how the asset utilization percentages were calculated may be found in the Appendix and on the following slide

41

DS4: Provide Primary Library Services• Primary library services are composed of instruction, reference,

mediated searching, and the print and electronic collection

• Asset utilization data for part of the instruction program, seminars held in the Library on Thursdays, was gathered

• The maximum number of potential class attendees were compared to the actual number of attendees

• 74% asset utilization = No. of actual class attendees (544)/No. of potential attendees (741) X 100

• Promotion of the seminars early in the year increased attendance and we increased the number of workstations from six to 20 during the summer

42

Instruction Program Asset UtilizationAsset Utilization: Percent Deviation from Plan for Classroom

Instruction Program FY02

-60%

-50%

-40%

-30%

-20%

-10%

0%

10%

20%

30%

40%

50%

Month

% D

evia

tio

n b

y M

on

th (

Bar

)

-60%

-50%

-40%

-30%

-20%

-10%

0%

10%

20%

30%

40%

50%

Cu

mu

lati

ve %

Dev

iati

on

(L

ine)

Deviation (Mon) -53% -44% -50% 11% 47% -40% -30% -33% 3% -10% -33% -58%

Deviation (Cum) -53% -48% -49% -33% -21% -23% -24% -25% -22% -20% -22% -26%

Oct Nov Dec J an Feb Mar Apr May J un J ul Aug Sep

43

Unique Financial Measures

--Unit Cost of Information

--Percent of Library Budget Devoted to E-resources

--Minimize Training Costs

44

Units of Information• Use of information units has

increased 26% in two years

• A unit of information may be a search, a document supplied, a customer trained, an article downloaded, or a visit to our web site

• Because e-resources are heavily used, more information is available to customers at a reduced cost per unit of information

Units of Information Provided

0

2

4

6

8

10

FY00 FY01 FY02

Fiscal Year

Un

its

in M

illio

ns

45

Unit Cost of Information• Objective: Maintain or reduce the

cost of information provide to NIH staff

• Measure: Change in unit cost of information

• The need to include rent and overhead in the FY01 budget increased the unit cost of information

• E-resources are more expensive than print resources; however, they are used more heavily. Thus the unit cost of information for FY02 is the same as for FY00

• See the Appendix for the definition of how the unit cost is determined

Unit Cost of Information

$0.91$1.03

$0.91

$0.00

$0.50

$1.00

$1.50

FY00 FY01 FY02

Fiscal YearU

nit

Co

st in

Do

llars

46

Percent of Library Budget Devoted to E-resources

• Objective: Increase availability of highly used e-resources

• Measure: Percent of library budget devoted to electronic versus print resources

• Result: The percent of the library budget devoted to electronic resources has increased 1,600% in the past six years. Percent of budget devoted to e-resources has risen 21% in the past year

47

Percent of Library Collection Budget Devoted to Electronic Resources

0

20

40

60

80

100

FY97 FY98 FY99 FY00 FY01 FY02

Fiscal Year

Per

cen

tag

e

48

Electronic vs. Print Journals

• The percentage of journals in electronic format has increased 29%

• However, many resources are received as print + electronic and these resources are still reflected in the print budget

Percentage of Journals in Print and Electronic

Format

0%

25%

50%

75%

100%

FY01 FY02

Fiscal Year

Per

cen

tag

eE-JournalsPrint + ElectronicPrint Only

49

Minimize Training Costs• Objective: Minimize training

costs by providing users with alternate means of training

• Measure: Percent of people trained in seminars or via tutorials versus the number of people using web-based training

• Result: Customers have clearly indicated their preference for web-based training which is available at any time

Number of NIH Staff Trained in Seminars &

Tutorials vs. Web-based Training

1152 9041219

525

4031

0

1,000

2,000

3,000

4,000

5,000

6,000

FY00 FY01 FY02

No.

Sta

ffWeb-based Training

Seminars & Tutorials

50

Conclusions and Recommendations

51

Conclusions from FY02 ASA• Our customers are very satisfied with the Library’s resources,

services, and staff

• Internal business processes are for the most part efficient and cost effective

• There will be a growing number of electronic resources and services and more customized services in the future and there need to be concomitant changes in staff skill sets

• Use of resources and services, particularly e-resources, continues to grow. We will continue to convert from a primarily print to an electronic library. While electronic resources are expensive, they are utilized much more heavily than the print collection thereby keeping the unit cost of information reasonable

52

Recommendations - Specific• Develop means to determine utilization of assets for

Provide Custom Research Assistance by March 2003

• Develop an evaluation form for Provide Custom Research Assistance projects by February 2003. Work with National Library of Medicine staff to evaluate Informationist program in 2003 and 2004

• Continue to collect data for Internal Business Measures for Provide Self-Service Copiers and construct a control chart for the ASA in November 2004

• Separately collect data for the time it takes to complete translations by in-house staff versus contractors to determine how service speed can be improved

53

• Continue with the redesign of the web site as recommended in last year’s ASA. Projected date for roll out January 15, 2003.

• Implement content management software for web site to ensure up-to-date and dynamic content. Build the databases needed to provide content. Projected date for completion January 15, 2003.

• Incorporate software called MetaLib and SFX into our web site This software will enable users to search across e-resources and link directly from databases to online journals. This recommendation partially replaces a similar one from last year’s ASA. Implementation began September 2002. Completion of initial phase of implementation March 2003.

• Develop an in-house training curriculum to ensure all staff are knowledgeable about Library services and to retrain staff for new work. Development will begin in January 2003. Full implementation will take two years.

Recommendations - Broad

54

Appendix

55

ASA Template – Page 2







Performance Measure

Customer segmentation by IC

Results from 2002 Telephone Survey




Type of position held at NIH from Telephone Survey


Customer satisfaction ratings from the ORS Customer Scorecard





Performance Objective

Increase understanding of customer base

Increase customer satisfaction




Results from class evaluations

DS4: Acquire electronic and print resources needed by users Percent of user electronic and print requests acquired

DS4: Maintain high user satisfaction

56

Percent of requests filled in 3 business days.

Percent of documents translated in 3 business days, protocols in 10 business days, articles in 30 business days

Process map of Discrete Service





Increase understanding of processes.


Internal Business Process Perspective

DS1: Provide Custom Research Assistance Process map of Discrete Service

Performance Objective Performance Measure

DS5: Provide adequate number of photocopy machines for users.

DS4: Respond to tutorial requests within one business day.

DS2: Maintain or reduce turn around time for filling document requestsDS3: Maintain or decrease the turn around time for translating documents

DS1: Increase the number of custom service projects



Percent increase in number of contracts established each year.

Percent of current journals subscribed to electronically.

Percent of requestors contacted within one business day.

DS4: Balance need for web-based training with hands-on classes.

DS4: Negotiate e-license agreements that include all of NIH.

Percent increase of license agreements that include all of NIH.

DS4: Provide 100% of current journals in electronic format.

Percent of courses that are web-based.

Ratio of photocopy machines to visits (average daily visits).

57

Awards/Recognition

Contacts/Complaints with EEO/ER/ADR

Sick Leave Usage

Turnover

DS4: Provide Primary Library Services - Maintain or reduce the cost of information provided to NIH staff

Maximize utilization of assets.


Enhance quality of work life for employees in ORS.


Maintain & enhance competencies for the future organization.




Change in Unit Cost

Minimize unit cost at a defined service level.

Performance Measure

Performance Measure

Analysis of Readiness Index


Change in Unit Cost

Change in Unit Cost

Change in Unit Cost




Change in the unit cost of information

Actual assets utilized/planned asset utilization


Percent of library budget devoted to electronic versus print resources.



Change in Unit Cost

DS4: Minimize training costs by providing users with alternate means of training.

Percent of people trained in seminars or via tutorials versus the number of people using web-based training.




DS4: Increase availability of highly used e-resources

DS4: Provide Primary Library Services Actual assets utilized/planned asset utilization


58

Customer Segmentation Graphs

--Provide Self-Service Photocopy

--Translate Documents

---Provide Custom Services

59

Provide Self-Service PhotocopyCustomer Segmentation

0

1,000

2,000

3,000

4,000

5,000

6,000

7,000

8,000

9,000

Customers

Nu

mb

er

of

Vis

its

FY01

FY02

60

Translate Documents


0

20

40

60

80

100

120

140

160

180

Customers

Job

s C

om

ple

ted

FY00 FY01 FY02

61

Custom Services FY02


0

1

2

3

4

5

NC

I

NIA

ID

NIN

DS

OR

S

NIC

HD

NIM

H

NID

DK

NID

A

NE

I

NIE

HS

CD

C

OR

I/OP

HS

NIS

T

Customers

Nu

mb

er o

f Jo

bs

62

Customer Satisfaction Graphs

--Translate Documents-- Seminar Evaluation Control Charts for

FY02 – no signals--Unique Customer Measure – Acquire

Electronic and Print Resources Need by Users

63

Translate Documents Scatter Diagram

5.00

5.50

6.00

6.50

7.00

7.50

8.00

8.50

9.00

9.50

10.00

5.00 5.50 6.00 6.50 7.00 7.50 8.00 8.50 9.00 9.50 10.00

Satisfaction

Imp

ort

an

ce

SATISFIED,IMPORTANT

Cost

Convenience

Responsiveness

Availability

Handling of Problems

Competence

Quality

Timeliness

Reliability


64

Seminar Evaluation Control Chart - Clarity of instructor’s presentation

FALSE


Moving Range: FY02 Presentation

0.00

0.20

0.40

0.60

0.80

1.00

Month

Mo

vin

g R

an

ge

(m

R)

Moving Range 0.07 0.14 0.37 0.43 0.10 0.38 0.33 0.19 0.08 0.16 0.42


UCL = .79

mR = .24

Individual Values: FY02 Presentation

3.00

3.50

4.00

4.50

5.00

Month

Ind

ivid

ua

l Va

lue

s (

X)

Mean Score 4.63 4.56 4.42 4.05 4.48 4.38 4.00 4.33 4.14 4.22 4.06 4.48


Mean = 4.3

LCL = 3.67

UCL = 4.96

Note: Monthly mean ratings are depicted based on a rating scale ranging from (1) Poor to (5) Excellent.

65

Seminar Evaluation Control Chart – Instructor’s knowledge of subject

Individual Values: FY02 Knowledge

3.00

3.50

4.00

4.50

5.00

Month

Ind

ivid

ua

l Va

lue

s (

X)

Mean Score 4.56 4.56 4.50 4.13 4.59 4.46 4.24 4.63 4.41 4.34 4.11 4.70


Mean = 4.4

LCL = 3.7

UCL = 5.0

FALSE


Moving Range: FY02 Knowledge

0.00

0.20

0.40

0.60

0.80

1.00

Month

Mo

vin

g R

an

ge

(m

R)

Moving Range 0.00 0.06 0.37 0.43 0.13 0.22 0.39 0.22 0.07 0.23 0.59


UCL = .85

medmR = .22


66

Seminar Evaluation Control Chart – Instructor’s responsiveness to questions

FALSE


Moving Range: FY02 Responsiveness

0.00

0.20

0.40

0.60

0.80

Month

Mo

vin

g R

an

ge

(m

R)

Moving Range 0.15 0.24 0.30 0.25 0.13 0.19 0.20 0.22 0.05 0.01 0.26


UCL = .59

mR = .18

Individual Values: FY02 Responsiveness

4.00

4.20

4.40

4.60

4.80

5.00

Month

Ind

ivid

ua

l Va

lue

s (

X)

Mean Score 4.63 4.78 4.54 4.24 4.49 4.62 4.43 4.63 4.41 4.46 4.47 4.73


Mean = 4.5

LCL = 4.1

UCL = 5.0


67

Seminar Evaluation Control Chart – Quality of Handouts

m

FALSE


Moving Range: FY02 Handouts

0.00

0.20

0.40

0.60

0.80

1.00

Month

Mo

vin

g R

an

ge

(m

R)

Moving Range 0.06 0.28 0.43 0.30 0.19 0.12 0.08 0.24 0.19 0.16 0.36


UCL = .72

mR = .22

Individual Values: FY02 Handouts

3.00

3.50

4.00

4.50

5.00

Month

Ind

ivid

ua

l Va

lue

s (

X)

Mean Score 4.47 4.53 4.25 3.82 4.12 4.31 4.19 4.27 4.03 4.22 4.06 4.42


Mean = 4.2

LCL = 3.6

UCL = 4.8


68

Seminar Evaluation Control Chart – Level to which course objectives were met

FALSE


Moving Range: FY02 Objectives

0.00

0.20

0.40

0.60

0.80

1.00

1.20

Month

Mo

vin

g R

an

ge

(m

R)

Moving Range 0.13 0.19 0.40 0.23 0.38 0.46 0.30 0.08 0.16 0.50 0.73


UCL = .1.1

mR = .32

Individual Values: FY02 Objectives

3.00

3.50

4.00

4.50

5.00

Month

Ind

ivid

ua

l Va

lue

s (

X)

Mean Score 4.57 4.44 4.25 3.85 4.08 4.46 4.00 4.30 4.38 4.22 3.72 4.45


Mean = 4.2

LCL = 3.4

UCL = 5.0


69

Seminar Evaluation Control Chart – Quality of classroom arrangements

FALSE


Moving Range: FY02 Classroom

0.00

0.20

0.40

0.60

0.80

1.00

Month

Mo

vin

g R

an

ge

(m

R)

Moving Range 0.22 0.62 0.28 0.42 0.09 0.32 0.16 0.22 0.61 0.20 0.37


UCL = 1.0

mR = .31

Individual Values: FY02 Classroom

3.00

3.50

4.00

4.50

5.00

Month

Ind

ivid

ua

l Va

lue

s (

X)

Mean Score 4.44 4.66 4.04 3.76 4.18 4.27 3.95 4.11 3.89 4.50 4.30 4.67


Mean = 4.2

LCL = 3.4

UCL = 5.0


70

Unique Customer Measure – Acquire Electronic and Print Resources Needed by

Users

42%

58%

Acquired Not Acquired

• Acquired 42% of print and electronic resources requested by customers in FY02

• Reasons for not acquiring resources are:

• Low usage• Cost• Unavailable in electronic

format• Lack of suitable license

agreement

71

Internal Business Performance Measures

--Block diagram and process charts

72

Provide Custom ResearchAssistance

Provide Copies ofPublications

Translate Documents

Provide Primary LibraryServices

Provide Self-serviceCopiers

Customer makesrequest

Staff identifiedand task

performed

Product deliveredto customer

Customersubmits request

Obtain documentDeliver

publication tocustomer

Receive requestTranslatedocument

Delivertranslation to

customer

Customer hasinformation need

Library providesinformation

resources andservices

Photocopymaterials

Customerreceives needed

information

Customerpresents

materials to becopied and NIH

ID

Customerreceives

photocopy

Provide Library Services Block Diagram

73

LibraryAdministrative

Officer

Custom Services Deployment Flowchart

AccountsPayable

Institute/CenterAdministrative

OfficerORS Budget

CustomService

DIRSAdministrative

OfficerCustomer

Staff SelectedFor Project

Custom ServiceRequested

ProposalDeveloped

ProposalAccepted

No

ContactTerminated

Proposal sentto AO

CustomService

Performed

ProductPrepared for

customer

Customerreceives product

AO initiatesInteragencyAgreement

(IAG)

AO obtainsrequired

signatures

IAGprocessed

Paymentmade toLibrary

Yes

IC AO willing toexecute IAG

AO obtainssignatures

Yes

No AO obtainsrequired

signatures

IAG verifiedand

processed

AO obtainssignatures

IAG verified

74

Provide Copies of Publications Deployment Flowchart

CustomerNIH Library Staff atNational Library ofMedicine (NLM)

Other Libraries andCommercial Sources

NIH Library Staff

Receipt ofRequest for Copy

of Publication

Request is forPublication Available at

NIH Library

No

Yes

Customer SubmitsRequest for Copy of

Publication

PublicationRemoved From

Shelves andTaken to Be

Scanned

Copy ofPublication

Scanned and SentElectronically or

By Mail toCustomer

Customer ReceivesPublication

Request is forPublication

Available at NLM

NIH Library StaffTakes Request toNLM to Process

Request sent toLibrary or

CommercialSource That Owns

Publication

PublicationRemoved From

Shelves andTaken to Be

Scanned

Copy ofPublication

Scanned and SentElectronically or byMail to Customer

Yes

Library orCommercial

Source SendsPublication to NIH

Library

Publication IsReceived and

Sent By Mail orCourier toCustomer

No

75

Translate Documents Deployment Flowchart

Translations staff ContractorLibrary Admistrative

Office StaffCustomer

IssueRequest

ReceiveRequest

ContractOut?

Yes

No

TranslateDocument

TranslateDocument

ReceiveTranslation

ReceiveTranslation

ReviewIn-House?

Yes

No

OutsideReview

by DifferentContractor

Accept?

In-HouseReview

Yes

No

MakeCorrections

CorrectIn-House?

Yes

No

FinalProcessing/

SendPaperwork

ReceivePaperwork

76

Provide Library Services Deployment Flowchart

CustomServices

Basic Library ServicesSelf-ServicePhotocopy

DocumentDelivery

TranslationsCustomers

Information RequestPresented

Does customerknow where to

go?

Access informationservices

Receive information orreferral

No

Yes Subroutine A:Request

Document

Subroutine B:Requst

Translation

Subroutine C:Request Custom

Service

Does Customerneed

photocopying?Yes

No

Subroutine D:PhotocopyDocument

Receive Training

Access print andelectronic resources

Does CustomerNeed Additional

Services?No

Yes

Yes

Yes

77

Provide Self -Service Copies Deployment Flowchart

Pay CopyingMachines

Self ServicePhotocopy Center IC Administrative OfficerCustomer

Customer RetrievesJournals and Goes to

Self Service PhotocopyCenter

CustomerPresents NIH IDand Journals to

Receptionist

NIH ID Valid

No

Customer GoesTo Copy Machine

Customer Goes toCirculation Deskfor Instructions

Customer Goes toAO for LibraryAuthorization

CirculationResolvesProblem

Yes

No

CustomerAuthorized

No

Yes

CustomerReceives Copy

A

Go ToA

Customer Goes toPay Copying

Machines

Yes

78

Internal Business Performance Measures

• --Provide Copies of Publications -Control Chart

79

Provide Copies of Publications -Control Chart

Note: Percentage of requests filled within goal of 3 business days is depicted by month. Goal = 96%

Individual Values: FY02 Requests

90

92

94

96

98

100

Month

Ind

ivid

ua

l Va

lue

s (

X)

% Meeting Goal 98.00 96.00 96.00 96.00 98.00 98.00 96.00 98.00 98.00 98.00 95.00 97.00


Mean = 97

LCL = 93.9

UCL = 100

Goal = 96

FALSE


Moving Range: FY02 Requests

0

1

2

3

4

5

Month

Mo

vin

g R

an

ge

(m

R)

Moving Range 2.00 0.00 0.00 2.00 0.00 2.00 2.00 0.00 0.00 3.00 2.00


UCL = .3.9

mR = 1.2

80

Additional Internal Business Performance Measures

Following are three additional Internal Business Performance Measures for DS4: Provide Primary Library Services

--Negotiate e-license agreements that include all of NIH

--Balance need for web-based training with hands-on training

--Respond to tutorial requests in one business day

81

DS4: E-Resource License Agreements

• Objective: Negotiate e-license agreements that include all of NIH

• Measure: Percent of license agreements that include all of NIH

• Results: At the end of FY02, 53% of e-resource license agreements included all of NIH – an increase in 8% over last year

E-Resource License Agreements

69102

8591

0%

25%

50%

75%

100%

FY01 FY02

Per

cent

Agr

eem

ents

Limited AccessAccess NIH-w ide

82

DS4: Web-based Training• Objective: Balance need for

web-based training with hands-on classes

• Measure: Percent of courses that are web-based

• Results: 37% of the 51 classes offered are web-based

• 61% of customers prefer web-based training but we will continue to provide hands-on classes as long as attendance indicates that customers want this type of training

Type of Training Class

2024

32

13

19

0

10

20

30

40

50

60

FY 00 FY 01 FY 02

Fi s c al Y ear

Web-based

Seminars and Tutorials

83

DS4: Tutorial Request Response

• Objective: Respond to tutorial requests within one business day

• Measure: Percent of requestors contacted within one business day

• Results: 100% of requestors contacted within one business day in FY02

84

Internal Business Perspective

--Control charts for Translate Documents showing percentage of protocols translated in 10 business days and articles in 30 business days – no signals

85

Translate Protocols in 10 Business Days

Note: Percentage of protocols translated within goal of 10 business days is depicted by month

FALSE


Moving Range: FY02 Protocols

0

20

40

60

80

100

Month

Mo

vin

g R

an

ge

(m

R)

Moving Range 43.00 80.00 30.00 21.00 38.00 34.00 0.00 50.00 0.00 50.00

1 2 3 4 5 6 7 8 9 10

UCL = 100

mR = 34.6

Individual Values: FY02 Protocols

0

20

40

60

80

100

Month

Ind

ivid

ua

l Va

lue

s (

X)

% Meeting Goal 57.00 100.00 20.00 50.00 71.00 33.00 67.00 67.00 17.00 17.00 67.00

Oct Nov Jan Feb Mar Apr May Jun Jul Aug Sep

Mean = 51.5

LCL = 0

UCL = 100

86

Translate Articles in 30 Business Days

Note: Percentage of articles translated within goal of 30 business days is depicted by month

FALSE


Moving Range: FY02 Articles

0

20

40

60

80

Month

Mo

vin

g R

an

ge

(m

R)

Moving Range 14.00 25.00 4.00 17.00 32.00 18.00 4.00 50.00 45.00 1.00 0.00


UCL = 65.7

medmR = 17

Individual Values: FY02 Articles

0

20

40

60

80

100

Month

Ind

ivid

ua

l Va

lue

s (

X)

% Meeting Goal 75.00 89.00 64.00 60.00 43.00 11.00 29.00 25.00 75.00 30.00 29.00 29.00


Mean = 46.6

LCL = 0

UCL = 100

87

Internal Business Perspective

--Provide Self-Service Copiers

88

DS5: Provide Self-Service Copiers• Ratio of photocopy

machines to visits is shown on the accompanying chart

• There were six copiers available in FY02

• Customers do not have to wait for a copy machine, therefore, the number of copiers is adequate

Ratio of Photocopy Machines to Visits FY02

0

20

40

60

80

100

120

140

0 5 10

Number of CopiersA

vera

ge

Dai

ly V

isit

s

89


--Learning and Growth Data Table

90

Learning and Growth Data Table

About 4.5 days sick leave per employee

About eight awards for every 10 employees

5% employee turnover

1 ER case out of 50 employees

1 ADR case out of 50 employees

Pop

ulat

ion

Est

imat

e

No.

of S

epar

atio

ns

Turn

over

Rat

eTo

tal H

ours

of S

ick

Leav

e U

sed

Ave

rage

Hou

rs o

f

Sic

k Le

ave

Use

d

Num

ber

of A

war

ds

Rec

eive

dA

vera

ge N

umbe

r of

Aw

ards

Rec

eive

d

Num

ber

of E

EO

Com

plai

nts

Ave

rage

Num

ber

of

EE

O C

ompl

aint

s

Num

ber

of E

R C

ases

Ave

rage

Num

ber

of

ER

Cas

esN

umbe

r of

AD

R

Cas

esA

vera

ge N

umbe

r of

AD

R C

ases

575 11 0 0.00 360 34 15 1.41 0 0.00 0 0.00 0 0.005752 13 1 0.08 359 27 11 0.83 0 0.00 0 0.00 0 0.005753 8 1 0.12 220 26 3 0.36 0 0.00 0 0.00 0 0.005754 18 0 0.00 864 49 12 0.68 0 0.00 1 0.06 1 0.06

Service Group 29

Total50 2 0.04 1803 36 41 0.82 0 0.00 1 0.02 1 0.02

91


--Analysis of Readiness Information

92

Several changes in service and resources in the Library will require staff with a different mix of skills

• Fewer document requests will be received as more e-resources are made available and there will be less work for the document delivery staff

• Implementation of NED results in less work for circulation staff

• More books and journals will be obtained in e-format which will require less clerical support

• Customers are answering many of their reference questions using e-resources and the types of reference questions we receive at the reference desk are less difficult and can be answered by a well trained paraprofessional. Information services need to be customized to users specific needs. Staff need to be trained to fully use e-resources effectively.

93

Several changes in service and resources in the Library will require staff with a different mix of skills - continued

• Customers are using the Library’s resources through their computers instead of walking into the Library. Library staff need to offer services that reach out to NIH staff in their labs and offices as the Informationist service does

• Librarians will be performing more custom services which will require more knowledge of biomedicine. Librarians will be asked to conduct more complicated database searches for animal alternatives and evidence-based medicine

• More translations are being requested

• Systems office staff will be needed to develop and program databases to track and make e-resources available and accessible

94

Changes in the way information is accessed will require a change in the way information is provided

• Customers are using the Library’s resources through their computers instead of walking into the Library. More services, such as reference service and instruction, need to be offered to staff electronically

• The growth in the number of e-resources requires the acquisition of software and systems that link databases to online journals and that enable customers to search efficiently across resources

• Users are requesting additional e-resources which are often more costly than print resources

• Specialized databases need to be created to organize and track e-resources and to populate the web site

95

Changes in service and resources will require that:

• Library technicians working in document delivery, circulation, and collection organization & maintenance will have less work and will need to develop new skill sets in order to remain employed in the Library

• Library technicians trained to respond to customer requests at the information desk will free the professional librarians to provide custom services

• The Library develop training programs to ensure that staff develop needed skills

• Librarians take classes from outside sources to develop more in-depth skills in database searching and knowledge of biomedicine

• Systems Office staff take training in database management and programming

• Content management software be implemented for the web site. This software will make it easier to update and maintain web site and free librarians for other activities

96

To ensure that staff are ready to take on new assignments, the Library is:

• Continuing to develop as a “Learning Organization”• Developing in-house programs such as the Reference Assistants

training program which is designed to train document delivery and circulation staff to work at the information desk

• Developing a training program for Informationists who will reach out to NIH researchers and clinicians to supply customized information assistance

• Appointing a Training Coordinator to organize training programs and develop ways to evaluate effectiveness of training

• Identifying courses and training programs that will prepare staff for new work

• Using COTA’s as appropriate to develop in-house talent• Continuing to use cross-functional teams to enable staff to learn

new skills and take on new assignments• Monitoring translation requests and hiring more staff with the

appropriate language skills as the situation warrants

97

Implications of Not Obtaining Right Mix of Skills or Materials

• The way in which information is being delivered has changed. If the Library is not able to provide e-resources to customers, the customers will be impeded in their work and ultimately will go elsewhere for the needed information

• Library technician work is decreasing, staff who are not retrained for new work will no longer be employed by the Library

• Custom services that reach out to customers to solve their unique information needs will be in demand. Professional librarians need to be trained to provide these services and need to be freed from lower level tasks so that they have time to provide them. If the Library does not provide these new services, customers will hire contractors to do the work.

• If professional librarians continue to provide services that can be transferred to well trained library technicians, the Library will not be operating in a cost efficient manner

98


Asset Utilization--Provide Custom Research Assistance--Provide Copies of Publications--Translate Documents--Provide Self-Service Copiers

99


• Asset utilization data has not been collected for this service

• We will research how to collect this data for next year

100

DS2: Provide Copies of Publications• The maximum number of hours that could have been

spent providing publications was compared to the actual number of hours spent on this service in FY02

• Included are government and contract staff hours

• 98% asset utilization = 37,435 actual hours/38,215 potential X 100

• In FY03 we plan to keep a log to sample the amount of time government and contract staff spend on this discrete service

101

DS3: Translate Documents• The maximum number of hours that the translators could ideally

devote to translating were compared to the number of hours they actually spent translating

• 74% asset utilization = translators spent 2,723 hours translating/3,680 hours that ideally could be spent translating X 100

• Hours not spent translating are spent attending meetings, contracting out translations, doing clerical work, and developing an ORS/ASA report

• In June, the Library’s labor contract was used to assume much of the clerical work and reduce the clerical work burden on the translators

• An asset utilization chart follows

102

Translation Asset Utilization GraphAsset Utilization FY 2002: % Deviation from Maximum Capacity

for Translate Documents

-60%

-50%

-40%

-30%

-20%

-10%

0%

Month

% D

evia

tion

by M

onth

(B

ar)

-60%

-50%

-40%

-30%

-20%

-10%

0%

Cum

ulat

ive

% D

evia

tion

(Lin

e)

Deviation (month) -4% -37% -39% -25% -21% -28% -19% -20% -52% -23% -9% -34%

Deviation (cumulative) -4% -21% -27% -26% -25% -26% -25% -24% -27% -27% -25% -26%

Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sept

Labor contract use begins

103

DS5: Provide Self-Service Copiers• The actual number of copies made on the six copies in the self-

service copy center were compared to the number of copies that could have been made, based on the normal rate at which customers make photocopies and the total hours the center is open

• 63% asset utilization = 1,498,192/2,384,100 X 100

• As shown in the internal business perspective, use of the self-service copy center is declining as customers use the e-resources

• Use of the self-service copy center will be monitored and hours reduced or the number of copy machines reduced as warranted

104


--Unit Cost of Information Data

105

Units of Information FY00 FY01 FY02Information requests 44,656 40,558 40,558 Searches performed 3,017 892 849 NIH staff trained 1,335 1,454 5,441 Books circulated/renewed 45,294 40,233 43,966 Documents supplied from collection 327,405 289,863 255,194 Home Page visits 646,550 842,856 1,055,001 Web of Science searches 4,495,507 5,036,855 5,137,149 Psychological Abstracts searches 9,277 10,020 12,097 Biological Abstracts searches 4,867 5,930 2,637 Ovid Medline searches 8,905 11,772 25,468 Other Ovid database searches n/a n/a 4,201 Articles downloaded from Amer Soc for Microbiology journals n/a n/a 319,614 Articles downloaded from Cell Press journals n/a n/a 105,678 Articles downloaded from JBC 166,476 188,544 225,580 Articles downloaded from Nature Journals n/a n/a 405,658 Articles downloaded from PNAS 94,725 125,380 149,544 Articles downloaded from Science 129,971 152,302 207,012 Articles downloaded from Elsevier/Academic Press 219,743 529,894 604,401 TOTALS 6,197,728 7,276,553 8,600,048

Primary Services Budget FY00* FY01** FY025,644,550 7,466,190 7,831,358

Unit cost of information FY00* FY01 FY020.91$ 1.03$ 0.91$

* FY00 budget did not include rent and overhead

** In FY01 NIHLB assumed added e-resources costs for Baltimore , Frederick and RML sites, plus paid rent and overhead

106

The End