calculating roi in special libraries and information centers: consequences of not having an...

Calculating ROI in Special Libraries and Information Centers: Consequences of NOT Having an Information

Center

José-Marie Griffiths, Ph.D.Sarah E. Aerni

University of North Carolina at Chapel Hill

June 2, 2007Special Library Association

Annual Conference

Topics Covered in Workshop

• Return-on-Investment (ROI) defined

• Two examples of ROI projects– ROI of entire special library– ROI of access to journal collections:

print and electronic

• ROI methods

• Communicating results

Role of Evaluation

• Planning

• Resource allocation and management

• Justification of new or existing resources

• Advocacy, marketing and public relations

Conceptual Framework for

Evaluation

Object of Evaluation

• Library

• Function

• Service

• Activity

• Resource

User Organization Industry/ Sector Society

Library

Measurement PerspectivesMeasurement Perspectives

Conceptual Framework for Library Economic Measures

Industry/ Sector

Inputs (Resources) Amount Cost Attributes


Specific MeasuresSpecific Measures



Library

Inputs (Resources) Amount Cost Attributes

Outputs (Products/Services) Amount Attributes Quality Timeliness Availability Accessibility





Library

Inputs (Resources)

Outputs (Products/Services)

Usage (Use & Nonuse) Amount Factors affecting use/nonuse Ease/cost of use (price paid) Available alternatives Purpose of use Importance of and satisfaction with attributes of output Awareness





Library

Inputs (Resources)


Usage (Use & Nonuse)

Outcomes (Consequences of Use and Nonuse) Time saved Improved learning Improved productivity Improved quality of work Improved timeliness of work Value derived Effects on organization goals Higher order effects





Library

Inputs (Resources)



Outcomes (Consequences of Information)

Domain(Environmental Characteristics) Target population User/nonuser population characteristics User/nonuser needs/requirements Externalities





Library

Derived MeasuresDerived MeasuresPerformance




Inputs (Resources)




Domain

Library User Industry/ Sector

Organization Society

Derived MeasuresDerived MeasuresPerformance Effectiveness




Inputs (Resources)




Domain






Cost-Effectiveness


Inputs (Resources)




Domain






Cost-Effectiveness Impact


Inputs (Resources)




Domain






Cost-Effectiveness Impact

Cost/Benefit; Return-on-Investment


Inputs (Resources)




Domain



Evaluation Involves Comparison

• Over time

• Between services

• Between the current and the anticipated

• Between inputs and outputs/outcomes

ROI is a Comparison

Compare Returns and Investments • Investments or Costs

- to the library (or service within) - to the library user- to the organization

• Returns- to the library- to the library user- to the organization

ROI is a Comparison

Investments or Costs • to the library (or service within):

$ spent

• to the library user:time and $ spent

• to the organization:total time and $ spent

ROI is a Comparison

Returns

Important to distinguish between:

outputs - produced as result of expenditure, e.g., larger collection

use - extent to which outputs are used, e.g., increased use of library materials

outcomes - consequences of use, e.g., learned something new, saved time, etc.

Use contingent evaluation

Return-on-Investment Defined

• Investment– Library expenditures and organization

overhead

– User time in wages and overhead

– Cost of other relevant resources

• Return– Contingent valuation of the additional cost to

users if there were no library services

– Changes in user outcomes e.g., productivity, information needs satisfied, and other relevant indicators

Definition of Contingent Valuation

• Contingent valuation is an economic method used to assess the benefits of non-priced goods and services (e.g., libraries or specific library services) by examining the implications of not having the product or service

Case Study 1:Library ROI (Availability & Use)

Investment - Total

• Library expenditures

• User expenditures

• Total organization investment

Special Library Investments

• Library expenditure– $610 per professional

• User cost to use library services– $1,090 per professional

• Total organization investment– $1,700 per professional

Contingent Valuation - Library Availability

• User cost to use library services

• User cost to use alternative source for service or information

• Net benefit or return (availability) is difference between current cost to use library and anticipated cost to use alternatives

Special Library ROI

Net benefit or return:User additional cost to use alternatives

$5,010 per professional

Library ROI (availability)($5,010÷$1,700) or 2.9 to 1

Returns from Library Use

• Amount of use of information resource or service

• Benefits derived from that use (time saved, $ saved, etc.)

• Assume a fixed willingness to invest

• When using costlier alternatives, some uses would be “lost”

Library ROI

• Benefits derived– $310 per journal article reading– $650 per book reading– $1,090 per internal report reading

• Total savings across all readings– $31,300 per professional for reading

journal articles– $28,000 per professional for reading

books– $42,500 for reading internal reports

Library ROI

• Labor ROI – Labor savings divided by time spent reading– 8.3 to 1 for reading journals articles– 7.9 to 1 for reading books– 14.2 to 1 for reading internal reports.– Overall ROI (Labor) is 8.3 to 1

• Lost Benefits$12,240 per professional

• Library ROI (Use)– Ranges from 5 to 1 …. 16 to 1

Outcomes of Special Library Use

• Special libraries help increase productivity– User time and/or other expenditures are

saved in over one-third of library uses– User time and/or other expenditures are

saved in about 40 percent of readings of library documents

– Five indicators of user productivity are correlated with amount of library

Outcomes of Special Library Use

• Special libraries contribute to users’ quality of work

• Users indicate that that the library services are absolutely essential to their work for nearly 40 percent of uses

• Users indicate that the library services improves their quality of work in nearly 60 percent of uses

• Users whose work has been recognized through awards, etc. use libraries more than cohorts and non-award winners

Case Study 2: Library Journal Collection ROI

Trends in Special Library Journal Collection Use

• Examine patterns of information seeking

• Examples of scientists and engineers

• Amount of journal reading by scientists is up

• Reading from special library journal collections is up

• Increases due in part to electronic journals

Patterns of Information Seeking

• Focus on article reading

• Time spent reading

• How read articles are identified

• How the articles are obtained

• Format of the articles

• A trend in the influence of libraries

• Effect of e-journals on library cost

Many Ways to Identify Articles

• Browsing through print or electronic journals (mostly for current awareness)

• Searching in search engines in bibliographic and e-journal databases (mostly for research and writing)

• Follow-up of citations in journals and other publications

• Recommendations from colleagues, etc.• Other (e.g., alerts, preprint services, etc.)

Many Sources of Articles

• Personal subscriptions• Library collections• Authors, colleagues, etc.• Other

Where Do Readers Obtain Articles?

Personal Subscriptions Down

Library Collections Up

From Another Person Same

Author Websites Small

Preprint and other databases Small

Age of articles is important

Trends

Amount of Reading Depends on Where Readers Works

• University scientists average reading more than non-university scientists served by special libraries– University scientists: 252 readings/year– Non-university scientists: 113 readings/year

HOWEVER• Non-University scientists account for 75 percent

of all reading in U.S. AND

• University scientists write nearly 75 percent of articles published by U.S. authors

Trends in Reading Patterns of Scientists Served by

Special Libraries

• The appear to be reading more

• They rely on libraries more

• Reasons for increased library use

Average Annual Reading by Scientists Served by Special

Libraries

75 80

106113

0

20

40

60

80

100

120

1985-89 1990-93 1994-98 2000-2001Ave

rage

Nu

mb

er o

f R

ead

ings

Per

Sci

enti

st

Year of Surveys

75 80

106113

0

20

40

60

80

100

120

1985-89 1990-93 1994-98 2000-2001Ave

rage

Nu

mb

er o

f R

ead

ings

Per

Sci

enti

st

Year of Surveys

Average Annual Reading by Scientists Served by Special

Libraries by Source of Articles Read

24

61

30

50

59

47

63

50

0

10

20

30

40

50

60

70

1985-89 1990-93 1994-98 2000-2001

Library CollectionsOther Sources

Year of Surveys

Ave

rage

Num

ber

of R

eadi

ngs

Per

Sci

enti

st

24

61

30

50

59

47

63

50

0

10

20

30

40

50

60

70

1985-89 1990-93 1994-98 2000-2001

Library CollectionsOther Sources

Year of Surveys

Ave

rage

Num

ber

of R

eadi

ngs

Per

Sci

enti

st

Reasons for Shift to Reading from Special Library Collections

• Decrease in personal subscriptions

• More reading of articles identified by online bibliographic searches

• Electronic collections have broadened access to articles

Average Number of Personal Subscriptions per Scientist

3.70

2.89

2.44

1.93

0

0.5

1

1.5

2

2.5

3

3.5

4

1985-89 1990-93 1994-98 2000-2001Year of Survey

Ave

rage

Sub

scri

ptio

ns P

er S

cien

tist

3.70

2.89

2.44

1.93

0

0.5

1

1.5

2

2.5

3

3.5

4

1985-89 1990-93 1994-98 2000-2001Year of Survey

Ave

rage

Sub

scri

ptio

ns P

er S

cien

tist

Average Number of Articles Identified by Online Search

1.5

5.6

12.813.9

0

2

4

6

8

10

12

14

1985-89 1990-93 1994-98 2000-2001Year of Survey

Ave

rage

Num

ber

of R

eadi

ngs

Per

Sci

enti

st

1.5

5.6

12.813.9

0

2

4

6

8

10

12

14

1985-89 1990-93 1994-98 2000-2001Year of Survey

Ave

rage

Num

ber

of R

eadi

ngs

Per

Sci

enti

st

Electronic Collections Contribution

• Personal subscriptions - 90% print

• Library collections– 80% electronic– Broadens journal availability– Saves readers about 20 hours per year

• Breadth of reading has increased– Read from about 13 journals in 1977– Over twice that amount now

Library Journal Collection Investment

• Library expenditure (amount allocated to faculty and staff use)– $1.87 million

• Faculty and staff cost to use the library collection– $1.56 million

• Total organization investment– $3.43 million

Library Journal Collection Return (Availability)

• User cost to use alternative sources of article information ($11.38 million in time, $2.1 million in purchases)

$13.48 million

• Return/net benefit ($13.48 million - $3.43 million)

$10.05 million

Library Journal Collection ROI (Availability)

• ROI (Availability)($10.05 million ÷ $3.43 million)

2.9 to 1 PLUS

• Potential lost benefits in savings, productivity, etc.

Library Journal Collection Return (Use)

If the journal collection were not available, professionals said they:

Did not know 37%

Would use another library 28%

Would use another source 21%

Would purchase the item 6%

Would take another course of action 8%


About 25% of readings of library-provided journal articles saved the user time and/or money

Average amount of savings$310 per article reading (special

library studies)$385 per article reading (national

studies of scientists and engineers)

ROI (Use)8.3 to 1


Reasons for savingsAvoided having to do some work 49% Provided confirmation of work

27%Stopped unproductive line of work

10%Modified research or design 12%Modified analysis methods 16%

Total Time Saved

• Total time saved– 250,000 hours– 114 FTEs

• Electronic remote access savings in time– 50,000 hours– 23 FTEs*

* Based on an average of 2,200 hours worked

ROI Methods

• Library ROI Methods

• Journal Collection ROI Methods

Library ROI Methods

• Mailed survey to organization professionals– Letter from high level executive– Publicity by library– Promise results to users– Reminders by cards

• Universe is visits (critical incidents)

Library ROI Methods

Observations include:

• Library use (including last use)– Amount of use– Services used– Time spent using library– Time and cost to use alternative sources

if the library were not there

Library ROI Methods

Observations include:• Section on purposes and

consequences of using the library– Purposes (e.g., research, writing, etc.)– Ways information/services affect

purposes– Savings in time and/or money from

information– Importance of information/services

Library ROI Methods

Observations include:• Section on demographics

– Education level achieved, discipline– Awards received– Authorships

Methods: Cost of Users’ Time

• We use the average salary plus an amount for overhead

• From other studies, we estimate that professionals average about 2,200 hours of work per year.

• This allows us to calculate an hourly rate per hour for users.

Survey of Special Library Use

Your responses are confidential and data will be reported only in aggregated form. Because your answers are extremely important to the accuracy of our study, please submit the questionnaire even if you are unable to answer all the questions. We have tried to keep the questionnaire as short and simple as possible and yet achieve our study objectives. If you have any questions, please contact….


Section 1: Library Use1. In the past year (12 months)

approximately how often have you used your (company, agency, laboratory) library? Uses include visiting the library in-person or remotely including access to the library’s bibliographic databases, electronic journal collections, to request services, etc.

Number of times the library was used the past year: _______ times


The following questions in this section refer to the last use of the library, whether in-person or remotely. Note that this last use may not be typical, but will help us establish the range of uses of your library.2. Was this last use in-person or remote?

– In-person– Remote

• By e-mail• By telephone

3. How long ago was this last use?

4. What services were used during this last use?


5. About how much time did you spend on this last use of the library?

6. If you did not have the library, what would you have done to obtain the information or service obtained during this last use?

– I would not bother getting the information (skip to question 7)– I need the information, but do not know where else to get it (skip to question 7)– I would obtain the information from another source. Please specify source here: _____________________

Journal Collection ROI Methods

• Survey of professionals

• In-depth cost of library collection purchase and processing

Reader Survey Methods

• Mailed survey…• Universe is readings (critical

incident)• See questionnaire sample in handout

Journal Collection Costs Include All Resources

Used

• Collection purchases

• Staff

• Facilities

• Equipment and systems

• Photocopies, binding, etc.

Allocating Library Staff Costs

• Prepare a list of relevant journal processing activities

• Prepare a staff activity log

• Instruct library staff for filling out the logs

• Establish detailed calculation methods

Total Cost and Cost Per Use of Collection Access Services

Life-Cycle Cost per Title

• Electronic collection - $180 per title

• Print collection - $580 per title– Current collection - $190 per title– Backfile collection - $390 per title

Annual Cost Per Reading• Compare the Unit Cost of Services

– Electronic - $3.00 per reading– Current Periodicals - $13.60 per

reading– Bound Backfiles - $15.60 per reading– ILL - $8.40 per item– ILB - $12.60 per item

Life-Cycle Cost Per Reading• Electronic - $7.30 per title

• Print - $23.50 per title

Communicating Results

Target Audiences

• Identify your target audiences – who do you want to pay attention to your information?

Target Audience Characteristics

Summarize key characteristics of your target audience(s)

• Current key issues and concerns

• Language/educational level

• Preferred communication media

Need “Sound Bytes”

• Core messages

• “So therefore…”

• Meaning and impact

• Comprehensibility

Core Messages

• List of 1-sentence statements of main points, NOT long paragraphs

• Maximum of 4 to 5 statements

• Sequenced from most important to least

• Need to have a 1-sentence summary for all

“So Therefore…”

• Your core messages must include not only facts and figures but also what it means

• This is true, so therefore– You should…– We will…– This proves…– Etc.

Core Messages: Meaning and Impact

• Your core messages must answer the question for your target audience:

“What does it mean and why should I care??”

Core Messages: Comprehensibility

• Simplify, simplify, simplify!• Just because you know big words doesn’t

mean you have to use them


“Insofar as the comprehensive research is too voluminous to delineate here, if you and/or your corporation, organization, etc. are apprehensive as to the pedagological or methodological veracity, authenticity or rationale supporting the aforesaid investigation please reference the relevant available electronic resource.”

Compared to: “Please check our website for more information.”


Avoid being misquoted in the press (including the academic press!) by asking the reporter to:

• E-mail you his/her questions before the interview.

• Repeat back to you what you have said

• Call you back with any additional questions, as necessary.

• Provide you with a copy of your quotes before the article is published. Quickly correct any inaccuracies that you find. (Some reporters wil not provide you with pre-publication quotes, but some will. And it won't hurt to ask.)

Craft Your Message

• Identify the desired results of your communication

Target Audience Know Feel DoOrganizational leadership

The results of the study: the numbers and their implications

The library/in-formation service is well worth the cost

Support additional funding requests

Craft Your Message

• Attention and sequencing - pyramid approach

• Find or create visuals

• Humanize your message

• Try out your message with your harshest critic and don’t argue with their comments - listen and learn

Attention and Sequencing

• Must have an opening line (headline, first line of interview,etc.) that is attention getting —focuses on impacts rather than data

• Need excellent headline/title

• Visuals

Visuals Communicate

Without Special Library With Special Library

Time of One Information Professional

Saves the equivalent of

FIVE professionals’

time

Follow Up

• Every communication is the foundation for the next

• Be clear about asking for what you want

• Be appreciative

José-Marie Griffiths, PhDSarah E. Aerni

• Email: [email protected]• Email: [email protected]

• School of Information and Library ScienceUniversity of North Carolina at Chapel Hill

• phone: (919) 962-8363

Thanks for coming!

• Please submit the evaluation forms to us so that we can send them back to SLA.

• Enjoy the rest of the conference!

calculating roi in special libraries and information centers: consequences of not having an...

Documents