Lynn Silipigni Connaway, Ph.D.Senior Research ScientistOCLCChair of ExcellenceDepartmento de Biblioteconomía y Documentación Universidad Carlos III de Madrid@LynnConnawayconnawal@oclc.org

May 15, 2014

Exploring Shifting Changes in User Engagement

Then & Now• Then: The user built

workflow around the library

• Now: The library must build its services around user workflow

• Then: Resources scarce, attention abundant

• Now: Attention scarce, resources abundant

(Dempsey, 2008)

The library? What’s that?

• Website hard to navigate• Inconvenient

• Limited hours• Distance to library• Physical materials

• Don’t think electronic resources are library resources• Associate with books

(Connaway & Dickey, 2010)

“…there was a book in the library that I just did not want to leave my house to go to. It is a 50-minute drive, I didn’t want to do that, but I was writing my paper and so I used Google books instead and really they only had a section of the book available but that was the section I used.”

(Digital Visitors and Residents, USG4, Female, Age 23, Latin American Studies)

Current Environment

• Challenges• Budget cuts• High retirement rates• Hiring freezes

• Opportunity• Best value for most use• Understand how, why, &

under what circumstances individuals use systems & services

Articulate the Value of Libraries

Why Assessment?

• Answers questions:• What do users/stakeholders want & need?• How can services/programs better meet needs?• Is what we do working?• Could we do better?• What are problem areas?

• Traditional stats don’t tell whole story

Importance of Assessment

“Librarians are increasingly called upon to document and articulate the value of academic and research libraries and their contribution to institutional mission and goals.”

(ACRL Value of Academic Libraries, 2010, p. 6)

Outcomes Assessment Basics• Outcomes: “The ways in which library users are

changed as a result of their contact with the library’s resources and programs” (ALA, 1998).

• “Libraries cannot demonstrate institutional value to maximum effect until they define outcomes of institutional relevance and then measure the degree to which they attain them” (Kaufman & Watstein, 2008, p. 227).

Interpreting

Analyzing

Assessment DefinedProcess of…

• Defining• Selecting• Designing• Collecting• Analyzing• Interpreting• Using information to increase

service/program effectiveness

Formal vs. Informal Assessment

• Formal Assessment • Data driven• Evidence-based• Accepted methods• Recognized as rigorous

• Informal Assessment • Anecdotes & casual

observation• Used to be norm• No longer acceptable

The Assessment Process

Identify purpose

Identify team

Choose model/approach/method

Training and Planning

Why?

Who?

How?

What We Know About Assessment

• Ongoing process to understand & improve service

• Librarians are busy with day-to-day work & assessment can become another burden

• Can build on what has already done or is known

Library Factors for Assessment

• Instruction: Games, single/multiple session, course embedded, tutorials

• Reference• Physical space• Discovery: Institutional web, resource guides• Collections• Personnel

Variety of Tools/Methods

• Survey• Interviews• Focus group(s)• Observation• Pre/Post test• Rubric• Student portfolio

• Research paper/project• Other class assignment• Test scores• GPA• Degree completion rate• Retention rate

Principles for Applying Outcomes Assessment

• Center on users• Assess changes in service/resources use

• Relate to inputs - identify “best practices”• Use variety of methods to corroborate conclusions

• Choose small number of outcomes• Need not address every aspect of service

• Adopt continuous process

How Individuals Work

• Convenience• Value human resources• Contextually based

rational decisions• Situational needs

determine search• Satisfice

(Connaway & Radford, 2011)

“I don't know how to access computer library service. When I need to look something [sic] up I use google.”

(Seeking Synchronicity, NOS-61939, Male Age 46-55)

VRS & SQA User Interviews:Finding What is Needed

6%, 3

36%, 2058%, 32

1 Rarely2345 Very often

When you search, how often can you usually find what you need?

VRS Potential User Online Survey: Prefer Face-to-Face Interactions

49%, 60

12%, 14

27%, 33

12%, 15

"I most enjoy using"

FtFPhoneEmailText Messaging

Digital Visitors & Residents: Means of Contact

Emerging (n=43)

Establishing (n=10)

Embedding (n=10)

Experiencing (n=10)

30%

40%

50%

60%

70%

80%

90%

100%

60%, 26

100%, 10 100%, 10 100%, 10

84%, 36

80%, 8

70%, 7

50%, 5

77%, 33

90%, 9

70%, 7

70%, 7

60%, 26 60%, 6

40%, 4

70%, 7 Email

Texting

Phone Calls

Face-to-Face

Information-Seeking Behavior

• Power browsing• Scan small chunks of

information• View first few pages• No real reading

• Squirreling• Short basic searches• Download content for

later use • Differ with discipline (Research Information Network, 2006)

(Consortium of University Research Libraries, and Research Information Network, 2007)

(Connaway & Dickey, 2010

Skills for Finding & Using Information• Students

• Determine credibility by:• Common sense (77%)• Cross-checking (69%)• Reputation of

company/organization (67%)• Credible recommendations

(48%)• Researchers

• Self-taught in discovery services• No formal training (62%)

• Doctoral students learn from dissertation professor

(Research Information Network, 2006) (De Rosa, 2010)

Digital Visitors & Residents: Authority, Time, Convenience

Emerging (n=43)

Establishing (n=10)

Embedding (n=10)

Experiencing (n=10)

0%

20%

40%

60%

80%

100%

120%

79%, 34

60%, 6

90%, 9

70%, 7

40%, 17 40%, 450%, 5

50%, 5

91%, 39100%, 10 100%, 10

90%, 9 Authority, Legitimacy

Available Time

Convenience, Ease of Use, Accessibility

Digital Visitors & Residents: Authority, Time, Convenience

Authority, Legitimacy Available Time Convenience, Ease of Use, Accessibility

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

81%25

39%12

94%29

75%9

42%5

83%10

Emerging 1st Group (n=31)

Emerging 2nd Group(n=12)

Tools Used: Students • Undergraduate Students

• Google, Wikipedia• Also use library website & e-

journals• Human resources

• Other students/classmates• Family & relatives • Friends

• Graduate students• Professors, advisors, mentors• Electronic databases (Connaway & Dickey, 2010)

(De Rosa, 2010)

27

“This year I don’t think I have ever picked up a book out of the library to do any research, all I have used is my computer.”

(Digital Visitors and Residents, USU1, Female, Age 19, Undeclared)

VRS & SQA User Interviews:Searching the Web

16%, 9

26%, 14

20%, 11

38%, 21

Occasionally1-3 searches per day4-6 searches per day7-10 searches per dayMore than 10 searches per day

How often do you search the web?

Tools Used: Researchers• Online resources

• 99.5% use journals as primary resource

• Google, Web of Science, PubMed, Science Direct, JSTOR

• Human resources• 90% mention expertise of

individuals as important resource

• Coworkers• Colleagues• Other professionals (Research Information Network, 2006)

(Connaway & Dickey, 2010)

Journals & Databases• Journals

• Access more important than discovery• Want full text, online versions• Expect seamless Discovery-to-Delivery• Backfiles difficult to access• Content often discovered through Google• Visit only a few minutes

• Databases• Electronic databases not perceived as

library sources• Frustration locating & accessing full-text

copies

(Research Information Network, 2006)

Digital Visitors & Residents:Digital Sources

Emerging (n=43)

Establishing (n=10)

Embedding (n=10)

Experiencing (n=10)

10%

20%

30%

40%

50%

60%

70%

80%

90%

19%, 8

50%, 5

80%, 8 80%, 8

33%, 1430%, 3

70%, 7

40%, 4

28%, 12

40%, 4

50%, 5

30%, 3

Databases

University databases

University websites

Digital Visitors & Residents:Digital Sources

Databases University databases University websites0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

30%20

41%27

26%1720%

21 17%18

32%33

UKUS

Examples of Outcomes

User matches information need to information resources

User can find appropriate resources

User effectively searches online catalog & retrieves relevant resources

User can organize an effective search strategy

What can we change?• Improved OPACs

• Community as content• Full text, online accessible• Seamless discovery to

delivery• Access more important

than discovery• Mobile access

• Presence in social networks• Facebook• Twitter

“Because I mean the thing that annoys me most is when these things are online, unlike library catalogues that’s supposed to be a really good way for looking for books but usually they are so bad that you are sort of stuck between the two worlds...”

(Digital Visitors and Residents, UKG5, Female, Age 25, Early Modern History)

The Simple Search Bar

Community is Content

• Social networks formed around social objects• Music, photos, videos, links• Reviewing• Tagging• Commenting• Rating

• Refines interaction with resources

(Dempsey, 2012)

What can we do?• Advertise resources, brand, &

value• Provide search help at time of

need• Chat & IM• Mobile technology

• Design all of our systems with users in mind• Familiar formats

• Model services on popular services

Amazon.com

Westerville Public Library

Familiar Formats

Continually evaluate services

Ask those who use and do not use them

infoKit

What is it?• Contains advice on evaluating digital/online services

within the broader context of traditional services. Why did we create it?• To understand the contexts surrounding individual

engagement with digital resources, spaces and tools.

Who will use it?• Librarians and information technology staff

(White, Connaway, Lanclos, Hood, & Vass, 2014)

©2014 OCLC. This work is licensed under a Creative Commons Attribution 3.0 Unported License. Suggested attribution: “This work uses content from [presentation title] © OCLC, used under a Creative Commons Attribution license: http://creativecommons.org/licenses/by/3.0/”

Thank You!

Lynn Silipigni Connaway, Ph.D.connawal@oclc.org@LynnConnaway

ReferencesAssociation of College and Research Libraries. (2010). Value of academic libraries: A comprehensive

research review and report. Chicago: Association of College and Research Libraries.Bertot, J. C., Berube, K., Devereaux, P., Dhakal, K., Powers, S., & Ray, J. (2012). Assessing the

usability of WorldCat Local: Findings and considerations. The Library Quarterly, 82(2), 207-221. Connaway, L. S., & Dickey, T. J. (2010). Digital information seekers: Report of findings from selected

OCLC, RIN, and JISC user behavior projects. http://www.jisc.ac.uk/media/documents/publications/reports/2010/digitalinformationseekerreport.pdf

Connaway, L. S., & Dickey, T. J. (2010). Towards a profile of the researcher of today: What can we learn from JISC projects? Common themes identified in an analysis of JISC Virtual Research Environment and Digital Repository Projects. http://ie-repository.jisc.ac.uk/418/2/VirtualScholar_themesFromProjects_revised.pdf

Connaway, L. S., & Radford, M. L. (2011). Seeking Synchronicity: Revelations and recommendations for virtual reference. Dublin, OH: OCLC Research. Retrieved from http://www.oclc.org/reports/synchronicity/full.pdf

Consortium of University Research Libraries, and Research Information Network. (2007). Researchers’ use of academic libraries and their services: A report. London: Research Information Network and Consortium of University Research Libraries (CURL).

Cunningham, S. J., & Connaway, L. S. (1996). Information searching preferences and practices of computer science researchers. In J. Grundy (Ed.), Proceedings: Sixth Australian conference on computer-human interaction, November 24-27, 1996, Hamilton, New Zealand (pp. 294-299). Los Alamitos, CA: IEEE Computer Society Press.

Dempsey, L. (2008). Always on: Libraries in a world of permanent connectivity. First Monday, 14(1). Retrieved from http://www.firstmonday.org/htbin/cgiwrap/bin/ojs/index.php/fm/article/view/2291/207

ReferencesDempsey, L. (2013, January 23). The inside out library: Scale, learning, engagement. Presented at

Hacettepe University, Beytepe, Ankara (Turkey). Dempsey, L. (2012). Thirteen ways of looking at libraries, discovery, and the catalog: Scale, workflow,

attention. EDUCAUSE Review Online. Retrieved from http://www.educause.edu/ero/article/thirteen-ways-looking-libraries-discovery-and-catalog-scale-workflow-attention

De Rosa, C. (2010). Perceptions of libraries: A report to the OCLC membership. Dublin, OH: OCLC Online Computer Library Center.

Dervin, B., Connaway, L. S., & Prabha, C. (2003-2006). Sense-making the information confluence: The whys and hows of college and university user satisficing of information needs. Funded by the Institute of Museum and Library Services (IMLS). Retrieved from http://imlsosuoclcproject.jcomm.ohio-state.edu/

De Santis, N. (2012, January 6). On Facebook, librarian brings 2 students from the early 1900s to life. Chronicle of Higher Education. Retrieved from http://chronicle.com/blogs/wiredcampus/on-facebook-librarian-brings-two-students-from-the-early-1900s-to-life/34845

Janes, J. (1999). On research survey construction. Library Hi Tech, 17(3), 321-325. Kaufman, P., & Watstein, S. B. (2008). Library value (Return on Investment, ROI) and the challenge

of placing a value on public services. Reference Services Review, 36(3), 226-231. Norris, D., Baer, L., & Offerman, M. (2009, September). A national agenda for action analytics.

Presented at the National Symposium on Action Analytics, St. Paul, Minnesota, http://lindabaer.efoliomn.com/uploads/settinganationalagendaforactionanalytic s101509.pdf

Priestner, A., & Tilley, E. (2012). Personalising library services in higher education: The boutique approach. Farnham, Surrey, England: Ashgate.

ReferencesRadford, M. L., & Connaway, L. S. (2005-2007). Seeking Synchronicity: Evaluating virtual reference

services from user, non-user, and librarian perspectives. Funded by the Institute for Museums and Library Services (IMLS). Retrieved from http://www.oclc.org/research/activities/synchronicity/default.htm

Radford, M. L., Connaway, L. S., & Shah, C. (2011-2013). Cyber Synergy: Seeking sustainability through collaboration between virtual reference and social Q&A sites. Funded by the Institute of Museum and Library Services (IMLS), Rutgers University, and OCLC. Retrieved from http://www.oclc.org/research/activities/synergy/default.htm

Research Information Network. (2006). Researchers and discovery services: Behaviour, perceptions and needs. London: Research Information Network.

Wasserman, S. (2012, June 18). The Amazon effect. The Nation. Retrieved from http://www.thenation.com/article/168125/amazon-effect

White, D., & Connaway, L. S. (2011-2014). Visitors and residents: What motivates engagement with the digital information environment. Funded by JISC, OCLC, and Oxford University. Retrieved from http://www.oclc.org/research/activities/vandr/

White, D., Connaway, L. S., Lanclos, D., Hood, E. M., & Vass, C. (2014). Evaluating digital services: A Visitors and Residents approach. Retrieved from http://www.jiscinfonet.ac.uk/infokits/evaluating-services/

Wong, W., Stelmaszewska, H., Bhimani, N., Barn, S., & Barn, B. (2009). User behaviour in resource discovery: Final report. Retrieved from http://www.jisc.ac.uk/whatwedo/programmes/inf11/userbehaviourbusandecon.aspx

Zickuhr, K., Rainie, L., & Purcell, K. (2013). Library services in the digital age. Washington, DC: Pew Research Center’s Internet & American Life Project.