Primož Južnič, Polona Vilar & Tomaž Bartol, University of Ljubljana, Slovenia

What do researchers think about altmetrics and are they familiar with their abilities?

LIBRARIES IN THE DIGITAL AGE (LIDA) 2014Zadar, Croatia

1. The debate on how to measure scientific

quality and quality of scientific research has been going on for decades.

Using citations as an absolute proof of quality has been frequently labelled “controversial”, either if used as an indicator of assessing the quality of research work, both directly (citations of individual papers and other publications) and indirectly (journal impact factors).

2. The rise of the Web 2.0 (Social web)

has given the main incentive to the creation of altmetrics, which are social web metrics for academic purposes.

It can be used in an evaluative role and as an information seeking aid, both tasks reserved until recently for traditional bibliometrics.

3. If altmetrics are to be trusted then

the claims about both of these tasks must be acceptable and verifiable.

Most of the research presenting arguments about the extent of the use and importance of social web simply quotes user statistics obtained from administrators of different social networking websites.

We have to find out more about the possibility of altmetrics, and to explore the applicability, use and acceptance of altmetrics sources and indicators in the scientific community.

Since it is still unclear how and to what extent the social networking platforms are used, by whom and for what purpose.

The objective of this study is to assess the representativeness and validity of altmetrics’ indicators with the help of scientific community.

Results are a part of extensive longitudinal survey of Slovenian scientists which investigated their information behaviour (preferences, opinions, use) and their views on research results evaluation.

The principal objective was to investigate whether Slovenian researchers essentially use social networking sites and perceive them as an important part of their professional work as social networking is regarded as a part of possible new metrics.

We had following research questions:1. Do younger researchers use social

networking tools more than older researchers?

2. Does the period of three years present an important difference in the acceptance of social networking tools among researchers?

3. Do the researchers regard altmetrics as alternative methods for evaluation of their research work?

Online survey (open April – May 2014) 10 content questions (eg. about information

resources they use/prefer and science metrics )

6 demographic questions (age, gender, employment, experience, research areas)

Random sample of active researchers (data from ARRS – Slovenian Research Agency.) E-mail invitation for every eighth (592

individuals)

114 acceptable answers

41 % female Age structure:

Research area (ARRS classif.):

Research area %

Natural Sciences 33

Social Sciences 19

Technical Sciences 13

Humanistic Sciences 9Interdisciplinary Research 16

Medicine 17

Biotechnical sciences 11

Age %

20-30 9

31-40 39

41-50 26

51-60 14

above 60

12

Heavy use of web search engines – general and scholar

(84% often/always), e-journal sites and specialised bibliographic

databases (WoS, Scopus…) (63% often/aways)

Library catalogue (COBISS) (43% often/aways)

They like electronic materials: (54% prefer e-) (43% cite over 80% resources

in electronic form)

which are not so surprising anymore, as the results are almost identical to ones three years ago.

They hardly use popular social networking tools:

Facebook (87 % never/ almost never), Twitter (89% never/almost never) blogs (77% never/almost never), forums (66% never/almost never),

Even specialized social networking tools are not so popular: Mendeley (87 % never/ almost never), CiteULike (93% never/almost never) LinkedIn (74% never/almost never), ResearchGate (60% never/almost never),

ResearchGate seems to be a little bit more popular than other tools

14% said they often use it, bot none (0%) always.

Age and gender were not an important factor in social network use.

Research discipline has some influence, but not really important one.

We asked the respondents to provide their opinion on the uses of altmetrics in the evaluation of scientific research.

Three possible answers were offered:

1. I am familiar with2. I am not familiar with but I'm

interested 3. I am not familiar with and I'm not

interested

19% of respondents (mostly male) reported on their familiarity with altmetrics.

Surprisingly many (71%) said that they are not familiar with it but are interested!

Age was not an influential factor. Research discipline also had some influence.

On average, fewer Natural scientists and more Medicine researchers are familiar with it; more scientists from Engineering fields are not familiar with it and are not interested.

There were also some correlations between the awareness of altmetrics tools and the use of Web 2.0 tools.

Many of those who claim to be familiar with altmetrics often use ResearchGate and LinkedIn, and occasionally use Facebook and Twitter.

We also wished to investigate the possible acceptance of different altmetrics' indicators on the part of the researchers as a measure for evaluation.

Number of downloads of articles from scientific journals/publications was the indicator agreed or partially agreed upon by the majority of respondents. Only 12% disagreed or had no opinion.

Similar answers were obtained related to the possibility of using the number of downloads of publications from repositories, although these received little more answers related to “No opinion”.

References to research results in mass media also received the same very positive acceptance (77%).

Two other indicators, references to research results in social networks and statistics from the programs such as ResearchGate and Mendeley were little less popular, as expected, but still receiving positive acceptance (51% and 66%).

Some respondents mentioned successful transfer of knowledge to industry as important reference.

Even if the researchers themselves are not very regular users of social netowork tools they do not completely reject such a possibility to evaluate research results. In fact, most researchers partially agreed with this option.

Researchers from the Humanities are more in favour of using the data on downloads, as expected as they usually oppose the use of citation data.

Slovenia has a very sophisticated system of tracking the publication patterns of scientists, and the respective citation impact, which is also very transparent as it is publicly available through two interconnected systems - COBISS and SICRIS.

The motivation for the updates on publishing activities is very strong among the researchers.

They regularly access COBISS and SICRIS, also in order to follow the publishing activity of their colleagues and associates.

A weak use of social networking tools does not seem to prevent the researchers from being open to the possibilities of employing new methods of research evaluation. Such non-use is more related to the lack of time in a highly competitive world of science, and also to pragmatism.

If the scientists do not perceive some concrete benefits, either in a better quality of information resources or improved prestige, they will not use such tools.