Research to articulate more clearly the return on investment from academic libraries’ provision of journals to support the core teaching and research activities in UK Universities.

Scholarly journal information-seeking: measuring activity and value

1. Background

 UK academic libraries make substantial investments in electronic serials collections. The move to electronic provision of journals, coupled with the huge expansion in the number of titles available, has led to a significant amount of investment in helping libraries to measure their usage of journals.  From this data, metrics like ‘cost per download’ have become routine to informing acquisition and retention strategies.

However, in some respects usage on its own remains a fairly blunt measure of value and other factors increasingly need to be taken into account, such as the purpose and outcome from reading journal articles and the actual impact of those readings on the core activities of the institution. Examples of the outcomes from reading journal articles might include helping to secure research grants, contributing to specific research outcomes, or supporting the teaching of a new income-generating course. In other words, we should not assume that journal article usage on its own is a sufficient measure of value without at least some understanding of the context of its actual use.

It is precisely this kind of qualitative as well as quantitative measures that are needed to move the debate away from simply ‘value for money’ towards a more informative analysis of ‘return on investment’ from libraries’ provision of access to scholarly journal articles.

This issue has become all the more urgent in the current economic climate where significant budget cuts are likely in Higher Education over the next few years. Along with other senior managers, Library Directors will be at the forefront of discussions between faculty and administrators about the retention of resources and decisions made on usage alone may not be sufficient to gain the trust of academics or to ensure that the correct decisions are made beyond the short term.

Not surprisingly funding and financial management issues topped the Sconul Top Concerns Survey again in 2010[1], as challenges will include greater scrutiny from university management and impending re-negotiations with the major journal publishers who currently dominate library materials budgets. Meeting these challenges will require new thinking and stronger advocacy.

In such a context, there is a need for additional research to articulate more clearly the return on investment from academic libraries’ provision of journals to support the core teaching and research activities in UK Universities. It is more important than ever to understand in more depth the relationship between scholarly information activity and research and teaching outcomes. Without such research, librarians will be left to rely on usage data and implicit correlations between journal provision and value which may not, in themselves, be sufficient evidence for justifying journal retention in a more business-orientated environment.

There have been a number of recent studies looking at the value for money from access to e-journals, most notably the RIN/CIBER study ‘E-journals: their use, value and impact’[2]. The research looked in detail at activity in a number of HE institutions, focussing primarily on deep log analysis of usage statistics. It included an estimate of the number of full-text articles downloaded per annum and their average cost per download and looked at correlations between “expenditure on electronic journals and the volume of downloads of articles per capita” with “papers published, numbers of PhD awards, and research grants and contracts income.” (ibid p. 8)

Whilst this report provided some evidence of value, it also recognised a need for more “qualitative research to gain a firmer understanding of the links between information behaviour and satisfaction” (ibid p. 46) i.e. the difference between ‘viewing’ an article and actually using the article.  Deep log analysis does not tend to differentiate between downloads by student or faculty member and cannot reveal the purpose of downloading an article. In other words, it is not enough just to know that scholarly articles are being downloaded (where the value is implicit), there is also a need for a better understanding of how articles are actually being used and the explicit value they are providing to “success in research”. (ibid p.45) 

 In the same month (April 2009), Carol Tenopir’s research team in the US were writing up the results of surveys undertaken at five US Universities[3]. Whilst sharing similar themes to the RIN/CIBER study, this sought to gain data about the usefulness and value of scholarly articles at a more granular level (i.e. to the individual researcher and, by implication, to the institution as a whole) and more qualitative level. The data focused on each participant’s most recent reading of an article which then informed an analysis that assessed:

·         the average number of hours spent on reading scholarly journal articles per faculty member per annum (compared with the amount of readings they access generally).

·         how much scholarly journal articles are used in comparison with other information sources.

·         how these scholarly journal articles were actually sourced (through library subscriptions, individual subscriptions, interlending or other sources).

·         how academics became aware of articles (e.g. searching bibliographic databases, current awareness, alerts, etc.).

·         an estimate of how many articles are read to support research activity and how many to support teaching activity and other academic support purposes such as current awareness, research administration, etc.

·         a measure of perception about how useful and valuable  these scholarly journal articles are.

·         an analysis of the actual benefits of reading an article such as “inspiring new thinking or ideas; improving the results; narrowing, broadening or changing the focus of research and teaching, etc.; or saving time or other resources.”  (For the UK this might also include anticipated contribution to grant proposals, enterprise & engagement activities, etc.).

·         the percentage of readings that are from articles published within the last six months (and the extent to which this varies across disciplines) compared with the percentage that are older than six months.

2. Aims and Objectives

There are many potential benefits of repeating the Tenopir and King studies, with some local variations, in the UK.

·         Additional qualitative data about the actual usefulness of scholarly journal articles and some explicit examples of the value that they provide to the research and teaching process, and their percentage use for research, teaching and other purposes.

·         A measurement of usage which goes beyond the number of articles viewed or downloaded by the institution and which instead estimates readings by faculty member per annum.

·         To provide additional qualitative data about the use and value of scholarly journal articles that can be take into account, alongside usage data, to inform future retention and cancellation policies (i.e. measures of actual rather than assumed value which may help to make the case for retaining some  low usage but high value journals.)

·         An assessment of these benefits within half a dozen specific UK institutions, thereby providing a UK specific context (which can then also be compared internationally which should strengthen and give further validation to its findings).

·         The value of scholarly journal articles in comparison with other readings (and where libraries might also add value to those other aspects of information provision).

·         To assess the ‘value added’ that academic libraries bring, not just in terms of subscription management, but also in terms of current awareness, alerts and information literacy, etc.

·         To provide qualitative data about whether there might be more appropriate business models for the provision of scholarly journal articles.

In addition, the research methodology has already been proven in other countries, so development and pre-testing is minimized.[4]

3. Overall Approach

Survey instruments for academic staff and students will be adapted based on previous Tenopir & King instruments[5]  in collaboration with JISC Collections and UK library participants. Instruments will be loaded online by the University of Tennessee team and Human Subjects permission will be obtained from the University of Tennessee Institutional Review Board.  Distribution strategy will be agreed on in consultation with the UT team.

An invitation email with an embedded link to an electronic survey instrument will be distributed to academic staff and students by the U.K. library participants. Nicholas Lewis, University of East Anglia and Hazel Woodward, Cranfield University will lead the efforts at their institutions and coordinate the addition of and distribution to other U.K. university participants in coordination with JISC Collections.  Distribution will be to the academic staff and students at a minimum of 3 and a maximum of 6 UK research intensive institutions. These will be representative of the variety of types, including the 94 Group and Russell Group.

Analysis and initial reporting of the results will be led by the University of Tennessee team, with data provided to the UK participants for further analysis as desired.

Time Line (November 8, 2010-November 7, 2011):

Phase 1: (November 2010-February 2011).  Coordination of project participants; survey instrument design, loading, testing, and revision; IRB Human Subjects permission obtained; distribution strategy finalized; analysis strategy developed.

Phase 2: (February-May 2011). Invitation with embedded survey link distributed to at least 3 U.K. universities. Data collected and backed up on the University of Tennessee server.              

Phase 3: (June-September, 2011). Data analyzed, reports written and distributed to participating libraries and JISC, data provided to participating libraries for further analysis as desired.

Phase 4: (September-Nov 7, 2011). Final reports completed and submitted.

4. Project Outputs

Outputs provided to JISC and UK librarians will include:

·         By project end: an open access report made available on the JISC web site and websites / repositories of project participants;

·         Subsequently: journal articles in international publications for both librarians and publishers;

·         Subsequently: presentations at conferences attended by librarians and publishers in both the UK and the US.

5. Project Outcomes

Understanding the value of reading and the contribution that libraries make will help inform library planning, collection decisions, and will contribute to the understanding of the value and role of the academic library by research communities, library constituents and funders. Results may stimulate academic faculty members to work more closely with academic libraries to better incorporate scholarly articles into teaching.

[2] Research Information Network, E-journals: their value, use and impact. 2009, London, Research Information Network. Available at: . See also: David Nicholas, Ian Rowlands, Paul Huntington, Hamid R. Jamali, Patricia Hernández Salazar. 2010. "Diversity in the e-journal use and information-seeking behaviour of UK researchers", Journal of Documentation, Vol. 66 (3) 2010 pp. 409 – 433 Available at: DOI: 10.1108/00220411011038476

[3] Donald W. King, Carol Tenopir, Songphan Choemprayong, and Lei Wu, 2009.  “Scholarly Journal Information Seeking and Reading Patterns of Faculty at Five U.S. Universities”, Learned Publishing, Vol. 22 (2) pp. 126-144. DOI: 10.1087/2009208

[4] Carol Tenopir, Concepción S.Wilson, Pertti Vakkari, Sanna Talja, and Donald W. King. “Cross country comparison of scholarly e-reading patterns in Australia, Finland and the United States,” Australian Academic & Research Libraries (AARL) 41 (1) March 2010: 26-41.

[5] Examples are available at:

Project Staff

Professor Carol Tenopir

Nicholas Lewis, University of East Anglia and Hazel Woodward, Cranfield University

Start date
1 November 2010
End date
31 October 2011
Funding programme
Information Environment Programme 2009-11
Scholarly Communications
Lead institutions

JISC Collections

University of Tennessee