JISC user behaviour observational study: User behaviour in resource discovery
Download this document
This report discusses the information-seeking behaviour of students and researchers working in the Business and Economics disciplines using subscribed and freely available Internet resource discovery systems in three UK HE institutions: Cranfield University, London School of Economics and Middlesex University. The institutions were chosen as exemplars of the Russell Group, the 94 Group, and the Million+ groups of universities in the UK.
The intention was to describe and gain a better understanding of:
- how different users (undergraduates, postgraduates and researchers) currently seek information on the existing resource discovery systems
- the roadmap used in a user’s information seeking journey
- their expectations and needs based on their understanding and experience of using the Internet to find information resources for academic study
In addition, we hope to inform JISC, publishers of electronic resources, and librarians working in the HE sector about user behaviours and the issues relating to resource discovery systems. The data was obtained from an observational study and in-depth interview of 34 participants. Each participant was studied on an individual basis and each session lasted up to two hours. The analysis of the data provided an insight on: resource discovery systems used by participants and what they found to be useful, users information search behaviour and strategies when seeking information, the issues that affect their searching behaviour, problems and difficulties users experienced with library resources, issues related to physical library services as well as recommendations for the publishers and librarians in order to improve the use of electronic resources.
In this study, we find that information literacy is often confused with information technology or digital literacy. Broadly, information literacy is about the abilities to know when to find, to search for, evaluate and make sense of the content. Information technology or digital literacy is about fluency with the procedures and knowledge of the underlying information technology. There is unfortunately some overlap between these definitions as the information that is sought is often contained within the systems. They closely intertwine, such that the way people formulate their queries to find information is highly dependent upon the functionality provided by the information technology. In addition, we have also found that the poor usability, high complexity, and lack of integration of many electronic resource discovery systems, have raised the entry threshold of information technology literacy. This acts as a barrier to information search and retrieval. This higher level of difficulty amongst electronic discovery systems distracts users from focusing on the content, analysis and evaluation that would help them learn and make sense of what they have discovered. In addition, we have also found evidence to suggest that information literacy skills are lacking. For example, at a simplistic level, many of the participants do not understand how to assess the quality of materials they find. Google or Google Scholar have lower thresholds of information technology literacy, and are considered their “… friends” because of the apparent higher yield or success rate. While we acknowledge that information literacy may not be a new or recent problem, the evidence suggests that it continues to be a problem. Being able to operate a search engine, does not mean that one is able to find the good quality information necessary to help us learn and to advance our society.
Students and researchers from Business and Economics use both resources subscribed to by the library and those freely available on the Internet when seeking information in the academic context. However, as the level of information literacy as well as the domain knowledge increases, there is an increased tendency to use better quality library resources. The most common library resources used by Postgraduates and Experts in our study were EBSCO, ProQuest or Emerald, whereas the Library Catalogue and federated search engines (CrossSearch at Cranfield, QuickSearch at London School of Economics and MultiSearch at Middlesex University) were observed to be more popular amongst Undergraduates. When using freely available Internet resources, Google is top of the list, followed by Google Scholar, Wikipedia and YouTube. Participants’ decisions about which resources to use were based on their prior knowledge and experience with a resource and a belief that resources provided by Google and Google Scholar are reliable and relevant and most of all always return a list of results. On the other hand, library resources were perceived as credible providing a quality material from a broad subject coverage.
Users find database structures hinder. They have to learn the procedural knowledge for using a particular database as well as have some basic knowledge of how the data table is organised and what subject matter the built-in thesauri refers to; both have limited transferability. The participants did not appear to lack information technology or digital literacy, as they had demonstrated they were able to use other internet-based search and retrieval tools.
The study groups very rarely applied only one search strategy (eg Simple Search) but their strategy changed during the information seeking process in relation to the results obtained (ie refine or re-formulate search, abandon search or resource or change resource). Participants usually carried out combined searches: for instance, re-formulated search where terms or concepts extracted from a document were carried out to pursue a new search. This search was often combined with a link search that gave the opportunity to follow hyper-links and extract new queries that were used as an input for the search terms in multiple fields. This group of users also followed the help suggested by a system (eg ‘Suggested Topics’, or Results by Source’). There are considerable differences between the user groups; and the ‘atomic’ or basic search components combined during the information seeking activities depend on their level of information literacy and domain knowledge. Expert users are more inclined to use Re-formulated Search, System Suggestions or Personal Knowledge and Experience, whereas UG groups tend to use more frequently a Link Search. Although, current systems functionally support these forms of searching, transitioning between them is currently not as “seamless” as it should be, making the search and retrieval process difficult and time consuming.
During information seeking activities, the groups we studied used different means of storing the material depending on the stage they were at. At the evaluation of results stage, they store material temporarily using the web browser’s tabs system. These tabs are then re-visited for quick evaluation and then the information may be stored permanently using other means (eg notes in a Word document, save downloaded material into a folder, bookmark using a browser’s feature and use of more sophisticated features provided by various resource discovery systems such as RefWorks, Endnote, My Reasearch). The study groups did not use the resource discovery systems storage features often, as they were not aware of their existence or how to use them. The current systems lack good ways of storing and retrieving documents allowing the users to create repositories of information that can be accessed easily and be transferable across different resources.
Publishers’ embargoes on different material raised an important issue for participants. They were irritated and frustrated when a promise to obtain a document was often not met. Cases like this lower the participants’ level of trust towards academic resources. These practices of embargoes and ‘free for the moment’ have encouraged participants to turn to the external sources like Google Scholar where, despite having no promise of the article, and where the scholarly quality cannot be assured, one still has a much higher chance of finding the article to download. The current ways of doing business can get in the way of finding quality scholarly materials through the respective resource discovery systems.