This strategic overview covers key findings and recommendations from several JISC-funded reports on user behaviour in resource discovery.

Improving the resource discovery experience: Executive overview

Download this document

This strategic overview covers key findings and recommendations from several JISC-funded reports on user behaviour in resource discovery.

Contents
Introduction User interfaces Benefits
Key findings & recommendations Search Where next
Learners Access management References

The JISC-funded reports include:

User behaviour in resource discovery (November 2009)

This project was a study of students’ and researchers’ use of information resources in business and economics using observational techniques and cognitive task analysis, and also drawing on analyses of usage data.

JISC user behaviour observational study (October 2009)

An evidence-based study on scholarly digital use and information seeking behaviour in business and economics.

JISC national e-books observatory project (November 2009)

This project undertook critical market research into the behaviours, attitudes and perceptions of students, teaching staff and librarians towards course text e-books. The project gained an in-depth understanding of how users discover, navigate through and use e-books to support learning and teaching.

Federated access management service provider interface study (August 2009)

A study that explored the approaches taken by service providers to implement federated access management and its effect on the end user experience. The aim of the study was to make clear recommendations regarding common terminology and practices across service provider platforms to both improve the user experience and ensure the maximum number of successful authentications.

Users’ trust in information resources in the web environment (April 2010)

The purpose of this study was to review existing research into how users place their trust in digital information resources in the web environment and current approaches to presenting such resources in ways that engender trust. Information resources in this context meant both formal (eg journal articles) and informal resources (eg blogs) used in learning, teaching or research.

Terminology

The terms information literacy, digital literacy and media literacy can be used interchangeably which can cause some confusion. Below are some definitions of what we mean by these different terms:

Information literacy

‘Information literacy is knowing when and why you need information, where to find it, and how to evaluate, use and communicate it in an ethical manner.

This definition implies several skills. We believe that the skills (or competencies) that are required to be information literate require an understanding of:

  • A need for information
  • The resources available
  • How to find information
  • The need to evaluate results
  • How to work with or exploit results
  • Ethics and responsibility of use
  • How to communicate or share your findings
  • How to manage your findings

The Chartered Institute of Library and Information Professionals, 2004

Digital literacy

The ability to locate, organise, understand, evaluate and create information using digital technologies, which requires an ability to understand and know how to use these technologies (Wikipedia, April 2010). An example of this is creating a document in Microsoft Office Word, being able to store it in an electronic folder and retrieving it for future use.

Learners

Higher education has generally attracted learners from a traditional educational background, those who have taken A-Levels and then enter an undergraduate programme before they are 20.

HEFCE as one of its strategic aims has been widening participation in higher education (HEFCE strategic plan 2006–2011) .

Widening access and improving participation in higher education are a crucial part of our mission and form one of our strategic aims. Our aim is to promote and provide the opportunity of successful participation in higher education to everyone who can benefit from it. This is vital for social justice and economic competitiveness.

Widening participation addresses the large discrepancies in the take-up of higher education opportunities between different social groups. Under-representation is closely connected with broader issues of equity and social inclusion, so we are concerned with ensuring equality of opportunity for disabled students, mature students, women and men, and all ethnic groups.

Higher education institutions have been recruiting non-traditional learners as part of their strategies to widen participation. These learners will often undertake degrees in further education institutions or while still in the workplace.

‘By the time an HE student arrives at university, searching habits and information seeking behaviours have already been adopted.’

Users’ trust in information resources in the web environment: a status report

Introduction

This strategic overview covers key findings and recommendations from several JISC-funded reports on user behaviour in resource discovery. It also discusses the implications of these findings and makes some recommendations on the design of user interfaces provided by both academic publishers and libraries.

Learners studying in UK colleges and universities have the potential to be able to access and use a wealth of online and digital resources. These range from resources generated by tutors that are stored and held in institutional repositories or virtual learning environments, to subscribed digital resources such as e-books, online journals, digitised collections and databases, to more general internet resources. However, many of the findings from the JISC studies confirm that although ease of resource discovery is important to learners, they often come across many difficulties which prevent them from finding and using valuable digital resources. Multiple routes to resources, complex navigation and display options will also confuse and distract learners from the core task of resource discovery. Learners are interested in the destination,not the journey to the resources.

Having poor information literacy skills is one of the barriers learners come across when they start studying in further and higher education. Learners will have developed resource discovery habits long before they enter college or university and these habits remain with them unless there is some level of intervention to change those habits. By the time a learner arrives at a college or university, they will have some digital literacy skills and may have adopted some searching habits and information seeking behaviours. These behaviours will have been picked up earlier in their education, sometimes through trial and error and sometimes through formal training. However, their search skills may not be sufficient to allow them to quickly and easily find relevant digital resources to support them in their studies. For example, learners may not know how to access digital resources or know where to locate them.

‘Course text e-books enable students and staff to fit work and study flexibility into their busy lifestyles.’

JISC national e-books observatory project

The complexity of some publisher and library interfaces can create another barrier for learners. Learners are confronted with a range of different interface designs which, according to users, are unintuitive and difficult to use. There are many reasons for this, but for many learners a key reason is that they do not use these platforms on a regular basis and will not have the time or the patience to learn how to use the platform or portal in order to gain access to resources.

In the face of such a complex and diverse range of user interfaces, learners tend to resort to using the simplest, easiest and most intuitive online search tools such as Google or Google Scholar as their primary method for discovering resources. Whilst these search tools may provide a relevant list of search results, they may not include some of the quality, content-rich resources that academic libraries subscribe to. The library catalogue or the Online Public Access Catalogue (OPAC) was also cited as a main way of finding resources.

Learners face a variety of access methods and search protocols in order to discover and find relevant resources. They have to find out which digital resources they have access to through their college or university library subscriptions, and which methods of access they have to each digital resource. Access methods range from IP authentication, to publisher username and password, to federated access management which allows a user access to multiple resources using their college or university username and password.

‘This is much more difficult to use (referring to Library Resources) ....Google in that sense is much easier to use.’

User Behaviour in Resource Discovery: Final Report

The challenge for publishers and libraries is that there is no single type of user and therefore standard approaches will not always meet the needs of all users. Learners at one institution will not necessarily behave in a similar way to learners at a different institution studying the same subject. Disciplinary differences mean that learners in one subject area will behave differently to those in others. There are also differences between learners in colleges, those in universities and those undertaking a higher education course in a further education college programme.

However publishers and librarians can improve this diverse range of users’ experiences of resource discovery by addressing common issues, for example by providing clear and consistent signage and common terminology.

The myriad of ways in which users want to use, share, copy and distribute resources, creates new challenges for publishers and librarians. The need for publishers and libraries to improve the learner’s experience of resource discovery is key to ensuring learners can search and access quality, content rich resources that are relevant to their studies, quickly and easily.

Within this overview you will find a summary of the key findings and recommendations from various JISC reports in the area of user behaviour in resource discovery. This overview provides a summary of the work that academic publishers and libraries may need to undertake to ensure that they can improve the user experience in resource discovery and use.

‘We had a previous training session by the librarian so she got us familiar with search engines like EBSCO etc.’

User Behaviour in Resource Discovery: Final Report

Key findings

  • Poor information literacy skills continue to be a problem in resource discovery
  • Information literacy skills vary depending on the type of learner. Whilst undergraduates are more comfortable with link searches using the library catalogue and federated search engines, researchers and other postgraduates (expert users) tend to use reformulated search, system suggestions, personal knowledge and experience on publisher search facilities
  • Google, followed by Google Scholar, Wikipedia and YouTube are the most widely used search engines because they are simple and easy to use and always provides a list of relevant results
  • Learners are changing the way they want to access, use, share, copy and distribute resources. In particular they want to be able to search for and access resources regardless of where they are and which device or platform they are using
  • Complex, unintuitive and non-standards based publisher and library systems make it harder for learners to search and access resources
  • Search is a key tool in resource discovery for learners who will use a combination of aggregated, differentiated and federated search for resource discovery
  • Library resources are perceived as credible and provide quality material in a broad range of subjects
  • Learners are not always aware of the support and training available from library staff

'I now own a Sony portable reader – this has revolutionised my reading habits. I find that being able to transport 160  e-books in one device invaluable. I read on trains and in cafes.’

JISC user behaviour observational study Scholarly digital use and information-seeking behaviour in business and economics: an evidence-based study

Key recommendations

  1. Encourage greater standardisation of platforms for all types of resources
  2. Include spelling auto suggestions when searching
  3. Provide an in-built dictionary or encyclopaedia
  4. Ensure keywords are present in every article
  5. User interfaces need to be simplified and use a standard language and terminology
  6. Make users aware of changes to database functionality
  7. Address the ‘time out’ issue
  8. Improve facilities to support dynamic searches, retrieval and re-access process
  9. Include facilities for annotation of resources and analytical support functions
  10. Search tools need to take account of the different subject knowledge, experiences and skills of users
  11. Search mechanisms should be regularly reviewed and evaluated, using where possible user analysis and end user feedback
  12. Implement federated access management which simplifies access for users
  13. It is important to provide support to users to improve both their digital literacy and information literacy skills throughout the duration of their study
  14. Usability and user testing should be undertaken on platforms and portals
  15. Accessibility is important for all stakeholders

Learners

The traditional learner is now just one of many types of learners who undertake resource discovery. Publishers need to be aware that postgraduates and researchers will have different needs and skills compared to undergraduates. They also need to consider that many digital resources will also be used by learners outside traditional higher education, for example in further education colleges, through work-based or distance learning.

Publishers need to recognise that UK colleges and universities now cater for non-traditional learners who have different needs to traditional learners. The non-traditional learner will often require remote access from home, from work, at wireless hotspots, from partner institutions and using mobile devices.

It’s not just non-traditional learners who access resources remotely. According to the studies undertaken by JISC, a significant amount of access by all users was in the evenings and on weekends which are outside the opening hours of many institutional libraries. For example, the JISC national e-books observatory project found that almost a third of page views of e-books were made away from the institutional campus.

Information literacy skills

One of the key implications from the JISC studies is the importance of information literacy skills in successful resource discovery.

The studies found evidence to suggest that information literacy skills are lacking. For example, many of the participants in the studies do not understand how to assess the quality of materials they find. Google or Google Scholar also have lower thresholds of information technology literacy, and are considered their ‘friends’ because of the apparent higher yield or success rate of search results. ‘Google books are good’ is a user perception; the reality may be different. Learners may not even be aware that their skills need improving or enhancing and so advice and training needs to be in put in place at a curriculum level as well in the library and on the resource platforms.

Library staff can be instrumental in providing the relevant support and training to improve and enhance the information literacy skills of learners. This support and training is not something that should be undertaken at induction, but should be available and supplemented throughout the time the learner is at the institution. As users improve their information literacy skills, more sophisticated searches become possible. It is therefore necessary then to ensure that users have the opportunity to gain these skills.

Learners’ relationships with library staff in many institutions are mundane, with users mainly using staff to help them find a book. Learners tend to go to their peers or social networks to source materials, or friends studying at other institutions. They often do not realise the potential support and training subject librarians can provide in helping users to search and access relevant digital resources and therefore do not discuss their needs with library staff.

With the increase in non-traditional learners in higher education, there are now many learners who may not have the time or resources to visit the institutional library on a regular basis. Learners will therefore benefit from accessing library support and training both on campus and from remote locations, such as from the library web pages and from portals.

Library staff may want to consider how they provide training and support to learners in using search, especially more advanced search. Effective searching routines depend on higher levels of information literacy and these skills can be provided by teams in the library. Learners should be provided with examples of how to carry out a successful search, including a demonstration of how to search more effectively.

‘Asking for help (referring to a librarian) is better than trying to do it yourself because I didn’t have a clue’.

User Behaviour in Resource Discovery: Final Report

Mobility

Learners and staff are now more mobile than ever before and access needs to account for this mobility. The ability to move resources to mobile devices is complicated by the sheer variety and quantity of devices owned by users. It is made more complex as different devices use different syncing protocols and there is no single process for rights management for mobile devices. Publishers may want to start planning how they will make the most of the potential of mobile devices that many learners now have in their possession.

However, other students and researchers said they preferred to access the library resources while on campus because it provided less distraction and being able to view a printed version of the material where an electronic version was not available was of high importance.

Copyright restrictions

The ways in which users want to use resources, share resources, copy resources and distribute resources, creates new challenges for publishers and librarians. Having gained access to and identified relevant resources, learners may want to use content from these resources to support their learning. Printing and copying restrictions in place to protect content need to be balanced against the needs of users to print, copy and share resources with other learners.

Learners may wish to print resources for those times when connectivity is poor, non-existent or expensive. They may not be able to access electronic resources in specific locations and at certain times. Learners may want to copy resources to mobile devices to allow them to access these resources whilst travelling, at work or at home. They may also want to share resources with others who will more than likely be learners or staff at the same institution who have the same rights to use those resources. User preferences change according to size. For example they want to be able to print large documents but are happy to read single pages in a web browser.

Most resources allow users to use them for their own use, but for very valid reasons do not provide rights for distribution.For example, learners may want to share resources with others from outside the institution such as researchers across a variety of educational institutions. These users may be at an institution that also has rights to use the resources, or they may be at an institution that does not have the rights or at an overseas institution.

Distribution also affects staff at institutions who may wish to place copies of resources on a virtual learning environment or similar protected online environment. Embedding URLs in many cases adds layers of complexity that learners see as barriers rather than as enabling access to resources. Linking also relies on permanent URLs and the library has to be involved in this area as there are constant updates.

Rights management often dictates how users should use platforms rather than being based on real knowledge of user behaviours. Any limitations placed on users need to be formulated following an analysis of user needs and these will probably vary from collection to collection.

Publishers need to balance the frustrations and irritation caused by multiple warnings with the need to protect and secure collections from illegal copying. Publishers need to recognise that though users will not have directly purchased the resource they are valid users of the collection.

‘I did do a library course on searching the various databases, but that was some time ago now. It was in 2001, probably out of date now. They described all the strategies and how different databases have different ways of working. I don’t care what method they use, it would just be nice to have a consistent one. But not many of them have, well some of them do have help, I tend not to use them unless I actually get stuck.’    

User Behaviour in Resource Discovery: Final Report

User interfaces

The JISC studies found that users often approach interfaces with minimal or non-existent training, which meant they faced a lot of difficulty and frustration when trying to use these electronic resource discovery systems. The range of publisher and library interfaces which all use different features, language and access controls often compounds this problem.

Design of portals and platforms needs to focus on the needs of the user, who rarely has the time or motivation to learn complex navigation or options. Interface designs need to provide the user with a seamless experience regardless of the type of device, browser, location and connection used. Most of all, though, interface designs need to consider the usability of the user interface and how easy and intuitive it is to use.

Many interfaces have evolved as collections grow, as technology changes and authentication processes are revised. Though this may sound obvious, as features and functionality are added these can reduce the usability to end users. The user interface of many platforms and portals are designed by technical staff whose primary focus is often on technical features and who do not always consider the user or how new users to their platform will engage with and use it.

Interface design should be focused on the process of resource discovery and how users navigate through the interface. Clear and graded routes to more advanced options will ensure that those users who wish to use these options will be able to use them effectively. Sophisticated and complex search routines, though useful for a small number of users, can alienate the majority of users and should be avoided.

In the past, publishers have been able to rely on a relatively closed and standard system that more traditional learners could use to access resources. It is also no longer possible to expect that the majority of users will be using a wide JANET connection on campus with Internet Explorer on a Windows desktop. In the recent past (2006) you could rely on 85% of internet users using Internet Explorer as their main browser. Today the figure is just over 50%. As the ways in which users engage with the internet change and new devices become available, a reliance or requirement on a single browser or operating system creates barriers for users in searching and accessing resources.

Learners need to be able to access platforms from remote locations using a diverse range of operating systems, browsers, connections and mobile devices. As remote access continues to grow, portals and platforms need to be designed to take this trend into account.

Simple interfaces

Too many options are putting users off and making searching appear overly complicated in the initial stages of a search process. Often search features are too complex and do not help inexperienced users to formulate their searches and select appropriate options in order to narrow the search results and obtain relevant documents. In comparison, Google’s simple search makes it a user-friendly search engine. The interface is clean and attractive. It is not cluttered or filled with any other unrelated information (when initiating a search). Users tend to know what to do intuitively. Where possible simple interfaces should be used as these are preferred by learners.

‘Going to the library database and then putting in some keywords, first of all there are so many options there, you know, do you want this, do you want that, I mean keep it simple!’

User Behaviour in Resource Discovery: Final Report

Any improvements to interfaces should learn lessons from how popular and effective Web 2.0 tools, services and social networking have designed their user interfaces.

Standardisation

While recognising the need for competitive differentiation and the difficulties that standardisation presents, the JISC studies recommended the adoption of interaction techniques that enable user interoperability between different resource discovery systems. For example, Google, Bing, Yahoo and AskJeeves are all different internet search engines, but they all use a very similar user interface interaction method: One text input field, and a ‘search’ button. The differences in their popularity lie in differences in the scope and power of the underlying search algorithms. Users, however, learn very easily how to operate between these different search engines according to their different needs, and without a user manual.

Navigating from one system to another – all of which have different functionalities and different bells and whistles with respect to searching, limiting/refining, indexing, saving and storage or exportation – is confusing for all users. Users literally have to ‘re-frame’ their minds when they move from one system to another and this requires patience and persistence because it is time-consuming.

Dynamic searches

Users rarely follow a static pathway or a pre-defined sequence when searching. Platforms need to be able to accommodate the various combinations of search activities that users will undertake.

Security and locking out

The balance between securing access to resource collections and ease of access by end users is a difficult one. Learners may want to access these resources in the future; this could be within the hour, on another day or later in the academic year. It is important that open logins are timed out and ‘closed’ to protect valuable digital resources. However, it also needs to be recognised that the process of re-accessing an existing resource needs to be simpler and easier. Going through the hurdles of access each time a user wants to access the same resource will create barriers to resource discovery.

Security and authentication processes that time out access to a resource need to be intelligent enough to remember users and where they were regardless of the device or location that was previously used. Platforms need to be flexible to enable users to move in and out of these resources and to return to and continue a previous search.

Publishers may want to evaluate and review how users use and view resources from their collections to ensure that users are not continually ‘locked out’ when using the platform to access the collection.

Many times I had been timed – when I tried to go and find the paper, there are many instances when (the collection) does not have it, I don’t want to mark it and have to come back later and lose my thread. You go off then the phone may ring and you come back and session has tied out and lost my marked list, so frustrating seems like wasted time.’

User Behaviour in Resource Discovery: Final Report

Usability testing

According to the JISC studies, designing platforms and portals is usually undertaken without sufficient user testing. The usability of a system is vitally important to the user who will be less interested in the process of discovery and more in the results. Libraries and publishers ensure they test the usability of their systems by different types of users. Such usability testing will ensure that the design of such systems meets the needs of the majority of users.

‘Don’t assume you know your users, know your users.’

Caren Miloy, JISC Collections

Accessibility

It is important that any system for resource discovery is accessible to users. A further consideration where accessibility takes on a new significance is the trend towards mobile access to resources.

Accessibility is important for all stakeholders. For institutions it influences an institution’s exposure under the Disability Discrimination Act (DDA). Previous work by the JISC national e-books observatory project reported that disabled users were facing many challenges in using e-book platforms. One of the key recommendations was to focus on user-centric design to enable all users to make use of e-book content. For publishers it influences customer experience and potential library procurement decisions.

When a resource becomes that significant to a learner’s success accessibility is more than an ethical consideration – it becomes a legal responsibility. Under the DDA education providers have very clear duties to provide an accessible service. There is an increasing (and justified) pressure on institutional procurement policies to ensure content services within the institution are fully accessible to all users.

‘In Aesop’s fable the Fox and the Crane, the crane invites the fox to supper providing refreshment in a narrow necked flagon. The crane – with her long slender beak – can easily reach the contents but the fox can only sit and watch in envy. For people with disabilities, e-books can evoke much the same feeling. The content, which opens up a feast of learning and enjoyment, is there: if only it can be accessed in the first place.

JISC TechDis (2010)

JISC TechDis have considered the following to be good practice. In terms of logging onto a platform:

  • Ensure password requirements are not over complex in terms of length or characters required
  • Ensure that tab order reaches the login box in minimal keystrokes
  • Ensure confirm buttons are close to the text field and of adequate size for users with poor mouse skills and let the Enter key work as confirm
  • Ensure text can be adequately magnified to support visually impaired users
  • Ensure all text entry fields are appropriately labelled
  • Ensure adequate visibility of the cursor position for users using keystrokes or who are visually impaired
  • Ensure that logging out is simple and that the logout button or link is situated prominently
  • Ensure any system messages are meaningful to the users

In terms of navigation:

  • Skip links can greatly enhance the experience of keyboard-only users as well as screen reader users
  • It is important to many groups of users that systems display consistency across all titles wherever practicable. Familiarity of layout and design increases usability
  • Icons should be consistent throughout the platform and not contradict common conventions
  • Avoid using coding that dynamically changes pages (eg updates information within an existing page without refreshing it). Where pages change dynamically some users, for example screen reader and screen magnifier users, will be unaware of the change. These users may need to be alerted in some other way or have an alternative means of accessing the updated page

Publishers are also recommended to:

  • Use skip links intelligently and consistently to allow keyboard-only users and screen reader users to navigate more effectively
  • Decide how font size and colour changes will be available to the user. Clearly signal this to the user where they need to go to get the information
  • Ensure text can be adequately magnified to support visually impaired users. A range from 8 point to 30 point (hard copy equivalent) is recommended
  • Ensure text reflows when magnified
  • If text-to-speech is provided, signpost clearly how it should be accessed. If text-to-speech is not provided ensure text is selectable for screen reader users and text-to-speech users
  • Make sure user testing includes disabled users

In terms of extracting content from a resource, publishers are recommended to:

  • Test the process of extracting content with disabled users
  • Clearly signpost what functionality is available; whether this is simple cut and paste or a more sophisticated system
  • Clearly signpost what functionality is not available. Where rights managements prevents users extracting text for their own use, users should be advised in order to avoid wasting their time with trial and error
  • Manage user expectations. A simple cut and paste solution that is accessible to all may be better than a sophisticated citation system that is inaccessible to assistive technology users. Or offer both and clarify the benefits of each
  • Focus on simple, robust functionality and do not assume that all users are mouse users

‘Google has come along and revolutionalised the whole search concept... Google is more dependable than anyone else’

User Behaviour in Resource Discovery: Final Report

Search

Search is a key tool in resource discovery for many learners. Once users have made their way through the access gateway they want to be able to find resources that are relevant to their studies, quickly and easily. How learners use search to discover resources needs to be better understood by service providers to ensure that any search process meets the search needs of the user.

Learners want to have an increased number of appropriate resources available to them and have the necessary skills to use them more effectively. Learners will often find that their searches fail to produce any results. This indicates a range of issues, from the information literacy skills of the learners, problems with search terms and the way in which the resources are catalogued.

Learners will face multiple search options covering the Online Public Access Catalogue (OPAC), federated search, institutional portals and repositories, publisher platform search and the ubiquitous Google. They may not realise that in order to search some resources they will need to authenticate and access a separate platform before those resources are either included in a search or often have to be searched separately.

There are similarities in the way in which learners use platforms and portals and the way in which users use Google. Learners will often make a single search and rely on a single result. They are unlikely to make multiple searches or click more than one result and advanced searching is rarely used.

Partly because of the ease of use of Google and Google Scholar, and in particular their reasonably high hit rate, many users have developed a naïve belief that Google is a reliable source, and that it has high quality articles.

In the UBiRD study it was also observed that users often gathered information from various resources and put it together in an easily accessible place. This was either a folder, a bookmark in a browser or a number of tabs.

Users also made use of the browser bookmarks or favourites, however no users in the UBiRD study used social bookmarking services on the web.

Types of search

Before undertaking any kind of search, a learner must first know which collections and other resources their college or university subscribes to, and which are provided externally. They also need to know whether they need to authenticate before certain collections are included in a search and the type of authentication method used. Problems can arise if different authentication processes are required for different collections.

Library search

The library search catalogue used to be a relatively simple affair, but with the increase in e-resources and digital collections, it has become more complicated. A balance has to be made between quality and convenience. A key consideration for librarians when choosing a library search catalogue is the desirability of a trusted portal that provides the ease of use associated with commercial search engines, whilst still providing more advanced retrieval, storage and analysis options. It should also enable easy transition between ‘internal’ resources (library catalogues, EBSCO, Science Direct), and ‘external’ resources (the internet, Google, Google Scholar, YouTube).

‘If I am just searching for a topic like this I would have to search through a lot of results in the library site.’

User Behaviour in Resource Discovery: Final Report

Aggregated search

Aggregated search is when a user can search multiple search engines and search tools with a single search term, avoiding the need to go to different and diverse locations to search multiple collections. However, the simplicity of the search may not provide the results that users are looking for and can often lead to complex results.

Users need to be clear about what search engines and search tools they will be using. A simple search box is all well and good, but unless that simple search box searches everything, users may have trouble determining whether they have undertaken a full search. Not all collections allow their indexes to be searched and therefore users may miss out on a relevant collection.

Publishers need to ensure that their indexes are open to aggregation. This will increase the visibility of their collections to users who are more likely to use simple aggregated search over more complex searches.

(This resource) has the worst searching capability ever. It doesn’t do the nearest match (ie smart text searching). You need the exact title and it isn’t easy to find that either.

User Behaviour in Resource Discovery: Final Report

Differentiated search

Aggregated search may seem like the obvious solution to many; however simple aggregated search terms can lead to complex results because it requires that the search covers all resources. Differentiated search, on the other hand, enables the user to reduce the number of collections being searched by using limiters.

Offering users a choice between aggregated and differentiated search can improve the chances of effective resource discovery. It may be necessary for search interfaces to have flexibility built in, to allow users to customise them as their information literacy skills increase and improve.

Differentiated search may be seen as a solution and is not an obvious barrier to effective searching. One of the disadvantages with differentiated search is that the proportion of searches undertaken is reduced at each differentiated stage. Users with effective search skills will often be able to differentiate their searches, whereas users without those skills will rely on a single search. This is apparent from Google which has a simple search on the front page, but has a range of different search types, eg images, news or blogs, in its navigation bar. However, these attract fewer searches than the main home search.

Federated search

Federated search is a method that combines a single search interrogating multiple collections and resources.

The advantage for the user is that they face a single search field whilst still maintaining a breadth of search. When approaching the design of a federated search interface, users need to be clear about what collections and resources they are searching. Do they need to authenticate before certain collections are included, do they know which collections are included and which are excluded?

Are they aware of what can be searched both inside and external to the institution? When federated search is performed against secured collections, will the appropriate authentication be passed to the relevant collections? Problems can arise if different authentication processes are required for different collections.

A well designed federated search tool will ensure that users have a seamless search experience. Users in the UBiRD study expressed a liking for the federated search engine. However, most users were unaware of where their resources were coming from; they did not understand how the federated search engines worked or what the differences were between the different sources of information, ie the different databases – which is perhaps as it should be.

‘… it brings to you all the related searches together’

User Behaviour in Resource Discovery: Final Report

Indexing and tagging keywords

It may be obvious, but publishers and librarians should check their search actually works and produces the results that they would expect. Publishers and librarians need to ensure that resources within their collections are tagged with suitable keywords. Human intervention in search indexing or tagging may be required in order to ensure that users have a successful search. An index needs to be continually evolving to ensure that it meets the changing needs of end users. Generally tagging or cataloguing is a one-time affair. However, enhancing and improving the keywords used within a collection based on user behaviour and on search terms gathered on previous searches can make it easier for future users to search and find appropriate resources.

Evaluating search queries

Users often ‘hit a wall’; that is they reached the point when, after performing a number of searches which returned unsatisfactory results and exhausting a number of resources that they believed were appropriate for finding information on a given topic, no further progress was made.

Users would get ‘massively irritated’ if search engines/ interfaces did not work in the way they expected.

Publishers and librarians should review and evaluate the search queries used to better understand how users are searching their catalogues and collections. They should use the outcome of these evaluations to improve and enhance search results for users. For example, publishers should regularly check their search algorithms so that they produce the results they would expect from the search terms used. Regular reviews and evaluations should be undertaken to ensure that search results are improved over time.

Advanced search for expert users

Learners have different levels of information literacy and this has an impact on their search and resource discovery skills. For example, the JISC studies found that in general postgraduates and researchers were more effective in searching and had higher level information literacy skills than undergraduates. These users also had a higher level of subject knowledge that additionally helped make them expert users in search.

Graded interfaces that allow users to add layers of complexity as they refine their searches will allow expert users to discover relevant and suitable resources, though these should be seen as additions to a simpler interface rather than the first interface users engage with. The studies showed that users rarely use advanced search. Therefore, more support and training is needed to support searching and the first level of search needs to be more effective in finding the right results from the search terms used.

'A key message for librarians is the desirability of a trusted portal that provides the ease of use associated with commercial search engines whilst still providing more advanced retrieval, storage and analysis options.’

Users’ trust in information resources in the Web environment: a status report

Access management

Access management is the term used to describe the process of permitting authorised users to access protected online information, usually in the context of web pages or web-based applications. It describes both the means by which an online information resource provider decides whether to allow access to a protected area, and also the administrative process of allowing access for approved individuals. Authentication processes need to reflect the reality of this mobility and to ensure a seamless experience for users. Yet access processes and gateways are perceived by learners as barriers to access and the term ‘access gateway’ would be considered by many as an oxymoron.

Multiple logins and login requests should be avoided and if possible integrated into a single request for the user’s credentials. Different processes off and on campus can confuse and frustrate users who may have learnt how to access a collection on campus in one particular way and will not understand why a different process is used off campus. In many cases there are valid reasons for different processes, using IP authentication for example, and it is useful to hensure that users are aware of the differences

Access processes need to be designed with end users in mind. They need to be simple, transparent and where possible integrated into the authentication process. Though authentication and access gateways are in place to protect collections, publishers need to ensure that overly secure restrictions and processes are avoided as this could result in less security, as users may use weak but memorable passwords. Publishers should also avoid multiple layers of authentication and where possible use the same authentication credentials for different collections to avoid confusing and alienating users.

Remote access is becoming more important to learners; they want to be able to access resources at a time and place to suit them, often on a device of their choosing. This places demands on portals and platforms in terms of access, authentication and usability.

IP authentication

In the past, publishers could often rely on IP authentication as virtually all users would access their platform from campus computers. Today learners are more likely to be accessing the same platform from their home desktop computer, a laptop in a coffee shop, or a mobile phone on the train. Learners are demanding a seamless and single approach to resource discovery and do not want to have to remember or recall different authentication processes depending on their location or their device.

IP authentication provides an easy way to identify an organisation and to ensure that no one outside the organisation can access the product. It relies on an institution having a fixed IP address or range of IP addresses through which it connects to the internet. Only requests from those IP addresses will be allowed. However it is impossible to identify parts of an organisation, say the department of chemistry, using IP authentication. Publishers have therefore been forced to make their resources available to the entire organisation, and have priced them accordingly.

Providing campus-only access to resources may limit the knowledge a non-traditional learner can acquire. By offering off-campus access, publishers can enhance the experience of both traditional and non-traditional learners.

Without using a proxy solution, users external to the institution will be unable to authenticate and access the resource. In these cases an alternative method of authentication is required. A proxy server is a server that acts as an intermediary between a workstation user and the internet. When a user connects to the proxy server and requests an online service (eg a file or a web page), available from a different server, the proxy server connects to the specified server and requests the service on behalf of the user. Federated access via an IP-Proxy Server solution can provide secure and seamless remote access for online resources that are only available through IP authentication.

Read JISC's IP Proxy Servers FAQ

‘…that’s what I call it, a dead end. I don’t know what to do.’

User Behaviour in Resource Discovery: Final Report

 

Federated access management

JISC has devoted significant funds to the development and implementation of the next generation access management system which has had significant implications for UK colleges and universities.

Federated access management builds a trust relationship between Identity Providers (an educational institution) and Service Providers (a publisher or software provider). It devolves the responsibility for authentication to a user’s home institution, and establishes authorisation through the secure exchange of information (known as attributes) between the two parties. How authentication is carried out by the institution and how rights management is carried out by the service provider is left up to the respective parties. In doing so, federated access management depends on a certain level of trust.

A federation is a group of institutions and organisations that sign up to an agreed set of policies for exchanging information about users and resources to enable access and use of resources and services. These trust agreements are managed by federations which are typically established at a national level. The UK federation is called the UK Access Management Federation for Education & Research.

The federation combined with identity management software within institutions and organisations can be referred to as federated access management.

The role of federated access management in improving resource discovery for users should not be dismissed by publishers. With users depending on their institutional credentials for authentication, publishers can spend less time on administering accounts or access processes.

Benefits for publishers and other service providers include:

  • No need to maintain a user database as authentication is performed by the institutional Identity Provider, reducing the need for storage and processing of personal data and therefore compliance with the Data Protection Act
  • Personalisation through finely controlled access to services and resources, allowing subscription and authorisation to individuals, departments or courses and being able to provide more accurate implementation of licence conditions
  • Improved security as users take better care of their username and password because it is also linked to their email account and other valuable resources and services
  • Organisations able to take better care of assertions because they are centrally controlled

Federated access management allows users to authenticate at their home institution. They can use the same username and password for both internal and external resources and the same authentication process can be used for resources from different publishers and from different collections. As a result institutions have greatly reduced user administration and enabled institutions to open up their own resources to other institutions more easily.

Federated access has many advantages to the user, institution and publisher, but has one major issue from the user perspective: the discovery problem.

Learners can search and access content using a range of different methods including direct access, via a library portal, through OpenURL resolvers, aggregators and search engines. Depending on which service the learner uses, the service provider will have to establish whether the user should be granted or denied access – they need to know some information about the user. The user is normally redirected to their home institution to log in. The difficulty for the service provider, however, is knowing which institution the user belongs to.

There are several solutions to this problem. The first is to avoid the discovery problem altogether, which can be done by the college or university listing all the resources available to users with carefully constructed links that sends the user to the institution’s login page. Once logged in, the user is sent directly to the resource. The problem with this solution is that a user may not initiate their search for resources using the library portal.

The second solution, client-less discovery, is when a service provider sends the user to a ‘Where are you from’ or WAYF page where the user selects which institution they are from. WAYFs can either be centrally provided by the federation or built into the service provider’s own platform. Service providers that are members of more than one federation have their own in-built WAYF. An in-built WAYF is preferable to a centrally provided WAYF because the user stays within the same site and the WAYF can be configured to only show a list of those institutions that actually subscribe to that resource, whereas the federation WAYF lists will list all of the institutions.

The problem with this solution is that as more institutions adopt federated access management worldwide, the list of institutions in the WAYF also grows. The UK federation alone, for example, has already reached the point where users have to find their home institution from many hundreds of options. This is not a good experience for the end user at the best of times; given the current increase in usage of mobile computing platforms such as smartphones and netbooks where this issue becomes a greater problem, it will only grow steadily worse. A predictive search box for both country and institution would help solve this problem.

The second of these issues is seen on those service providers that provide their service to customers worldwide, and therefore have to exist on multiple federations (since federations are currently largely geographically based). In this case, before the service provider can ask the user to indicate which institution they are from, they usually first narrow down the list of options by first asking them which federation they belong to. This is also not a particularly good experience for the end user; not least because the vast majority of users have no idea what a federation is, let alone which one is the correct choice for them. While federations are largely geographically based, however, this issue can be somewhat mitigated by asking the user what country they are from in order to deduce the correct federation – a question that makes infinitely more sense to the average user.

Client-mediated discovery solves the main issue of client-less discovery – the fact that a user has to manually indicate where they are from – through the use of a client that essentially answers that question on behalf of the user or removes the need to ask the question in the first place. The user needs to have a client installed and configured to do this for them.

Client-mediated discovery is undoubtedly the best actual solution to the discovery problem since it is a fairly intuitive and effortless process (other than initial setup of the client) for the user, thus providing a good end user experience. However, the user needs to have a client installed and configured correctly. At the present time, such clients are relatively uncommon, and thus client-less discovery will remain important for at least the short to medium term future

Other problems identified by the Service Provider Interface Study included:

  • The number of login steps a user has to make to gain access which varies according to the WAYF
  • Differences in language and terminology in the user interfaces and WAYF
  • No options for WAYFs ‘remembering’ users’ country and institution from a previous session

The main conclusions from the JISC Service Provider Interface Study is the lack of consistency in the way that every step in the process is presented to users. One of the main recommendations from this study is to produce a set of guidelines on how best to implement a WAYF service or a client-mediated plug in.

'Students don’t like ‘multiple passwords’, they want their access kept simple.’

Elizabeth Malone, Head of Content Development, Kingston University

Benefits

By implementing some of the recommendations in this strategic summary, the following benefits can be obtainable:

For publishers
  • More visibility for collections
  • Increased use of resources
  • Sharing of resources and sources of resources
For librarians
  • Better use made of collections by learners and staff
  • Expanded collections
  • More effective use made of subscriptions and e-book collections
  • Better experience for learners
For institutions
  • Improved attainment and achievement by learners
  • Enhanced assessed work using a wider variety and selection of resources
For learners
  • Easier and faster resource discovery
  • Wider range of resources available

Where next?

JISC is funding further work in this area under the Enhancing library management systems programme.

This programme will include a series of case studies that will be available later in 2010. The case studies will cover implementing and using one of the new resource discovery interfaces and evaluating the extent to which they meet learners’, teachers’ and researchers’ needs.

The programme will also fund projects that are intended to explore existing but still emerging means of enhancing existing library catalogue web interfaces and the visibility of library resources and services. The primary purpose of these projects is to provide practical examples which others can follow or use to inform their decision-making. The projects will investigate how to enhance existing Online Public Access Catalogue interfaces by using Application Programming Interfaces (APIs) and Web Services to include additional content or services. A further set of projects will look at developing widgets and/or portlets to embed access to library resources and services within institutional, social and/or personal web environments.

  • Increased understanding of the possibilities for enhancing existing library catalogue web interfaces and the visibility of library web resources and services within the current library systems environment
  • Increased adoption by institutional libraries of existing technologies and approaches for enhancing the usability of library catalogue web interfaces and the visibility of library web resources and services
  • Outputs from these projects will, therefore, have a ‘how you can do this’ focus
Key outputs from these projects include
  • Guides to what has been learnt during the projects and how enhancements were made which others can follow
  • Re-usable code (where applicable) with supporting documentation

Under the same programme, JISC is also funding a series of institutional case studies which will illustrate practical issues libraries face in a number of ‘problem areas’ currently associated with library management systems. The purpose of these case studies is to identify and document specific constraints which prevent libraries from developing or providing services as they would wish, to illustrate how improved systems integration could support learning and research and to document any ‘workarounds’ which individual libraries may have found to address the issues.

JISC Collections is working with publishers to raise awareness of these issues and to encourage the development of effective resource discovery and interface design.

References

  • Browne, T., Hewitt, R., Jenkins, M. and Walker, R. (2008) 2008 Survey of Technology Enhanced Learning for higher education in the UK Oxford: UCISA
  • Connaway, L. and Dickey, T. (2010) The Digital Information Seeker: Report of the Findings from Selected OCLC, RIN, and JISC User Behaviour Projects. OCLC Research and JISC.
  • Fast, K.V. and Campbell, D.G. (2004) I still like Google: University student perceptions of searching OPACs and the Web. Proceedings of the American Society for Information Science and Technology, 41 (1), p. 138.
  • Nicholas, D., Rowlands, I., Clark, D., Nicholas, T. and Jamali, H. (2009) JISC user behaviour observational study: Scholarly digital use and information-seeking behaviour in business and economics: an evidence-based study. JISC.
  • Pickard, A., Gannon-Leary, P. and Coventry, L. (2009) Users’ trust in information resources in the Web environment: a status report. JISC.
  • Sharpe, R., Beetham, H., Benfield, G., DeCicco, E., Lessner, E. (2009) Learner Experiences of Elearning Synthesis Report: Explaining Learner Differences JISC.
  • Smith, R. (2009) Server Provider Interface Study. JISC. Wong, W., Stelmaszewska, H., Bhimani, N., Barn, S. and Barn, B. (2009) User Behaviour in Resource Discovery: Final Report. JISC.
  • NISO/UKSG KBART Working Group. (2010) KBART: Knowledge Bases and Related Tools. NISO. JISC national e-books observatory project final report. (2009) JISC.

Web references

Documents & Multimedia

Bookmark and Share