Science advances by providing insights into the natural world and theories are built on robust observations. There is now growing interest in turning the scientific method on to itself and using this approach to understand the factors that influence the behaviour of scientists, and the robustness of the research they generate.
There has been a lot of interest in research reproducibility over the last ten years, particularly in the biomedical sciences, and this debate is now mainstream. Recently, the focus has shifted towards identifying ways in which science can be made more efficient, by improving the quality of the work we produce and the speed with which self-correction occurs. Jisc has also just published a report on the digital tools and services which will help the replicability of research (pdf).
So, what can we do to ensure that the practice and methods of scientific research are rigorous and the outputs robust and reproducible?
The power of networks
A group of researchers recently launched the UK Reproducibility Network, supported by Jisc and a range of other stakeholders, including funders and publishers.
Our aim is to bring together colleagues across the higher education and research sector, forming local networks at individual institutions to promote the adoption of initiatives intended to improve research.
There are many people in the UK who have been working on this topic, but we sensed a need to bring them together and harmonise activity to make the most of our collective efforts. This is very much a peer-led, grassroots initiative that will allow academics to coordinate their efforts and engage with key stakeholders.
The network will operate across three main areas:
- Meta research: we will conduct research into the factors that influence research reproducibility, and the effectiveness of initiatives intended to improve this
- The network itself: it will help advocate locally and nationally for the adoption of key initiatives such as the use of open research practices
- Providing training, disseminating better practice, and engaging with stakeholders: the success of the network depends on having a range of different people from different disciplines involved to meet the needs of all stakeholders, and ensure we can learn from each other
Reproducibility in a nutshell
If you’re reading this blog as a researcher, and this issue is one that could impact your career, then you may find our manifesto for reproducible science, published in Nature Human Behaviour last year, interesting.
The proposals within the manifesto give all of us in research professions and academic leadership positions a helpful reference list to keep reproducibility at the forefront of our approach:
- Protect against cognitive biases
- Improve methodological training and independent support
- Collaborate through the team science consortia
- Promote study pre-registratio
- Improve the quality of reporting
- Protect against conflicts of interest
- Encourage transparency and open science
- Diversify peer review
- Reward open and reproducible practices
Reporting and dissemination
Part of the issue with ensuring that research is reproducible is in how it’s published.
For example, results that are viewed as “uninteresting” (eg null results) are less likely to be published and the data and analysis code underlying published results is often not available for scrutiny.
Various solutions to these problems have been proposed, such as the pre-registration of study protocols and analysis plans, and the publication of data alongside articles. However, these approaches require training as well as platforms to support, for example, open data. It’s also important to ensure that these approaches actually improve the quality of our work, without creating unnecessary bureaucracy.
This is just one example that illustrates the need for a peer-led network that works with stakeholders.
If new initiatives are introduced that have not been developed in collaboration with end-users, they are much less likely to succeed. Widespread adoption of these incentives will require a cultural change that will be accelerated if there are researchers actively advocating for them.
How do we incentivise change?
There is some positive work already taking place, from the Royal Society’s campaign and guidance around changing the research culture, to projects such as the research data champion scheme run by Jisc.
The champions are working to support colleagues within institutions and spread the word about how we make research open. Jisc has also just launched their open research hub service which will make it easier for researchers and research managers to showcase open data. If data are findable and reusable, then we are one step closer to being able to assess the reproducibility of research and reassess the value of research outputs.
People choose careers in science to have impact, improve lives, and to tackle some of the major challenges of our times. As a community, we need to make sure that the methods we ask early career researchers to adopt open doors for both them and their research. Reproducibility is central to making this a reality and to creating a sustainable research economy as we exit Europe.