Research Waste...Too much Quantity where's the Quality?

Ioannidis JPA et al. Increasing value and reducing waste in research design, conduct, and analysis.The Lancet, 2014: 383:166 - 17
Correctable weaknesses in the design, conduct, and analysis of biomedical and public health research studies can produce misleading results and waste valuable resources. Small effects can be difficult to distinguish from bias introduced by study design and analyses. An absence of detailed written protocols and poor documentation of research is common. Information obtained might not be useful or important, and statistical precision or power is often too low or used in a misleading way. Insufficient consideration might be given to both previous and continuing studies. Arbitrary choice of analyses and an overemphasis on random extremes might affect the reported findings. Several problems relate to the research workforce, including failure to involve experienced statisticians and methodologists, failure to train clinical researchers and laboratory scientists in research methods and design, and the involvement of stakeholders with conflicts of interest. Inadequate emphasis is placed on recording of research decisions and on reproducibility of research. Finally, reward systems incentivise quantity more than quality, and novelty more than reliability. We propose potential solutions for these problems, including improvements in protocols and documentation, consideration of evidence from studies in progress, standardisation of research efforts, optimisation and training of an experienced and non-conflicted scientific workforce, and reconsideration of scientific reward systems.

The recommendations were


Ten options to improve the quality of animal research
Protocols and optimum design

Use of realistic sample size calculations
Effect-to-bias ratio

Programmes for continuing professional development for researchers

Reproducibility and reward systems

If you put your head above the parapet, people are out there often will take a pot-shot. Sometimes these shots can be fair-do's and often quite destructive and hurtful. Sometimes truth hurts, sometimes people talk mushroom food.

The link to this paper was provided in response to one of our posts, maybe by an MSer or maybe by a have-a-go Researcher. There is a need to up the quality of some papers, so researchers have a read and for the non-researchers if you read science papers have a think about some of the point to help decide if the contents are quality or not.

These are opinions and people will not agree with all of them, including me. There were many comments to this paper made here (click). However, there is a point. As you can see on the Blog there are vast differences in the quality of studies reported, including many clinical studies. Many animal studies are not intended to spawn clinical trials, but some are. This is perhaps where the authors of the comments are aiming the majority of their comments and so get some views on aspects of animal work wrong. However, canning non-reproducible stuff early saves lots of money in the long run and stops us providing false hope.

There are now thousands of journals, many online only and as they can charge for publication, an income generator. This means that they need to fill pages. As such quantity appears to be the name of the game and quality can go out of the window. As such much research with go nowhere and will not translate into human benefit.

What is quality? This will mean different things to different people, some we can agree is lower quality than other studies. However some of the apparent quality work published in quality journals is never reproducible and we see this time and time again. This is the nature of Science, however the referees for (quality) journals would do well to read and adhere to some of the comments, and then quality would go up if more quality control is put into the system.

However, time and time again we see that many studies involving MSers are too small to give a definitive answer, so it needs to be done again wasting years in the process. 

Should this be supported or should researchers be encouraged to do it right first time. I have heard one Genetic researcher state that unless 10,000 samples are looked at then the data will not tell us much. The literature is full of genomic studies involving less than 10,000 samples.

I think TeamG are not holier than though and sometimes one strikes a balance between information, logistics and cost. Small studies can inform whether it is cost effective to do a larger study. It does however, waste time.

Labels: