4-Experimental design and statistical analysis

Glossary of terms used in design and analysis
An overview of concerns in the literature about poor experimental design

Pre-registration of animal research

The pre-registration of protocols for animal research is gaining momentum, enabling peer review and as part of the work to reduce bias:

• Preclinicaltrials.eu (pdf of a presentation)
• The Animal Study Registry (animalstudyregistry.org), Germany (see also Bert et al., 2019)
• PROSPERO: An international prospective register of systematic reviews, established by the National Institute for Health Research (NIHR) in the UK
• Preregistration - Center for Open Science (COS)
• Anderson & Kimmelman (2015): Should preclinical studies be registered?
• van der Naald et al. (2021): A 3-year evaluation of preclinicaltrials.eu reveals room for improvement in preregistration of animal studies
• Pre-registration and Registered Reports: A Primer from UKRN
• Chambers (2019): What's Next for Registered Reports?
• A guide to preregistration and Registered Reports
• Data sharing: A Primer from UKRN
• Open Code/Software: A Primer from UKRN
• Preprints: A Primer from UKRN
• Open Access: A Primer from UKRN
• Further advice on protocol registration

Depositories for online protocols

• Protocol Exchange from Nature.com
• protocols.io
• protocol-online.org
• Open Wetware

Registration of accidents or critical incidents

If unexpected or undesirable incidents occur during a study, scientists have at the very least a moral obligation to help prevent this from happening in future research. There are now centres which allow registration of such incidents, such as CIRS-LAS. These should be consulted to see if incidents relevant to a study have been reported.

Experimental design

In a paper entitled "The well-built research question", Penny Reynolds describes how research ideas can be converted into actionable plans by decomposing the study into 5 components, using the mnemonic PICOT (Platform, Intervention, Comparators and Controls, Outcome, Time).

The October 2024 issue of Laboratory Animals contains a large set of papers on biostatistics, including discussions of sample size, p-values, blinding, randomisation, correlations, pilot studies and how to deal with heterogeneity.

TextBase contains many books about experimental design, including:

• The Design of Animal Experiments (Festing et al., 2016)
• A Guide to Sample Size for Animal-Based Studies (Reynolds, 2023)
• Experimental Design for the Life Sciences (Ruxton & Colegrave, 2016)
• Research Methods for the Biosciences (Holmes et al., 2016)
• Power Analysis: An Introduction for the Life Sciences (Colegrave & Ruxton, 2020)
  
  

Further references

• Longitudinal Studies in Animal Research: Benefits and Challenges (AI-assisted blog by Jivago Rolo, 2024) 
• Study design: think 'scientific value' not 'p-values' (Reynolds, 2024)
• Gurjanov et al. (2024): The Road to Virtual Control Groups and the Importance of Proper Body Weight Selection 
• Establishing effective cross-disciplinary collaboration: Combining simple rules for reproducible computational research, a good data management plan, and good research practice (Stawarczyk & Roos, 2023)
• Recommendations for improving the use and reporting of statistics in animal experiments (Rowe, 2022)
• Power to the People: Power, Negative Results and Sample Size (Gaskill & Garner, 2020)
• Guidance on Statistical Reporting to Help Improve Your Chances of a Favorable Statistical Review (Harhay & Donaldson, 2020)
• Guidelines for the Design and Statistical Analysis of Experiments Using Laboratory Animals (Festing & Altman, 2002)
• A collection of papers on principles of statistics (Editors Doug Altman & Martin Bland) in the British Medical Journal
• Experimental replications in animal trials (Frommlet & Heinze, 2020)
• Reporting guideline checklists are not quality evaluation forms: they are guidance for writing (Logullo et al., 2020)
• A multi-batch design to deliver robust estimates of efficacy and reduce animal use – a syngeneic tumour case study (Karp et al., 2020)
• Best practice for the design and statistical analysis of animal experiments (Palarea-Albaladejo & McKendrick, 2020)
• Rein in the four horsemen of irreproducibility (Bishop, 2019)
• Statistical experiment design for animal research (Sorzano & Parkinson, 2019, 317 pages)
• The reign of the p-value is over: what alternative analyses could we employ to fill the power vaccuum? (Halsey, 2019)
• The practical alternative to the p value is the correctly used p value (Lakens, 2019) 
• Moving to a world beyond "p<0.05" (Wasserstein et al., 2019)
• How the Animal Welfare Body can help with reproducibility (Hawkins, 2019)
• Two level factorial experiments (Smucker et al., 2019)
• On determining sample size in experiments involving laboratory animals (M. Festing, 2018)
• How to decide your sample size when the power calculation is not straightforward (Bate, 2018)
• Guidelines on statistics for researchers using laboratory animals: the essentials (Gosselin, 2018)
• What exactly is 'N' in cell culture and animal experiments? (Lazic et al., 2018)
• Sex bias in preclinical research and an exploration of how to change the status quo (Karp & Reavey, 2018)
• Reproducibility vs. Replicability: A Brief History of a Confused Terminology (Plesser, 2018)
• Comparing phenotypic variation between inbred and outbred mice (Tuttle et al., 2018)
• Why null results do not mean no results: negative findings have implications for policy, practice and research (Miller-Halegoua, 2017)
• The reproducibility of research and the misinterpretation of p-values (Colquhoun, 2017)
• Introducing Therioepistemology: the study of how knowledge is gained from animal research (Garner et al., 2017)
• The Weak Spots in Contemporary Science (and How to Fix Them) (Wickerts, 2017)
• Targeting next generations to change the common practice of underpowered research (Crutzen & Peters, 2017)
• Statistical tests, P-values, confidence intervals, and power: a guide to misinterpretations (Greenland et al., 2016)
• Where Have All the Rodents Gone? The Effects of Attrition in Experimental Research in Cancer and Stroke (Holman et al., 2016)
• The development of response surface pathway design to reduce animal numbers in toxicity studies (Dewi et al., 2014)
• Threats to validity in the design and conduct of preclinical efficacy studies: a systematic review of guidelines for in vivo animal experiments (Henderson et al., 2013)
• Six red flags for suspect work (Bagley, 2013)
• Sample size calculations: should the emperor's clothes be off the peg or made to measure? (Norman et al., 2012)
• Author guidelines for displaying data, data analysis and statistical methods, from the American Society of Pharmacology and Experimental Therapeutics (ASPET)
• Why most published research findings are false (Ioannidis, 2005)