Design and reporting of animal experiments

This page supplements advice given in Section 4 of the PREPARE guidelines. PREPARE covers all aspects of design (including animal and facility related issues).

Resources about contingency plans and preparedness, relevant to the Covid-19 pandemic and other emergencies.

This page contains a collection of links to resources on the more "mathematical" aspects of experimental design, including statistical analysis.
Direct oversight by a statistician is likely to be more effective than reliance on self-education from interactive tools, textbooks and websites, although many of these are excellent. The following points should be considered:

1. A clear hypothesis and descriptions a priori of primary and secondary outcomes, to avoid HARKing (see also Bishop, 2019).
2. Steps to minimise numbers and suffering of animals by appropriate statistical analysis, including the use of pilot studies.
3. Decisions on the power and significance levels to be used.
4. Definition of the experimental unit and number of animals in each unit
5. Choice of sample size and gender, age and/or developmental stage
6. Avoidance of bias, including “blinding” and randomisation (the procedure should be specified)
7. Inclusion and exclusion criteria

Guidance on reporting of animal experiments has been moved to a separate section. The section on common concerns about experimental design should also be consulted.

The journal PLOS has published guidance on how to report statistical analyses.

Resources for experimental design

• Tools for experimental design:
  • SMAFIRA
  • SHADE
  • 3Ranker
  • EDA (Experimental Design Assistant)
• Recommendations for improving the use and reporting of statistics in animal experiments (Rowe, 2022)
• Explanations, examples and references from the NC3Rs about study design, sample size, inclusion and exclusion criteria, randomisation, blinding, outcome measures and statistical methods
• Berdoy (2019): Experimental design and the avoidance of unnecessary suffering (recorded webinar)
• Advice on how to write up a power analysis, produced by Guy Prochilo
• Strategic planning poster from FRAME
• The Focus on Alternatives (FOA) initiative has produced two excellent posters outlining means of addressing the three R's when planning animal experiments.  A general poster about early planning and one illustrating a worked example within oncological research can be downloaded here in pdf format.
• NC3Rs Guidelines for conducting a pilot study
• Dr. Michael Festing has made enormous contributions to the science of experimental design and statistical analysis. Some of his resources are mentioned below (a more complete list can be viewed here):
  • The Design of Animal Experiments: Reducing the use of animals in research through better experimental design (Festing, 2016). An interactive CD based on the first edition of the book has been produced. Topics on the CD include: Choice of animal model; The experimental unit; Use of valid statistical methods; Increasing precision; Increasing the use of statistical methods; Planning and organising; Self-study and evaluation; Software and references.
  • Guidelines for the design and statistical analysis of experiments using laboratory animals (Festing & Altman, 2002)
• The Experimental Design Assistant (EDA) is an online tool to help researchers design animal experiments with a minimum of animals and apply appropriate statistical methods when analysing the results.

More references

• Amrheim et al. (2017): The earth is flat (p>0.05): significance thresholds and the crisis of unreplicable research
• Karp et al. (2022): A qualitative study of the barriers to using blinding in in vivo experiments and suggestions for improvement
• Lakens (2021): Sample size justification
• Bliss-Moreau et al. (2021): Improving rigor and reproducibility in nonhuman primate research
• van Eenige et al. (2020): RandoMice: a novel, user-friendly randomization tool in animal research
• Derek Fry (2014): Teaching Experimental Design
• Bate & Clark (2014): The Design and Statistical Analysis of Animal Experiments. The authors have developed a statistical package for animal experiments, InVivoStat.

Material from courses

• Videos of presentations at a workshop in May 2018 on experimental design, aiming to improve peer review of in vivo research proposals, highlight some of these issues.
• Norecopa arranged a seminar on 11 June 2009 where Dr. Manuel Berdoy, Oxford University, gave three lectures entitiled "Statistics: The Good, The Bad and The Ugly". The lectures discussed, among other topics, p-values, power analysis, calculation of group size, experimental design and control of variation. His lecture notes can be downloaded here.
• Online courses arranged by the University of Oxford, including statistics training and data analysis
• FRAME Training Schools (Report from the meeting held in Voss, Norway, 1-3 February 2016)

Design of animal facilities
Facility design will obviously have a major impact on the outcome of animal studies, or indeed the feasibility of conducting them at all. There are a large number of guidelines, which should be read in the context of the country in which they are issued and its legislation (e.g. the EU Directive 2010/63/EU and CoE convention ETS123), as standards may vary. Many of the guidelines are general, or have wide ranges for critical parameters such as room/water temperature, air humidity, environmental enrichment and stocking densities. These may need to be adjusted to suit the requirements of specific experiments.
Guidance includes:

• NIH Design Requirements Manual for Biomedical Laboratories and Animal Research Facilities (DRM)
• CCAC Guidelines on laboratory animal facilities - characteristics, design and development
• The UFAW Handbook on the Care and Management of Laboratory and Other Research Animals
• Health and Safety in Laboratory Animal Facilities
• Design Standards For Small Animals (Rodents) Vivarium (MD Anderson Cancer Center, University of Texas)
• Noise, Vibration and Ultrasound Design Guide (MD Anderson Cancer Center, University of Texas)
• The answer to the question "What is the best housing temperature to translate mouse experiments to humans?" is: thermoneutrality (Fischer et al., 2019)

 
Other literature and resources

• An Unforeseen Events Report Form (see also CIRS-LAS)
• An End of Study Report Form
• Science Fictions (2020) by Stuart Ritchie
• The American Statistical Association has issued a statement warning scientists about undue trust on p values when evaluating treatment effects
• A statistical guide for authors, produced by the journal Animal
• Statistical test, P values, confidence intervals, and power: a guide to misinterpretations (Greenland et al., 2016)
• A Practical Primer to Power Analysis for Simple Experimental Designs (Perugini et al., 2018)
• Principles of Experimental Design for the Life Sciences by Murray R. Selwyn
• Experimental Design and Statistics - volume 55, issue 3 of the ILAR Journal (2014)
• Degrees of freedom in Planning, Running, Analyzing, and Reporting Psychological Studies: A Checklist to Avoid p-Hacking
• Detecting and avoiding likely false-positive findings - a practical guide.
• The seven deadly sins of statistical misinterpretation, and how to avoid them
• The NOAH (National Office of Animal Health, UK) Compendium of Veterinary Datasheets, Withdrawal Periods and Veterinary Poisons Information Service
• Practical guides produced by ECHA to requirements under REACH, CLP and BPR, including How to use alternatives to animal testing to fulfil your information requirements for REACH registration
• Statistical Analysis Must Improve to Address the Reproducibility Crisis: The ACcess to Transparent Statistics (ACTS) Call to Action (Gosselin, 2019)
• Replicating Effects and Biases (Rosenbaum, 2001)
• Applets for power and sample size calculations prior to experiments
• PRIVAT (Peer-Review In Vitro Appraisal Tool) to facilitate consistent, comprehensive, and exacting peer-review of in vitro studies
• Advice on how to write scientific paper