Worked out a cracking program of research? ✅ Identified the first question to answer? ✅ Ready to jump in and get started with the first project? ❌
There are two words that nearly all our clients hate to hear:
Statistical planning.
Every time that phrase slips into a conversation, a sense of gloom suddenly takes over the room. It’s the necessary evil that no one likes to talk about, let alone do. Yet it is the one thing that will ensure meaningful patterns and trends can be derived from your data and will actually answer the questions you have asked. Are my patients doing better with a new model of care? Is this device superior to others on the market for my patients?
Granted, statistical analyses can be a little confusing. For sure, planning an analysis is something most people don’t consider. We’ve been conditioned to start thinking about dusting off the statistics textbooks once we’ve got the data in hand, but the reality is, statistical planning should be undertaken at the very beginning of a research study, to inform every aspect of it.
Why, you ask? I’ll give you three good reasons:
1. Planning the statistical analysis will essentially drive the tactics and techniques you implement
Below, you can see the key components of a research study. When you’re putting your stats analysis plan together, there are questions you need to answer concerning each of those components. What you’re essentially doing is building a model of your data, and this underpins nearly every aspect of your approach to answering your question.
2. It will dictate your resource requirements
So you’ve got a question that you know you can answer, and have put together a set of variables that you need to collect. At the statistical planning stage, you identify that the sample size required to detect differences between the groups will take about 20 years of recruitment at your current clinical volume. Do you run the study for 20 years, or do you turn it into a multi-centre study?
Simply running through the motions of statistical planning will help refine your study design, and flag issues in your tactics and techniques. Armed with that information, you’ll be in a safe position to make decisions around the allocation of resources and budgets, as well as the investment of your own time and effort.
3. It is necessary for open and transparent science
Evidence-based medicine is only as effective as the evidence that’s used to inform change. As doctor-turned-bad science critic Ben Goldacre aptly explains in his TED talk, even the best-designed studies can be rigged to produce favourable results. Removing negative data, changing the outcomes when the study yields undesirable results, being opaque in terms of what the outcomes are until they actually see the data, and hypothesising after results are known (a term called HARKing), are all completely unethical ways to do science.
Putting a statistical analysis plan in place before conducting a study establishes transparency and accountability around the results. It also helps others in the community provide important feedback on how the results have evolved and what future efforts may need to take into account.
Personally, we’re a little biased about statistical planning because we have years of experience with it. But when we’ve shown our clients how they can get the most out of their data or saved them money by advising against non-viable projects, they’ve come to appreciate the power of robust statistical planning. In the words of Dwight D Eisenhower, “plans are worthless, but planning is everything”