Making changes to your organization is foolish without having the proper data to tell you the impact that the changes may have. A tool called a “detectable effect size” can help in making that determination.
It is nice to believe that certain changes that are proposed will have an impact on our business, but with so much at stake, merely having a belief is generally not good enough. Gathering information through experiments can show you the detectable effect size that these proposed changes may have.
Overview: What is a detectable effect size?
A detectable effect size can be defined as the noticeable degree of impact that a change or experiment will have.
3 benefits of a detectable effect size
There are some clear benefits of detectable effect size that are worth exploring:
1. Defining parameters
By working with detectable effect sizes, you can easily set the parameters for a proposed change.
2. Setting a minimum
When testing for the impact that a change will have, if test results keep showing up below a defined minimum detectable effect size, you can know that making the proposed changes may not be worth the resources. This can prevent a lot of time and money waste.
3. Seeing the potential power of change
When you see a large detectable effect size, you can know that the power behind a proposed change can be significant.
Why is detectable effect size important to understand?
Detectable effect size is important to understand for the following reasons:
Power calculations
Understanding detectable effect size is important because it is an input that is critical for power calculations.
Close relationship to key areas of interest
Having an understanding of detectable effect size is important due to its close relation to areas of interest such as sample size, project budgets, and surveying.
Impact
Having a clear comprehension of the detectable effect size is important because it can help determine the impact changes will make and can help aid in making better decisions for your organization.
An industry example of a detectable effect size
A toy company has decided to refurbish its image by focusing more on its social media presence. The organization decides to see how much more engagement with their website comes a significant increase in their social media posting. The researchers assigned to the task set the minimum detectable size at 5%, since the CEO believes that any effect lower than that is not worth paying attention to.
3 best practices when thinking about a detectable effect size
Here are some tips that are worth keeping in mind when working with detectable effect sizes:
1. Use detectable effect size to prioritize changes and experiments
By using detectable effect size, you can determine the relativity between cost and likely value for proposed changes or experiments. It can also give you a better framework to base decisions as opposed to simple guesswork.
2. Convincing stockholders and team leaders
Using detectable effect size can hold valuable sway when trying to convince stakeholders, team leaders, and other relevant parties of the benefits or drawbacks of making potential changes.
3. Benchmarking
Incorporate detectable effect sizes into your analysis for benchmarking how long to conduct an experiment in order to see desirable results or a return on investment.
Frequently Asked Questions (FAQ) about a detectable effect size
What is the link between sample size, minimum detectable effect size, and your baseline?
In order to maintain a statistical significance percentage when looking at lower minimum detectable effect sizes, the sample size will need to increase.
What is the link between a lower minimum detectable effect size, statistical significance, and sample size?
In order to keep a statistical significance percentage maintained when looking at lower minimum detectable effect sizes, the sample size will need to increase.
Can there be a large detectable effect size and there still be a low level of significance?
It is rare, but it is possible for there to be a high detectable effect size while there is little to no significance.
Detectable effect size can help you make the right decision
There are many factors to consider when considering making changes to your organization. Working with detectable effect sizes can give you the insight needed to see the amount of impact that proposed changes will have. With this knowledge, you are in a position to make smarter choices for your company than you would be able to otherwise.
