Statistical Hypothesis Testing Framework
Six business questions, each answered with a test chosen to fit the data: two-sample t-test, one-way ANOVA, chi-square and Mann-Whitney U. Normality was checked before any parametric test was trusted, and every result was translated into a plain-language recommendation.
The Challenge
Six business questions, six tests chosen to fit the data. Three of the six nulls were retained, including a competitor pricing comparison (p = 0.151) and a two-campaign engagement comparison (p = 0.528). Reporting "no evidence of a difference" is what stops noise being sold as an effect.
Each scenario came with a decision attached: whether a store's prices differ from a competitor's, whether productivity differs across three departments, whether age group relates to preferred social media platform, whether defect rates differ across three production lines, which book category earns more revenue, and which of two marketing campaigns drives more customer engagement.
The first five scenarios used small sample datasets supplied with each brief. The campaign comparison used a 200-record engagement dataset, 100 customers per campaign.
Approach
Results
Rejected nulls: productivity differs across sales, marketing and finance (one-way ANOVA, F = 143.0, p < 0.001); defect rates differ across three production lines (chi-square = 11.11, p = 0.004); non-fiction titles earn higher mean revenue than fiction (F = 92.5, p < 0.001; with two groups this is equivalent to an independent t-test).
Retained nulls: no significant price difference between the store and its competitor (t = -1.48, p = 0.151), and no significant engagement difference between Campaign A and Campaign B (Mann-Whitney U = 4,741, p = 0.528).
Retrospective note: the social media scenario ran a chi-square test on only five observations, well below the minimum expected count of five per cell that the test needs. Its result (p = 0.441) cannot support any conclusion either way; Fisher's exact test or a larger sample is the right route.
How it was tested
- A null and alternative hypothesis stated for each scenario before any test, at α = 0.05.
- Normality checked with histograms, Q-Q plots and Shapiro-Wilk; both campaign groups failed, so Mann-Whitney U replaced the t-test.
- The social media scenario ran a chi-square test on only five observations, below the test's minimum expected count, so its result (p = 0.441) supports no conclusion.
- Five of the six scenarios used small sample datasets supplied with each brief.
- Fisher's exact test or a larger sample is the right route for the social media question.