35 Binomial Test
Binomial Test
The Binomial Test is a specific type of statistical test that falls under the category of non-parametric tests.
It is used to analyze data that result from a series of experiments or trials that have two possible outcomes (e.g., success/failure, yes/no, heads/tails).
The test determines whether the observed distribution of outcomes significantly deviates from a theoretical distribution, usually specified by the null hypothesis.
Key Features:
- Data Type: Used for binary data or data with two possible outcomes.
- Purpose: To determine if the observed proportion of outcomes significantly deviates from the expected proportion under the null hypothesis.
Here’s an overview of binomial tests in hypothesis testing and their applications:
35.1 Understanding Binomial Tests:
Null Hypothesis (H0): The null hypothesis typically assumes that there is no difference between the observed proportion of successes and a specified proportion or that the probability of success is equal to a certain value.
Alternative Hypothesis (H1): The alternative hypothesis states that there is a significant difference between the observed proportion of successes and the specified proportion.
Test Statistic: The test statistic used in binomial tests is based on the binomial distribution, which models the number of successes in a fixed number of independent Bernoulli trials (experiments with two possible outcomes).
P-value: The p-value represents the probability of observing the obtained result or more extreme results, assuming that the null hypothesis is true. A small p-value suggests that the observed proportion is significantly different from the hypothesized proportion, leading to the rejection of the null hypothesis.
35.1.1 Formula
For a sample of size \(n\) with a hypothesised success probability \(p_0\), the probability of observing exactly \(x\) successes is given by the binomial probability mass function:
\[ P(X = x) = \binom{n}{x} \, p_0^{\,x} (1-p_0)^{n-x} \]
The two-tailed p-value for the binomial test sums the probabilities of every outcome that is at least as extreme as the observed count, in either direction. Because this involves summing several binomial terms, the p-value is computed directly with statistical software rather than by hand — shown for R and Python later in this chapter.
35.2 Binomial Test Example Problem
A call-centre manager claims that 85% of customer complaints are resolved on the first call. A quality analyst samples 20 recent complaints and finds that only 13 were resolved on the first call. Does this sample give evidence, at the 5% significance level, that the true first-call resolution rate differs from the claimed 85%?
Hypotheses
- Null Hypothesis (\(H_0\)): \(p = 0.85\). The true first-call resolution rate is 85%.
- Alternative Hypothesis (\(H_1\)): \(p \neq 0.85\). The true first-call resolution rate differs from 85%.
Here the sample size is \(n = 20\), the observed number of successes is \(x = 13\), and the hypothesised proportion is \(p_0 = 0.85\). The observed sample proportion is \(\hat{p} = 13/20 = 0.65\).
Interpretation
Running the exact binomial test on \(x = 13\), \(n = 20\), \(p_0 = 0.85\) (shown in R and Python below) gives a two-tailed p-value of approximately 0.022.
- Since \(0.022 < 0.05\), the null hypothesis is rejected.
- The sample provides statistically significant evidence that the true first-call resolution rate differs from the claimed 85% — and, since the observed rate (65%) is well below the claim, the manager’s figure appears optimistic.
35.3 Binomial Test calculation using R and Python
35.4 Applications of Binomial Tests:
A. Quality Control:
- Binomial tests are used in quality control to assess whether the proportion of defective items in a sample is significantly different from a target proportion. For example, in manufacturing, a binomial test can be used to determine if the proportion of defective products in a batch exceeds a specified threshold.
B. Market Research:
- Binomial tests are employed in market research to evaluate the success of marketing campaigns, product launches, or customer satisfaction surveys. Marketers can use binomial tests to determine if the proportion of respondents who exhibit a desired behavior (e.g., making a purchase, providing positive feedback) differs significantly from expectations.
C. A/B Testing:
- In online experiments and A/B testing scenarios, binomial tests are utilized to compare the success rates of different versions of a webpage, advertisement, or user interface. By comparing the proportions of conversions or clicks between experimental groups, businesses can determine which version performs better.
D. Medical Research:
- Binomial tests play a vital role in medical research for assessing the efficacy of treatments, drugs, or medical interventions. Researchers use binomial tests to determine if the proportion of patients experiencing a positive outcome (e.g., recovery, symptom relief) differs significantly between treatment and control groups.
E. Political Polling:
- Polling organizations use binomial tests to analyze survey data and assess public opinion on various political issues or candidates. Binomial tests help determine if the proportion of respondents supporting a particular candidate or policy differs significantly from a specified threshold or from the proportions observed in previous polls.
35.5 Considerations:
Sample Size: Larger sample sizes generally provide more reliable results in binomial tests, allowing for more precise estimation of proportions and detection of smaller differences.
Type I and Type II Errors: Like other hypothesis tests, binomial tests are subject to errors, including Type I (false positive) and Type II (false negative) errors. The choice of significance level (alpha) and statistical power influences the likelihood of these errors.
In conclusion, binomial tests are valuable tools in statistical hypothesis testing for assessing proportions and categorical data in various fields such as quality control, market research, medical research, and political polling. By applying binomial tests appropriately, researchers and practitioners can make informed decisions based on rigorous statistical analysis.
- Both Nominal and Binomial tests serve critical roles in statistical analysis by allowing researchers and analysts to test hypotheses and draw conclusions about their data when the data is categorical or binary in nature. Choosing between these tests depends largely on the type of data at hand and the specific research questions being addressed.
Summary
| Concept | Description |
|---|---|
| Foundations | |
| Binomial Test | Non-parametric test that compares an observed proportion of successes to a hypothesised reference proportion |
| Binary Outcome | Data with exactly two possible outcomes, such as success/failure or yes/no |
| Bernoulli Trial | An independent trial with two outcomes and a constant probability of success |
| Test Mechanics | |
| Null Hypothesis (H0) | States that the observed proportion equals the hypothesised reference proportion |
| Alternative Hypothesis (H1) | States that the observed proportion differs significantly from the hypothesised reference |
| Test Statistic | Number of successes in a fixed number of independent trials, modelled by the binomial distribution |
| Binomial PMF Formula | P(X=x) equals n-choose-x times p0 to the x times (1-p0) to the (n-x) |
| P-value | Probability of observing the data or something more extreme when the null hypothesis is true |
| In R and Python | |
| R binom.test() | Base R function that performs the exact binomial test and returns a confidence interval for p |
| Python scipy.stats.binomtest() | SciPy function that performs the exact binomial test and can return a confidence interval for p |
| Applications | |
| Quality Control | Used to test whether the defect rate in a batch exceeds a specified tolerance |
| Market Research | Used to evaluate whether a campaign response rate exceeds historical benchmarks |
| A/B Testing | Used to compare conversion proportions between two experimental variants online |
| Medical Research | Used to judge whether a treatment produces significantly more positive outcomes than a control |
| Political Polling | Used to test whether support for a candidate differs significantly from a reference level |
| Considerations | |
| Sample Size | Larger samples produce more reliable estimates and detect smaller differences |
| Type I and Type II Errors | False positives and false negatives that depend on the chosen significance level and statistical power |