Recently, Trone Research + Consulting (TR+C) was tasked with conducting a quantitative study to help a client better understand where and how they should invest in their facilities. This client had a sizable customer base of more than 1.1 million customers, comprised of both individual consumers and businesses.
Data Collection
Sample data was obtained by emailing invitations to our client's customer database. As is often the case with old lists, there proved to be some difficulty in reaching customers. Some email addresses were no longer valid. Other emails were being delivered to spam or junk filters. Plus, since many of our client’s customers were businesses, sometimes the email address on file was for the person who had been in contact with our client to make arrangements but not necessarily the same person who had used our client’s services. After many interventions, we ended up with 296 completed surveys.
While consulting with our client’s executive board about the survey results, a pressing question was raised. Is a sample quantity of 296 large enough to gather reliable information on the entire client base and make important investment decisions?
Can less than 300 people represent more than a million customers?
In this case, the answer is, yes.
Statistical Analysis
Two inferential statistics useful in answering this question are the level of confidence and the margin of error. In survey research, these give us a range within which a measured observation would fall if the question used to obtain the measure was asked across 100 samples. For example, suppose we obtain 95% confidence with a margin of error of +/-5 and observe that 75% of respondents to our survey find Brand X appealing. What this tells us is that the actual appeal of Brand X lies between 70% and 80%. Furthermore, we’re 95% confident that this is the case. In other words, if we conducted our study 100 times, in 95 of those 100 times, we would observe values between 70% and 80%. The other 5% of the time, we’d observe data outside this range.
The following table details margins of error at various levels of confidence common to market research given a sample size of n=296 pulled from a universe size of 1,136,898.
| Universal Size = 1,136,898 | |
|---|---|
| Sample Size n=296 | Margin of error +/- |
| 95% confidence | 5.70 |
| 90% confidence | 4.78 |
| 85% confidence | 4.18 |
Per the table above, a sample of n=296 will yield 90% confidence with a margin of error of +/-4.78 which, in TR+C’s judgment, is generally acceptable for market research. Looking at 95% confidence, we observe a margin of error of +/-5.70. This is also relatively decent for most market research but the acceptable margin of error varies by industry. If we were conducting clinical trials research, for example, we would desire greater precision.
To quote other research professionals:
- The margin of error in social science research generally ranges from 3% to 7% and is closely related to sample size. (National Institutes of Health)
- In general, the precision of an estimate is related to the square root of the sample size—in other words, to double the precision, the sample size must be quadrupled. As a general rule, sample sizes of 200 to 300 respondents provide an acceptable margin of error and fall before the point of diminishing returns. (Kevin Lyons, Lipman Hearne)
Ultimately, it is up to you to determine the margin of error with which you can live. This decision is affected by several factors that need to be considered, including the desired level of precision, feasibility and the cost of sample as well as the investment of time.
Tom Minsel, Ph.D., of Trone Research + Consulting, works with a wide array of clients, bringing diversified experience in data-driven business strategy, database analysis and marketing, predictive modeling and research design.
