It’s been said many times but needs to be said again. The offer is one of the most important ingredients of any customer marketing program. The purpose of offer testing is to determine which offer drives the most response.

Best Practice
A best practice is to test various offers against each other and against a “no offer” control. This is particularly important when you are selling something that the target values. Through such testing you may find out that an offer of a free gift with purchase – for example – actually does little to drive incremental response.

Planning
An example below is a test to determine for differences in response rates for two different offers (Offer #1 – Personal Offer; Offer #2 – Job Support Offer) versus a Control group (no offer). The minimum sample size for each test cell is determined by the expected response rate, confidence interval, and allowable percentage error. A handy sample size tool to calculate the minimum sample size can be found by clicking here.

For each campaign, a diagram needs to be constructed that visually depicts the testing we plan within each segment. Click here for an example of what we mean.

Backend Analysis
By properly designing test cells in the planning stage, backend analysis becomes simpler and allows us to learn with a high degree of confidence.

Using a confidence interval worksheet, we can determine whether observed response rates between the three are statistically significant. The confidence interval worksheet can be found by clicking here.

For this example, at the 95% confidence level, consumers receiving the job support offer had an expected response rate between 1.13% and 1.27%. Consumers who received the personal offer had a response rate between 1.42% and 1.58%. The “No Offer” control group had an expected response between .98% and 1.22%.

Business Impact
Here, we can say at the 95% confidence level that the target responded best when given the job-support offer. The job support offer (Offer #2) outperformed the personal offer (Offer #1) in a way that was statistically signifcant. However, the personal offer worked about as well as no offer at all. In other words, the difference in response rate seen between these two cells was not statistically significant at the 95% level.

Keep in mind that a 95% confidence level doesn’t mean that you are 95% sure. It means that if you repeated this particular test 100 times, you would get the same results 95 times out of 100. Go to the glossary to learn more about the confidence interval and the margin of error.