The goal of this kind of testing is to determine whether using a different form factor will lift response.  The most common form factors used in direct marketing include: 

• #10 letter mailing
• self mailer
• over-sized postcard mailing
• dimensional mailing

Best Practice
A best practice here is to know that any test you are fielding isn’t testing form factor alone but also the creative execution devised to make the most of that form factor.  For this reason, we refer to this type of testing as a “package test”:

DM Package = Form Factor + Creative Execution

Planning
The minimum sample size for each test cell is determined by the response rate, confidence interval, and allowable percentage error. Click here for a tool that calculates minimum required sample size.

Backend Analysis
By properly designing the test cells in the planning stages, the back end analysis becomes simpler and allows us to learn with a higher degree of confidence.

Using a confidence interval worksheet, we can determine whether observed response rates are affected by package. The confidence interval worksheet can be found by clicking here.  For this example, at the 95% confidence level, the letter mailer had an expected response rate between 1.11% and 1.29%. Similarity, the expected response for the self mailer is between .82% and .98%.

Business Impact
Here, the letter mailer outperformed the self mailer.  The difference in response rate seen between the two mailings was statistically significant at the 95% level.