How To Create Catalog Split Test Scenarios That Matter (part 2)


This week in the final installment of this two-part series on the value of creating mail tests that produce measurable, and telling, results for your catalog, I provide takeaway lessons from last week’s example of how one catalog company tested the profitability of using an upgraded paper stock in its catalog. I’ll also list some tips to help ensure your company is conducting productive mail tests. 

The Moral of the Story
Even scientific tests often succumb to the subjective. Once all of your scientific testing is done, the art of interpreting the data takes over. Returning to last week’s example, as a direct marketer I never would’ve rolled out the higher-grade paper without additional testing. I would’ve wanted to confirm my results by running the same test over again. Even if the test were a runaway success, I would’ve proceeded cautiously. 

In the catalog/multichannel industry, there are infinite varieties of tests that can be configured, measured and, ultimately, interpreted. 

So how do you become successful and improve your company via testing? I believe the following seven principles, that if adhered to, will allow you to hedge your bets.

1. Always test against something. Your current catalog, e-mail, Web site, merchandise, among other things, can be used as your control. 

2. Always create a testing hypothesis. Use this example: If we do X, we expect Y to occur. 

3. Always set up logical tests based on scientific principles.

4. Test only one variable at a time. If you test more than one, you’ll never know which variable made the difference. See below for a list of variables you can test.

5. Always do your math up front. Calculate all expenses in advance of the test and set up a pro forma profit and loss and break-even analysis. 

6. Always do the math on the back end. Before the creative side of testing begins, have the math in front of you. Many times the numbers will drive decisions. 

7. Always test until you’re satisfied. Don’t roll out any changes to your catalog unless you’re certain of the outcome through testing, retesting and further testing. 

Variables You Can Test Today (a checklist to get you started)

List Variables:

* Recency — the most recent names from a new list; if your list is “working”; older segments of the list; average order sizes.

* New lists in your product category.

Creative and Printing Variables:

* Paper, trim size, pages, formats, photography, copy, ink-jet messages, cover versions, dot whacks, inserts.

Offer Variables:

* Gift with purchase, dollar or percentage off, free shipping.

Merchandising Variables:

* New or different products; repositioning older or non best-sellers via creative execution.

We love comments. Go for it. We dare you...

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s