Search TXNP

< More How to's

Wednesday, January 17, 2018

Share: facebooktwitterdigg

Getting the most from testing
Mal Warwick

August, 2009

WHY SHOULD a minimum suggested gift of $20 yield both higher response and a higher average gift than a $25 Ask? Blatantly illogical, isn't it? Yet I've seen that happen, as often as not.

Too bad. There's no way to avoid the ambiguities and uncertainties that testing delivers: If you want to make more money for your organization through direct mail, you've got to test.

But be careful when you test. One of the all-time masters of direct mail made the following claims some years ago:

  • That he got good results in the mail only when he used #10 or 6 x 9" envelopes.
  • That he could remember only a single time in his career when a list test was successful and the retest results were poor.
  • That if a test lifted results, the rollout always reflected the lift.

Not one of those three assertions is consistent with my experience. Granted: Usually, a #10 envelope is the best choice. And usually good test results point the way to successful rollouts. But there's not much certainty in my world when it comes to testing.

From my perspective, the key to successful testing in direct mail fundraising is to minimize the ambiguities by following seven simple, common-sense guidelines:

1. Select statistically valid test panels. Follow rigorous procedures to ensure statistical validity and thus the comparability of the test panels you'll be mailing, and mail every test panel against a "control."
2. Mail test panels on the same day from the same place. Our complex world is different from one day and one place to the next. Tests are unreliable if test panels are mailed on different days or from different places.
3. Mail test panels calculated to produce 100 responses. Decide how many names to include in each test on the basis of the likelihood that each panel will produce 100 responses at the rate of response you might reasonably expect.
4. Don't trust other mailers' test results. One organization's test results don't necessarily apply to another's circumstances. In fact, they fail to translate with such frequency that you would take another's test results as gospel only at great risk.
5. Above all else, test lists. My rule of thumb is to devote 20% of each acquisition mailing-20,000 out of each 100,000 names-to test new lists. The list is the single most important variable in a fundraising mailing.
6. Test elements that can dramatically affect results. Next to lists, the most important elements to test are:
• Whole new packages (as opposed to individual package components); and
• Significantly different offers, including membership or donor benefits, involvement devices, front-end or back-end premiums, or Ask amounts.
7. Retest if the confidence level of results is less than 95%. Given all the little things that can go wrong and skew test results, that "unlikely" tenth chance out of 10 seems to come along much too frequently for my taste!
Even if you follow all these seven guidelines to the letter, you'll probably stumble across your share of brain-teasers when you test. But this straight-and-narrow path will minimize the confusion and doubt you're likely to encounter -- and maximize the chances you'll raise more money for your organization.

This article is excerpted and condensed with permission from Testing, Testing, 1, 2, 3 . . .: Raise More Money Through Direct Mail Tests, to be published in 2003 by Jossey-Bass Publishers. Copyright © 2002 by Mal Warwick.


Your TXNP Weekly E-Newsletter is made possible by the generosity of:

FROST in many Texas cities

TXNP Professional Members Are Dedicated to Texas and Texans.

Aurora Grants & Consulting |Dawson Murray Teague Communications | ELITE Research | FOR THE PHILANTHROPIST | Graystone Consulting | J A Churchill Associates | John F. Lewis PC | McConnell & Jones LLC

Sign up for your personal TXNP E-Newsletter

at-t Meadows Foundation express news HOBLITZELLE FOUNDATION v greenly zachry foundation w b h b bank of america southwest airlines Sid W. Richardson Foundation forst