Your Institutional Memory

“It is not deeds or acts that last: it is the written record of those deeds and acts” -- Elbert Hubbard, American philosopher and writer

How's was your weekend?

A well needed rain fell on the Baltimore area cooling the air and soaking the plants. It didn't stop me from continuing my vegetable garden project. I was just forced to work on it in a steady rain. I'd say it's 90% complete... with just the finishing stones to go on top. The planting is the easy part.

Let's get back to testing... Today we're going to discuss two more tips.

First off, it's important that you clarify what you are testing before getting started. The best way to do this is to begin by documenting what you want to test.

The what, why and how.

A short pre-test summary will keep you from repeating mistakes or wasting time on questions already answered. It will also avoid any confusion as to the sole purpose of the test.

AND after writing it down you may realize you are testing too many things or you're not testing enough, i.e., meaning the test may not be big enough.

So I suggest before the test, write down a clear hypothesis of what you're trying to prove or disprove. Here's an example:

Test #6, October 2006: Need to improve the click through rate of Google PPC ads - A/B test new teaser copy. Test “Free Research Reports” headline vs. control.

A) 2007 Best Stocks To Buy
10 Low-Priced Stocks You Must Own
Free Report on Profitable Stocks
www.web-site.com

B) Free Research Reports
2007 Best Stock Recommendations
10 Low-Priced Stocks You Must Own
www.web-site.com

Before the test, you should also record your decision metrics and the roll-out plans.

The next thing, as if you couldn't guess, is to record the results.

Once you have statistically value data... record your date, sample size, numeric results, your interpretation, and suggestions for the next steps. For more on the validity of testing data, please read the CVoD issue Statistically Valid Data.

For example:

Results: 4,812,768 impressions - The “Free Report” headline did not beat control. The control “2007 Best Stocks To Buy” is a clear winner. See results below:

A) Impressions 2,436,124, clicks 4,854 (0.20%)
B) Impressions 2,376,644, clicks 1,237 (0.05%)

Many marketers also like to record the results in terms of dollars and cents instead of percentages. You can't spend percentages, and it's not always a valid comparison. Note it as a cost savings or improved ROI... after all it's all about making money, right?

I like to put both these summaries in one notebook. It's the diary of your day in and day out testing.. in a sense it's your marketing department's shared institutional memory.

The testing diary will also help you implement test results. No one will forget the results... or what was tested. It's here for all to see. Black and white, easy to reference and easy to learn. AND no one will take the knowledge away with them when they move on.

What more can you ask for?

Just like rain on a June weekend...

AP