Testing a marketing campaign by rotating different versions of the creative, copy and offer/premium has been a widely used tactic since the early days of advertising. The digital revolution and market challenges of the recent years, however, put testing into a high-profile, increasingly strategic position. From the days of behind-the-scenes testing, it emerged as a mainstream phenomenon. Running the same marketing creative, copy and using same tactics and delivery methods over and over almost always proves to be a recipe for disaster in terms of response rates and eventually sales. On the other hand, the dynamics of segmentation, growing sophistication of marketing channels and more format options in the creative area complicated test designs further. So, although testing constantly has become an indispensable component of a successful marketing campaign, implementing a systematic, scientific testing strategy is still a challenge many marketers face.
Not quite.
Steps to Build a Successful Testing Strategy
The planning, design and implementation of a testing strategy is usually a “half-art, half-science” process. There are multiple disciplines and teams involved in the process. The party that is most intimate with the product and the market (product managers, account managers, the client, key decision makers) need to define the testing objectives. What are we trying to get out of the testing exercise? What are the key response differentiators and among which of these key differentiators the interactions are mostly expected? How do we define success? What are the market conditions/regulations and corporate constraints that should be taken into account in the test design? You need to bring the creative team to the table to develop relevant variations for each factor in the test. You also need your Analytics team to build a design of experiment (DOE) and compute statistical significance to identify which test cells perform better where performance is explained by the impact of the factors in test.
The design of experiment is at the heart of a successful testing strategy, especially when multi-variate testing is involved. For online testing, there are many tools available that do the heavy lifting, such as Google Website Optimizer, Adobe Test & Target, Optimost. In the case of offline and e-mail campaign testing, you need to utilize a DOE software and feed the testing factors and their variables in the software and also input expected interactions among factors in the test. Some popular DOE software that also works within Excel platform are QI Macros, DOE PRO XL. On the higher end, statistical packages such as SPSS and SAS also have DOE components. Based on test specs, the DOE software will output the necessary test runs that will enable you to observe each factor’s main effect and the impact of interactions among factors. These packages will also give you the flexibility to modify the combinations selected by the software, e.g. a certain combination of the test elements may not make sense to run, so you will be able to exclude this from the test.
The landing page and website testing involves the arduous task of properly tagging the landing pages and conversion page with the tags of the testing software of choice. Although tagging is usually a straightforward process, it is important to be test properly and continuously QA the tags, as we have seen in many instances tags being dropped or inadvertently modified. Another aspect of tracking that can get tricky is to properly track all the channels that a prospect can use, not just the medium in which the testing is performed and also keep in mind the sales cycles that might be longer than anticipated.
Testing Phases
One of the key decisions in a testing strategy is to identify the testing methodology to be used. Although a specific methodology can solely be used in a given campaign (such as using multi-variate in an e-mail campaign design), utilizing different methodologies in succession is a common practice. A/B (also referred as split run) is the first phase in which a new creative, a new product or a new marketing channel is tested against the control. If this new version provides a statistically significant result, the next phase is to run a multi-variate campaign to test different components that make up the winner of the A/B test. So far, we have tested the elements of the marketing communication though. We are answering the question: “Which version of the marketing communication get the prospects to take action?” However, the outcome (whether the prospect becomes a customer) is also substantially affected by post-click (for online) or post-initial action (for offline) elements of the conversion funnel. Therefore, this sequence of identifying a winner by an A/B test first (e.g. a winning landing page), and then optimizing the individual elements of this winner should be used not only for the marketing communication, but also for the elements of the conversion funnel.
