What is "Mom and Pop" split testing?
When small and large businesses start thinking of Conversion Rate Optimisation, the first thing they do is get testing technology. The wonderful sales pitch of these software vendors is that hey! its a super powerful mega testing machine that can test everything you want.
Many businesses get these great tools, and some of them are truly good, tools like, Optimizely (unreliable), Google Experiments (free but limited), Maxymizer (expensive and getting outpaced), Test & Target (only good if you have the entire Adobe suite), Visual Website Optimizer (our preferred).
If you are mostly testing this... You are Mom and Pop testing:
When you are simply testing images, content, headlines, buttons, instructional copy then you are Mom and Pop testing and it typically looks like this:
What is advanced ABn and MVT testing?
Advanced testing is when you go way beyond the simple swapping of content assets in your AB or MVT tests. Advanced split testing is when you are re-engineering functionality, page structure, entire or partial storylines, modifying filters, redesigning entire pages restructuring forms and so on.
Advanced testing doesn't use the standard GUI interface of these testing tools, your testing tools is a just a way of injecting custom code that transforms your pages. Advanced split testing enables the delivery of custom code changes directly in the users browsers and it can do this across multiple page and template types not just on a single defined url.
Example 1: advanced split test using custom code for search modules
The ability to restructure and design a complex search module from this control to a redesigned variation with functional, content and design changes through advanced experiment coding:
The test control:
One of the 3 test variation:
Advanced testing example 2: Advanced testing for list modules on list pages
In this example, we are testing the structure and thought sequence at the individual test module on list pages. You know list pages, the pages that show all the product choices you have based on filter criteria or search terms and categories on retail style websites.
Well, in this experiment, on the list page, we had to take into consideration the different types of content and of course how content changed in these list modules when special offers where displayed. The experiment also had to run on all the product types and across all product categories regardless of the URL structure of these pages.
Your control(s): With and without special offer treatment
List module variation 1:
And the variations 2: there were 6 variations in total...
Advanced testing example 3: Advanced code for offer selection pages
In this example, we are putting together a number of learnings from both our research as well as previous experiments. This advanced test made a number of changes required quite significant custom coding to deliver this ABn test.
A large number of webpage variables were changed, including, progressive page interactions, rebuild of messaging, how incentives are presented and over all the entire user interaction was modified.
The control page: The select offer page has a number of different dynamic content, offers and functional elements.
Experiment variation 1: There were 3 test variations in total
Advanced testing example 4: Advanced code for download landing page
In this last example, the client tech team did not want to build alternative URLs for testing, so this is not a Split URL test. The AB test below runs on the same page link. The control and variation shown below are in fact the same page. The test variation is an advanced coded test that transforms the page structure, design as well as the content, thought sequence and form all in less that 0.5 seconds.