A/B testing has been around for a long time now. Some say it was introduced by Google in 2000. Today, it is used by most website owners and a lot of tools are available to make that task as easy as possible. The bravest ones will even do some multi-variable testing and push the optimization to its limits.
However, 12 years later, a lot of companies still rely on their instincts or the creativity of their director of marketing to make critical decisions such as the photo, colours or texts to be used in landing pages or signup forms. Problem is, these instincts are usually dead wrong and data-driven methods often show unexpected results. The most explicit example in porn is the choice of a girl to place next to a signup form. In companies that are not data-driven, it will usually be one of the managers, or even the CEO who will decide and choose a girl who think will be a killer to generate conversions.
In companies that are not data-driven, it will usually be one of the managers, or even the CEO who will decide.
Data-driven companies however will not choose any girl, they will put a dozen of them in rotation and let statistics decide. Nine times out of 10, the girl that would have been chosen instinctively generates very poor results and it’s another one that nobody would have expected that generates huge conversion rates.
You may think that these optimizations are not so important. But the differences in conversions can be very significant. Using the right girl might increase your conversions by five percent, changing the button colours another five percent and using the text “Buy Now” instead of “Submit” another slight increase. Overall, once you are done with your testing, you might see impressive improvements in conversions, with a massive impact on your product and company.
In short, stop thinking your instincts are great and start thinking Why You Should Trust Data Instead of Guts relying on data. Even if you are not the decision maker, gather some data and show your boss the data proves he’s wrong on the choice of girl! He won’t be able to fight it and he will leave all the design choices to numbers after that.
