CIRCA:A/B Testing


Jump to: navigation, search



An experimental approach to web design (especially user experience design), which aims to identify changes to web pages that increase or maximize an outcome of interest. (Wikipedia)

A/B Testing is a simple way to test changes to your page against the current design and determine which ones produce positive result. It is a method to validate that any new design or change to an element on your webpage is improving your conversion rate before you make that change to your site code. (Optimizely: What is A/B Testing)

Conversion Rate

In internet marketing, the conversion rate is the proportion of visitors to a website who take action to go beyond a casual content view or website visit, as a result of subtle or direct requests from marketers, advertisers, and content creators.


A/B Testing on eBay:

Video: How A/B Testing tools work (Visual Website Optimizer)


  • Determine a target
  • Come up with hypothesis of factors
  • Build a variation page
  • Show the two pages randomly to users.
  • Compare conversion rate of both sites.


  • Test your software: Before you do the A/B testing, do an A/A testing first.
  • Focus on one goal per page: More than one goal will mix the testing result.
  • Test small elements: Small elements can also bring amazing conversion rate.
  • Do not expect large results from small changes: If all testing results are not satisfying, maybe it is time to change the whole interface. A/B testing is not omnipotence for developing conversion rate.
  • A/B Testing has its limitations: It only gives data, reasons are not included.

Companies well-known for using A/B Testing

  • BBC
  • eBay
  • Google
  • Microsoft
  • LogMeln
  • Netflix
  • Platdom
  • Zynga

A/B Testing Tools


Wikipedia: A/B Testing
Optimizely: What is A/B Testing
23 tips on how to A/B testing like a badass
Bella Martin, Bruce Hanington. (2012). Universal Methods of Design. Rockport Publishers.
Seven A/B testing mistakes you need to stop making in 2013
Wikipedia: Conversion rate

Personal tools