How to do A/B Testing in WordPress

We recently posted about how 37signals increased their sign-ups by 200% with A/B Testing.

A lot of you are using WordPress, which makes A/B testing pretty tricky, so we’ve recorded a screencast tutorial to show you how to do it.

Editor’s note: We’ll be covering topics like A/B testing and marketing your web app at The Future of Web Apps London.

If you’re new to A/B testing, here are a few resources to get you started:

The Video

Please note, if your goal page is on a different domain or sub-domain, you need to make small modifications to your tracking code.

Treehouse

Our mission is to bring affordable Technology education to people everywhere, in order to help them achieve their dreams and change the world.

Comments

67 comments on “How to do A/B Testing in WordPress

  1. Good screencast, I’ve just been setting up this system for a client application I’m currently building around WordPress.

    Amazing how easy it is to get addicted to A/B testing!

    • I agree, Ross, A/B Testing is a necessary function but I have wrestled with the questions that this article has answered. Our clients should appreciate the value of this tool.

    • Hey John – I used iShowU – a great little app for the Mac. For the video of me talking, I just used PhotoBooth.

  2. Great overview on A/B testing Ryan. Was just starting to get into this for ourselves and have had some clients asking about it, so your video is right on time. Thanks for pulling this together!

  3. Thanks for this video! I’ve been trying to figure out how to do this for some time.

    Additional question: What if my goal is getting them to click an outbound link? (I’m an author marketing my books on Amazon.)

    Since I can’t put tracking code on an Amazon page, would it work to make a page on my site that just redirects to the Amazon page, but also includes the optimizer javascript?

    Something like: mysite.com/book.html (and the file would consist of the optimizer javascript, followed by the redirect javascript)?

    • That should work fine, however, I’ve never done it.

      Also, you can connect Google Analytics with Google Website Optimizer, so you can create Goals and measure those. You’d need to get Amazon to send buyers back to a thankyou page on your site though, and I don’t know if that’s possible.

      • Cool. Thanks for the reply–I’ll give it a go tonight. :)

        (As to getting Amazon to send people back to my site, I can’t imagine how I’d do that. They’re not the most responsive to authors/publishers/vendors. They know they’re holding all the cards, so to speak.)

  4. Pingback: Neue Links

  5. Pingback: OUVYT » Blog Archive » Daily Del.icio.us

  6. Pingback: Daily Links for Tuesday, July 28th, 2009

  7. Pingback: Waynes Weekly Notable Links | The Bliss of Being

  8. Hi Ryan,

    thank you so much for this video, it made me go all-in on testing in WP.

    Unfortunately WP (TinyMCE?) keeps removing from the code, even when I have disabled the visual editor. I have to manually edit the post in the database (phpmyadmin) to be able to add the tag.

    Did any of you experience this, and i that case: what did you do to prevent it?

  9. Right post as right time (atleast for me).

    Thanks for the super awesome screencast Ryan.

    With the information provided by your blog post and google love, I can easily checkout what variables work and what doesn’t work for my business. :)

  10. Pingback: My del.icio.us bookmarks for July 28th through July 30th | B-WORX blog

  11. Wow, this was a great post! I’ve actually been meaning to toy around with Google Website Optimizer after I had signed up for it about 3 months ago. I started watching this video and decided that now is as good a time as any and set up a heading test, so thank you very much for the push :)

    I had one question. Once it’s set up, can we re-enable the visual editor. Was that only so that we could get the full code instead of what WordPress shows in the HTML editor?

  12. Pingback: lillbra » Blog Archive » links for 2009-07-30

  13. Thanks for the video.

    I hadn’t heard of the Website Optimizer service before. Seeing as I also use WordPress, I think I’ll include a few experiments with my planned redesign.

    Thanks again =)

  14. Great Video !!

    I know Website Optimizer for a long time but I never know how to use it. Today I know because of you Ryan. Thanks a lot.

  15. Pingback: Links of the Week 8 - Inspect Element

  16. Pingback: Cómo hacer A/B Testing con Wordpress | Bloguismo

  17. Pingback: Weekly Digest for August 3rd » Alex Jones

  18. That was a great watch! Learnt a lot, thanks.
    Are there any security risks?
    What security do most people run on WP?

  19. Pingback: A/B Testing with Wordpress (h/t Carsonified) — Social Mallard

  20. Pingback: Daily Digital Recap for August 6th | Digital Gilbert

  21. Pingback: Links de agosto 2009: O que é necessário para ter um blog wordpress bem bacana. | the worst kind of thief

  22. I enjoyed this screencast very much. I found it very educating. I actually had not used Google Website Optimizer but now i am going to use it in future projects. Thank you Ryan. By the way, the music heard at the end of screencast sounds like Stray Cats.

  23. Pingback: Pruebas A/B con Wordpress

  24. Pingback: 30 Simple Ways of Improving Bounce Rate and Conversion Rate | SEOptimise

  25. Pingback: Ultimate A/B Testing resources and how I used them

  26. Pingback: AB Testing with Google Website Optimizer and WordPress | Fresh Tilled Soil

  27. I just love to split test my site’s pages. But having a rough time with some pages in wordpress that require forms.

    But thanks anyway for the insights.

    Cheers

    Joana

  28. Pingback: links for 2009-10-14 « Köszönjük, Emese!

  29. Pingback: Perfecting User Experience Design with A/B Testing - Freelance Web Design Belfast Northern Ireland - Lee Munroe

  30. That’s a really useful video, not just on how to do A/B testing in WordPress, but A/B testing with Google Website Optimizer generally! Thanks.

  31. Pingback: User testing and thirteen ways of selling a book - paul olyslager

  32. Kim Roach has been pushing all of us at her membership site to do testing, but I have opted to use WordPress, so it seemed insurmountable. TILL NOW!!

    I’ve shared this with Kim already, and will be doing so in a post to hers and Chris Farrell’s forums next. Thanks so much!

    Terry Britton

  33. Pingback: The Importance of a/b Design Testing | Design Shack

  34. Hi Ryan. Thanks for the video… you make it look so simple. And once trying it once or twice, it really is simple.

    I was wondering if it’s possible to do a test on a page that has multiple conversion goals. Like a page with product a, b, and c?

  35. GREAT tutorial!! Thanks.

    I’m having trouble getting this to work for a home page with an Aweber opt-in form. I’m basically enclosing the entire page in the control script and loading variations of video, text, and an aweber form. For some reason the forms are showing up in IE. When I preview them they work fine but once live the forms don’t show up at all.

    I realize this is a pretty vague problem and a long shot anyone has had this some issue, but just thought I’d see if anyone had and has a suggestion. I’m basically just wanting to test various video/text/image combinations on a wordpress home page to test conversions and figured this would work.

  36. Hi there Ryan,
    Great tutorial. I had tried before to do A/B testing in WordPress, but never succeeded at it. I guess it was because I didn’t consider multivariate testing the way you show it on your tutorial.
    I’m no techie, so I guess I better start testing simple things and then move on from there. Like using aweber forms and stuff like that.
    Thnx for the light!!

  37. Bit late to the party on this article, but thank you very much for the useful video. I’m just starting on A/B testing and was looking for ideas on how to test on a WordPress blog. Your video answered that perfectly. Thank you

  38. Before watching your video I had a tough time wrapping my head around how multivariate testing would work with WP. Now I see that it’s not so tough. Thanks for the run-through, Ryan!

  39. Új weboldalunkat a felhasználók igényei szerint kívánjuk alakítani. Ehhez fontos tudni, mik működtek jól eddig, és min kell változtatni. Az A/B teszt jó eszköz lesz.

  40. So I followed you directions exactly(except that google now seems to combine the control and tracking scripts into one), but when I validate pages I get these:

    Control and Tracking script-
    Expected to find: function utmx_secti
    Found on line 284: /*<![CDATA[*/functi

    Page sections-
    { – Unexpected or missing characters – Line: 284 Expected: utmx_section Found: /*<![CDATA[*

    Any ideas on how to fix this?
    Thanks

  41. Just a quick tip:

    In the video you disabled the visual editor. You can use WordPress shortcodes to insert the section into your blog post using the visual editor. Just add a couple of lines to your functions.php in your theme:

    add_shortcode( ‘gwo’, ‘gwo_shortcode’ );

    function gwo_shortcode( $atts, $content = null ) {

    $t = “utmx_section(“”
    .esc_attr($atts[section])
    .”")”
    .$content
    .”";

    return $t;
    }

  42. Hi guys.. This is a commandable tutorial and would like to share my experience. Initially I had trouble with AB testing, as I failed to modify tracking code on my page Cashiers nc, now it is fine but need more knowledge.

  43. HI I’m more than halfway through the video but I had a question before I continue…Do both pages have to be in wordpress? i.e. Would your blog post and your about page both have to reside in wordpress? I would like to do it using a blog post in wordpress and an about page that is not in wordpress.

  44. I agree, Ross, A/B Testing is a necessary function but I have wrestled with the questions that this article has answered. Our clients should appreciate the value of this tool.