How to do A/B Testing in WordPress

Treehouse●Design 66 Comments

We recently posted about how 37signals increased their sign-ups by 200% with A/B Testing.

A lot of you are using WordPress, which makes A/B testing pretty tricky, so we’ve recorded a screencast tutorial to show you how to do it.

Editor’s note: We’ll be covering topics like A/B testing and marketing your web app at The Future of Web Apps London.

If you’re new to A/B testing, here are a few resources to get you started:

The Video

Please note, if your goal page is on a different domain or sub-domain, you need to make small modifications to your tracking code.

Start Learning Web Design, Coding, How To Start A Business & More for Free!

Learn More

Treehouse

Our mission is to bring affordable Technology education to people everywhere, in order to help them achieve their dreams and change the world.

Comments

  • http://www.rossbearman.co.uk Ross Bearman

    Good screencast, I’ve just been setting up this system for a client application I’m currently building around WordPress.

    Amazing how easy it is to get addicted to A/B testing!

    • Ryan Carson

      I know – it makes you want to start testing everything :)

    • http://plano.maidbrigade.com Angela McCall

      I agree, Ross, A/B Testing is a necessary function but I have wrestled with the questions that this article has answered. Our clients should appreciate the value of this tool.

  • http://johndturner.com John Turner

    Google Optimizer is cool, I didn’t even know it existed.

    By the way , what screencast software do you use?

    • Ryan Carson

      Hey John – I used iShowU – a great little app for the Mac. For the video of me talking, I just used PhotoBooth.

  • http://unmatchedstyle.com Gene Crawford

    Great overview on A/B testing Ryan. Was just starting to get into this for ourselves and have had some clients asking about it, so your video is right on time. Thanks for pulling this together!

  • http://www.jeromecastaneda.com/ Jerome

    thanks for the great lesson, Ryan.

    wordpress is really cool for non-techie like me.

  • Benjamin Alijagić

    Great video! Please more testing tutorials!

  • http://www.obliviousinvestor.com/ Mike P

    Thanks for this video! I’ve been trying to figure out how to do this for some time.

    Additional question: What if my goal is getting them to click an outbound link? (I’m an author marketing my books on Amazon.)

    Since I can’t put tracking code on an Amazon page, would it work to make a page on my site that just redirects to the Amazon page, but also includes the optimizer javascript?

    Something like: mysite.com/book.html (and the file would consist of the optimizer javascript, followed by the redirect javascript)?

    • http://carsonified.com Ryan Carson

      That should work fine, however, I’ve never done it.

      Also, you can connect Google Analytics with Google Website Optimizer, so you can create Goals and measure those. You’d need to get Amazon to send buyers back to a thankyou page on your site though, and I don’t know if that’s possible.

      • http://www.obliviousinvestor.com/ Mike P

        Cool. Thanks for the reply–I’ll give it a go tonight. :)

        (As to getting Amazon to send people back to my site, I can’t imagine how I’d do that. They’re not the most responsive to authors/publishers/vendors. They know they’re holding all the cards, so to speak.)

  • Pingback: Neue Links

  • http://www.bronsonharrington.com Bronson

    This stuff is priceless, many thanks for the great tips.

    Stumbled!

  • http://brendanwenzel.net Brendan Wenzel

    Thank you so much for sharing this. I have been trying to figure this out for the past few days and this helped me tremendously.

  • Pingback: OUVYT » Blog Archive » Daily Del.icio.us

  • Pingback: Daily Links for Tuesday, July 28th, 2009

  • Pingback: Waynes Weekly Notable Links | The Bliss of Being

  • http://michaelkjeldsen.com/ Michael Kjeldsen

    Hi Ryan,

    thank you so much for this video, it made me go all-in on testing in WP.

    Unfortunately WP (TinyMCE?) keeps removing from the code, even when I have disabled the visual editor. I have to manually edit the post in the database (phpmyadmin) to be able to add the tag.

    Did any of you experience this, and i that case: what did you do to prevent it?

  • http://michaelkjeldsen.com/ Michael Kjeldsen

    “… removing from the code” (never mind the gaps in the tag – the comment system removed the tag entirely in the above comment :-)

  • http://michaelkjeldsen.com/ Michael Kjeldsen

    Oh my, your system is teasing med.

    In the above two post, guess where to insert [/noscript] :-)

  • http://designtowp.com Haris

    Right post as right time (atleast for me).

    Thanks for the super awesome screencast Ryan.

    With the information provided by your blog post and google love, I can easily checkout what variables work and what doesn’t work for my business. :)

  • Pingback: My del.icio.us bookmarks for July 28th through July 30th | B-WORX blog

  • http://www.seohats.com Alex Juel

    Wow, this was a great post! I’ve actually been meaning to toy around with Google Website Optimizer after I had signed up for it about 3 months ago. I started watching this video and decided that now is as good a time as any and set up a heading test, so thank you very much for the push :)

    I had one question. Once it’s set up, can we re-enable the visual editor. Was that only so that we could get the full code instead of what WordPress shows in the HTML editor?

  • Pingback: lillbra » Blog Archive » links for 2009-07-30

  • http://www.wiredvision.com Keith McLaughlin

    Thanks for the video.

    I hadn’t heard of the Website Optimizer service before. Seeing as I also use WordPress, I think I’ll include a few experiments with my planned redesign.

    Thanks again =)

  • http://www.dominixz.com/blog DominixZ

    Great Video !!

    I know Website Optimizer for a long time but I never know how to use it. Today I know because of you Ryan. Thanks a lot.

  • Pingback: Links of the Week 8 - Inspect Element

  • Pingback: Cómo hacer A/B Testing con Wordpress | Bloguismo

  • Pingback: Weekly Digest for August 3rd » Alex Jones

  • Gavin Steele

    That was a great watch! Learnt a lot, thanks.
    Are there any security risks?
    What security do most people run on WP?

  • Pingback: A/B Testing with Wordpress (h/t Carsonified) — Social Mallard

  • Pingback: Daily Digital Recap for August 6th | Digital Gilbert

  • Pingback: Links de agosto 2009: O que é necessário para ter um blog wordpress bem bacana. | the worst kind of thief

  • Stergios

    I enjoyed this screencast very much. I found it very educating. I actually had not used Google Website Optimizer but now i am going to use it in future projects. Thank you Ryan. By the way, the music heard at the end of screencast sounds like Stray Cats.

  • Pingback: Pruebas A/B con Wordpress

  • Pingback: 30 Simple Ways of Improving Bounce Rate and Conversion Rate | SEOptimise

  • Pingback: Ultimate A/B Testing resources and how I used them

  • Pingback: AB Testing with Google Website Optimizer and WordPress | Fresh Tilled Soil

  • http://aparelhosdentarios.com Joana p.

    I just love to split test my site’s pages. But having a rough time with some pages in wordpress that require forms.

    But thanks anyway for the insights.

    Cheers

    Joana

  • Pingback: links for 2009-10-14 « Köszönjük, Emese!

  • Pingback: Perfecting User Experience Design with A/B Testing - Freelance Web Design Belfast Northern Ireland - Lee Munroe

  • Angelo Cioffari

    nice one ryan! thanks for sharing…

  • http://www.carmasnack.com Carmen

    Great article! I love your video on A/B split testing.

  • http://garyjones.co.uk Gary

    That’s a really useful video, not just on how to do A/B testing in WordPress, but A/B testing with Google Website Optimizer generally! Thanks.

  • Pingback: User testing and thirteen ways of selling a book - paul olyslager

  • http://terrybritton.com Terry Britton

    Kim Roach has been pushing all of us at her membership site to do testing, but I have opted to use WordPress, so it seemed insurmountable. TILL NOW!!

    I’ve shared this with Kim already, and will be doing so in a post to hers and Chris Farrell’s forums next. Thanks so much!

    Terry Britton

  • http://www.professional-copywriter.co.uk Professional Copywriter

    I love it Ryan!!!! A/B testing is an obsession of mine! and to have a WP plugin as well makes life so easy!

  • Pingback: The Importance of a/b Design Testing | Design Shack

  • http://davyia.dk/ David Yiadom

    Hi Ryan. Thanks for the video… you make it look so simple. And once trying it once or twice, it really is simple.

    I was wondering if it’s possible to do a test on a page that has multiple conversion goals. Like a page with product a, b, and c?

  • jared

    GREAT tutorial!! Thanks.

    I’m having trouble getting this to work for a home page with an Aweber opt-in form. I’m basically enclosing the entire page in the control script and loading variations of video, text, and an aweber form. For some reason the forms are showing up in IE. When I preview them they work fine but once live the forms don’t show up at all.

    I realize this is a pretty vague problem and a long shot anyone has had this some issue, but just thought I’d see if anyone had and has a suggestion. I’m basically just wanting to test various video/text/image combinations on a wordpress home page to test conversions and figured this would work.

  • http://www.claususonline.com Jorge R.

    Hi there Ryan,
    Great tutorial. I had tried before to do A/B testing in WordPress, but never succeeded at it. I guess it was because I didn’t consider multivariate testing the way you show it on your tutorial.
    I’m no techie, so I guess I better start testing simple things and then move on from there. Like using aweber forms and stuff like that.
    Thnx for the light!!

  • http://www.itsdigitalmarketing.co.uk/ Gary Robinson

    Bit late to the party on this article, but thank you very much for the useful video. I’m just starting on A/B testing and was looking for ideas on how to test on a WordPress blog. Your video answered that perfectly. Thank you

  • http://joeyreed.com Joey Reed

    Before watching your video I had a tough time wrapping my head around how multivariate testing would work with WP. Now I see that it’s not so tough. Thanks for the run-through, Ryan!

  • ldexterldesign

    So which did get more clicks out of interest?

  • http://twitter.com/szodaviz szodaviz

    Új weboldalunkat a felhasználók igényei szerint kívánjuk alakítani. Ehhez fontos tudni, mik működtek jól eddig, és min kell változtatni. Az A/B teszt jó eszköz lesz.

  • Bupham

    Thank you.

  • http://pulse.yahoo.com/_UGCPDB6GMOXXI5CBMHCBXHAJDM Halley

    So I followed you directions exactly(except that google now seems to combine the control and tracking scripts into one), but when I validate pages I get these:

    Control and Tracking script-
    Expected to find: function utmx_secti
    Found on line 284: /*<![CDATA[*/functi

    Page sections-
    { – Unexpected or missing characters – Line: 284 Expected: utmx_section Found: /*<![CDATA[*

    Any ideas on how to fix this?
    Thanks

    • http://carsonified.com Ryan Carson

      Hi Halley – I’d recommend switching to http://optimizely.com now – much
      easier than this method I used to use

  • Pieter

    Hi,
    Great tut, thanks!
    One question: is it also possible to do this with images or widgets?

  • http://www.jamiemchale.co.uk jamiemchale

    Just a quick tip:

    In the video you disabled the visual editor. You can use WordPress shortcodes to insert the section into your blog post using the visual editor. Just add a couple of lines to your functions.php in your theme:

    add_shortcode( ‘gwo’, ‘gwo_shortcode’ );

    function gwo_shortcode( $atts, $content = null ) {

    $t = “utmx_section(“”
    .esc_attr($atts[section])
    .”")”
    .$content
    .”";

    return $t;
    }

    • http://carsonified.com Ryan Carson

      Great tip! Thanks Jamie

  • http://lastdropofink.co.uk/ Matthew Ogborne

    Just to say thanks, I had this on my todo list for ages and just never got around to it

  • http://plano.maidbrigade.com Angela McCall

    I agree, Ross, A/B Testing is a necessary function but I have wrestled with the questions that this article has answered. Our clients should appreciate the value of this tool.

  • http://3store.uk.com 3 Store UK

    Can the goal be an event in Google Analytics. I too run an affiliate site where multivariant testing will be invaluable