LearnImprove your app with Pre-Alpha Testing


writes on April 5, 2011

We’ve been working hard on the re-brand and re-build of Think Vitamin Membership (soon to be called Treehouse). If you’re building a web app, I’d like to share a tactic that we used that has helped us a lot: Pre-Alpha Testing.

A screengrab of the user unlocking the 'Super Alpha Tester' badge

Here’s the main problem: You need to do user testing on your app, but you’re a long way off from having a Beta ready. The answer is Pre-Alpha Testing.

How it works

We’re building a learning game engine and we needed to do user testing and make sure it actually worked well and helped people learn. We couldn’t wait to design and build a Beta, only to discover there were fundamental problems with the way the game engine worked.

We solved the problem with what I call a ‘Pre-Alpha User Test’. Here’s how we did it …

1. Created a User Flow Diagram

User Flow Diagram showing the question, answer and quiz process of the gaming engine

Allison created a basic UFD that explained how the gaming engine would work. She used LovelyCharts but I’m a big fan of Google’s super-awesome online drawing tool (which has live-collaboration and powerful flow charting capabilities – all for free).

2. Built basic views

Once we agreed on the UFD, Allison marked up very basic HTML and CSS views of each page. These views are basically un-styled pages that our Developer, Alan, can then start hooking up.

Screengrab of first commit

3. Created a rough working mockup

Once Allison committed the HTML/CSS views to GitHub, Alan then used JavaScript and HTML5 localStorage to create a working mockup of the app.

There were no user accounts or server-side databases. It’s all just client-side code to give the appearance of a working app.

Alan then deployed the app to Heroku.

4. Styled the views

Once the app was working, Allison went back in and added more styling and spent more time on UX to make the app more usable and life-like.

Screenshot of the rough demo quiz game

5. Ordered tests on UserTesting.com

We ordered 3-5 tests on UserTesting.com, specifying the type of user we were looking for ($29 per test). We also asked some of our loyal Think Vitamin Members to help us out as well, by conducting the tests with ClearLeft‘s SilverBack app.

6. Reviewed the tests and iterated

Once the user testing videos had been completed, we each watched them and made notes. We then had a Skype meeting where we discusses usability problems with the demo, and what actions we’d take to fix them.

We’d then update the app and visuals over one to two days, and then order another set of user tests, and then repeat again.

Overall, we did three rounds of user testing, in just two weeks. It was amazingly valuable and I’m SO glad we did it.

We’re going to throw away all the code and start over, but we now know that the fundamental functionality of our app is usable and successful. Bam! 🙂


Learning with Treehouse for only 30 minutes a day can teach you the skills needed to land the job that you've been dreaming about.

Get Started

2 Responses to “Improve your app with Pre-Alpha Testing”

  1. Anonymous on April 5, 2011 at 1:20 pm said:

    Awesome stuff! I have a web app in mind, and I will likely try to get some of my friends to do some pre-alpha testing with it.

    I am really looking forward to what you guys come up with for Treehouse. By the time it is out, I may even be in the correct financial state to afford a membership. Keep up the great work!

    Oh, heads up: “There was no user accounts or server-side databases.” Perhaps “were” works better here. 😉

Leave a Reply

You must be logged in to post a comment.

man working on his laptop

Are you ready to start learning?

Learning with Treehouse for only 30 minutes a day can teach you the skills needed to land the job that you've been dreaming about.

Start a Free Trial
woman working on her laptop