IMC Vancouver Jared Waxman: Rapid Testing

Testing for Conversions

The second standout presentation from IMC Vancouver was given by Jared Waxman from Intuit who related some of the company's experiences with testing. Intuit has built a testing framework that enables them to accomplish one day testing for new ideas. Quick testing means that they are able to try plenty of ideas and quickly reject bad ones.

Ten Pieces of Rapid Testing Puzzle

Loosely paraphrased the ten steps for quick testing are:

1. Create a testing strike force that has complete access to the site and the ability to make changes where they are needed.
2. Appoint one "approver" for what goes up. Have a single person approve the tests to expedite the decision making process.
3. Create one straightforward criteria to measure test success.
4. Report at the "go live" point of the test and then again when the results of the test are in. Between these two points, let the test run unhindered.
5. Develop ideas fully before implementing tests. While the testing team can tighten ideas, these ideas should be developed to their fullest potential before reaching the testing team.
6. Base test ideas on a reasonable theory and customer data.
7. Manage deadlines with quality and reduce scope as necessary to facilitate running tests and getting results in one day.
8. Take ideas from anywhere and everywhere, but allow the test team to maintain control of those ideas.
9. Maintain a flexible testing calendar. When one line of testing proves unfruitful, an open testing calendar will allow you to change your strategy and implement a new line of testing.
10. Practice one day test sprints. One day test sprints allow for the testing of many ideas and make testing a part of regular business rather than something only done once in a while.

Faith-Based Business?

Both Jared and Avinash emphasized that much of the work that we do on websites is unnecessarily faith-based. There's no need for this approach. Available tools (like Google Analytics and Google Website Optimizer, which are both free) tell us what actions site users are taking and provide information to help us hypothesize what changes we can make to improve the function of the site. All that's left for us to do is to create the tests to ensure changes are positive ones and use the results of those tests effectively and to our advantage.

Bookmark It

Hide Sites