Optimal Usability: Tree testing: evaluating your site’s organization

Some years ago, Donna Spencer pioneered a simple paper-based technique to test the organisation of websites. This month, Optimal Workshop launched a new tool called Treejack to make this “tree testing” automated and easy.

This article briefly describes tree testing, how the Treejack tool works, and what we’ve learned from running tree tests for several large clients.

The problem

Suppose you’re designing (or redesigning) the overall structure of a website. You’ve come up with a new structure – depending on the coffee, you may have come up with two or three. But are they good? Which one is best? And are they better than the old structure? Until recently, there wasn’t a quick way to find out, short of building the site and testing it after the fact. Tree testing is a simple technique that takes a site structure and evaluates its findabilityand labeling, answering the basic questions of site organisation:

• Can users successfully find items in the site?
• Can they find those items directly, without having to backtrack?
• Can they choose between topics quickly, without having to think too much?
• Overall, which parts of the tree work well, and which fall down?

Donna did her testing on paper, but it struck us that an online web tool would make the technique much quicker and easier. So we built Treejack.

Treejack - a tree-testing tool

Tree testing is a simple technique, so we made Treejack a simple, focused tool. Its sole purpose is to improve site organisation through quick, iterative testing. In a typical tree test, about 50 participants click a web link to start a Treejack session:

• Each participant is shown a website task (e.g. “Sign up for the company newsletter.”)
• They see a list of the site’s top-level topics.
• They click a topic, and see the subtopics under it.
• They continue clicking down until they find the answer. They can also back up and try a different path, or give up and move on to the next task.
• After about 10 tasks, they’re done. Elapsed time: 10-15 minutes.

Skimming the high-level results

After those 50 people have tried 10 tasks each, clicking through your site structure, Treejack can show you how well the structure performed, and more importantly, show you which parts failed with users. It summarises the results into 3 values:

• Success – The % of users who found the correct answer.
• Speed – how fast users clicked through the tree. Quick choices suggest high confidence, while hesitation suggests that topics are not clear enough.
• Directness – how directly users reached an answer. Ideally, users reach their destination without having to backtrack and try a different path.

These are measured for each task, with an overall score for the whole test. In a way, it’s like “unit-testing” your site’s organisation and labeling. Or as Treejack manager Sam Ng put it, “Think of it as analytics for a website you haven’t built yet.”

Lessons learned

We’ve run several tree tests now for large clients, and we’re very pleased with the technique and the tool. Along the way, we’ve learned a few things too:

• Test a few different alternatives. Because tree tests are quick to do, we can take several proposed structures and test them against each other. This is a quick way of resolving opinion-based debates over which is better. In a recent government project, one proposed structure showed much lower success rates than the others, so we were able to discard it without regrets or doubts.
• Test new against old. You don’t just want your new site structure to be different, you want it to be better. Tree testing is a great way to determine this. In the same government project, the original structure had a 31% success rate - ugh. Using the same questions, the new structure scored 67% - a solid quantitative improvement.
• Lather, rinse, repeat. Everyone talks about being agile and iterative, but schedules and budgets often quash that ideal. Tree testing, however, has proved quick enough that we’re able to do two or three revision cycles for a given tree, using each set of results to progressively tweak and improve it.

Summary

Tree testing has become a big part of our information architecture work with clients. And Treejack has given us the automation we were after – a quick, simple-to-use tool that generates clear results quickly.

Like user testing, tree testing shows us (and our clients) where we need to focus our efforts, and injects some user-based data into the site-design process. The simplicity of it means that we can do variations and iterations until we get what we came for – a tested, effective site structure.

Tree testing is still a relatively new technique, and Treejack as a tool is still in its early days, but we’re hoping that both earn a place in your organisation’s design toolkit.

 

The complete Boxes & Arrows article is at:
http://www.boxesandarrows.com/view/tree-testing