tree testing

You have lots of potential starting points for improving this IA, so keep it simple by focusing on the big numbers. And finally, we needed to test the proposed trees against the existing tree. very inspiring! * Give clear results that could be acted on quickly. | J'en profite ! Reusing users: Just like traditional usabilty testing, we aim to use fresh users each time, for just the reason you mention. This page was last edited on 11 January 2020, at 20:31. In this example, we can see three green lines that lead to three yellow circles — this is because this task had three correct destinations on the tree. You can filter the table by indirect and direct success, indirect and direct failure, and indirect and direct skips, depending on the scores you consider are worth exploring in depth. Il s'agit de se projeter et d'imaginer un site avant même sa mise en ligne. In addition to the task-based exploration, I also begin each testing session by simply asking the participant to look at each primary (and sometimes, secondary) navigation category and tell me what they *think* they would find underneath each. Because tree tests are quick to do, we can take several proposed structures and test them against each other. We posted an article looking at C-Inspector in September, including comparisons with your Treejack tool: Test your information architecture using C-Inspector A tree test shows participants only the task instructions and a stripped-down menu of category labels, as you can see in this task from a UserZoom tree test. Twitter | While that gets results sooner, it also means creating a throw-away set of pages and running an extra round of user testing. But to date, the IA community hasn’t found an effective, simple technique (or tool) to test site structures. You have control over who to include and exclude. You can test any size in between. Treejack and UserZoom, the two most common tree-testing tools, each use a slightly different style for presenting these metrics, but both provide these quantitative measures for each task in your study: Success rate: The percentage of users who found the right category for that task, Directness: The percentage of users who went to the right category immediately, without backtracking or trying any other categories, Time spent: The average amount of time elapsed from the beginning to the end of the task. Put methods into practice using our tools, Explore articles, news and product updates, Discover the key features of Optimal Workshop, Experience how our tools work in practice, Sign up for a free account and create a tree test, View a sample tree test from the participant’s perspective, View sample results from a researcher’s perspective, Read more about tree testing in our Help Center. This table gives you the first-click data straight. It is sometimes described as "backwards card … Kathryn Whitenton is Nielsen Norman Group's Director of Digital Strategy. Tasks per participant. * Speed – how fast participants clicked through the tree. If they didn’t find a topic that satisfied them, they could backtrack (go back up one or more levels). The. You can’t select category labels (or ‘parent nodes’) as correct destinations because although we do want to know if our category labels make sense, we ultimately want to know if people can navigate the labels to find the right information. However, this format does not capture the full context of user behavior (such as comments made while performing a task) and you can’t ask personalized follow … Getting the labelling and hierarchy of your content right will reduce the need for people to contact you or use the search function, so that’s something to keep in mind as well. Do you want to test your whole website in one go, or create a separate test for each part? We assist in developing screeners, scheduling participants, and managing participant honorariums and communication. Do you know any sites that have tried it? After you’ve launched your tree test, you’ll be given the option to recruit participants via our integrated panel from within your Optimal Workshop account. The good news is that our miserable overall score of 5/10 is often some 8’s and 9’s brought down by a few 2’s and 3’s. The quality of your participants, and therefore the quality of your data, is an important thing to consider when you start recruiting. We needed some real data – data from users. Donna was careful to point out that this technique only tests the top-down organisation of a site and the labeling of its topics. Information Architecture,Navigation,Research Methods, 10 Usability Heuristics for User Interface Design, When to Use Which User-Experience Research Methods, Empathy Mapping: The First Step in Design Thinking, Between-Subjects vs. Within-Subjects Study Design, UX Mapping Methods Compared: A Cheat Sheet, Average UX Improvements Are Shrinking Over Time, 3 Types of Roadmaps in UX and Product Design, Compensatory vs Noncompensatory: 2 Decision-Making Strategies, State-Switch Controls: The Infamous Case of the "Mute" Button, The Visual Principle of Scale in User Interface Design, Tree Testing Part 1: Fast, Iterative Evaluation of Menu Labels and Categories, Utility Navigation: What It Is and How to Design It, Audience-Based Navigation: 5 Reasons to Avoid It, Tree Testing: Fast, Iterative Evaluation of Menu Labels and Categories, Intranet Information Architecture Design Methods and Case Studies, Marketing Email UX - User Research Methodology, Conversion Rate Optimization and Applied UX Research, Digital Diary Studies for Longitudinal Field Research, The importance of that task to the overall user experience, How each success rate compares to other similar tasks (e.g., tasks which target content at the same level in the hierarchy), The success rate for the food-stamps task to that of another task which also targets content that is 6 levels down; or, The success rate of the food stamps task performed on two different trees with different labels — one which uses the term. And three people see that link and think ‘Yes, that’s exactly what I need!’. Here’s a few things you could do next to explore this technique in more depth. 1.) Remember that, by its very nature, tree testing eliminates many helpful design elements, such as the search function, secondary navigation options (like related links), and any context cues from the visual design or content. And as each task is scored individually, you can have a few as one task if you have one specific part of the tree you want to test. We jumped in and did some research, including card-sorting exercises with various user groups. Task difficulty: We use the same post-task, First instincts: Where users click first to find the item. Denver, Colorado 80206 Deux outils indispensables pour les pros du commerce connecté et pour les prestataires de la filière ecommerce. Tree testing, on the other hand, has proved quick enough that we’ve been able to do two or three revision cycles for a given tree, using each set of results to progressively tweak and improve it. Finally, for each task, we select the correct answers – 1 or more tree topics that satisfy that task. Whatever you can find out, the better. The pilot test can expose any unintended problems with your task wording early enough to correct them. How many tasks? Let’s say you have a link on your website labelled ‘Application form for X’. July 9, 2017. You can do this any time during your analysis. Then for each task, people will see the labels arranged in a different order. Your tree is a text-only version of your website structure (similar to a sitemap). With these needs in mind, we looked for a new technique – one that could: What kind of evidence do I need so I can apply for this funding. After all, we hadn’t just contracted to deliver a different IA – we had promised a better IA, and we needed a quantifiable way to prove it. It didn’t take too much to imagine all this turned into a web app – both for the information architect running the study and the participant browsing the tree. A verb or adjective in another place can jog a participant’s memory, suddenly “solving” a problem… that’s why I’m a little cautious when you said that 10 tasks were enough. A tree test is a method that quantifies the findability of an interface (usually a website or software product) by using the labels and structure of the navigation. Each task also receives an overall score out of 10 (with an 8 or above score considered well-performing). The tree is presented as a simple list of top-level topics: They click down the tree one topic at a time. Two pieces of research (including one using Treejack data from millions of participants) have illustrated that when participants get their first click right in task-based usability tests, they have at least double the chance of success, and sometimes even triple. Go through the tree thoroughly to make sure you get them all. Every time we run a tree test, we get a few seemingly bizarre choices, and it’s good to be able to ignore them with confidence. Tree testing is a new concept and activity for many people, which makes the clarity of your instructions particularly important. On the surface, the success rate seems simple: higher is better. Les tâches demandées sont réalistes. Brian: Love the idea of using a nested folder structure in a file browser – use the simplest tool that gets you the answers you’re looking for. At MindTree, we use this often. Page Laubheimer: You must have javascript and cookies enabled in order to display videos. The red represents an incorrect path, and the lack of blue shows that people kept going further before going back up the tree. Like usability testing, we may also do discount testing with surrogate users, but we always avoid people who know too much (e.g. After a certain amount of debate, it became clear that debate wasn’t the way to decide. That piggybacking saves effort, but it also means that we don’t get to evaluate the IA until later in the design process, which means more risk. 2. You’ll get answers to your ‘Why’ from things like analytics, repeated support queries, phone calls to customer support, customer studies, user interviews, stakeholder feedback, client specifications, and so on. About Us | We test the tree by getting participants to look for specific things – to perform “find it” tasks. Half of the successful users had to retrace their steps at least once before locating it — although the information was actually available in 3 different locations in the tree. a high Speed score), while hesitation suggests that the topics are either not clear enough or not distinguishable enough. As a bonus, we’ve found that participants usually find tree tests less taxing than card sorts, so we get lower drop-out rates. * Provide a way to compare alternative trees Basically, it involved modeling the site structure on index cards, then giving participants a “find-it” task and asking them to navigate through the index cards until they found what they were looking for. After hitting Go, you can sit back and watch the results come in while you get on with other work. You ask participants to complete tasks by clicking through your tree and nominating the information they think is correct. I hope you don’t misunderstand me – I’m really excited about your tool, and I’m just thinking that it has a huge potential to convey even more information about user decisions . Poor findability is a major reason for a poor user experience. Rights & Permissions | To do more detailed analysis on the low scores, we can download the data as a spreadsheet, showing destinations for each task, first clicks, full click paths, and so on. After working with the results in Excel, it became very clear which parts of the trees were failing users, and how they were failing. This ensures that the structure is evaluated in isolation, nullifying the effects of navigational aids, visual design, and other factors. Hmm…. If the site had a high return-visitor rate (e.g. So write in a natural, plain English style, and introduce a hypothetical scenario for people to bring to mind. And which new one was best? The only downside was that it took more effort to analyze the data than your tool would. At this early stage, we weren’t concerned with visual design or navigation methods; we just wanted to test organisation – specifically, findability and labeling. For online tree testing, we aimed for something that was: This makes the tedious chore of IA testing a lot faster and more scalable. Time spent recruiting: For this type of unmoderated testing, where the participant can do it remotely on their own time, it’s usually not hard to recruit users. If they couldn't find items, where did they go astray? Subscribe to our Alertbox E-Mail Newsletter: The latest articles about interface usability, website design, and UX research from the Nielsen Norman Group. Participants are asked to find items or functions in a navigation structure devoid of any design elements or search features. Tree testing is a usability technique for evaluating the findability of topics in a website. This pattern suggests that lower-level categories overlap too much. If you need to test more than 10 tasks, recruit more users, and use the tree-testing tool’s randomization feature to assign only 10 tasks to each participant. So if you have the phrase ‘application form’ in your task and on your tree, you risk ‘giving away’ the answer. The success rate for that task indicates the percentage of users who found the correct location in the tree and identified it as the right place to complete that task. Refresh. Success rate and directness tell you whether a category is findable; detailed pathway analysis helps you figure out how improve categories that don’t work well. You can recruit participants in a bunch of different ways, and how you do so will depend on a few different factors. think aloud) and ask about their more curious departures from the “correct” paths. You can set 1 task, or up to 10. If this issue occurs for many tasks, consider changing the overall organization scheme. You may have also heard this method described as ‘reverse card sorting’, or possibly ‘card-based classification’. In general, we’ve found that tree-testing results are much easier to analyse than card-sorting results. D'autres tests de programmation aidant à améliorer l'architecture d'un site existent, comme le Card Sorting (tri par cartes) qui favorise l'ergonomie et le confort visuel. great article! If they still couldn’t find what they were looking for, they could give up and move on to the next task. While the success rate is high, it would be worth looking more closely at why 20% of people had to click back through the tree before they selected a correct destination, and in particular, what made 5 people nominate the wrong destination as correct. Showing user paths graphically – Yes, results that are more graphical (like what you suggest and a few other ideas we’re playing with) are definitely on the wish list. Instead of expecting to achieve a 100% success rate, use a more realistic frame of reference to evaluate what success rate is acceptable for each task, taking into account: For example, consider two tasks and their respective success rates in the table below. Tree testing has given us the IA method we were after – a quick, clear, quantitative way to test site structures. For most sites – particularly large ones – this means creating a hierarchical “tree” of topics. She mentioned closed card sorting, but like us, she wasn’t satisfied with it. We can see quickly that only 15% of participants selected the correct answer, which is made up of 7 people (Indirect Success) who had to go back up the tree before they found the answer, and only 1 person who went directly to it. In tree testing, the first click is always a top-level category, because none of the subcategories are visible until a parent category is selected. Le tree testing est apprécié pour sa rapidité d'exécution et son coût peu élevé. Although tree testing yields quantitative data, the conclusions are by no means black and white. Getting the tasks right is vital for gathering useful data. This is a quick way of resolving opinion-based debates over which is better. For example, if you test 30 people, and 1 chooses a given incorrect topic, that can be safely ignored. We wanted to know: That way, you can spot outliers more easily. Once the IA is frozen, we then use the same pages to develop clickable wireframes. Pilot test. In general, we try to get at least 30 users of a given type – that seems to be the general statistical number to see medium-sized effects in any study. Tree testing is a usability technique that can help you evaluate the findability of topics on a website. Even luckier was that we got to hear about it firsthand from its inventor, Donna Spencer, the well-regarded information architect out of Australia, and author of the recently released book “Card Sorting”:http://rosenfeldmedia.com/books/cardsorting/. It’s like unit-testing a site’s organisation and labeling. Getting your timing right — running tree tests at critical stages of your project — will give you a whole series of results visualizations and benchmarked data to compare, use to inform design decisions, and present as progress and return-on-investment reports to stakeholders. * Quick for an information architect to learn and get going on We can find out more about these things on the Pietree, the Paths tab, and the First Click tab. The first click operates in the same way. Some years ago, Donna Spencer pioneered a simple paper-based technique to test trees of topics. * the visual design and layout of the site (However, this increase would happen only if the rest of the design was well executed; a bad search implementation or poorly designed menus can also reduce success rates below levels observed in a tree test.). Clicking a link in that email takes them to the Treejack site, where they’re welcomed and instructed in what to do. You can’t just sit through a day of testing and jot down notes, but instead you need to take a systematic analysis to identify data trends and evaluate their significance. In the following sections, we refer to two examples: results from a high-scoring task, and results from a low-scoring task. And when we look at the paths of the five people who went directly to an incorrect destination, we can ask ourselves which label made me them think they were on the right track. The thinking goes that if there is general agreement about which cards go in which buckets, then the buckets (the categories) should perform well in the delivered IA. See those branches going out in every direction? Tree testing can answer questions like: Tree testing has two main elements: your tree, and your tasks. For that, information architects have traditionally turned to closed card sorting, where the user is provided with predefined category “buckets” and ask to sort a pile of content cards into those buckets. Then you’ll have data showing you which trees perform best, and the labels and groupings that trip people up. Introducing tree testing. Offering an incentive is a good way to let your participants know how much you value their contribution, as well as ensuring they are committed to complete the whole activity. i had some really good ideas while i was reading your article. If we leave them in, it makes it too easy for users to choose them as alternatives to browsing the tree. Raised and educated in Toronto, Canada. Ideally, they reach their destination without wandering or backtracking. * Directness – how directly participants made it to the answer. so i am looking forward to solve a problem i was carrying with me for some time. And see that very skinny green line? A quick and dirty way of doing this is in Dreamweaver. The results are designed to give you quick, actionable insights and enable you to delve deeper into the data. If you’re improving a website that already exists, running a tree test to start with will give you insights into what works and what needs work from the start. What do I have to do to get funding for this project? We could look at the tiny yellow circles at the end of the branching gray lines, and analyze why people might have incorrectly nominated these answers. Usually our clients have lists that we can use. 2.) For the personality test also known as tree test, see, Learn how and when to remove this template message, Tree Testing for Websites: A Comprehensive Guide, Tree Testing: A quick way to evaluate your IA, https://en.wikipedia.org/w/index.php?title=Tree_testing&oldid=935300062, Articles needing additional references from August 2018, All articles needing additional references, Creative Commons Attribution-ShareAlike License. Once this analysis is complete you can identify appropriate solutions. Just like with usability testing, a good tree-test study must recruit representative users as study participants, particularly for products with specialized target audiences. Recent refinements to that method, some made possible by online experimentation, have now made “tree testing” more effective and agile. In this example, which is filtered, we can see that although Participant 14 got to the correct destination, they moved around the tree a lot before they got there. At some point, your customers will get to benefit from all the work you’ve put into tree testing, so it’s vital to establish who these intended users are, before you run your tree tests. Even though tree-testing tasks can be completed quickly, it’s still not a good idea to have people do 30 tasks in a row. Once they start the test, they’ll see a task to perform. For example, if, when asked to find information about the New Mexico state library, 67 out of 100 participants selected the correct location, the success rate for that task is 67%. Full disclosure: As noted in his “bio”:http://boxesandarrows.wpengine.com/person/35384-daveobrien, O’Brien works with Optimal Workshop. Ps. Are you working on a complete overhaul, or just tweaking the information in one category? We recently explained the process for designing a tree test; once you’ve planned your study, the next step is to collect data and interpret the results. It is also known as reverse card sorting or card-based classification. * Could they find those items directly, without having to backtrack? That is, if we prompt the participant to think a bit about each top-level heading up front (before they have a task to do), might this improve their performance (or even hurt it, depending on how well their preconceptions matched up with their later thinking)? The Directness score is the average percentage of participants who selected a destination without backtracking. Do you have just one structure to test to start with, or four variations to compare? Sign-in, Boxes and Arrows 2001-17, Boxes and Arrows LLC 2017-18 - All Rights Reserved. On top of this, it’s very difficult to know exactly what people have in their minds when they arrive on our website — and almost impossible for our labels to mimic the language each of our users have when they visit our websites. Because tree tests are so basic, success rates are often much lower than in regular quantitative usability studies. In any event, I propose a “splay menu system” that would have the most popular links appearing in order (top/down, left/right, whatever). Depending on the type of tree and tasks in a study, some of these metrics may be more useful than others at predicting how well the information architecture will perform in real life. And when we filter by Indirect Success, we can look in more detail at the labels that people clicked and then moved back from. Along with card sorting, tree testing has the power to eliminate confusing categories or labels – both of which can … You can do the same thing with the text you write in your invitation email, your tweet, and your facebook post. You might have a vague idea of who you’re designing for, and a few questions as well. * where the participant hesitated or backtracked. Subsequent menus would appear likewise but further down the link tree. Her user studies have included general audiences as well as specific consumer types, business segments, children, and seniors. 3.) Afin de recevoir un lien de réinitialisation, veuillez renseigner votre adresse e-mail: Votre demande a bien été prise en compte. We created a rudimentary application as a proof of concept, running a few client pilots to see how well tree testing worked online. Tree testing data is presented in a series of tables and visualizations in Treejack, and is downloadable in either a raw or customized spreadsheet format. i was thinking about your conclusion: doing a better IA, not a different! A 67% success rate on a tree test could easily become a 90% success rate on the final design. 100% sécurisé, votre adresse ne sera pas diffusée, Alexandre Vannier, UX Strategist et CEO d'Octave Octave, Pendant le confinement, profitez des magazines numériques, Les tendances de la pub digitale vues par Octave Octave, Le guide du CMO pour la refonte d’un site web.