"It works" and other Usability Measurement Myths

The world is cluttered. It is cluttered with cheap plastic trinkets and toys, and late night cable shoping channel purchases. The world is also cluttered by the intangible. We are constantly bombarded with dust covered efficient promises that were abandoned long ago by practical obvious failure; we are cluttered with bad ideas and most importantly cluttered by words. But is it not the fault of words that we are cluttered. It is the fault of the people who wield them. The world is cluttered with dust covered words that are big on promise and even faster on excuses once failure is certain. And failure is not to blame either, or to fear. Failure simply marks the map and tells us where success cannot be found. Failure is our best friend if we will let it be that. The trouble is, words get in the way. Let me explain.

In my field of work I am constantly designing interfaces for human interaction (specifically on the web.) It is a requirement of my profession to shorten the time between needing an interface and finding one that is actually usable. If Usability (word #1) in its simplest form means, “Something that can be utilized by someone with reasonable intuition in determining how to interact with it,” then usability is a huge issue for people who have to design interfaces and an “interactive experience.” I would say that Usability as a requirement in that process goes without saying. Here rests part of the problem:

People confuse Usability with Pleasant Aesthetics

I have heard a metric ton of debate on this topic here. Because the issue of usability goes without saying, people do not spend nearly enough time thinking specifically about usability. What they do spend time on is the aesthetic. If Aesthetic (word #2) in its simplest form means, “The shape a thing takes, typically defined in terms of tone or beauty,” then you can start seeing the problem with confusing the two (important) issues. Let’s just back to the title of this blog post.

It works!

When discussing an interface the most common measurement I hear from folks is the ever unquantifiable “It works,” or its evil twin “It doesn’t work.” The problem is this is typically an emotive response to the aesthetic and not the usability. Let me explain, again.
Referring back to the definition of usability, when examining the intuitiveness of an interface, we should be measuring, at the simplest level, recognition time (emphasis on word #3: time.) Said another way, we should be able to ask someone “Using this interface, how do I navigate to my personal preferences?” and the time it takes for the average person to answer that question gives us a hint to the usability of that interface. Imagine if I asked the question again and the response was “It works.” What would that tell me? The fact is “It works” is a conclusion at best. At best it is a description of the interfaces functional accuracy in doing the job it was created to do. Frankly, it tells me almost nothing about whether or not the interface is intuitively usable. Which leads us to usability measurement myth number two:

Usability means it either “it works” or “it doesn’t work”

Usability is a scale, a percentage and not a Boolean (i.e. true or false.) If someone asks “Is this interface usable?” the answer is almost never “no” simply because it is the wrong question. Usability “works” in percentages. The higher the percentage the better the product. It is really that simple.

So why do people, once they see a product simply declare “It works” or “It doesn’t work” by sight alone?

Well, because they are likely confusing usability with the way they feel about the aesthetic of the interface. Answers about usability should arrive as a result of analyzing the speed of recognition within an individual goal of the interface. People study this stuff by examining stuff like search patterns in humans, latent associations between interface elements, position of elements, color, contrast, etc. Ah, now we are starting to see how aesthetic can be confused with usability. There is some cross over, but typically only in terms of ones affect upon the other. It is a balancing act. Here is an example.

Imagine you are attempting to create an interface for a fun playful toy manufacturers website. Specifically you will be building the menu system. To add a little bit of fun to the site, you create big colorful menu buttons (this would be both an issue of aesthetic as well as usability.) You decide that to stay consistent with the playful theme once they click the menu item, a little magical twinkle happens and a fairly flies around the menu for a second or two (this is not interactive but simply an issue of aesthetic.) After consulting with your client you decide that each menu item should look like a toy that corresponds with the menu items. To navigate to the menu item, you would drag the toy (the menu item) onto a graphic that looks like a old pirates chest. It seems fun and exciting! You might even decide that it does in fact successfully perform the goal of navigating to the menu items location. It definitely dovetails into the fun playful aesthetic of the site. But the average human struggling to figure out that interface might actually lower the overall usability score for the menu system. Now the aesthetic nature and the quality of usability are actually competing.

Now, just because usability and aesthetic can compete doesn’t mean it is wrong, it just means that they are not the same thing. It also demonstrates that there is not always a positive correlation between a solutions aesthetic and its usability. But again, maybe you want the user to work for it a little. That is OK as well. The key is to know that they are not the same and that both should be intentionally designed and brought into the right kind of balance.

So how do you test the two? Here is how I would do it.

Testing the Aesthetic:

Aesthetics don’t spring to life without goals. Likewise, reviewing an aesthetic should mean that you examine the shape, tone, beauty of the solution with a specific and intentional eye to its goals. The workflow for review might look like the following:

1. Review each goal.
2. Examine the aesthetic and discuss how the aesthetic is meeting the needs of the goals.
3. Discuss any areas of the aesthetic that are not meeting the goals.
4. Create an action plan to modify the aesthetic to match the remaining goal needs.

Testing the Usability:
Usability is directly related to various level of functional goals within a solution. Usability should be reviewed within the scope of those various functional goals. That workflow looks like:

1. Select a functional goal to test.
2. Ask a person to attempt to perform a functional goal (you don’t tell them how, rather you tell that what the goal is and ask them to perform it within the interface.)
3. Observe how they interact within the interface: record the speed of performing the task; record the patterns by which they attempt to identify the correct interface elements; record how many times they have to start over to figure it out, etc.
4. Offer an opportunity to let them tell you how the interactive experience felt to them.
5. (Repeat step #2 above until you have tested all of the relevant functional goals.)

In conclusion, usability and usability testing is a known science. If someone claims to be a usability expert then that means they should know how to test for usability and draw quantifiable conclusions about its quality. A usability expert is not, however, someone who knows how to bake up a really tasty interface. That might just be called a really good designer. But the best designers are folks who design with an eye to both usability as well as aesthetic and know how to draw up the right balance within a given project. If someone claims “usable design experience” that should mean that they have actually subjected their interfaces to real usability testing. Now that would be a great designer!