Going further with crowdsourced user testing: The System Usability Scale

Title graphic bar


Opening your ears and eyes to input from every level is critical to staying innovative – a lesson the folks behind TryMyUI put into practice with crowdsourced web usability testing that connects businesses and organizations with real users and their concerns and insights. Video, audio, and written feedback are all invaluable in optimizing your website for the customer, but they don’t fill all the gaps in your self-understanding.

Imagine an Olympic swimmer that watches video to improve his form, invests in the newest and most advanced swimwear, and trains in the best of facilities. Every time he beats his personal record, he is making progress; but it’s hard to know what that progress means until he compares his time to the other top swimmers’ personal bests. In the same way, it is easier to understand and make the most of usability feedback when it is placed in the context of the bigger picture – How does your website chalk up to the myriad others? In what aspects is it stronger, or weaker? Grounding your user feedback in a broader context allows for a complete understanding of the nuances of not just your own system, but also of the global system of which it is a part.

Fortunately, a tool exists already that has been used for decades to this very purpose. The System Usability Scale (SUS) is a widely respected questionnaire that quantifies and standardizes usability data, allowing UX researchers to make meaningful comparisons between feedback that, in its video/audio form, is subjective and non-measurable. Today something of an industry standard in the usability field, SUS has long been a favorite for its simplicity and accuracy: ten questions, a five-point “strongly agree” to “strongly disagree” response system, and a quick scoring algorithm yield an extremely reliable score for your website on a scale of 0 to 100.

1. I think that I would like to use this system frequently.
2. I found the system unnecessarily complex.
3. I thought the system was easy to use.
4. I think that I would need the support of a technical person to be able to use this system.
5. I found the various functions in this system were well integrated.
6. I thought there was too much inconsistency in this system.
7. I would imagine that most people would learn to use this system very quickly.
8. I found the system very cumbersome to use.
9. I felt very confident using the system.
10. I needed to learn a lot of things before I could get going with this system

SUS response scale

With thousands of previously documented uses to compare to, SUS gives you a solid idea of users’ overall satisfaction with your website, and can even be broken down into usability and learnability components. The percentile ranking contextualizes your raw score, allowing you to understand how your site performs relative to others; and some researchers have tried, with some success, to map adjectives like “excellent,” “poor,” or “worst imaginable” to individual scores for extra insight.

SUS quartiles and adjectives

By various accounts, the mean SUS score hovers around 68-70.5 (a score that roughly corresponds, as it happens, to the adjective “good,” though falling quite short of “excellent”). Normalizing score distribution with percentiles therefore makes a 68 (or a 70.5) into a 50% – better than half of all other systems tested, and worse than the other half.

Though described by its inventor as a “quick and dirty” measure, studies have found SUS to be among the most accurate and reliable of all usability surveys, across sample sizes. It has today become one of the most successful metrics for quantifying system satisfaction, with thousands using it to gauge user-friendliness over a wide range of products online and off.

It is these qualities that make SUS so key in getting a holistic picture of your website. By aggregating and synthesizing a diverse array of tester responses into a concise portrait of website usability, SUS brings a deeper understanding of what your user feedback really means. If individual test videos are the trees, SUS shows you not only the forest, but the entire ecosystem into which your system fits; with a widely-trusted industry standard to rely on, you can take a step back from your own company and see how you fit into the broader world that surrounds you.

To learn more about the System Usability Scale and its application in UX research, join TryMyUI and Measuring Usability’s Jeff Sauro for the SUS Webinar on October 9.


Comments are closed.