Usability testing doesnāt have to be complicated. There are dozens of options for a design team that wants to remotely test their solutions. To prove that, weāve redesignedĀ Yelp with the help of remote usability tests.
We ran remote usability tests because theyāre fast and affordable. All of these tests were unmoderated, which means the users ran them in the comfort of their own homes. While this gives us less control, it provides for more natural results. Because their reactions were recorded, we encouraged them to think out loud.
Itās vital to understand that there is no single magic approach.Ā There are a wide selection of tactics ranging from usability lab studies to simple email surveys, and they all have their time and place. We went with a remote usability test because it strikes a nice balance between speed, accuracy, and cost.
In this article weāll explain why qualitative research matters, explain the insights, and show how they were built into the new design.
How to approach qualitative analysis
When it comes to qualitative analysis, you canāt just ask users to talk about their experiences.
As Jakob Nielsen, Partner at the Nielsen Norman Group,Ā points out, you should never listen only to what users say. The wrong approach would be creating a few designs and then asking users which one they like the most ā users havenāt tried the design, so they can only comment on what they see on the surface.
The correct approach is watching what users do and then asking them the bare minimumĀ Single Ease Question, which is: āHow easy, or difficult, was the task?āĀ This helps reduce bias and gets to the bottom line of UX analysis: how did the users accomplish their tasks, and how easy or difficult was it? Our screen recording also captured audio (since we encouraged people to think aloud), because otherwise itās easy to miss why certain behavior occurred.
How we analyzed the user videos
Distinct patterns emerged in the videos of user interactions with the Yelp website. Unsurprisingly, we learned that the search bar was one of the most essential and easy-to-use. Of course, this makes sense because Yelp is essentially a business search engine with some added social features. However, other features werenāt as intuitive, as youāll see below.
Lesson 1: The search bar was the preferred starting point
All five test participants relied heavily on the search bar, even for tasks that could easily be completed by browsing through the categories instead (like finding a nice restaurant or bar without being given any specific criteria). In fact, four out of the five participants went straight to the search bar to find a restaurant. Only one user started browsing through the categories, and she quickly found them āoverwhelmingā and ended up falling back on the search bar instead.
Itās important to mention that in our test instructions, we asked users to āfindā a restaurant, not to āsearch forā a restaurant, because we wanted to observe how they would naturally complete this task without biasing them toward a specific function.
Interestingly, when the users were given specific parameters (like the budget, ambiance, and type of restaurant, or the name of an individual business) they almost always ignored everything on the homepage except for the search bar. In our redesign, we made sure that the search bar was the most visually prominent feature.
Lesson 2: Events arenāt very noticeable
In one task, we asked the two users without Yelp accounts to find an interesting event in their area this weekend. We wanted to learn whether they would use the events tab at the top of the page.
Surprisingly enough, nobody used theĀ eventsĀ tab. When asked to find an interesting event in their area this weekend, one test participant used the search bar while the other navigated through theĀ arts & entertainmentĀ category in theĀ best of YelpĀ section.
Because holding events is an important part of Yelpās business, our redesign would pull the events section out from the sidebar and make it part of the primary scroll.
Lesson 3: Bookmarking was confusing, and no one used lists
We were curious to see how users would choose to save locations for later reference. In Yelp, there are two ways to do this: users with existing accounts can eitherĀ bookmark a locationĀ orĀ create a list.Ā We simply asked group 1 (three users with Yelp accounts) to āsaveā a number of locations to look into later so that our wording wouldnāt mention any features that could bias their actions.
Of the three users who were given this task:
- One saved the businesses using bookmarks but complained that the process took a long time
- One started to save businesses using bookmarks but gave up because it took too long
- One was not able to figure out how to save businesses and gave up on the task
The two users who used bookmarksĀ both remarkedĀ that it would be nice to be able to bookmark a business from the search results page, rather than having to go to each businessā page separately.
It would be nice to allow users to have an easier and more intuitive method of saving businesses to return to later, so we added a one-click save feature in the redesign. This new feature lets you save a business straight from the results page without diving deeper.
Lesson 4: Itās fast and easy to search for a specific venue
All five users were given a task to find a specific business and find out if it was open at a certain time. They all successfully completed this task, and rated the task as āvery easyā. As mentioned previously, all five used the search bar to accomplish this task.
Since searching for a specific business is working so well, we decided that the functionality didnāt need any changing.
Lesson 5: Photos are the top choice for determining ambiance
When asked to find a restaurant with a certain ambiance, none of the five users attempted to use the search bar. Instead, threeĀ users looked through photosĀ of the restaurant on Yelp, one visited the restaurantās website, and the last stated that the price symbols ($, $$, $$$, $$$$) was enough to indicate if the restaurant had the right ambiance.
This brought up two insights:
- Photos are an essential part of the Yelp experience, and they are critical for users to choose a business.
- To prevent cluttering the interface, the solution could be as simple as just enlarging the photos when you get to the search results page.
Lesson 6: Users relied on filters, but they need to be revamped
All five users were asked to find a restaurant for a group of 15, three of the five participants used theĀ good for groupsĀ filter, while one used theĀ make a reservationĀ feature and scrolled down until she found a restaurant that could seat the group.
At another point,Ā one user attemptedĀ to select two categories to filter his results, but one of his choices disappeared when he clicked the other.
While filters are important, we learned that they could be greatly improved. To dig deeper, we decided to to run aĀ card sortĀ on all of Yelpās current filter options to determine which ones are actually useful to users.
For the redesign, we decided that we would categorize filter categories in groups of four based on how users prioritized them. For example, we learned that āOpen Nowā and āAccepts Credit Cardsā were some of the most important filters, but it can take several clicks to access them.
Lesson 7: Price categories werenāt clear
When users were searching for the restaurant with specific parameters, one of the requirements was to find a restaurant within a $20/person budget. Two of the five users were confused by whether their $20 restaurant budget would fall into the $, $$, or $$$ category. One user stated that she didnāt know what the symbols meant, and another clicked the wrong category. The other three correctly chose the $$ category.
The definition of the symbols does not display when users select filters; it only displays when the user navigates to a particular restaurantās page. Since price expectations are highly subjective, it was unclear to users which category they should choose.
In the redesign, we solved this by placing the exact values in parentheses next to the symbols. This is especially helpful as you move between categories, since $$ has different meaning if youāre choosing a mechanic or burger joint.
Design driven by usability insights
Once it was time to design, we followed an approach based on the last few steps of theĀ Google Ventures designĀ process. UXPin CEO Marcin Treder started with numerousĀ informal sketches before a team decision helped cull it down to the top 2-3 sketches. To prevent design by committee, Marcin had the final say regarding which sketches would progress into wireframing and prototyping.
We created a wireframe to incorporate most of the design changes, then added some interactions and animations to turn it into a low-fidelity prototype. Once the animations were smoothed out, we added detail for a high-fidelity prototype.
Low fidelityĀ prototypes
High fidelity prototypes
Determining all dimensions of usability
What users say and what users do should serve as checks and balances during user testing. While you donāt need to necessarily be present during the test, an audiovisual recording is mandatory, otherwise you might miss out on the context of actions. When you combine qualitative analysis with quantitative analysis , youāll get an even clearer idea ofĀ whyĀ andĀ howĀ to fix a problem, as well asĀ how manyĀ usability problems your design needs to solve.
