Clear results

December 31, 2015 10:29 AM didn’t want any surprises for mobile shoppers when it planned to replace its separate desktop and mobile sites in late August with a responsive e-commerce site that adjusts to the size of the screen the consumer is viewing. In particular, the retailer wanted to ensure that nothing was lost for mobile shoppers when the site adjusted to their smaller screens.

That led the online-only seller of costumes and decorations to use several full-time employees to test its site on mobile devices, says Troy Eaves, vice president of marketing at, which operates the site.

“We handed our testers a device and said ‘Buy this.’ Sometimes we’d find things didn’t auto-scroll, or if there was an error when they entered their data, they wouldn’t find out until they had filled out the entire form,” he says. In the run-up to the site going live, the tests led the retailer to tweak the form so the page auto-fills based on a shopper’s previous purchases and made a dial pad automatically appear on a user’s phone instead of forcing the consumer to switch to the numbers mode on her keypad, Eaves says.

Those types of changes were necessary, Eaves says, because the retailer’s broad customer base means the site has to work for all shoppers, regardless of device or browser. That requires plenty of testing and tweaking. “Whenever we do a facelift or small redesign, we have to make sure the mobile looks and feels the same as the desktop,” he says.

As the number of consumers browsing and buying on mobile devices continues to grow, retailers have little choice but to ensure they offer shoppers a high-quality online shopping experience. Mobile design, more than desktop design, is tied to user experience and conversion, says Jason Wong, research analyst at consultancy Gartner Inc. Consumers have their smartphones in hand and retailers are seeking new ways to engage with them, when they’re in stores or elsewhere, which means it’s crucial that the sites are easy to navigate. And that often requires retailers to conduct mobile usability testing. Those efforts can range from large budget experiments in a retailer’s innovation lab or technology center or lower-key efforts, such as working with vendors to observe consumers using a retailer’s site or app.

The methods a retailer uses to test are less important than how the retailer uses the insights it garners from testing to revamp its mobile site or app, experts say. “If you understand what the five things are that 80-90% of your customers want to do, you can design for that flow,” says Bill Albert, director of Bentley University’s Design and Usability Center.

Testing is just one piece of a larger puzzle, Wong says. Rather than an annual or occasional task, testing, combined with analytics, works best when retailers integrate it into their everyday business. “Testing goes hand in hand with analytics, especially for mobile sites or apps,” he says. “It’s essential from an operational and behavioral perspective.”

On the operations side, a retailer can use site analytics to discover that shoppers using certain devices are experiencing problems, which can help it react quickly, Wong says. From a behavioral perspective, a retailer testing its checkout process, for example, may see a jump in cart abandonment. The retailer can troubleshoot for latency issues or determine if the abandonment rate is user-driven, which could mean, for example, a consumer is in a certain geographic location and receiving offers from a local retailer so he isn’t completing the purchase, Wong says.

Mobile user testing doesn’t have to be complicated. A standard test from, for example, takes about an hour and can be done remotely, says Michael Mace, vice president of mobile at the user research platform provider. A retailer can buy individual tests for $50 per respondent if it wants to try out a tool and gauge reaction. Panelists who agree to test sites can load an app when they are ready to do a test and follow the instructions to, for example, shop for boots on a retailer’s site. The app records the user’s smartphone screen and records audio as she goes through the process. Blue dots illustrate how the user swipes and touches the site on her smartphone. then compiles and analyzes the data, or it can go straight to the retailer for analysis, Mace says.

By using relatively inexpensive methods to test their sites and apps, retailers can test often to ensure that the regular tweaks and adjustments they make are having the desired impact on their customer experience. Revolve Clothing, for example, uses to regularly test its mobile and desktop sites, says Grace Hong, vice president of product and design.

A recent test to help reshape the web-only apparel retailer’s checkout process revealed an issue with product category navigation, Hong says. Revolve had added a small icon in the header that opened up more category options for shoppers, but “unfortunately, it wasn’t obvious enough for our users,” says Alex Park, the retailer’s product manager for mobile and optimization.

When the retailer was revamping the category navigation for mobile shoppers, Hong and her colleagues had thought the changes were clear. But they weren’t. When the retailer examined videos of its tests it found shoppers completely bypassing the navigation options.

In addition to testing its own site, Revolve observes how testers interact with competitors’ sites, which it uses as benchmarks, Park says. Revolve, which has an annual contract with but declines to say what it pays, uses five testers in each usability session and reviews the screen captures and video and audio results, he says. Often Park compiles snippets into a highlight reel to share with designers and other mobile team members so they can see and hear user experience firsthand.

“We will submit a request for a test and get results within a day,” Hong says. “It’s invaluable and easy to add into our process.”

Not every retailer relies on an outside vendor to find what’s working and what isn’t. Take, an online-only seller of underwear and sleepwear that never tests its site.

“We trust our partners for search and design to follow best practices,” says Terri Hunsinger, WebUndies’ co-owner, referring to vendors she pays for site design, personalization and other services. “I personally read all feedback from customers regarding the process from sources like Reseller Ratings reviews, social media, direct emails from them asking for assistance. I pay close attention to those feedback loops and take action as needed if we find a source of friction or frustration.”
The system works, she says. For example, having determined from customer feedback and her own experience that mobile shoppers were frustrated with the site, WebUndies spent about $25,000 to launch a responsive design site.

The redesign, which rolled out in August 2014, eliminated unnecessary elements on the mobile site to help shoppers quickly find what they’re looking for. Take the bright but stark home page, which features three images, one each for women, men, children, along with a “What’s hot now” category. The only other elements on the page are a search bar, a three-line “hamburger” icon that shoppers can click on for a dropdown menu and bar at the bottom of the page where consumers can enter their email address and subscribe.

“We made the pictures bigger and on certain pages where we would’ve had an image, price and text, we just made it a photo. If the picture is engaging, they’ll take that next step and click to learn more. People almost never read the text,” she says. Without testing, it’s difficult to conclusively say that the revamped site eliminated shoppers’ frustrations, but the changes seem to have helped. “There’s definitely uptick in mobile conversion since we made the changes,” she says.

WebUndies missed a cutoff date to change code on the site before the holiday shopping season, but it plans to update product pages early this year to include a clearly labeled “Add to cart” icon instead of the unique duck-feet buttons the retailer uses to denote its sizes on products (its brand icon is a duck in boxer shorts). The change is coming thanks to shoppers writing and calling to complain that they didn’t know how to add an item to their cart.

Testing, however, eliminates much of that type of wait-and-see approach. And with site and app development moving much more quickly now than it did a couple of years ago, with updates occurring in weeks not months, testing is becoming increasingly important, Gartner’s Wong says. “It’s so competitive that you have to use some level of A/B or multivariate testing to understand your users better,” he says. “Analytics can also help automate and streamline the process about getting feedback and prioritizing resources.”




Top Solution Providers