updated: April 16th, 2012 / Ross Johnson / 8 Comments

Clark is Wrong About Nielsen Being Wrong About Mobile

Earlier last week, I was pleasantly surprised when Jacob Nielsen posted a report on mobile web design which closely mirrored my own philosophy. The key takeaway of the article is that mobile specific sites are more effective than full content sites with mobile friendly presentation. After reading the article, I hoped the findings would put the “responsive design” craze in check. I was wrong.

If you aren’t familiar with my stance on responsive design, it would be worth your while to see my previous post on the subject. If that sounds like too much work, I will summarize it for you. Responsive design is a great tool, but it has a time and place, neither of which are “always” or “everywhere.” In my experience mobile specifics site are more effective when dealing with a broad audience. Mobile optimized presentation is often better than nothing, but it rarely results in the best experience for all users. Nielsen’s research lead to a similar conclusion. Our thoughts are not shared by everyone however. Shortly after Neilsen’s post, designer Josh Clark posted a response entitled “Neilsen is Wrong on Mobile.” After reading the article I felt Clark had some valid points and I began rethinking my approach to mobile as a results. During that process I decided to reread his article, and now I am no longer sure I agree with Clark’s conclusions.

Josh Clark’s Issue with Nielsen’s Findings

The bases of Josh’s response is largely fueled by two studies that report 25% of smart phone users access the web on the their cellphone more than any other device. I couldn’t find the On Device Research study referenced, but I was able to find the PEW study. According to this study, 35% of Adults in the United States own what they would consider a “smart phone.” 25% of those smart phone users report they access the internet from a mobile device majority of the time. When I do the math, that means 123.865 million adults in the US own a smart phone, and out of those people 30.9 million claim to primarily access the internet from their mobile device. This ends up being 11% of US Adults.

Ten percent is a significant portion of the population and seems to be a magic number in web design. We (web designers) have justified accessible websites because 10% of the population has some disability. We often choose which browsers, technologies and resolutions to support based on close proximity to a 10% adoption rate. With that in mind, I can’t deny that mobile designers should consider “mobile only users” (ie: users who only browse the web using mobile devices,) but to take this study and say the mobile experience should be designed specifically for them is extreme. Specifically I have two issues with this conclusion. Does this study really prove 11% of US Adults only browse the web on a mobile device? and if so, are full feature/content websites reorganized for smaller screens really the best experience for all mobile users? Let me explain further.

What The Study Really Says

While I am a web designer and not an anthropologist, my degree is in social science. For that reason I am often skeptical of survey results. In this case, I find the conclusions of the PEW study premature. If this study is going to be the guide determining how mobile websites should be designed, we need to be sure. Having looked through the study, I don’t see it addressing mobile websites specifically at all.

In Clark’s response, he states:

“…11 per cent of adults in the US, or about 25million people, [who] only see the web on small screens”

Josh uses this conclusion to argue that mobile web browsing isn’t about short, on the go and simple tasks. He claims mobile browsing is now on the couch, in the kitchen, airport layovers, etc… In short, he is saying this study proves mobile users are looking for a full featured and full content web experience.

Remember, we are talking about the design of mobile websites. So in order for this survey to have relevance to the topic at hand, it needs to cover mobile websites. However, if you look at the question this conclusion is based on it doesn’t reference mobile websites at all. The question the survey asked is says “internet usage,” not “web usage.” The exact question is detailed below:

“Overall, when you use the Internet, do you do that mostly using your cell phone or mostly using some other device like a desktop, laptop or tablet computer?” – PEW Internet Survey Questions

“Use the internet” is very different from “Browse the web.” There are hundreds of activities that constitute using the internet that don’t involve a mobile browser. E-mail, directions, music streaming, video streaming, app downloads, instant messaging, etc… etc… How can we use one study as a guide for mobile website best practices when it doesn’t even address websites specifically?

Furthermore, even if the study did prove 11% of US Adults only browse the web using their mobile device, how does that impact mobile web best practices? Does it really mean we should have one website for all devices with altered presentation? I think not.

Prioritizing Mobile Design

While 11% is a significant portion of the US population, it seems backwards to cater towards them over the other 88% who don’t use their cellphone as their primary internet device. Going back to Jackob Neilsen’s original test, after testing hundreds of mobile websites they sites with reduced functionality and features offered a better mobile experience. Disputing Neilsen’s field research because a small percentage of the population only browses on cell phones or does so for extended periods of time is illogical. To further say we should design mobile sites focusing on fringe cases rather than the majority is also illogical.

This doesn’t mean designers shouldn’t account for the fringe cases, by all means we should. Neilsen takes this into consideration by mentioning a “Full Site” link at the very beginning of his article. Doing so in an easy and obvious manner accounts for the fringe cases while catering to the most common use cases first. We can always make considerations for the anomalies where necessary. For example, you could provide a link to the full site which also employs responsive design. This is a more effective prioritization of users.

The Future of Mobile

Despite my disagreement with Josh, I admire his mobile education efforts. Mobile web design is very young, the way people use mobile is constantly changing. We may have strategies for mobile design but we have yet to find which strategies work best and why. This is why I also value Neilsen’s work. There is a lot of speculation and opinion surrounding mobile design, Neilsen does research. Facts and field research will ultimately uncover the best approach, not surveys and speculative conclusions. This is why Nielsen’s findings are ultimately right and should weigh in on the mobile design approach.



8 thoughts Clark is Wrong About Nielsen Being Wrong About Mobile

  1. Clark is Wrong About Nielsen Being Wrong About Mobile: Earlier last week, I was pleasantly surprised when Jacob … http://t.co/LuP2cuHy

  2. My take on the full content vs simplified mobile site debate – http://t.co/3YG2MtPf

  3. This is a great post. I have found it very useful in my research work. Thanks!

  4. While you are correct when you say that Internet usage is not equivalent to browsing the web, I have a strong suspicion that if you were to revisit that study with modified language or conduct another survey about how the average person interprets the phrase “use the Internet,” you would find that the vast majority of people, who are non-technical, conflate the terms.

    After extensively considering Jakob’s article and numerous conversations with many of the vocal and outspoken industry figures who disagree, here’s what I believe is the crux of the disagreement with Jakob. At a fundamental level, here are two recommendations that are mutually exclusive:

    1) Aim for content and feature parity between a mobile and a desktop web experience. Do the best you can to provide as much content parity as possible, in a presentation that is optimized to view all the same content on a mobile device. If, for some practical reason, you are unable to achieve content parity, provide a link to view the rest of the content in a presentation that is not optimized for mobile; a desktop site for example.

    2) Do not aim for content and feature parity. Trim content so that you think you have provided all of the information a user might actually care about, and provide that content in a presentation that is optimized for view on a mobile device. For the rest of the content, provide a link to view the content in a presentation that is not optimized for mobile; a desktop site for example.

    While, functionally, many businesses will ultimately find themselves unable to provide content parity, the differences in these two recommendations is that one recommends to provide access to all content and optimize the presentation of all of that content, and the other recommends to provide access to all content, and do not optimize the presentation of all content.

    As a researcher and a source of usability recommendations, Nielsen should be *recommending* the optimal presentation, not a likely but less desirable pragmatic fallback.

    Nielsen himself says in the article, “True, we’ve seen some underpowered and poorly designed mobile sites that would hardly satisfy anybody’s mobile needs. But bad design that misinterprets a guideline is no reason to throw the baby out with the bathwater.”

    However, he makes this exact mistake, throwing the baby out with the bathwater, when he says the simple truism, “The design challenge is to place the cut between mobile and full-site features in such a way that the mobile site satisfies almost all the mobile users’ needs. If this goal is achieved, the extra interaction cost of following the link to the full site will be incurred fairly rarely.” Of course if you do it right, you will have done it right. But he provides no argument for why it’s better to cut content and hope no one will notice over including all content and knowing no one will notice.

    The unspoken premise seems to be the argument or claim that there is content that cannot be optimized for mobile, which is an entirely different argument than any Nielsen makes, and is one that should be viewed extremely skeptically in lieu of extensive research and proof.

    Matt Menzer
  5. Nicely considered aticle about the backlash from Nielsen’s mobile post. http://t.co/YVJatmjK

  6. Hi Matt, first thank you for your long thought out response. I appreciate your time to read through my thoughts and formulate an intelligent, articulate response.

    It probably won’t surprise you that I stand by my conclusion. Neilsen’s thoughts are based on field research and testing where everyone else is speculating. As a social scientist, I can say with certainty that speculating a similar response with different language is pretty bold. Furthermore, surveys themselves are inherently flawed. The people who choose to respond might be a select group that is different than the majority, people lie about their answers and often times how they actually behave and how they think they behave are extremely different.

    If the question becomes “should content be cut at all?” then I ask, what is design? My definition (and the dictionaries agree) is “creation intended for a specific purpose.” Thereby design is not a practice of compiling anything that can be included, rather deciding what should be left out. Whether you are designing for a mobile device or a desktop you need to decide what should be included and what shouldn’t. This is why we develop personas and research target markets. So we can produce a design that is more effective for the primary user base, not the fringe cases. This is a topic that has been beaten to death for the past twenty years, it puzzles me to see it contradicted when it should be reenforced.

    Is there content that cannot be optimized for mobile? Maybe, but that’s irrelevant. More pressing would be, is there content that should be optimized for mobile? Every page on the site, every link in the navigation, every element in the viewport makes the website more complicated and difficult to use. Trying to design for everyone rather than the primary users isn’t design, it’s a “catch all.” Include all content and knowing no one will notice? People will notice. They will notice it’s harder to use, more difficult to find what they are looking for and that they have to speed precious time sifting through irrelevant content using a difficult device to find what the content they need right now. The exception might be early adopters, but they again are not the majority.

    Ultimately I trust Neilsen’s findings and conclusions. In this debate he is the only one who has performed field research and his experience is more than enough to validate his methodology and findings.

    I certainly welcome further discussion. Thanks again Matt.

    Ross Johnson
  7. Clark is Wrong About Nielsen Being Wrong About Mobile http://t.co/wFITjx6c

  8. Clark is Wrong About Nielsen Being Wrong About Mobile #webdesign http://t.co/sKSThwmr

Comments are closed.