Can Blind People Use Social Media Effectively? A Qualitative Field Study of Facebook Usability

Rakesh Babu

  Open Access OPEN ACCESS  Peer Reviewed PEER-REVIEWED

Can Blind People Use Social Media Effectively? A Qualitative Field Study of Facebook Usability

Rakesh Babu

School of Information Studies, University of Wisconsin-Milwaukee Milwaukee, United States

Abstract

Social media allow people to communicate, collaborate and socialize for personal and professional matters. However, their sight-centered design can present access and usability problems for the blind. Existing quantitative approaches to usability testing do not provide in-depth assessment of the problem. This paper presents a qualitative approach to test social media usability, and illustrates its application to evaluate Facebook for the blind. Think-aloud observation of six blind participants generated verbal evidence of their Facebook interaction experiences. Verbal protocol analysis explained the nature of interaction challenges in performing common Facebook functions. Design standards analysis explained design errors in Facebook interface. It helped identify remedial measures to potentially improve Facebook usability. Findings demonstrate the utility of the qualitative approach to feasibly evaluate social media usability for blind users. It shows how blind users think, act and perceive in performing common social media functions non-visually. This has implications for the design of non-visual user interfaces to access social media through ‘Internet of Things’ and in multi-tasking situations.

At a glance: Figures

Cite this article:

  • Babu, Rakesh. "Can Blind People Use Social Media Effectively? A Qualitative Field Study of Facebook Usability." American Journal of Information Systems 2.2 (2014): 33-41.
  • Babu, R. (2014). Can Blind People Use Social Media Effectively? A Qualitative Field Study of Facebook Usability. American Journal of Information Systems, 2(2), 33-41.
  • Babu, Rakesh. "Can Blind People Use Social Media Effectively? A Qualitative Field Study of Facebook Usability." American Journal of Information Systems 2, no. 2 (2014): 33-41.

Import into BibTeX Import into EndNote Import into RefMan Import into RefWorks

1. Introduction

Social networking sites (SNS)such as Facebook and Twitter offer a conducive platform to communicate, collaborate and socialize – personally and professionally [1]. There are 2.3 billion registered users of the ten most popular SNS [2]. SNS is used all over the world, by all types of users for numerous reasons. Government agencies such as the Central Intelligence Agency and the Environmental Protection Agency use SNS as a productivity tool [3]. Merchants such as McDonalds, Whole Foods, Best Buy and Zappos use SNS to sell their merchandise [4]. It is reasonable to expect that SNS will remain a mainstay of the future information society. However, the myriad benefits of social media that the sighted world continues to enjoy are not available to people who are blind. This is primarily because Web 2.0 technologies are sight-centered by design and lack the needed accessibility and usability. Accessibility allows blind users access to all features and functionality of a website [5]. Usability is how well a website fits with a blind user's conceptualization of completing online tasks it supports [5]. Both accessibility and usability are necessary to derive the intended utility of a website. [6] This research is concerned with blind people’s ability to derive SNS utility.

The blind are a significant user group comprising 39 million totally blind and 246 million partially blind. [7] They access SNS and other web sites primarily by listening to content read aloud by screen reader (SR) software. They conceptualize Web interactions differently than sighted users. Extant literature recognizes that SR-mediated SNS interaction is inherently problematic. [1, 8] These problems may result from poor design choices that necessitate visual interaction with content and controls. [8] However, it is unclear as to how these interaction problems hamper goal accomplishment for blind SNS users. Existing approaches to usability evaluation such as automated testing, heuristics evaluation and user testing fail to provide an in-depth assessment of the problem. [9] Critical questions that remain unanswered include: What is the nature of blind users’ SNS interaction challenges?; and What can we do about these challenges?

This paper presents a qualitative approach to SNS evaluation for blind users and demonstrates its application to understanding the nature of their accessibility and usability problems in performing common SNS functions. We conducted an exploratory field study with six blind Facebook users. Think-aloud observations generated rich verbal evidence of their thoughts, perceptions and actions in performing three common SNS functions. Verbal protocol analysis [10, 11] and an integrated problem-solving framework [11] generated in-depth, contextually-situated, experiential knowledge of their interaction challenges in searching for people, communicating with others, and planning social events. Design standard analysis [12] identified design errors responsible and potential remedial measures.

What follows is a description of our methods, participants, material and procedure. Subsequently, results and discussion is presented. Finally, we discuss contribution, implications and future work.

2. Materials and Methods

2.1. Methodology

We used concurrent verbal protocol analysis (VPA) to collect and analyze context-rich evidence of blind users’ SNS interaction experiences. In this technique, participants work on tasks and concurrently verbalize their thinking. [5] This technique is suitable for in-depth examination of user-system interaction [13] and evaluates systems usability effectively. [14] Think-aloud verbalizations of participants are captured in audio recordings, which are transcribed and decomposed into segments. Each segment is categorized using a coding scheme. [5] Babu, Singh and Ganesh [15] adapted VPA for a closer examination of blind users’ actions in completing online tasks. Babu [11] developed a coding scheme to capture the nuances of blind users’ interaction challenges. Babu and Singh [12] employed design standards analysis to determine the accessibility and usability character of an interaction challenge. We use a combination of the adapted VPA, the Babu [11] coding scheme, and design standards analysis for an in-depth, contextually-situated, experiential understanding of blind users’ SNS interaction challenges.

2.2. Participants and Site

We recruited six participants, mean age 19 years, with one to two years of familiarity with SNS. Three participants were students at the Texas School for the Blind and Visually Impaired (TSBVI) and three at the Michigan Commission for the Blind Training Center (MCB/TC) at the time of the study. These institutions have technology instructors to train students in the use of screen readers, computers and the Web. They facilitated recruitment of participants for our study. Given it was a field study, TSBVI and MCB/TC campuses were the research sites.

2.3. Tasks and Material

We designed three Facebook tasks as the context for investigating participants’ SNS interactions. Task choice was guided by the researchers’ knowledge and experience in working with blind and sighted SNS users. The tasks were:

1. Searching for people;

2. Communicating with others;and

3. Planning social events.

2.4. Searching for People

A first and basic task every social media user undertakes is searching for friends and choosing a sub-set from the search results. This task requires a search query on Facebook where users locate the SNS search field; type appropriate search term(s); and activate the search button. The SNS search field is one of two search fields on the account home page along with the “Web Search” field. Appropriate terms to search for Facebook users include full name and email. The search button immediately precedes the search fields. Successfully executing the search query generates a list of results with links to profile pages of users with matching names, followed by an “Add Friend” button.

2.5. Communicating with Others

Social media is used to communicate with friends and family by exchanging messages. This task on Facebook requires: navigating to a user profile page; navigating to her Timeline; locating the message text box; typing a message; and activating the Post button. Successfully posting a message displays the message text on the Timeline along with other posted messages.

2.6. Planning Social Events

Social media allow subscribers to plan adhoc events. Facebook allows users to accomplish this task by: navigating to the Events page; activating the Create Event; filling in each field in the Create Event form with relevant information; and activating the Create button. Successfully creating an event generates a notification for all invitees regarding the name, location, timing and guest list for the event.

Material included an SNS platform, an observation study protocol, and computers equipped with an SR and Internet connection. The SNS platform comprised multiple Facebook pages supporting the three tasks described above. The observation study protocol included a Word document describing the research objective and think-aloud technique. It included a think-aloud practice exercise for an online task. It also included instruction to complete the three Facebook tasks. This document was available on the home screen of the study computer.

2.7. Procedure
2.7.1. Data Collection

After receiving IRB approval, we emailed a flier seeking research participants for the field study to US institutions serving the blind. TSBVI and MCB/TC agreed to help by facilitating recruitment and providing lab space. Students who agreed to participate completed a questionnaire seeking demographic and background information. They scheduled study sessions during summer 2012 at the site closest to them during after-school hours. Each session commenced with a familiarization of the research, and the think-aloud technique. They reviewed the observation study protocol and practiced thinking aloud while performing the practice task. They logged on to their Facebook accounts, visited each task environment and completed the tasks while thinking aloud. Each participant took roughly two hours to complete the three tasks. We audio recorded each participant’s verbal protocols and transcribed verbal protocols including participant verbalizations, participant-investigator conversation and screen reader audio.


2.7.2. Data Analysis

We employed two analytic techniques—verbal protocol analysis and design standard analysis.

We performed verbal protocol analysis on the rich verbal evidence to examine participant thoughts, perceptions and actions in Facebook interactions. We decomposed each transcript into segments representing single units of thought. Three independent coders coded nearly 21,000 segments using categories of an integrated problem-solving framework. [11] Coded segments accounted for roughly 13 hours' worth of data. Coders were free to assign multiple codes to a segment if necessary. We used Cohen's kappa to assess inter-rater reliability, which yielded a reliability measure of 0.8 (N = 16,748). We identified segments representing interaction problems that manifest as dissonance or failure. [11] Problems where the situation was not comprehensible to the participant due to inadequate system feedback were labeled as Dissonance. Problems corresponding to situations where participants’ action did not yield expected outcome were labeled as Failure. We analyzed only codes that all three coders agreed upon and identified segments representing interaction problems that hampered participants’ ability to perform tasks effectively.

We performed design standard analysis on the VPA results to determine the accessibility and usability character of dissonance or a failure. Design standards include Web Content Accessibility Guidelines success criteria and usability heuristics. [16] This analysis provided a basis to compare our results with the WCAG guidelines – the de facto standards on Web accessibility and usability for users with disabilities. We retraced participants’ interaction paths and identified associated design elements and mapped problematic design elements and the accessibility and usability criteria violated. Together, VPA and design standard analysis provides our research the needed user-centered understanding of SNS accessibility and usability problems in social learning tasks.

Figure 1. Verbal evidence of confusion about relevant search field for people search

3. Results

Our analysis showed that basic SNS functions that typical sighted users take for granted proved problematic for participants and were perceived as vexing. Multiple interface elements (content and controls) that make up the task environment for an SNS function created dissonance or hampered goal accomplishment for participants. Since these interface elements failed to meet their objectives for blind users, they were in ‘error’. Hence, we refer to such problematic controls and content presentation as ‘design errors’. Each design error represents a violation of extant accessibility and usability principles based on the W3C Web Content Accessibility Guidelines. Based on normative design principles, we suggest design improvements to the task to alleviate blind users’ interaction challenges, and facilitate their effective participation in online social networking. In the following, we describe and discuss these problems grouped by task.

3.1. Difficulties in Searching for People

Analysis showed that participants faced multiple problems searching for people on Facebook. These problems included confusion about relevant search fields, ambiguity about appropriate search terms, and inability to understand reasons for failure. We present evidence of these problems using verbal reports comprising participant utterances and screen reader announcements (in angular brackets). To help the reader better appreciate the problem, we indicate participant pauses (reflecting additional information processing and cognitive effort) and screen reader silences (representing lack of system feedback).

We first present evidence of the confusion about the relevant search field. The participant logged on to her Facebook account and intended to search for one of the authors.

Figure 1 shows the participant’s confusion about which search field to use to search for a friend on Facebook. It indicates that the home page contained at least two search fields - one for a regular Web search and another for specific SNS search. As the participant browsed down this page from the top left corner, she first came across the Web search field and could not “see” that there was another search field, due to the sequential nature of non-visual interaction. The Web search field lacked a descriptive caption to communicate its purpose. Consequently, the participant assumed that Web search was the relevant search field and was baffled by the failure to obtain relevant search results.

Figure 2 presents evidence of ambiguity about appropriate search terms. The participant located the relevant search field, and typed in the search terms to find the author.

Figure 2. Verbal evidence of ambiguity about appropriate search terms for people search

The participant could not determine an appropriate term to search. The label indicated that a user could be searched by name or email. She tried searching for the author first using name and then using email. However, the query did not yield any result for either term; she consistently got the message “no results found …” The participant suspected she may have made a typo and tried to rectify this unknown error. She could not tell what search terms could possibly generate the desired result.

Figure 3 presents evidence of inability to understand the reason for a failed search. The scenario is the participant tried to search the author by name and by email without luck; the search query did not yield any result. She decided to give it one more try.

Figure 3. Verbal evidence of inability to understand reason for failed people search

The evidence in Figure 3 shows the search query using email did not yield any result. Importantly, the participant did not receive any explanation as to why the query failed - she just heard “Check your spelling or try another term.” However, the participant had spelled the name and email correctly and was appropriately baffled by the failure.

Participants faced problems searching for people on Facebook. Problems include confusion about which search field to use, ambiguity about what search term is appropriate and an inability to tell why a search query failed. These problems correspond to three design errors: improper labeling of the Web search, lack of clarity about appropriate search terms, and incomprehensible explanations for a failed search query. According to the Web Content Accessibility Guidelines (WCAG), these design errors represent accessibility and usability problems. Improper labeling of the Web search field is a violation of WCAG Success Criterion (SC) 1.1.1 that requires a descriptive label conveying the utility of the field. Ambiguity about appropriate search term violates SC 3.3.2 and 3.3.5. SC 3.3.2 requires supplementing SNS search field with clear instruction on appropriate search terms and avoiding common mistakes in searching for people. SC 3.3.5 requires provision of “Context-Sensitive Help” that guides users in executing an effective SNS search without getting distracted from the search task. Random explanation for failed search query violates Success Criteria 3.3.1 and 3.3.3. SC 3.3.1 requires displaying a text message following the activation of Search button clearly explaining the reasons for the failed search query (e.g. incorrect search term used). SC 3.3.3 requires this text message to include suggestion on how to fix the faulty SNS search. Random explanation for failed search query also violates Norman’s [17] Feedback Principle that requires full and continuous feedback about SNS response to an SNS search attempt.

3.2. Problems in Communicating with Others

Analysis showed that participants faced problems in communicating with others on Facebook. This is due to the problems in locating the Timeline—Facebook’s message board. Problems included in ability to identify the Timeline section and difficulty perceiving the Timeline input area. We present evidence of these problems using verbal reports that include participant utterances and screen reader announcements. The participant navigated to her friend’s Facebook profile with the intention to say “Hello.” She browsed the friend’s profile page looking for the Timeline.

Figure 4. Verbal evidence of problems locating the Timeline to communicate with others

As Figure 4 reveals, the participant could not locate her friend’s “Timeline” after scanning her profile page multiple times. She came across a section of the page with messages from other people. She supposed this possibly to be the friend’s Timeline. However, she could not tell this with certainty as there was no descriptive label to identify the message board. Moreover, she could not find the input area to type a message.

Participants faced problems communicating with others on Facebook. Their real problem was locating the Timeline. This problem arises due to the absence of any descriptive label identifying the Timeline section, and the obscurity of the input area to type a Timeline post. WCAG explains that the difficulty communicating with other SNS users represents accessibility and usability problems in Facebook design. The absence of a descriptive label for the Timeline violates Success Criterion 2.4.10. This requires the message board content to be organized under a separate section identifiable through its descriptive title. Obscurity of the Timeline’s input area violates WCAG Success Criterion 1.1.1. This requires a descriptive caption for the input field clearly communicating its purpose.

3.3. Problems in Planning Social Event

Analysis showed that participants faced problems planning an event on Facebook. Specifically, they could not schedule the event effectively. This is due to the problem choosing a desired date. We present evidence of this problem using verbal reports that comprises participant utterances, SR announcements (enclosed in angular brackets), and system-generated non-verbal sounds (enclosed in square brackets). The participant navigated to the “Create Event” page on Facebook. She wanted to plan for a get-together with selected friends on Memorial Day.

Figure 5. Verbal evidence of problem in scheduling event

As the evidence in Figure 5 reveals, the participant could not edit the date field in the “Create an Event” form. The default value for this field was “Today”. The participant tried to delete this default value in order to input the desired date. However, she could neither delete “Today” nor insert the desired date - May 31. Her action to delete the default value with the use of Backspace resulted in backward page navigation. By the time she realized, she had navigated multiple pages back. A closer examination of the evidence shows that the Backspace occasionally behaved like the browser’s back button. So, the first few Backspace key presses did not do anything. But subsequent key presses triggered backward page navigation. And all this while, the participant had no clue what was going on.

Participants could not plan an event effectively on Facebook. They faced difficulty editing the Date field of the Create Event form to schedule the event. In addition, they dealt with the nuisance of backward page navigation when trying to delete a default field entry. These problems correspond to two design errors— date field not operable through keyboard and backward page navigation triggered by Backspace without explicit user request. WCAG explains these design errors represent accessibility and usability problems in Facebook design. Specifically, the lack of keyboard operability of the date field is a violation of Success Criterion 2.1.3. This requires the field to be editable through key commands. The backward page navigation triggered by Backspace is a violation of Success Criterion 3.2.5. This requires that only an explicit user request should trigger a page change.

4. Discussion

The qualitative evaluation of Facebook usability provided an in-depth, contextually situated and experiential knowledge of participants’ interaction challenges in performing basic SNS functions and identified the responsible design elements. Each of the three SNS functions examined was problematic. Participants faced multiple types of problems performing these functions. Problems included challenges that require greater time and effort, and roadblocks that necessitated sighted assistance to continue. These problems were linked to poorly designed controls and content presentation used to design these basic SNS functions on Facebook. In fact, these design errors that created roadblocks and challenges for participants represent violation of multiple accessibility and/or usability criteria as defined by WCAG and other design standards.

Searching for people on Facebook was problematic. Participants could not tell which search field to use, what search terms would be appropriate, and why a search query failed. These problems resulted from design errors such as improper labeling of the Web search field, ambiguity about appropriate search terms, and random explanation of failed search queries. Communicating with other Facebook users was problematic. Participants could not locate the Timeline to post their messages. The problem resulted from design errors such as lack of organization and descriptive section header for the Timeline, and lack of descriptive caption for the input field. Planning an event was also problematic. Participants could not use the Create Event form effectively. Specifically they were unable to set the schedule of the event. This resulted from two design errors--date field not operable through keyboard and backward page navigation triggered by Backspace without explicit user request.

A secondary yet valuable outcome of our qualitative evaluation was a set of design improvements to potentially improve the accessibility and usability of these three SNS functions of Facebook. For example problems in searching for people can be reduced by providing (1) a descriptive label for the Web search field communicating its purpose unequivocally; (2) instruction accompanying SNS search field describing appropriate search terms and common errors; (3) context-sensitive help that describes the process of effective SNS search; (4) complete, accurate and continuous feedback about SNS response to a search query; and (5) descriptive error message suggesting how to rectify it following a failed search query. Problems in communicating with other Facebook users can be reduced by providing (1) Timeline as a distinct section having a descriptive section header; (2) instruction on how to access the Timeline’s input area; and (3) descriptive label for the Timeline’s input area. Problems in planning social events on Facebook can potentially reduce by providing: (1) keyboard support for operating the date field; (2) instruction about performing edit functions in date field using key commands; and (3) page navigation only when the user requests. Our ongoing research examines the feasibility of these proposed design improvements through interviews with web developers first and then the development of a prototype system to validate this claim. Examination of multiple instances of failures and development of interventions to remediate them is the focus of our on-going work, based on the research presented here.

5. Conclusions

The central premise of the research reported here is that millions of blind people cannot utilize social media effectively due to significant accessibility and usability barriers. To resolve this issue, we need answers to two critical questions: What is the nature of blind users’ SNS interaction challenges? And, what can research do to reduce or eliminate these challenges? To answer these questions, we need to carefully examine blind users’ thoughts, perceptions and actions in SNS interaction tasks, and analyze their challenges vis-à-vis extant design standards on Web accessibility and usability. The qualitative approach forwarded in this paper helps bring closure to this research problem.

The approach evaluates SNS accessibility and usability in a more comprehensive manner. Its hallmark is a combination user-centered and task-based approach, [15] with verbal protocol analysis, [5] an integrated problem-solving framework [11] and design standards analysis [12] for an in-depth, contextually-situated and experiential understanding of the problem. The paper demonstrated the feasibility and utility of this technique through an exploratory field study that examines the Facebook interaction experiences of 6 blind users. Three basic SNS functions—searching for people, communicating with others, and planning social events were studied. Results illustrate the nature of users’ interaction problems, why these challenges arise and what can be possibly done about these challenges. It shows that blind users cannot effectively search for people, communicate with others or plan events on Facebook. They experienced dissonance and often failed to achieve their goals due to poor design choices. It explained what they go through in dealing with these dissonant conditions or failures. Additionally, it identified potential design modifications to reduce or eliminate these challenges. Such understanding is needed to develop design principles on SNS accessibility and usability for non-visual interaction.

Researchers in Information Systems, Cognitive Science, and Human-Computer Interaction could utilize our qualitative approach to investigate accessibility and usability of other Facebook functions (e.g., inviting friends to a network, joining interest groups, sharing multimedia content), other SNS (e.g., Twitter, MySpace, Orkut), and other social media genre (e.g. weblogs, wikis, social bookmarking sites). They could use this approach to better understand how blind users conceptualize a problem situation, and how they deal with it. Such understanding is necessary to develop (a) blind-minded help mechanisms to guide the blind in effective SNS interactions; and (b) design principles on systems accessibility and usability for non-visual interaction. Nonvisual interaction assumes significance for the use of Internet of Things and in multi-tasking situation where one task demands constant visual attention.

Understanding blind users’ SNS interaction experiences represents the first step in creating a user group profile for the blind. User group profiles are an essential component of user-centered design that define the unique characteristics of a specific user type. [18] User profiles of under-studied user groups such as the blind aid in design decision-making for universally usable systems. [19] A blind user profile that explicates the unique accessibility and usability needs of this atypical user population is helpful to design more accessible and usable websites. [20] Our findings provide important clues to the accessibility and usability requirements to make SNS functions blind-friendly. This can form the basis for developing a blind user profile for social media. Developers and designers can use such a blind user profile to significantly enhance SNS utility for the blind.

Our future research will further investigate the problems identified in this study to develop a more robust and in-depth understanding of the nature of blind users’ SNS accessibility and usability problems. Specifically, our ongoing and future research will create a more comprehensive understanding of these problems by replicating this study with a larger set of participants with varying skill levels and age ranges. In addition, we will conduct future research using other common SNS functions and other SNS platforms to create a knowledge base of blind users’ accessibility and usability needs and challenges. Findings from these ongoing and future studies will allow greater generalizability of our results.

6. List of Abbreviations

MCB/TC: Michigan Commission for the Blind Training Center.

SC: Success Criterion.

SNS: Social Networking Site.

SR: Screen Reader.

TSBVI: Texas School for the Blind and Visually Impaired.

VPA: Verbal Protocol Analysis.

W3C: World Wide Web Consortium.

WCAG: Web Content Accessibility Guidelines.

References

[1]  Wentz B., and Lazar J., “Are separate interfaces inherently unequal? An evaluation with blind users of the usability of two interfaces for a social networking platform,” in Proceedings of the Conference 2011, ACM, 91-97.
In article      
 
[2]  Qualman, E., “Social network user statistics,” Socialnomics, August 16, 2011. [Online]. Available: https://www.socialnomics.net/2011/08/16/social-network-users-statistics/. [Accessed September 18, 2013].
In article      
 
[3]  Herman, J., “Social networks and government,” How To. gov, April 19, 2013. [Online]. Available: https://www.howto.gov/social-media/social-networks. [Accessed July 21, 2013].
In article      
 
[4]  Jewell, M., “American Express, Twitter team up on retail deal,” Huffington Post, March 6, 2012. [Online]. Available: https://www.huffingtonpost.com/2012/03/07/american-express-twitter_n_1326342.html. [Accessed July 21, 2013].
In article      
 
[5]  Babu, R., Developing an understanding of the accessibility and usability problems blind students face in web-enhanced instruction environments. [Doctoral Dissertation]. Pro Quest Dissertations and Theses. (Accession Order No. AAT 3473492), 2011.
In article      
 
[6]  Babu, R., “Understanding challenges in non-visual interaction with travel sites: An exploratory field study with blind users,” First Monday, 18 (12), December 2013. Available: https://firstmonday.org/ojs/index.php/fm/article/view/4808. [Accessed February 2, 2014].
In article      CrossRef
 
[7]  World Health Organization, “Visual impairment and blindness,” WHO. int, October 2013. [Online]. Available: https://www.who.int/mediacentre/factsheets/fs282/en/. [Accessed December 19, 2013].
In article      
 
[8]  Buzzi, M.C., Buzzi, M., Leporini, B., and Akhter, F., “Is Facebook really ‘open’ to all?” in IEEE International Symposium on Technology and Society (ISTAS), NSW, 327-36.
In article      
 
[9]  Mankoff, J., Fait, H., and Tran, T., “Is your Web page accessible? A comparative study of methods for assessing Web page accessibility for the blind,” in Proceedings from the CHI 2005, ACM, 41-50.
In article      
 
[10]  Isenberg, D.J., "Thinking and managing: A verbal protocol analysis of managerial problem solving," Academy of Management Journal, 29 (4), 775-788, August 1986.
In article      CrossRef
 
[11]  Ericsson, K.A., and Simon, H.A., Protocol analysis: Verbal reports as data, MIT Press, Cambridge, MA, 1984.
In article      
 
[12]  Babu, R., and Singh, R., “Enhancing learning management systems utility for blind students: A task-oriented, user-centered, multimethod evaluation technique,” Journal of Information Technology and Education-Research, 12, 1-32, 2013. [Online]. Available: https://www.jite.org/documents/Vol12/JITEv12ResearchBabu001-032.pdf. [Accessed August 7, 2013].
In article      
 
[13]  Todd P., and Benbasat, I., “Process tracing methods in decision support systems: Exploring the black box,” Management Information Systems Quarterly, 11 (4), 493-512, December 1987.
In article      CrossRef
 
[14]  Cotton, D., and Gresty, K., “Reflecting on the think-aloud method for evaluating E-learning,” British Journal of Educational Technology, 37 (1), 45-54, January 2006.
In article      CrossRef
 
[15]  Babu, R., Singh, R., and Ganesh, J., “Understanding blind users' Web accessibility and usability problems,” AIS Transactions on Human Computer Interaction, 2 (3), 73-94, July 2010.
In article      
 
[16]  World Wide Web Consortium, “Web Content Accessibility Guidelines (WCAG) 2.0,” W3. Org, December 11, 2008. [Online]. Available: https://www.w3.org/TR/WCAG20. [Accessed January 9, 2014].
In article      
 
[17]  Norman, D.A., The design of everyday things, Basic Books, New York, 2002.
In article      
 
[18]  Nielsen, J. and Landauer, T.K., “A mathematical model of the finding of usability problems,” in Proceedings of the INTERACT' 93 and CHI' 93 conference on human factors in computing systems, ACM, 206-213.
In article      
 
[19]  Obrenovic, Z., Abascal, J., and Starcevic, D. "Universal accessibility as a multimodal design issue," Communications of the ACM, 50 (5), 83-88, May 2007.
In article      CrossRef
 
[20]  Henry, S.L., Martinson, M.L. and Barnicle, K. “Beyond video: accessibility profiles, personas, and scenarios up close and personal,” in Proceedings of UPA, 2003.
In article      
 
  • CiteULikeCiteULike
  • MendeleyMendeley
  • StumbleUponStumbleUpon
  • Add to DeliciousDelicious
  • FacebookFacebook
  • TwitterTwitter
  • LinkedInLinkedIn