Usability: A Process of Inclusion

With due reverence to E&S TAP (think aloud protocol) I would like to argue for the first part of my discussion that B&S TAP should be adopted as the practitioner’s method for TA in usability testing as it is more rhetorically sensitive to users.

Due to lack of alternative or better method(s) in usability testing, TAP seems to take the center-stage no matter what approach is adopted. E&S approach being the father of TAP naturalizes itself into a standard for usability testing although as Boren and Ramey pointed out the E&S approach tend to leave the system out of the equation. It is true that in usability we are ultimately trying to shape the product in terms of the user and hence any adherents of the E&S would naturally argue the position of user as central and therefore any external parameter (for instance practitioner query) might pollute the centrality. But on the other hand, the more I think about user the more I think about the nature of interaction of the user. For me, the E& S method with its “Keep talking” cue seems to ignore the entire notion of interaction with the system and/or product. Moreover, the E&S cue with its mechanistic persistence precludes the role of the facilitator, who is one of key components of usability (as Barnum would argue). On the other hand the E& S TAP considers the user activity as given and in isolation from the activity which again tend to obscure the notion of satisfaction and utility since the idea of a product or a system is minimized or absent. In the event of breakdowns, is it natural for a user to “keep talking.”

Krahmer and Ummelen admit the drawback of B&S method saying, “that subject’s performance (in terms of task success and lostness) is actually influenced by the way thinking aloud protocol is administered […]” and they hope to minimize the “undesirable side-effect” (116) but nonetheless they evince that the method does have the proactive and reactive measures built into it. On the other hand, the E&S TAP with its subliminal-kind-of-a- reminder seem to throw unnecessary roadblocks in at least two potentially real situations— (a) when the user is confounded and (b) when the conceptual design/metaphor is incomplete. How can E&S TAP account for these types of criticality?

The whole TAP in usability hinges on the assurance that the user is not the topic of test but at best a domain expert feeding insights to practitioners. Given this role playing between the user and practitioner one has to account speech as a manageable series of interpolated collocation consisting of (user) utterances and (practitioner) assurances in terms of back channels and continuers. To this end Boren and Ramey rightly point out that “any time words are spoken knowingly for another’s benefit, the roles of speaker and listener exist: both parties are aware of and are reactive to each other” (267). In E& S this reactive notion is replaced by incantations which blur the context of manageable speech acts.

Further, there is an immense rhetorical significance inherent in any testing situation and more so in usability testing since there are definable exigences, purposes, contexts and constraints. All these aspects obviate the need for agents and agencies where the user act as a defined agent, TAP being the agency, and the facilitator being the co-agent or the neutral agent who negotiates the constraints (system breakdown, incomplete metaphor, etc.) As such it is important that the facilitator as co-agent take part at least in the ways suggested by Boren and Ramey—“ […] practitioner necessarily slipping into the more technical role of troubleshooter (and possibly the reassuring role of apologetic host)” (272). In E&S the role of the facilitator is more passive than neutral and therefore tends to ignore the rhetorical aspect of the process. Is it then possible to address the rhetoric within the E&S TAP framework? or, Does E&S TAP framework completely ignore the rhetorical aspects fundamental to a test situation?

Given these necessary usability entailments and others, such as lack of importance to user satisfaction (E&S consider this as l3 data), I tend to view B&R TAP as better adapted to usability testing and rhetorically more responsive. As a result, the descirtptive-orinetation embedded within usability should necessiate more inclusivity than exclusivity if it has to develop as a discovery tool.

Norgaard and Hornbaek reflect on seven important issues about the process of usability testing.

They are:
Lack of immediate post-session follow ups
Tasks are designed as confirmation of preexisting notions
Contexts influence tests
Questions were hypothetical and less experiential
Usability perceived as an experimental in nature
Measure of utility is infrequent
Asymmetric data analysis

I have few specific questions related to some of these foregoing observations:

-How does immediate post-sessions evaluation help in usability given the constraints of time and productivity?
-How can we apply methods of discovery to task designing?
-If rhetoric is important, how can we control certain variables?
-Is it really possible or practical to control some of the situational factors?

Finally, I think as part of the usability testing we ought to broaden the scope of MEELS by including both utility and rhetoric as part of the discovery process since both product and test/user are affected by U and R respectively.