Part 2: Types of Studies Laboratory Studies: " Taking users out of the normal work environment and into a controlled environment " Appropriate if controlled context is needed and if you want to manipulate the context · Pros: o Specialist equipment available o Uninterrupted environment . Cons: o Lack of context o Difficult to observe several users cooperating Field Studies: · Taking designer out into the user's environment to observe the system in action · Appropriate when we do not want to take people away from their jobs, and when context is crucial for longitudinal studies · Pros: o Natural environment for observing interactions in real contexts with users doing what they would normally do o Context retained o Longitudinal studies possible . Cons: o Distraction, e.g. noise, movement, interruptions o Observation may influence participants
Part 3: Experimental Factors Participants: " Representative - participants should match the expected user population as closely as possible, e.g .: o Age o Education levels o Experience with computers o Systems related to the one tested o Experience/knowledge of the task domain Variables: · Things to modify and measure to test a hypothesis · Independent Variable (IV) o Characteristics that are manipulated or changed to produce different conditions for comparison o E.g. interface style, number of menu items, etc. · Dependent Variable (DV) o Characteristics that are measured o E.g. time taken, number of errors Hypothesis: " Expectations/predictions of the outcome of the experiment " Framed using the independent and dependent variables, e.g. a variation in the IV will cause a difference in DV · Example - error rate will increase as font size decreases Experimental Design: · Within groups design (repeated measures) = participants perform each different condition o Learning effects possible o Less costly; fewer participants needed o Less likely to suffer from user variation Between groups design = participants/groups perform under only one condition o No transfer of learning o More users required o User variation can bias results Data Analysis: " Choice of statistical technique depends on the type of data and the information required " For comparing two groups (between group design) = Independent samples t-test " For comparing two measures of the same group (within groups design) = Paired samples t- test
Part 4: Evaluation through Expert Evaluations Instead of recruiting users for evaluation, we can recruit experts instead Cognitive Walkthrough: · Evaluated design on how well it supports users in learning task · Usually recruits cognitive psychology experts · Experts walks through design to identify issues using psychology · Forms used to guide analysis · For each task consider: o Impact of interaction on user o Cognitive process required o Potential learning problems Heuristic Evaluation: · Usability criteria (heuristics) identified . Design examined by experts to find violations of heuristics · Example heuristics: o System behaviour is predictable o System behaviour is consistent o Feedback is provided · Heuristic evaluation "debugs" design Review-Based Evaluation: . Uses literature to support/refute design aspects · Need to ensure transfer of