HCI - Lecture 6: Evaluation Techniques - Quantitative Methods Part 1 --- What are Quantitative Methods? 3 types: Interval/Scale; Ordinal; Categorical/Nominal --- Questionnaires = Set of fixed questions; no text/free text questions Pros = quick data collections and typically low-cost Cons = restrictive questions/answer options and only predictable results Questionnaires in HCI -the design process: · Requirements in gathering participants · Testing a theory, prototype, and final design Designing a questionnaire: " Research requirements · Style of questions o Open ended, general, multiple choice, ranking/drop-down o Scalar - Likert and Visual Analog Scalar Questions: · Likert o Designed by Rensis Likert to measure attitudes in a scientific way o Typically with 5 or 7 options o Most common form is 'strongly agree -> strongly disagree' Format can be used for anything · Visual Analog Scales (VAS): o Used in Psychology, less so in HCI o Lewis & Erdinc (tti017): " No difference between VAS & 7/11-point Likert " No difference in reliability and validity Likert Scales - other options: 3-point o Clear meaning of scale (yes-neutral-no) o No varying levels; forcing extreme choice . 10-point o Too many numbers; difficult to interpret o Subjective issues · 4/6-point o No neutral, so forced choice o Not used as often " Pros - simple to create, easy for participants to use/understand " Cons - limited answers, unsure on participants' interpretation, Central Tendency bias, validity? Multiple Choice Questions: " Explicit response choice; can be tti or more answers · Useful in gathering demographics Ranking Questions · ftood for gathering opinions on a large range of topics · Can be difficult to analyse due to relativism Drop Down Questions: · ftood for demographics
· Allows for wide range of responses Part 2 --- Physiological Measures Eye Tracking: . Head/Desk mount for the stability of tracking · Eye movements can indicate cognitive processing and areas of interest · Measures: o Saccades (quick eye movements) o Fixations (stable eye positions Pros: o Direct measurements o Ca be highly reliable and valid o Objective measurements Cons: o Can be expensive o Difficult to interpret meanings from the data Heart Rate, fflood Pressure, etc .: " Emotional response linked to physical changes Can measure: o Heart activity - ffPM, blood pressure o Activity of sweat glands - ftalvanic Skin Response o Electrical brain activity - ElectroEncephaloftram (EEft) · Pros: o Relatively objective o Range in price Cons: o Unclear what emotion is being measured o Unclear what triggered the response o Sensitive to external influences
--- Standardised Measurable Tests In HCI context, these are used to assess performance on a HCI task Measures: . Accuracy · Skill/performance · Speed · Recording computer use Accuracy - Smith & Chaparro (tti015): Compared older and younger accuracy on a range of text inputs o 50 participants - tti5 older; tti5 younger o 5 text input methods - physical QWERTY, onscreen QWERTY, tracing, handwriting, voice Findings: o ffoth groups made less errors in voice and physical QWERTY o Older adults made more