• Home
  • The University of British Columbia
  • Research Methods
  • Manipulation Checks and Measurement Techniques in Research

Manipulation Checks and Measurement Techniques in Research

Monday, May 17, 2021 WEEK 2 Lecture 3 How to tell if manipulation worked? - Manipulation check: A method by which researchers can quantify the effectiveness of a manipulation. - Have 2 conditions (1 will be playing a violent game, 2 will be playing a non-violent game) - Condition 1 seen as more violent? Condition 2 seen as less violent? Types of measures - Self-report measure: Participants' explicit attitudes, judgments, thoughts, or characteristics - Behavioral measure: Observing and coding participants' behaviors - Physiological measure: Biological data (e.g. heart rate, sweat levels on skin ... ) How do we ask questions? - Open-ended questions - Close-ended questions (rating scale) Rating scales - Rating scales often ask participants about "degrees" of judgments o How anxious do you feel right now? Not at all 1-2-3-4 Extremely o Numbersftalternatives 1-5 o Anchorsftlabels (not at allftExtremely) o Likert scale (9-item scale with anchors spread out) vs. Likert-type scale (Strongly Disagree - Strongly agree, more specific) - Consideration for rating scales o How many numbersftalternatives do you provide? · 1,2,3,4 1,2,3,4,5,6,7 o How many anchorsftlabels do you provide? " None, a little, some, more than some ... None, some, a whole lot (1-11) o What do you actually give as anchors? None 1, a little 2, some 3, a lot 4. None 1, sometimes 2, often 3, always 4. - Other type of scales o Semantic differential scale · Opposing poles o Non-verbal scale . Cartoon faces for kids, more realistic faces for adults. o Forced choice " "On Saturday nights, I like to: - Pillage an abandoned farm -party like it's 2009 " On which issues does the mainstream media do the worst job on representing Republicans? - Immigration -Economics -Pro-life values - More considerations o Issues to keep in mind when writing questions: · Double-barreled questions (do you believe that political correctness has created biased news coverage of both illegal AND radical Islamic terrorism? - yes -no -no opinion) " Loadedftleading questions (do you believe that the mainstream media actually cares about working Americans? - yes -no -no opinion) " Negative wording (do you believe that contrary to what the media says, raising taxes does not create jobs? - yes -no - no opinion -other) · Yea-sayingftnaysaying (someone who disagrees (or agrees) with all items) (we should not have strict gun control agree 1 2 3 4 5 disagree) yay-saying- agreeing with the questions regardless of the content. Naysaying - disagreeing with the questions regardless of the content. " Fence-sitting - being neutral with the questions regardless of the content. - Get rid of neutral alternatives o Strongly disagree 1, disagree 2, neither agree nor disagree 3, agree 4, strongly agree 5. Type of Studies Between-subjects design o Recall studies playing a game, checking for behavior. - Post-test-only design o You do not know the participants before manipulation - Pretest-post-test design o Test them first, then give them manipulation, then test them again. This is to see how their responses might change. - Framework: