Chapter 5 Reliability of Measures 1. True score 2. Measurement error: less variability = more reliable Pearson product moment correlation coefficient o ris on a scale of -1.00 to 1.00 o r = 0 means two variables are unrelated Reliability coefficient o ris on a scale of 0 to 1 Indicators of Reliability: Consistency · Test-Retest Reliability: how consistent is measure overtime? ? Give same treatment twice o Correlation of score at time 1 and time 2, scores should be similar Measure a group of people on day 1 and 1 week later · Internal Consistency Reliability: how well do set of items relate to each other? o Responses gathered at one time ? Cronbach's alpha: av. of all inter-item correlations . Interrater reliability: extent to which raters agree in their observations · Cohen's kappa Validity of Measures Construct validity: degree to which measure measures the construct it is intended to measure Indicators of Reliability: Truth · Face Validity ? Measure reflects construct being measured · Content Validity ? Content of measure captures all necessary aspects and nothing more · Predictive Validity ? Use measure to predict some future behavior · Concurrent Validity Examine relationship between measure and behavior at the same time People who plagiarized vs. people who didn't on measure · Convergent Validity o Extent to which scores on the same question are related to scores on other measures
· Discriminant Validity o Tests whether measurements that are not supposed to be related, are actually not related Reactivity of Measures Reactivity: what the person is like when they are aware of being observed Scale of Measurement . Nominal: Categories with no numeric scales o M/F, Experimental/ Control · Ordinal: Rank ordering o 2- 3- 4- star restaurants, birth order Was the service better than the last time you were here? · Interval: Numeric properties are literal, no true zero/ zero # absence ? Temperature, Intelligence · Ratio: Zero indicates absence of variable measured, equal interval b/t values, true 0 o Reaction time, Age, Frequency of behaviors
Chapter 7 Survey Research Panel Study: (longitudinal study) people are studied at two points in time; · Response set: tendency to respond to all questions from a particular perspective ? Social desirability: answer in most socially acceptable way Constructing Questions to Ask 1. Define research objective a. Attitudes and Beliefs b. Facts and Demographics c. Behaviors 2. Question Wording a. Loaded Questions b. Negative Wording c. Yea Saying/ Nay saying 3. What kind of Data are you seeking? a. Close-ended i. Easier to code ii. Response alternatives are same for everyone 1. Rating scales: "how much judgement" a. Response Alternatives b. Graphic Rating Scale c. Semantic Differential Scale d. Non-verbal Scale b. Open-ended i. Take more time to code ii. When you need to know what people are thinking 4. Administering Surveys a. Questionnaires i. Personal administration ii. Mail Surveys 1. Low Response rate iii. Internet Surveys 1. Inexpensive and Easy 2. May be inconsistent to results collected from tradition methods 3.