EMSA 4 Questionnaires and Sampling 4 Important elements for a good questionnaire 1. Discrimination - Can it differentiate between individuals who differ on a variable of interest? (e.g. high or low levels of aggression) 2. Standardisation - Ensuring questions can measure intended population and ensure consistent administration procedures. Do not discriminate or bias against some groups of people (ethical). - pilot study 3. Reliability - includes internal and external reliability. The consistency of the scale (in the same context will the same participant score identically?) External is stability across time, measures should consistently produce the same results for the same people across time. You check external validity by test-retest reliability. Internal reliability is the consistency of measure, achieving the same results You test internal reliability using split half reliability (divide test items in half, scores on each half should be comparable) and alpha reliability (Cronbach's alpha is an average of every possible half of the items correlated with every possible half of the items). Good reliability has a correlation of 0.75-1. 4. Validity - Does your questionnaire measure what it set out to measure? To check validity, use face validity (do questions look like they are measuring the construct?) or content validity (ask an expert in the field as to whether they think that questionnaire neglects elements of the topic under investigation), criterion validity (applies questionnaire to a group who are known to have the characteristics under investigation Checking validity 1. Face validity - do questions look like they are measuring the construct? 2. Content validity - ask an expert in the field as to whether they think that questionnaire neglects elements of the topic under investigation 3. Criterion validity - applies questionnaire to a group who are known to have the characteristics under investigation - Concurrent validity compares to existing scale and predictive validity can predict an outcome. 4. Construct validity - long term process where one tries to show that proposed hypothetical construct measured in the questionnaire actually exists. It requires a lot of experimental work. Questionnaires: Response format Closed - rate your satisfaction (-3 to +3) Open - describe how you are currently feeling (more detail so less chance for ambiguity but they are difficult to quantify) Questionnaire Construction Issues 1. Complexity - respondents may become confused 2. Technical - avoid specialist terms that the respondents may not know 3. Ambiguity - questions should not have multiple interpretations 4. Double-barrelled answers - respondent may agree with one part but not the other 5. Double-negatives - can be confusing 6. Emotive language - can be leading 7. Leading questions - can encourage a particular response fi. Invasion of privacy - unless fundamental for the study, avoid private questions
EMSA 4 9. Balance of scale items - for a Likert scale you should not have four positive and one negative option 10. Sensitivity of response format - dichotomous options versus a continuous scale Demand characteristics and social desirability Participants may provide the responses that they think you want to