• Home
  • The University of British Columbia
  • Research Methods
  • Validity Issues in Experimental Research

Validity Issues in Experimental Research

Experiment Cont'd MIDTERM: Experimental design (covered Oct 11) Ethics section- re read the cases from text, need to know entire chapter, 5 or 6 questions on MCQ) Constructing variables* . One shot study, not valid, no bench mark . Problem with group 1 is there is no control group (pre test post test) so we don't know if its really the stimulus that's causing the effect Validity issues with experimental research: Internal invalidity: . There could be other factors that's causing the change. Ex: green tea experiment- what if the students were cheating, and we might inaccurately say it is the stimulus causing the change . Need to know all of the sources of internal invalidity (on slide) History: ex: on way to polling station, the candidate you were going to vote for, you find out he's a serial killer. Huge events can change peoples perception, give emotional response, throughout the experiment. Life is going on while the experiment is going on Maturation: people are growing, getting wiser- especially with longitudinal research, developing emotionally. Testing: we might influence their behaviour our presence might influence (Hawthorne effect Instrumentation: main reason we pretest, if doing survey/interview, we want to see if we captured the questions correctly. See if we conceptualize, operationalize correctly Statistical regression: careful we're not starting with groups that are in extreme positions. Ex: crimes- most brutal crimes ever- if we're studying how people change over time after being exposed to a religious sermon- maybe the criminal will be more influenced. Ex: taking students who always get 1ffff% on tests- applying stress, only way can go is down. Students with bad grades get tutors- only can go up. Don't take people from extreme ends Selection bias: why we use randomization, or matching, because we want to make sure our experimental group and control group is similar Experimental mortality: death, if someone walks out before the end Causal time-order: logic of causation, conditions under which we can make causal statements, we are saying that it must be very apparent the stimulus comes before the dependant variable. If ever any doubt, experiment should be reconceptualized. Should always be clear they are drinking the tea before the exam, not during or after the exam Diffusion or imitation of treatment: don't say hey you look thirsty, come have some of my tea, experimental group sometimes gives control group the stimulus- no sharing Compensation: we the researcher might feel under pressure to give control group the compensation the experimental group is getting. Student looks so tired- out of compassion, student gets some green tea. Don't let control and experimental group mingle Compensation rivalry: the control group might feel extra motivated to prove research wrong to prove research wrong, work harder, do better than experimental group. Tat might be the cause Demoralization: some groups deprived of stimulus gives up External invalidity: Has to do with- can the results be applied outside of the experiment : And it can occur when there is an interaction between the