In advance of the 1936 Presidential Election, a magazine...

In advance of the 1936 Presidential Election, a magazine titled Literary Digest released the results of an opinion poll predicting that the republican candidate Alf Landon would win by a large margin. The magazine sent post cards to approximately 10,000,000 prospective voters. These prospective voters were selected from the subscription list of the magazine, from automobile registration lists, from phone lists, and from club membership lists. Approximately 2,300,000 people returned the postcards. a. Think about the state of the United States in 1936. Explain why a sample chosen from magazine subscription lists, automobile registration lists, phone books, and club membership lists was not representative of the population of the United States at that time. b. What effect does the low response rate have on the reliability of the sample? c. Are these problems examples of sampling error or non sampling error? d. During the same year, George Gallup conducted his own poll of 30,000 prospective voters. His researchers used a method they called "quota sampling" to obtain survey answers from specific subsets of the population. Quota sampling is an example of which sampling method described in this module?

In advance of the 1936 Presidential Election, a magazine titled Literary Digest released the results of an opinion poll predicting that the republican candidate Alf Landon would win by a large margin. The magazine sent post cards to approximately 10,000,000 prospective voters. These prospective voters were selected from the subscription list of the magazine, from automobile registration lists, from phone lists, and from club membership lists. Approximately 2,300,000 people returned the postcards.
a. Think about the state of the United States in 1936. Explain why a sample chosen from magazine subscription lists, automobile registration lists, phone books, and club membership lists was not representative of the population of the United States at that time.
b. What effect does the low response rate have on the reliability of the sample?
c. Are these problems examples of sampling error or non sampling error?
d. During the same year, George Gallup conducted his own poll of 30,000 prospective voters. His researchers used a method they called "quota sampling" to obtain survey answers from specific subsets of the population. Quota sampling is an example of which sampling method described in this module?

Key Concepts

Sampling Bias

Sampling bias occurs when the sample collected is not representative of the population intended to be analyzed. In the context of the 1936 poll, using magazine subscriptions, automobile registrations, phone books, and club memberships inherently excludes segments of the US populace that were less likely to have access to these services or privileges, such as poorer or rural populations. This leads to a systematic skew in the results, as the sample does not capture the diversity of opinions present in the overall population.

Non-Response Bias

Non-response bias happens when individuals selected for the sample do not participate, and their opinions or characteristics differ from those who do respond. With a return rate of only about 23% from the mailed postcards, the reliability of the survey is compromised because the opinions of the non-responders, who may have differing political views, are not represented. This low response rate reduces the confidence that the measured outcomes accurately represent the broader population.

Sampling Error vs. Non-Sampling Error

Sampling error refers to the natural variability that arises from observing a sample rather than the full population, while non-sampling errors are biases not related to the act of sampling itself. In this scenario, the issues of non-representative sampling and low response rate are examples of non-sampling errors because they introduce systematic biases into the data collection process, rather than just random errors that occur when a sample does not perfectly mirror the population.

Quota Sampling and Non-Probability Sampling

Quota sampling is a non-probability sampling technique that involves segmenting the population into exclusive subgroups and then selecting a predetermined number of subjects from each subgroup. Gallup’s use of quota sampling in 1936 was intended to ensure that specific subsets of the population were represented in his poll. Despite its attempt to mimic stratification, because not every individual has a known or equal chance of being selected, it is classified under non-probability sampling methods rather than true stratified sampling.

Transcript

00:01 This is a problem that's looking at a survey from 1936, there are not a survey, an opinion poll, looking at who was going to win the presidency.

00:14 So this was right before the 1936 election.

00:17 This is when fdr ended up winning the presidency.

00:21 He was reelected.

00:23 But there was a magazine titled the literary digest that was releasing the results of the opinion poll.

00:28 And they said that the republican candidate was going to win.

00:32 And not fdr.

00:35 So if we look into more detail into the actual opinion poll, they sent postcards to about 10 million people, okay, that were potential voters, right? and these voters were found from various lists of things.

00:49 So they found people that had to subscribe to the magazine itself.

00:54 They looked at some phone lists, some automobile lists, some club lists, and a variety of different places, and they sent out all these postcards to voters.

01:03 As a result, they got about $2 .3 million, not million dollars, million people to return them.

01:10 And the question is just kind of analyzing the sampling right in the process and seeing which problems may exist.

01:19 So the first question is looking at the context, right, of being in the 1930s in the united states.

01:26 So thinking about historical events that were happening during that time, this was kind of right in the middle of the great depression.

01:32 A lot of people were struggling financially, right, in not great situations.

01:38 So if we think about the sample chosen from places like magazine lists for literary magazine, right, an academic, more academic based magazine, right? automobile lists, these are all things that are kind of signs of wealth, right? if you belong to these, you had a lot of money in your family to be a part of these different clubs.

02:02 To have a phone in your house that was working, right, to have your own automobile.

02:07 These were all big deals, especially in the midst of the great depression.

02:11 So the sample, right, of people that they were choosing was not representative of the population.

02:18 And the reason is because the population, the majority, were struggling, right? so choosing just kind of the list of people from all of these more higher class types of groups is not representative of the entire population.

02:34 Okay, so looking at our response for a, the idea is that in the middle of the great depression, right? there were many financial struggles, and the lists used to sample pulled from groups that only be wealthy could be a part of.

03:14 Okay.

03:15 So again, contextually, we're in the middle of the great depression, right? we have a lot of financial struggles in the nation.

03:22 So pulling from things that would be a sign of wealth, right? we're not representative of the entire population.

03:27 We're pulling from a very small group of people.

03:31 How does the low response rate relate to reliability? so we got a little over 2 million responses of this 10 million postcards, right, that went out to all these people.

03:44 Because it's such a low response rate, it affects the reliability, right? so the lower the response rate, we could say it this way, the weaker reliability, right? so because we don't have as many people responding, even though the sample isn't great, right? the lower amount of people we have responding, the weaker it's going to be, right? we can't really rely on this.

04:11 And the data, once we try to analyze the data and come up with some conclusions and results, it's going to be pretty severely skewed.

04:18 And again, not representative because of the few amount of people that responded...

Please give Ace some feedback