A university is applying classification methods in order to identify alumni who may be interested in donating money. The university has a database of 58,205 alumni profiles containing numerous variables. Of these 58,205 alumni, only 576 have donated in the past. The university has oversampled the data and trained a random forest of 100 classification trees. For a cutoff value of 0.5, the following confusion matrix summarizes the performance of the random forest on a validation set:
Actual
Predicted
Donation No Donation
Donation 269 19
No Donation 5,526 23,288
The following table lists some information on individual observations from the validation set:
Observation ID Actual Class Probability of Predicted
Donation Donation Class
A Donation 0.9 Donation
B Donation 0.8 Donation
C Donation 0.3 No Donation
(a) Choose the correct explanation for how the probability of Donation was computed for the three observations.
(i) The probability of Donation for each observation is the ratio of the individual classification trees that classified the observation as "Donation" and those that classified it as "No Donation."
(ii) The probability of Donation for each observation is the ratio of the individual classification trees that classified the observation as "No Donation" and those that classified it as "Donation."
(iii) The probability of Donation for each observation is the proportion of the 100 individual classification trees that classified the observation as "Donation."
(iv) The probability of Donation for each observation is the proportion of the 100 individual classification trees that classified the observation as "No Donation."
- Select your answer - â–¾
Why were Observations A and B classified as Donation and Observation C was classified as No Donation?
If required, round your answers to one decimal place.
The probability of Donation for Observation A is
It is - Select your answer - than 0.5, so Observation A is classified as Donation by the random forest.
The probability of Donation for Observation B is
It is - Select your answer - than 0.5, so Observation B is classified as Donation by the random forest.
The probability of Donation for Observation C is
It is - Select your answer - than 0.5, so Observation C is classified as No Donation by the random forest.
(b) Compute the values of accuracy, sensitivity, specificity, and precision. Explain why accuracy is a misleading measure to consider in this case. Evaluate the performance of the random forest, particularly commenting on the precision measure.
If required, round your answer to three decimal places.
Accuracy =
If required, round your answers to the nearest whole percentage.
Accuracy is not the best measure to use for unbalanced data sets because less than % of the alumni in the data have donated.
If required, round your answers for Sensitivity and Specificity to three decimal places and round your answer for Precision to four decimal places.
Sensitivity =
Specificity =
Precision =
The value of precision seems disturbingly - Select your answer - The precision measure represents the percentage of alumni classified by the random forest as - Select your answer - that are donors. Comparing the value of precision with the proportion of observations corresponding to donations, there - Select your answer - a tremendous improvement in the ability to target alumni who may be more likely to donate.
