STA30004: Data Mining Tutorial Week 7 a) Installing R, then RStudio and finally rattle R and RStudio have already been installed in the computer lab but you may need to install them at home. We recommend the Melbourne(1) CRAN. (See https://www.r-project.org/ and https://www.rstudio.com/products/rstudio/download/). Now instructions for loading rattle. This has to be done in the computer lab and at home after entering RStudio. It does not work for me if I do not have RStudio open. Choose the appropriate platform. In the lab we are using Windows. Use the Melbourne(1) CRAN mirror when prompted. You may be prompted to load additional packages. The previous problem with Rattle has been fixed so the following should now work on any Windows platform. Windows > install. packages ("rattle") b) In RStudio command line enter the following one line at a time. > library(rattle) rattle() c) Click on Execute in the GUI and ask for weather.csv i) Check that Rattle has correctly guessed the data type for each variable. ii) Use the View button to see the worksheet. iii) Be careful with the Edit button. Document any changes. d) Assign appropriate roles i) RainTomorrow is the Target Variable. ii) Risk_MM=amount of rain tomorrow identified as a RISK variable by name iii) Date has an Identity role. iv) Location must be ignored because there is no useful information in this variable. Always set equal to "Canberra". v) All other variables should have an Input role vi) Ignore the categorical variable (WindDir and WindGustDir) 1
vii) Use a 70/15/15 partition for the data. 2
e) Run a linear regression model with Rain Tomorrow as target using a logit link function. Is this model overfitted? Why? What is the AIC? f) Refit this model using only the significant predictors (p <. 05). What is the AIC? Smaller is better. g) Calculate and interpret the odds ratios for the predictors in this new model. Variable Coefficient (B) Exp(B ) Interpret odds ratios g) Using these same predictor variables develop a support vector machine (SVM) to identify days with rain tomorrow. What is Training error rate? How many support vectors were used? h) Now use the Evaluation Tab to compare the results for the linear model in (f) and the SVM in g) using the validation data. What are the overall error rates, areas under the ROC curves and the confusion matrices for these two models? Overall Error Rate Area Under ROC Curves Observe no rain tomorrow TN=0 Observe rain tomorrow FN=10 Logistic Regression Predict no rain tomorrow Predict rain tomorrow FP=1 TP=0 Support Vector Machine Predict no rain tomorrow TN=0 FN=10 Predict rain tomorrow FP=1 TP=0 3