| SOCR ≫ | DSPA ≫ | Topics ≫ |
Data Science and Predictive Analytics (UMich HS650)
Assessment: 13. Evaluation of Model Performance
SOCR/MIDAS (Ivo Dinov)
June 2017
1 The ABIDE case-study
These data include imaging, clinical, genetics and phenotypic data for over 1,000 pediatric cases - Autism Brain Imaging Data Exchange (ABIDE).
Apply C5.0 to predict on part of data(training data).
Evaluate the models performance, using confusion matrices, accuracy, \(\kappa\), precision and recall, F-measure, etc.
Explain and compare each evaluation.
Use the ROC to examine the tradeoff between detecting true positives and avoiding the false positives and report AUC.
Finally, apply cross validation on C5.0 and report the CV error.
You may apply the same analysis workflow to evaluate the performance of alternative methods(e.g., KNN, SVM, LDA, QDA, Neural Networks, etc.)
Try the procedure on other data in the list of Case-Studies.
