Monday, February 05, 2007
Poll: Model Selection
In the spirit of the latest posts on model selection, here is a poll to get feedback on that question. I understand that few practitioners always use the exact same metric to select models. This poll is only asking which one is used most often when you need a single number to select models (and input variables don't matter as much).
Subscribe to:
Post Comments (Atom)
Posts
2 comments:
For classification problems, my usual practice is to assess class separation via AUC (area under the ROC curve) and probability accuracy via informational loss or MSE.
For numeric problems, I lean toward mean absolute error, or, when relative error is more important, mean absolute percent error.
For risk scoring it's nearly always KS (Kolmogorov-Smirnov). For forecasting, MAPE.
Post a Comment