Monday, February 05, 2007

Poll: Model Selection

In the spirit of the latest posts on model selection, here is a poll to get feedback on that question. I understand that few practitioners always use the exact same metric to select models. This poll is only asking which one is used most often when you need a single number to select models (and input variables don't matter as much).

Create polls and vote for free.


Will Dwinnell said...

For classification problems, my usual practice is to assess class separation via AUC (area under the ROC curve) and probability accuracy via informational loss or MSE.

For numeric problems, I lean toward mean absolute error, or, when relative error is more important, mean absolute percent error.

Jeff Zanooda said...

For risk scoring it's nearly always KS (Kolmogorov-Smirnov). For forecasting, MAPE.