Data Science and Big Data Search Trends

These are from google trends.

Data science is growing, but still way behind other traditional terms for our field such as data mining, predictive analytics and machine learning. Big data on the other hand is growing rapidly and outpacing other fields.

This page for now is just an FYI that I may refer to in today's DM Radio show, "Blinded by Data Science"  (http://bit.ly/KfK74l ). I hope to turn it into a more cogent blog post soon (but no promises!)

“data science”

“data science” vs. “data mining”

“data science vs. predictive analytics”

“data science” vs. “machine learning”

“data science city”

“big data”

“big data” city

“big data” vs. “data science”

“big data” vs. “data mining”

“big data” vs. “predictive analytics”

“big data” vs. “machine learning”

“predictive analytics”

“big data” vs. “data science” city

“big data” vs. “data mining” city

“predictive analytics” city

Is Data Mining still on the rise?

Another very interesting and thoughtful take on Predictive Analytics and Data Mining from Mark Madsen can be found here. I've never met him before, but I think I'd like to since he is a TDWI kind of guy, obviously well informed, and I'll be in the same location this May in Boston teaching a data mining course at the next TDWI conference in Boston on the 17th, which is Thursday.

But back to the article...Mr. Madsen writes that Predictive Analytics

rated by the Executive Summit attendees as the number one item expected to have the most impact over the next several years.

Well, that's good news, and I think it makes sense because most companies I deal with are just starting to use predictive analytics. There will always be the powerhouse, large companies that have large data mining teams. They make for great case studies. But we'll know that data mining has "made it" when small companies can have one person working part time doing their analytics, and being effective with it. I know several companies like this already, but it takes some investment in training to get there.

Future Data Mining Trends

In his latest post, Sandro has a nice summary about future data mining trends here. I'm with him that being a prognosticator is not something I do a lot of, but I do have one idea that I still think will happen.

First, let me say that of the references provided by Sandro, the Tom Dietterich one is something I like very much, especially his treatment of model ensembles.

At the 1999 or 2000 KDD conference in San Diego, I think there was a roundtable discussion on the future of data mining with the particular emphasis revolving around whether or not data mining will occur inside the database or external to the database. The general consensus was that mining will move more inside the database, and I frankly agreed. This has not materialized nearly to the degree I expected, though it has progressed especially in the past couple of years with improvements to Oracle Data Miner and SQL Server 2005 Business Intelligence. (I'm not familiar with the current state of DB2 Data Warehouse Edition, and I don't think there has been much work done in recent years on the Teradata Warehouse Miner product, formerly TeraMiner).

However, most folks I know who do data mining still pull data from a datamart or warehouse, build models in a standalone app, and then push models and/or scores back up to the warehouse. I think this is going to move more and more into the warehouse either through improved software in the warehouse (like what we're seeing with Oracle and Microsoft), or, perhaps more likely, through improved interfaces to warehouse functions by standalone data mining software. For example Clementine from SPSS allows you pushback database function to the database itself rather than operating on data that has been pulled from the warehouse. This speeds up basic data processing considerably I've found. I think the latter is the more likely area of growth in data mining software and how practitioners use data mining software.