tag:blogger.com,1999:blog-5652924.post6419682830366547419..comments2024-03-28T23:07:52.202-07:00Comments on Applied Data Science and <br>Machine Learning: Do Predictive Modelers Need to Know Math?Dean Abbotthttp://www.blogger.com/profile/16818000233889520746noreply@blogger.comBlogger11125tag:blogger.com,1999:blog-5652924.post-39181859450809391052014-04-15T17:35:56.754-07:002014-04-15T17:35:56.754-07:00Philip: Thanks for your comments, and I take them ...Philip: Thanks for your comments, and I take them seriously coming from a fellow mathematician (and I wasn't even a "proper" mathematician--I was an Applied Mathematician!<br /><br />However, I still disagree that one needs to know how algorithms work mathematically and have seen it in action. I've had great data miners working for me who don't have a math or stats degree, Dean Abbotthttps://www.blogger.com/profile/16818000233889520746noreply@blogger.comtag:blogger.com,1999:blog-5652924.post-20984995683167996772014-04-15T15:12:08.203-07:002014-04-15T15:12:08.203-07:00You pointed out that understanding the mathematics...You pointed out that understanding the mathematics of an algorithm can help you understand the nuances about the algorithm and results the algorithm gives you. However you imply that this is an extra bonus; someone should know how this works but not necessarily the one using the model. I am definitely biased because I am trained as a mathematician but I would argue that this insight is not just aPhilipnoreply@blogger.comtag:blogger.com,1999:blog-5652924.post-92122877309380330402013-12-18T07:54:13.281-08:002013-12-18T07:54:13.281-08:00This comment has been removed by the author.Anonymoushttps://www.blogger.com/profile/03400286026862727810noreply@blogger.comtag:blogger.com,1999:blog-5652924.post-23704546508494393292013-04-25T08:03:38.473-07:002013-04-25T08:03:38.473-07:00I think that this very issue is being actively deb...I think that this very issue is being actively debated and decided. As a mathematician, when I hear this question my response is “Of course you need math!” Really though, this question goes a step deeper - what we are really asking is “What is a predictive modeller?” It doesn’t help that data mining, machine learning, big data, etc. are buzz words that people are trying to latch onto as the Jaredhttps://www.blogger.com/profile/03028239538126703007noreply@blogger.comtag:blogger.com,1999:blog-5652924.post-7250524783330239362013-04-20T22:11:21.869-07:002013-04-20T22:11:21.869-07:00In response to the previous post, is there a diffe...In response to the previous post, is there a difference between the fundamentals and the mathematical background? Math explains why certain techniques are better in certain situations, or the fundamentals as defined in the comment. One can identify general rules for techniques, such as this algorithm is better on this type of data, but to really understand a model, I think that one has to Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-5652924.post-73563470641269282602013-04-15T22:50:42.445-07:002013-04-15T22:50:42.445-07:00I donot agree completely with you, as in 90% of ca...I donot agree completely with you, as in 90% of cases a Analyst can build models without even understanding any of the mathematical concepts behind (As most Models application are at crude level). But if you need really good model than you need to throughly understand pros and cons of all techniques being weighed, without which you are bound to make mistake. <br />So knowing fundamentals is what Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-5652924.post-50385391679928666672013-04-11T02:42:16.010-07:002013-04-11T02:42:16.010-07:00You're right Will, the concepts, even explaine...You're right Will, the concepts, even explained using English words are still math. What I wanted to say is that equations are not needed if you explain concepts in English. I guess math is needed anyway. <br /><br />Could a concept such as overfitting be explained without Math? You need to understand signal, noise and fitting (i.e. Math) to understand overfitting, right? So you need to know Sandro Saittahttps://www.blogger.com/profile/17682082649770875583noreply@blogger.comtag:blogger.com,1999:blog-5652924.post-73779156735082632032013-04-07T16:40:38.423-07:002013-04-07T16:40:38.423-07:00I only partially agree with the three of you. I th...I only partially agree with the three of you. I think it should be plain that the more math one has at one's disposal, the better a job can be done. I do concede that the math threshold for getting started in this field is probably lower than most people might think.<br /><br />On the other hand, some of the things you describe as "understood using English words" (to use Sandro'Will Dwinnellhttps://www.blogger.com/profile/03379859054257561952noreply@blogger.comtag:blogger.com,1999:blog-5652924.post-76763958092578085482013-04-07T03:00:08.717-07:002013-04-07T03:00:08.717-07:00Nice post Dean! Let me answer in three points.
F...Nice post Dean! Let me answer in three points. <br /><br />First, I agree on the fact that understanding equations is not needed for data mining "users". As long as concepts are understood, for example, using English words, everything is fine. I agree with John: statistics is more important than calculus. If one understand Gini's diversity index concept, there is no need to Sandro Saittahttps://www.blogger.com/profile/17682082649770875583noreply@blogger.comtag:blogger.com,1999:blog-5652924.post-31296292273123244082013-04-03T08:27:38.283-07:002013-04-03T08:27:38.283-07:00John: great to have you visit, and better still, t...John: great to have you visit, and better still, to agree! I was struck years ago by your description of economist Julian Simon and resampling stats (<a href="http://www.juliansimon.com" rel="nofollow">juliansimon.com</a>). This was maybe in the mid-90s while he was still alive. I found <a href="http://www.juliansimon.com/writings/Resampling_Statistics/Part_III/LESSON.txt" rel="nofollow">this Dean Abbotthttps://www.blogger.com/profile/16818000233889520746noreply@blogger.comtag:blogger.com,1999:blog-5652924.post-55730072524283866242013-04-03T07:56:10.933-07:002013-04-03T07:56:10.933-07:00I agree! The level of Math needed is not high. A...I agree! The level of Math needed is not high. And Statistics - far more important and useful than Calculus! - could be much better understood and used if the math parts were de-emphasized. They are only there because geniuses a century ago figured out shortcuts to get to answers (under strict assumed conditions) that they never would have needed to figure out if they had computers and could John Elderhttp://www.datamininglab.comnoreply@blogger.com