"Beware of geeks bearing formulas," quipped Warren Buffett about the risks of relying on arcane financial models to make investment decisions.
Not only do we enjoy the classical allusion in Buffett's observation, but this illustrates one of the important philosophies behind Aventine's approach to marketing analytics and modeling.
In short, a model isn't a black box. It's a mathematical description of patterns of behavior that should be developed and used in conjunction with qualitative descriptions of cause and effect (or at least relationships between events or conditions). In other words, I should be able to explain in layman's terms what is going on, and it should make sense.
We're not unique in this regard, but we find more and more modeling done by people who don't understand the business but know how to throw data into a stats package and get an answer. They then pass it off with great claims of "accuracy" to marketers who now have a new way to target but lack real insight into what is really going on.
(And let's not even start with the opposite absurdity championed by many software vendors who want to "empower" marketers with no statistical training to build predictive models using their handy-dandy do-it-yourself predictive modeling module. Encouraging customers to make marketing investment decisions based on the result of a modeling "wizard" is malpractice, and should be criminal.)
Coming from a corporate and investment banking background, I've tried to keep tabs on the world of quantitative finance and look for ties to what we do in marketing. Analytics and modeling in the financial world have in many ways developed far beyond the marketing analytics realm, having benefited from massive amounts of money that still today slosh around in that system.
Paul Wilmott runs a great site on this topic and has been a critic of modelers run amok in his industry. In the aftermath of the financial meltdown, he recently posted the
Financial Modelers' Manifesto. This extended appeal to be humble in the face of complex reality and responsibly practice within your limitation is also relevant to us in the marketing analytics world.
Here is an excerpt that could be easily translated to any marketing model situation:
Models are at bottom tools for approximate thinking; they serve to transform your intuition about the future into a price for a security today. It’s easier to think intuitively about future housing prices, default rates and default correlations than it is about CDO prices. CDO models turn your guess about future housing prices, mortgage default rates and a simplistic default correlation into the model’s output: a current CDO price.
Our experience in the financial arena has taught us to be very humble in applying mathematics to markets, and to be extremely wary of ambitious theories, which are in the end trying to model human behavior. We like simplicity, but we like to remember that it is our models that are simple, not the world.
In short, if there is no intuition or insight, no hypothesis about behavior driving your model, beware.
Comments