Aventine Partners

Limits to Data

Via the Freakonomics Blog I ran upon the departure announcement from Doug Bowman,  the head of the Google visual design team.  What is one of the driving forces behind his leaving Google?  Too much reliance on data to make decisions.

Here at Aventine we naturally highly value data.  But we also often talk about the limits of analytics as a science, which is to say experience and even instinct bring much to the process.  Typically experience makes the difference not so much between right and wrong or more or less accurate, but between more and less useful.

However, the Bowman post prompted some new questions, though he doesn't frame them this way much less answer them.  Is the problem with too much data and analysis:

1. It doesn't always lead to the "right" answer?
2. It is dehumanizing?
3. It is a nuisance?

My instinct (if that's allowed) is that it's a bit of all three, but especially one and two.  There is much that can't be measured and reduced to formulae, and part of our dignity as human beings lies in the fact that we do not mechanistically respond to events or stimuli like inert matter or mere animials. 

This is very relevant in the realm of predicting behavior, as we often do at Aventine, but even more so when dealing aesthtic questions and some the cognitive processes that good information visualization strives to enable.  In a funny way the power of our models keeps us employed, and the magnitude of the error terms keeps us human.

Topgrading Marketing Spend

As I noted in my post To Cut or Not to Cut?, this is a smart time for most companies to reduce marketing spend, notwithstanding desperate pleas from marketing services providers that you double-down in the downturn.

Is there a smart, systematic way to do this?  Probably, but it's elusive, as confirmed by the many stories I hear of the across-the-board, arbitrary 10%, 15%, 25% cuts many marketers are trying to cope with.

A recent white paper from former colleagues at Deloitte Consulting prompted my writing this post, so let's look at their approach, which they call Structural Cost Cutting.  

The premise of the Deloitte's Structural Cost Cutting is that the big opportunities for big near-term reductions in spend come from challenging basic assumptions about the business and how marketing supports it.  The savings will emerge from multiple categories of marketing spend.  The Deloitte hit list of areas to examine are:

More on R and Alternative Statistical Analysis Packages

As a follow up to my prior post on the R statistical analysis tool, there have been a couple of interesting recent blog posts on the same topic, notably:

Comparison of data analysis packages: R, Mathlab, SciPy, Excel, SAS, SPSS, Stat, from from the AI and Social Science Blog, and

How Google and Facebook are Using R from the Data Evolution Blog.

Some quick takeaways:

• Google and Facebook are big users of R, which is the most popular analytics tool at Google.
• They use R for development, but not implementation of models, for which Python or C++ are the tools of choice. 
• The biggest debate seems to rage between users of R and SAS.  Many, many interesting takes on this question if the comments to the first post. 
• Ability to handle large data sets is one of the big differentiators and drivers of preferences.
• Some contend there is an age split, with younger users tending to prefer R and older users tending to prefer SAS.    

In any case, these make interesting reading for those facing these decisions.  Enjoy. 

Learning from the Quants

"Beware of geeks bearing formulas," quipped Warren Buffett about the risks of relying on arcane financial models to make investment decisions.

Not only do we enjoy the classical allusion in Buffett's observation, but this illustrates one of the important philosophies behind Aventine's approach to marketing analytics and modeling.  

In short, a model isn't a black box.  It's a mathematical description of patterns of behavior that should be developed and used in conjunction with qualitative descriptions of cause and effect (or at least relationships between events or conditions).  In other words, I should be able to explain in layman's terms what is going on, and it should make sense.

We're not unique in this regard, but we find more and more modeling done by people who don't understand the business but know how to throw data into a stats package and get an answer.  They then pass it off with great claims of "accuracy" to marketers who now have a new way to target but lack real insight into what is really going on.

(And let's not even start with the opposite absurdity championed by many software vendors who want to "empower" marketers with no statistical training to build predictive models using their handy-dandy do-it-yourself predictive modeling module.  Encouraging customers to make marketing investment decisions based on the result of a modeling "wizard" is malpractice, and should be criminal.)

Coming from a corporate and investment banking background, I've tried to keep tabs on the world of quantitative finance and look for ties to what we do in marketing.  Analytics and modeling in the financial world have in many ways developed far beyond the marketing analytics realm, having benefited from massive amounts of money that still today slosh around in that system.  

Paul Wilmott runs a great site on this topic and has been a critic of modelers run amok in his industry.  In the aftermath of the financial meltdown, he recently posted the Financial Modelers' Manifesto.  This extended appeal to be humble in the face of complex reality and responsibly practice within your limitation is also relevant to us in the marketing analytics world.

Here is an excerpt that could be easily translated to any marketing model situation:

Models are at bottom tools for approximate thinking; they serve to transform your intuition about the future into a price for a security today. It’s easier to think intuitively about future housing prices, default rates and default correlations than it is about CDO prices. CDO models turn your guess about future housing prices, mortgage default rates and a simplistic default correlation into the model’s output: a current CDO price.

Our experience in the financial arena has taught us to be very humble in applying mathematics to markets, and to be extremely wary of ambitious theories, which are in the end trying to model human behavior. We like simplicity, but we like to remember that it is our models that are simple, not the world.

In short, if there is no intuition or insight, no hypothesis about behavior driving your model, beware.

Predictive Fund Redemption Behavior

It's always exciting to be able to apply analytics to a problem in real-time.  So often we are looking in the rear view mirror to analyze the events that took place 12 to 24 months ago and applying the insights to the current situation.

When the financial tsunami struck at the end of last  year, one of our clients became acutely concerned about taking action to manage attrition in its investor base.   Amid the swirling market uncertainty, they had limited resources for this unplanned initiative late in the fiscal year, and asked Aventine to develop a model to identify the most at-risk investors.  

Among the challenges was determining the right metric target behavior (unusual, out-sized fund redemptions) and accounting for the dramatic change in the environment in 2008.  Nonetheless, we developed a model that has proven effective, and will continue to refine the approach in 2009.

We've just added a case study describing this project, and a PDF version is available for download.  Hopefully this might spark thinking about what new phenomena in your business would lend themselves to using predictive models as part of the response.  Certainly, we all have plenty of new challenges to confront these days.