Analyst Perspectives

In this second in a blog series on business analytics I focus on the increasingly important area of predictive analytics. Our benchmark research into predictive analytics shows that while the vast majority of companies see this technology as important or very important for the future of their organizations, most are not taking full advantage of it. This finding suggests that there is an opportunity for companies to gain competitive advantage by implementing predictive analytics in the near term.

Earlier this year I spoke at the Predictive Analytics Summitin San Diego as part of a panel entitled “Winning with Data Science: Transforming Complexity into Simplicity”. Listening to my fellow panelists and presenters as well as speaking with vendors at their booths confirmed that the category of predictive analytics is still being defined. In fact, the environment reminded me a bit of the dot-com era in its energy as well as its disorder. But the two are different: The dot-com era was built on a “field of dreams” where companies built massive web properties but the consumers they expected to come never arrived. The value for the consumer was not obvious. The value of predictive analytics is much clearer and much more rooted in the realities of business. This is confirmed by our benchmark research, in which more than two-thirds of companies view the use of predictive analytics as conferring a competitive advantage.

Because the term predictive analytics is sometimes confused with others, let’s take a moment to define it. In its simplest sense, predictive analytics is about using existing data to predict future outcomes. For example, in database marketing it is often associated with scoring a customer record with the probability (or likelihood) of a desired behavior such as purchasing a particular product. Predictive analytics differs from descriptive analytics in that the latter is about describing existing data and examining how an existing dataset behaves. Descriptive analytics is exploratory in nature, and it is the basic approach used with legacy BI systems as well as with the new class of visual discovery tools. In descriptive analytics, the data is what it is; in predictive analytics, we use the existing data and the laws of probability to predict the future.

The market for predictive analytics is best understood by viewing it as divided into three subcategories: business operations and financial predictive analytics, industry-specific predictive analytics and customer behavior and marketing predictive analytics.

Operations and financial predictive analytics includes the use of predictive analytics in areas such as financial planning, workforce management, IT and supply chain operations. Financial forecasting (the domain of my colleague Robert Kugel) has been a part of the predictive analytics world for a long time. Financial predictive models utilize many different factors, including past company performance and leading economic indicators, to predict revenues and to budget more effectively.

More recently, in areas such as supply chain management, predictive analytics is allowing companies to match their stock with customer demand, thereby reducing inventory costs. In such a system, a manufacturer may collaborate with the retailer to look at run rates and predict stock-keeping unit (SKU) levels. Traditionally this was done with a store manager’s guess or by applying uniform assumptions across all inventories. By applying this type of predictive analysis, companies are able to reduce the inventory levels needed by their partners and segment the market to more efficiently align to an increasingly niched retail market environment.

Predictive analytics has applicability across an array of other areas as well. Workforce management systems, for example, use predictive analytics to understand the staying power of an employee based on his or her job history, or to plan capacity and rationalize new hires. IT is using predictive analytics to analyze log data and automate systems, thereby reducing the time it takes to manage the company IT infrastructure.

Industry-specific predictive analytics encompasses niche undertakings like fraud prevention, risk analysis and disease prediction. Police departments, for instance, do a better job of matching resources to threats when they use predictive models to determine when and where a violent crime might occur. Niche applications in sports analytics (think “Moneyball”) are changing how teams recruit players and even play their games. Healthcare companies and practitioners increasingly are predicting the occurrence of diseases and using these predictions to shape clinician behavior, drug production priorities and treatment protocols.

Big data approaches make possible interesting predictive analytics opportunities in specific areas such as Internet security. Predicting and preventing security threats, for example, is complex since it involves multiple variables that are constantly changing and new variables that are constantly being introduced. The ability to analyze the large volumes of network flow, log and new malware data to understand the different patterns and threat vectors now makes it possible to build predictive algorithms that can be used to recognize and score potential harm to the system.

Customer behavior and marketing predictive analytics is the area that likely hits closest to home for the many business managers who have been hearing that big data and predictive analytics are changing the world. In fact, according to our benchmark research, revenue-producing functions are the business areas where predictive analytics are being used most, with 65 percent of organizations using the technology in marketing and 59 percent in sales.

Loyalty and customer analytics are hot topics right now, and analytical CRM frameworks married with the right toolsets are providing sophisticated ways of not only predicting attrition but preventing it from happening. Companies are looking at individual-level behavior and wallet share across both online and offline environments. This individual-level view currently predominates, and it is proving to be a powerful tool when supported by the right data.

One area in particular where this sort of modeling is effectively being used is sales attribution, which is a major component of return on marketing investment (ROMI). The adage often attributed to John Wanamaker that “Half the money I spend on advertising is wasted; the trouble is I don't know which half” has been applied to marketing spend as well, but it may not necessarily be true any longer in the era of big data and predictive analytics.

Implications and Recommendations

The adoption of predictive analytics, particularly in the important areas of marketing and sales, is forcing an uneasy partnership between CIOs and CMOs. This is because data quality and information management issues, traditionally the domain of the CIO, need to be resolved in order to realize the true value of predictive analytics. From the CMO’s perspective, predictive analytics has enormous power to predict things such as the next best customer offer, but if the product or customer data is incorrect, the value of the prediction is severely diminished.

On the flip side, some marketing services categories and approaches face disruption due to the emergence of predictive analytics. Media buying is an obvious one, but also impacted is the lesser known cottage industry around market-mix modeling. This modeling technique uses multivariate regression techniques to predict the impact of various promotional channels (that is, of the market mix) on future sales. As companies are able to do predictive behavioral modeling on an individual basis, they can fine tune how they tie together promotions and sales. This diminishes the need for less precise aggregate approaches such as market-mix models.

Increased reliance on predictive analytics may also result in realignment of business processes and roles. As business decision makers are able to do their own exploratory analysis and predictive “what-if” modeling and take immediate action based on that, sophisticated BI tools used by those executives may begin to replace the traditional analyst. However, with executive level baby boomers extending their stay in corporate America and the first generation of “digital natives” just graduating from school, such a scenario isn’t likely for mainstream businesses anytime soon.

As organizations move forward with their predictive analytics initiatives, I recommend they think broadly about how the models will be integrated into their existing systems, what type of modeling is needed, and how complex the models need to be. And organizations should by all means explore making use of the support offered by the vendors of the applications and tools that are deployed. At the moment IBM’s SPSS and SAS are leaders in the predictive analytics space with a broad range of tools and models addressing a wide range of use cases. MicroStrategy takes a different approach with its 9.3 release, allowing R to be programmed inside the software so that the functions run in an embedded manner. Many providers including those mentioned above support Predictive Modeling Markup Language (PMML), an XML-based standard that allows predictive models to be shared across applications. For big data initiatives, there are some interesting offerings: Datameer has partnered with Zementis, for example, to develop a universal PMML plug-in that allows SPSS, SAS, and R to be integrated with their Hadoop-based engine.

Approaches such as these help overcome the challenge of architectural integration, which was one of the key obstacles to the deployment and use of predictive analytics identified in our benchmark research. The integration of models is difficult because the statistician who builds the model rarely has the skill set to code the model, and so there is often much misalignment between the intent of the model designer and what the model actually does once it is implemented. With more and more vendors embedding analytic support in their portfolios and further adoption of the PMML standard, this should become less of an issue.

Companies should also pay close attention to the human factor when rolling out a predictive analytics initiative. Our Predictive Analytics Maturity Index shows that of the four dimensions (People, Process, Information, Technology) in terms of which we evaluate maturity, the People dimension is the least mature when it comes to predictive analytics. This issue, which largely is about available skills sets, potentially can be addressed through hiring of recent graduates, as many schools are teaching the R language and graduates are coming out with an appreciation of its power. Open source R is an increasingly popular language that is in many ways the “Lingua Franca” for predictive analytics. Going one step farther, IBM is working with schools such as Northwestern University to put SPSS and other of its advanced analytic tools in the hands of educators and students. SAS, meanwhile, has a very strong and loyal user base already resident in many of today’s corporations.

Predictive analytics initiatives should involve only organizational data sources in which managers have complete confidence. In the longer term, the right way for companies to do this is first to address the adequacy of their information management before embarking on wide-ranging predictive analytics initiatives. Our recent benchmark research into Information Management shows that organizations continue to face information management and data quality challenges. These result from the heterogeneous environment of disparate systems in organizations, the lack of a common metadata layer, and most of all a lack of attention and budget resources available to tackle the issue. My colleague Mark Smith offers a more in-depth look at the data quality and information management issues in his recent blog post. The net/net is that predictive models are no different than any model; if garbage goes in the front end, it’s garbage that comes out on the other side.

As organizations address these issues and respond to competitive market pressures, operational intelligence and predictive analytics inevitably will gain center-stage. Though businesses are still early in the maturity cycle with respect to predictive analytics, we at Ventana Research see companies capitalizing on the market advantage that predictive analytics provides. In some industries – financials, insurance, telecommunications, and retail, for example – things are moving quickly. Companies in these industries that are not currently taking advantage of predictive analytics or are not actively evaluating their options may be putting their businesses at risk.

What’s your thinking on the deployment and use of predictive analytics in your organization? Let me know!

Regards,

Tony Cosentino

VP and Research Director