Big Data Will Make Elephants Dance

IBM’s Big Data and Analytics Analyst Insights conference started me thinking about the longer-term potential impact of big data and related technologies on business management. I covered some of the near-term uses of big data and analytics in an earlier perspective. There are numerous uses of big data that can provide incremental improvements to existing processes and practices. Some of these will have a significant impact on changing business models, enabling new classes of products and services and improving performance. As well, the technology will have more profound, longer lasting effects. The ability to analyze large quantities of business-related data rapidly has the potential to set in motion fundamental changes in how executives and managers run their business. Properly deployed, it will enable a more forward-looking and agile management style even in very large enterprises. It will allow more flexible forms of business organization. None of these changes will be universal, and the old school will be with us for some time. Technology, however, will give executives and their boards of directors a powerful tool for strategic differentiation to achieve a sustainable competitive advantage.

A skeptic might assert that an era of big data will expand the scope of “paralysis by analysis” simply by adding so much more grist to the mill. In an environment where it takes days or at least hours to get an answer back from some bit of analysis, doing another iteration provides a handy excuse for delay. While managers who hesitate to act could exploit big data to delay decisions, they would do so in spite of what’s feasible. In-memory processing of analytics on large data sets can support a much faster decision-making – even in large organizations – because business feedback loops will shrink. However, when answers come in seconds and when systems can almost immediately provide a range of potential solutions to a business issue with supporting considerations (as IBM Watson has demonstrated), active managers will be able to make more objective, numbers-based decisions faster; there’s less excuse for delay or no need for gut reactions. Predictive analytics, which are built on large data sets, have the ability to spot meaningful divergences from expected results, enabling companies to anticipate changes in markets and their environment. More decisive management styles will evolve because using big data with in-memory analytics better supports a fail-early-and-adapt approach to running a business. While this has always the case for small entrepreneurs, it’s increasingly possible for large organizations. (Watch for the strategic consultants to emphasize this point.)

A second area where change will occur is in the forward-looking aspects of management. Today, budgeting is the main form of collaborative business planning that’s done across the entire enterprise. Budgeting is a financial process where the main purpose is to control spending. Planning, however, is a process of determining the best way to achieve business objectives and, in the process, optimize spending. The two are intertwined, and both are necessary. That said, today most companies spend too much time budgeting and not enough time doing collaborative, integrated business planning. This situation is partly due to habit but also the result of practical limitations in technology and data. As can be seen in the chart, our research shows that having a more integrated planning environment promotes better business coordination. More than one-fourth of companies that have no integration of individual plans experience a lack of coordination compared to just 7 percent of those that integrate these the details of individual plans into an enterprise view. The same research reveals that companies with limited ability to calculate the impact of sudden major changes in their business environment have to base their decisions on what to do next on simplistic calculations or just “wing it.” By making it feasible to build scenarios rapidly using integrated business models, big data and in-memory technologies offer the potential to make more informed decisions about how to adapt to changing business conditions. This approach can shift the focus of reviews to a forward-looking, what-to-do-next mindset. That, in turn, can change the emphasis from creating an annual budget to integrating business planning and accelerating reaction cycles.

The ability to harness and share large data sets, along with an expanded ability to interoperate remotely, will support a third change to business organizations: increased dis-integration of enterprises as more companies embrace a federated “Hollywood model” of business organization. In the 1950s, television and the court-ordered divestiture of theaters in the United States forced the movie studios to abandon their factory model. They eliminated their contract workforces and a large portion of their employees. Motion picture production evolved into a system that today brings together loose confederations of professionals, skilled trades people and highly specialized small or midsize businesses. They operate as independent contractors collaborating on a project-by-project basis. This model is increasingly the norm for startups and small businesses, especially in the technology sector, because it is economical and offers greater flexibility. Even though they are less rigid, federated business relationships tend to persist because quality and personal dynamics are important to making them successful. Big data and analytics can support more flexible organizational structures, even for larger corporations. While this model is unlikely to supplant the standard corporate model, it will define an increasing share of the North American economy (and perhaps others) as it allows companies to grow in a more organic fashion. Many successful small and midsize businesses today could not have existed 20 years ago because their business structure would have been unworkable without today’s information technology. It would have been too difficult for them to assemble their talent (employees or contractors) or implement their business model with the cost and technical limitations of the IT and network technologies available in that era.

So far, much of the focus on the impact of big data and related technologies has been on refining or extending existing use cases or tasks. This is understandable since this is where the immediate impact will fall. Over the coming decade, though, technologies for acquiring, organizing and analyzing ever-larger data sets will alter how executives manage business, continuing a long trend toward a more data-driven management style. Compared to even ten years ago, leaders pay considerably more attention to objective measures and metrics and rely less on gut feelings in making decisions. As companies acquire the ability to harness large quantities of business-related data and analyze it rapidly, good management teams will be able to get better results more consistently. Smaller companies will find they can have more of the capabilities of large organizations while large corporations will be able to become more agile in responding to markets and better coordinated in their responses. Technology can enable even elephants to dance.

Regards,

Robert Kugel – SVP Research