Our recently released benchmark research on information optimization shows that 97 percent of organizations find it important or very important to make information available to the business and customers, yet only 25 percent are satisfied with the technology they use to provide that access. This wide gap between importance and satisfaction reflects the complexity of preparing and presenting information in a world where users need to access many forms of data that exist across distributed systems.
Information optimization is a new focus in the enterprise software market. It builds on existing investments in business applications, business intelligence and information management and also benefits from recent advances in business analytics and big data, lifting information to higher levels of use and greater value in organizations. Information optimization also builds on information management and information applications, areas Ventana Research has previously researched. For more on the background and definition of information optimization, please see my colleague Mark Smith’s foundational analysis.
The drive to improve information availability derives from a need for greater operational efficiency, according to two-thirds (67%) of organizations. The imperative is so strong that 43 percent of all organizations currently are making changes to how they design and deploy information, while another 37 percent plan to make changes in the next 12 months. The pressure for such change is being directed toward the IT group, which is involved with the task of optimizing information in more than four-fifths of organizations with or without line of business support. IT, however, is in an untenable position, as demands are far outstripping its available resources and technology to deal with the problem, which leads to dissatisfaction with the IT department in two out of five organizations, according to our research. Internally, many organizations try to optimize information using manual spreadsheet processes and are confident in their ability to get by 73% of the time. But when the focus turns to the ability to make information available to partners or customers, an increasingly important capability in today’s information-driven economy, the confidence rate drops dramatically to 62% and 55% respectively.
A large part of the information optimization challenge is users’ different requirements. For instance, the top needs of analysts are extracting information, designing and integrating metrics, and developing access policies. In contrast, the top needs of business users are drilling into information (37%), search capabilities (36%) and collaboration (27%). IT must also consider multiple points of integration such as security frameworks and information modeling, as well as integration with operational and content management systems. This is complicated further by multiple new standards coming into play as customer and financial data – still the most important information systems in the organization – append less structured sources of data that add context and value. SQL is still the dominant standard when it comes to information platforms, but less structured approaches such as XML and JSON are emerging fast. Furthermore, innovations in the collaborative and mobile workforce are driving standards such as HTML5 and must be considered carefully when optimizing information. Platform considerations are also affected by the increasing use of analytic databases, in-memory approaches and Hadoop. Traditional approaches like an RDBMS on standard hardware and flat files are still the most common, but the most growth is with in-memory systems and Hadoop. This is interesting because these technologies allow for multiple new approaches to analysis such as visual discovery and machine learning on large data sets. Adding to the impetus for change is that organizations using an RDBMS on standard hardware and flat files are less satisfied than those using the more innovative approaches to big data.
Information optimization also encounters challenges associated with data preparation and data presentation. In our research, 47 percent of organizations said that they spend the largest portion of their time in data preparation, but less than half said they are satisfied with their process of creating information. Contributing to this dissatisfaction are lack of resources, lack of flexibility and speed of integration. Lack of resources and speed of integration tend to move together. That is, when more financial and human resources are dedicated to the integration efforts, satisfaction is higher. Adding more human and financial resources does not necessarily increase flexibility. That is a function of both tools and processes, and we see it as a result of divergent data preparation workflows occurring in organizations. One is a more structured approach that follows more traditional ETL paths that can lead to timely integration of data once everything is defined and the system is in place, but is less flexible. Another data preparation approach is to merge internal and external information on the fly in a sandbox environment or in response to sudden market challenges. These different information flows ultimately have to support specific forms of information presentation for users, whether that be the creation of an analytic data set for a complex statistical procedure by a data scientist within the organization or a single number with qualitative context for an executive on a mobile device.
Thus it is clear that information optimization is a critical focus for organizations; it’s also an important area of study for Ventana Research in 2014. Our latest benchmark research shows that the challenges are complex and involve the entire organization. As new technologies come to market and information processes must be aligned with the needs of the lines of business and the functional roles within organizations, companies that are able to simplify access to information and analytics through the information optimization approaches discussed above will provide an edge on competitors.
Regards,
Tony Cosentino
VP & Research Director