Generative AI Makes Natural Language Analytics a Reality

We’ve been saying for years that natural language processing (NLP) and natural language analytics would greatly expand access to analytics. However, prior to the explosion of generative AI (GenAI), software providers had struggled to bring robust natural language capabilities to market. It required considerable manual effort. Many analytics providers had introduced natural language capabilities, but they didn’t really resonate with enterprise requirements. They required significant effort to set up. They weren’t accurate enough in interpreting the natural language queries. In many cases, they were restricted to or required specific keywords in the queries.  

The advent of GenAI has changed all that. Nearly every analytics provider has GenAI capabilities that incorporate NLP either in preview or generally available. Enterprises are adopting GenAI as well. In fact, 85% of enterprises believe that investment in GenAI technology in the next 24 months is important or critical (from the ISG 2023 Future Workplace Study). And the recently completed ISG Buyer Behavior research on AI shows enterprises are experiencing positive outcomes from their AI investments. Nearly 9 in 10 (88%) report positive outcomes when using AI for search that proactively answers questions. A similar proportion (87%) report positive outcomes in the interpretation of tabular data.

In addition to responding to natural language queries, there are many parts of the analytics process where NLP can be applied, in particular, generating entire reports, dashboards or presentations. For those who work directly with SQL, GenAI can convert natural language requests to generate SQL statements. Natural language can be used to prepare data for analysis. It can document and explain the lineage of data being used in analyses. It can be used to help tag data and analyses for governance purposes. For these reasons, we expect that by 2026, more than one-half of enterprises will use search and natural language generation to increase the reach and consistency of their analytics.  

Despite the fact that GenAI has dramatically improved NLP, it is not a panacea. The issue of hallucinations is well known. Techniques like vector search and retrieval augmented generation have helped improve trust in GenAI. But enterprises must be aware of these issues and have the skills to address them, which unfortunately are in short supply. Two-thirds of enterprises (65%) report they don’t have the AI skills they need to be successful.  

But there is still more work to do, and we expect NLP capabilities will continue to evolve. Software providers are racing to bring GenAI-enabled features to market. Enterprises are increasing their investment in AI, driven largely by GenAI. As you evaluate different sets of capabilities, be sure to consider what is on the roadmap. We expect to see more multilingual capabilities enabled by GenAI. Very few analytics providers had made much progress on multilingual capabilities, but with GenAI they should become much more widely available. We also expect more advanced analytics to be supported with NLP. In the same way that GenAI can be used to generate SQL, it should be able to produce forecasts and predictive models.  

GenAI and NLP are changing the way we work. Enterprises should expect that many systems will evolve to conversational experiences. These changes are likely to result in improved productivity; therefore, enterprises would be wise to embrace NLP early and explore software providers that are as well. To the extent enterprises plan custom NLP implementations, they should also be considering the skills gap issue and determining how they will address it.  

Regards,

David Menninger