SingleStore Positions Hybrid Data Processing for Data Intensity

I recently described the use cases driving interest in hybrid data processing capabilities that enable analysis of data in an operational data platform without impacting operational application performance or requiring data to be extracted to an external analytic data platform. Hybrid data processing functionality is becoming increasingly attractive to aid the development of intelligent applications infused with personalization and artificial intelligence-driven recommendations. These applications can be used to improve customer service; engagement, detect and prevent fraud; and increase operational efficiency. Several database providers now offer hybrid data processing capabilities to support these application requirements. One of the vendors addressing this opportunity is SingleStore.

SingleStore was founded in 2011 to create a new relational database management system. Initially known as MemSQL, the company originally focused on in-memory high-performance transactional workloads, taking advantage of fast data ingest, high-performance memory and skiplist indexing. Over time, the company added support for tiered storage, utilizing columnar, disk-based storage as well as cloud-based object stores. The combination of row- and columnar-based data processing is what initially enabled the company to support applications requiring hybrid operational and analytic data processing. SingleStore is not alone in that regard. I assert that through 2026, operational data platform providers will continue to invest in hybrid operational and analytic processing capabilities to support growing demand for intelligent operational applications infused with personalization and AI-driven recommendations.

While other vendors continue to use dual engines to support hybrid data processing, SingleStore has differentiated itself, thanks to the development of a single Universal Storage table type that evolved the column store to be capable of performing both transactional and analytic processing. In 2020, the company adopted the name of that approach – SingleStore – as its overall brand. Today, SingleStore offers SingleStore DB for self-managed deployment on-premises or on public cloud infrastructure as well as the SingleStore Managed Service. While SingleStore DB is a scalable SQL database, it has also evolved over time to support multiple data types, including JSON documents, geospatial, time-series and key-value as well as relational data.

We continue to see distinct requirements that drive organizations to adopt operational data platforms to store, manage and process data to support worker-, customer- and partner-facing operational applications, as well as analytic data platforms for business intelligence and data science workloads. However, there is growing demand for intelligent operational applications infused with the results of analytic processes, such as personalization and AI-driven recommendations. These applications rely on real-time analysis of operational data, making them unsuitable for traditional approaches that involve the extraction, transformation and loading of data from operational applications into separate analytic data platforms. Almost one-quarter (22%) of participants to Ventana Research’s Analytics and Data Benchmark Research analyze the data they collect in real time. A growing set of operational application use cases are being improved through the infusion of intelligence.

Given the traditional dominance of separate data platforms for operational and analytic data processing, adoption of databases capable of providing hybrid data processing is still in the early stages. Indeed, so dominant is the traditional architectural approach that many organizations may not understand how or why they might need hybrid data processing. SingleStore explains the need in the context of data intensity, encouraging organizations to evaluate the data requirements of any given application based on five variables: data volume, query latency, query complexity, data ingest speed and concurrency. While the devil is in the details, the general advice is that, if an application exceeds certain thresholds related to two or more of those variables, the organization should at least consider an architectural alternative to traditional separate operational and analytic data platforms. In addition to the hybrid data processing capabilities provided by its Universal Storage, SingleStore’s alternative architecture also includes separation of compute and storage to take advantage of cloud object storage, with the additional distinction that SingleStore ingests data into memory and disk prior to ingestion into object storage to avoid analytics latency.

Having a differentiated offering is generally a benefit, but being different is not always easy, especially when competitors include some of the biggest names in data platforms and data processing. In addition to customer case studies from the likes of Fiserv, Kellogg and Uber, SingleStore has also recently received validation from data and analytics heavyweights IBM and SAS Institute. IBM has made an investment in SingleStore and is offering SingleStore DB with IBM. SAS has integrated SingleStore DB with its SAS Viya massively parallel analytics engine to accelerate data processing and time-to-insight for high-concurrency queries with reduced storage footprint.

Adoption of hybrid data processing is still in its infancy. It relies on organizations understanding the availability of alternatives to traditional approaches that split data processing between operational and analytic data platforms. Awareness of the development of applications to take advantage of hybrid data processing functionality also plays a role. Adoption is expected to grow, however, given the increased interest in intelligent applications infused with personalization and AI-driven recommendations. Any organization considering the development of applications fitting that description are advised to consider the data intensity requirements and the potential role that SingleStore could play in addressing them.

Regards,

Matt Aslett