DataOps.live Enables the Development of Data Products

I wrote recently about the role that data intelligence has in enabling enterprises to facilitate data democratization and the delivery of data as a product. Data intelligence provides a holistic view of how, when, and why data is produced and consumed across an enterprise, and by whom. This information can be used by data teams to ensure business users and data analysts are provided with self-service access to data that is pertinent to their roles and requirements. Delivering data as a product requires more than self-service access to data, however. Also needed are cultural and organizational approaches that encourage domain-oriented data ownership (which makes business departments responsible for managing the data generated by their applications and making it available to others) and product thinking (which prioritizes treating consumers of data as customers). Data operations vendors such as DataOps.live are addressing these requirements with software that is specifically designed to provide an environment for developing and delivering data as a product.

DataOps.live was founded in 2018 with the goal of helping enterprises to reduce time to insight and accelerate time to value from analytics and data initiatives. The company aimed to do this by bringing the principles of DevOps to data management, and it was an early participant in the DataOps movement, which focuses on the application of agile development, DevOps and lean manufacturing by data engineering professionals in support of data production. In 2020, the company shared details of its TrueDataOps philosophical approach, as well as the first iteration of its DataOps.live platform, providing DataOps capabilities for Snowflake’s Data Cloud. DataOps.live remains focused on Snowflake environments today, and Snowflake is also an investor in the company, having contributed to DataOps.live’s $10.3 million seed funding round in January 2022, as well as its $17.5 million Series A funding round in May 2023. Other investors in the company are Notion Capital and Anthos Capital. The DataOps.live platform has been enhanced since it was introduced and now provides a combination of data pipeline orchestration and management, federated governance and data observability, and data platform automation and cost optimization. In combination, these capabilities are positioned as providing an environment for building, testing, and deploying data as a product.

The terms “data as a product” and “data product” are often used interchangeably but are slightly different. Data as a product is the process of applying product thinking to datasets to ensure that they can be discovered and consumed by others, while a data product is the outcome of this process. Traditionally, data initiatives are delivered on a project-by-project basis. While the resulting software, such as a data warehouse or dashboard, is designed to serve the specific requirements of the project, often little or no attempt is made to ensure the data can easily be accessed and used for other purposes without duplication. By applying product thinking to datasets, data as a product ensures that the outcome of an initiative, such as a data warehouse or dashboard, is considered a data product that can be discovered and consumed by users inside or outside the business. I assert that by 2027, more than 6 in 10 enterprises will adopt technologies to facilitate the delivery of data as a product as they adapt their cultural and organizational approaches to data ownership in the context of data mesh. As I previously explained, data mesh is an organizational and cultural approach to data ownership, access and governance that enables enterprises to shift away from centralized approaches to analytics built around monolithic analytic data platforms and toward distributed ownership of data and metadata. Data as a product is one of the four principles of data mesh, alongside domain-oriented ownership, self-service data infrastructure, and federated governance.

DataOps.live Enterprise is a cloud-based software platform designed to help enterprises embrace data as a product and deliver data products. It is described as a unified control plane for Snowflake Data Cloud that enables enterprises to build, test and deploy data products and applications on Snowflake. DataOps.live Enterprise provides data product management functionality to capture change requests and track development and branches of code and data through continuous development and deployment processes. It features a DataOps Development Environment used to define data product specifications, test backward compatibility, automate deployment into production and publish the results to data catalog products from vendors including Data.world and Collibra. Users are also able to monitor and manage attributes and data quality key performance indicators while DataOps.live Enterprise also offers observability of data pipelines, workflows, and processes; late 2023 saw the addition of the generative AI (GenAI)-based DataOps.live Assist intelligent assistant. DataOps.live also offers Spendview, a free service that provides cost optimization capabilities for Snowflake, and in March 2024 announced the launch of DataOps.live Professional Edition, which is designed for small teams of developers and provides a managed and supported version of the open source dbt Core analytics deployment tool that enables them to ingest, transform, and visualize data in Snowflake.

As noted above, DataOps.live has so far focused its attention on users of the Snowflake Data Cloud. While Snowflake is widely adopted, this is naturally limiting the company’s addressable market and leaves it at risk of being disrupted by Snowflake developing or acquiring similar functionality. As such, DataOps.live would be wise to expand its focus to address other data platforms providers sooner rather than later. In the interim, I recommend that enterprises using Snowflake and interested in delivering data as a product evaluate DataOps.live and its capabilities for building, testing, and deploying data products and applications.

Regards,

Matt Aslett