Data is the coin of the realm in IT and will continue to grow in importance as the industry moves into the future. Businesses are being swamped by the amount of data they are generating and the need to find the bits of business-critical information that will help them remain competitive and drive services and products to their customers.
That means being able to manage all that data at a time when it is being created and housed in myriad places, including on-premises data centers and multiple public clouds. The various locations where data is kept and the expanding nature of that data – structured and unstructured – make managing it all a challenge.
This has put a focus on the growing number of data management companies that are entering the fray with promises of not only enabling organizations to collect, store, analyze and act on that data, but also to ensure that they stay in compliance with the various governmental regulations regarding management, security and privacy, such as the European Union’s General Data Protection Regulation (GDPR).
The recent $1 billion investment in Databricks, a company founded in 2013 by the creators of the Apache Spark open-source framework, is the latest example of the value the IT industry is putting on data management firms. Initial reports of the Series G funding – which included more than a dozen investors and drove the San Francisco-based company’s post-money valuation to an astonishing $28 billion – surfaced in late January and was more widely reported this month.
The funding round was led by global investment firm Franklin Templeton, an existing Databricks investor, as well as organizations such as Fidelity Management and Research, T. Rowe Price, BlackRock and software-as-a-service (SaaS) and cloud services providers like Microsoft, Amazon Web Services (AWS) and Salesforce Ventures. Also investing was CapitalIG, the growth fund for Google parent company Alphabet. That’s about as close as you can get to an A list of venture investors and tech giants.
Databricks offers a data management platform that leverages artificial intelligence (AI) and offers what the company calls an open cloud-based “lakehouse” architecture that takes advantage of the benefits of data warehouses and data lakes. Its central product is called Delta Lake, which is designed to help businesses scale their data lakes and takes advantage of the Apache Spark framework for distributed environments.
Like other vendors, including Snowflake and C3.ai – both of which filed for IPOs in 2020 – Databricks marries data management with AI to help organizations extend management capabilities into the public cloud. The company’s technology is available as a service and can be found on Microsoft Azure and AWS, which is important given the accelerated adoption of cloud services driven by the COVID-10 pandemic.
Given the amount of data being created – IDC analysts said they expect 59 zettabytes of data to be created in 2020 and have said that could reach 175 zettabytes in 2025 – it shouldn’t come as a surprise that investors are grabbing onto up-and-coming data management companies, according to Charles King, principal analyst with Pund-IT.
“You could call it the latest chapter in the ‘business data is valuable’ story,” King told Internet News. “This time around, the constantly growing volumes and variety of data and the speed at which it’s being created are making effective management more and more difficult. Any vendor that can offer companies workable solutions to contend with those challenges is going to get serious attention.”
Rob Enderle, principal analyst of The Enderle Group, told Internet News that there is a lot of demand for data management that and that “many companies’ initial efforts weren’t well thought out, so they are looking for a solution that better meets initial expectations and are a lot less labor-intensive.”
AI now ‘required’ in enterprise solutions
AI will be a key to data management platforms going forward, particularly given the heightened focus on automation driven by the pandemic, Enderle said, adding that “AI is pretty much a required part of any new large-scale enterprise solution right now.”
King said the technology will have to be part of data management equations in the future given the massive growth of data both in volume and variety, which is “leaving many traditional management approaches in the dust. AI offers a means of effectively analyzing data, enabling decision-making and executing management and compliance practices.”
The IPOs of Snowflake and C3.ai illustrated the demand for such data management vendors. Snowflake went public in early September and sold 28 million shares, raising $3.4 billion. C3.ai’s IPO was in early December and the company’s opening share price of $42 quickly shot up by more than 140 percent, bringing in almost $10 billion on initial trading.
Databricks founder and CEO Ali Ghodsi said in a statement that the $1 billion investment will help drive future innovation. He told VentureBeat that it also in part will finance an M&A strategy aimed at data startups and machine learning technologies. In his statement, Ghodsi said that the company’s “lakehouse paradigm is what’s fueling our growth, and it’s great to see how excited our investors are to be a part of it.”
Pund-IT’s King noted that the goal of the Lakehouse architecture is to “enable companies to better use cloud object stores and achieve value similar to what they would gain from expensive, often proprietary data warehouses. That’s a worthy effort that should gain the interest and support of many business customers.”