Data Virtualization, Reimagined: Reviewing the Past, Looking to the Future

We’re happy to report that CData’s recent digital event, Data Virtualization, Reimagined, was a success, with hundreds of IT and data professionals from around the world joining live to hear from leading experts about data virtualization’s evolution. Miss the live event? No problem – get the details of how data virtualization is changing to meet the needs of today’s data-centered organizations in the on-demand webinar.

Watch the webinar

So, what is data virtualization?

To kick things off, Will Davis, CData CMO, and Amit Sharma, CData co-founder & CEO, presented an informative session on data virtualization, its history and evolution, the challenges with traditional methodologies, and how CData is addressing those challenges for modern data teams.

Data virtualization is an approach to data management that allows applications to retrieve and manipulate data without having to build a data pipeline to integrate it. Data virtualization accomplishes this by creating an abstraction layer that simplifies access to data, allowing applications to directly connect to and work with the data without creating copies. Unlike traditional ETL/ELT (extract, transform, load/extract, load, transform) processes, the data remains where it is. It’s not copied or moved in any way.

Data virtualization solves several data management challenges, including managing data among different sources, applications, formats, and semantics. It also provides various extraction and transformation techniques behind the scenes, so users don’t have to understand how the data gets to them—it just does.

Davis observed that many data-driven businesses feel they need to choose between accessing live data through virtualization or replicating data. But, “one without the other is incomplete and doesn't solve the needs of an organization.”

“A lot of other vendors in the market have a singular view of either data virtualization or ETL/ELT data pipelines. In reality, organizations need a combination of both – they should have access to live data when and where they need it, and they need replicated data for different use cases, like analyzing aggregated historical data.”

– Will Davis, Chief Marketing Officer, CData

Past insights and future direction

Sharma has witnessed the transformation of the data management landscape over the last 20 years. He shared his perspective and insight on the beginnings of data virtualization and what the future holds for organizations that depend on data to inform decision-making.

The history of data virtualization

Data virtualization originated from a simple need: To help enterprise IT teams publish data from a variety of enterprise applications that ran on different types of relational databases (e.g., SQL Server, Oracle, etc.). It was particularly helpful for defining logical views to simplify data understanding, setting permissions for data access, and establishing a clear separation between physical and logical views of the data.

“I think it was and remains to be a very successful approach for some use cases,” says Sharma. “But over time, it has had to evolve.”

Sharma believes that the next stage of evolution in data virtualization is in self-service access to data from a broader array of sources, without the technical rigmarole required by traditional virtualization solutions.

"The data ecosystem has changed. The users of data have also changed. Traditionally, data architects and engineers needed access to this data, and they were serving it to other parts of the organization. Now, business users are very savvy, and they want to use it and interact with it directly, using their tools of choice."

– Amit Sharma, co-founder and CEO, CData

What’s ahead for data virtualization

As we look at what's next for data virtualization, some key questions have emerged. How can organizations bring in data from everywhere, not just the usual places like data lakes and databases? Consider all the data management and access tools organizations use. How can business users get this data quickly and easily without jumping through hoops?

As data generation and consumption increases, complexity magnifies. Accessing, managing, and using data from disparate sources, along with inconsistencies in the way APIs are built among different standards and formats, presents ongoing challenges for organizations trying to simplify processes. Sharma explains that his goal is to create some consistency in the world of data management.

Data engineers are no longer the only ones who need data. People in all departmental functions need access to all their data to make better decisions, find new opportunities, and get their work done more efficiently. “And so, we have what we have done at CData is standardize access around SQL,” Sharma explained. SQL eliminates the complexity of building and maintaining integrations for multiple different APIs, removing the need for IT teams to build custom integrations for every application, and allowing data-savvy business users to build simple connections themselves rather than waiting on IT to fulfill their data requests.

“This is really the secret sauce of CData – the ability to provide access to a wide variety of data sources in a common SQL-based access layer. We make all these APIs look consistent, and the beauty of SQL is that it's not just well understood by the industry and data engineers, but every single tool that was built knows how to use SQL.”

– Amit Sharma, co-founder and CEO, CData

As the evolution of live data and data virtualization continues, adopting better technology is only part of the equation. Organizations need to make a cultural shift toward data democratization—making it just as easy for a salesperson to get the data they need as it is for a data scientist. Sharma believes that this is where the market is headed, and CData is pioneering this major initiative by reimagining data virtualizaton.

Watch the full session, ‘Data Virtualization – A Look Back and Look Forward,’ here.

CData is shaping the future of data virtualization

Data virtualization has undergone significant changes from its beginnings and continues to adapt to the evolving data landscape. The future involves innovations in technology and culture. Expanding access to a broader range of data sources by more people is critical for organizations in a data-centric business environment. Be sure to watch the on-demand webinar to learn more.