What is Data Mesh? A Simple Introduction

Data mesh is a concept that helps enterprises guide the use of decentralized information by building a use case–driven data economy. Instead of compiling their data architecture into a centralized warehouse or data lake, organizations that embrace data mesh treat data as a part of individual teams’ “domains” – to be managed on a team-by-team basis.

In essence, data mesh proposes that siloed teams become an advantage by reframing how teams answer business questions and organizing data around this approach. Instead of opening up access to a central data store for all employees to access, individual data teams must curate and mold the data within their domain to share specific datasets with the rest of the organization.

The idea of “data mesh” was introduced in 2019 by consultant Zhamak Dehghani at ThoughtWorks. It intends to break assumptions that centralized data control is best for every organization. According to Dehghani, central management can bottleneck updates to data workflows, blocking data from fitting evolving use cases.

Data mesh encourages enterprises to distribute data management roles to those who know the data best. Enterprises lean into data mesh to embrace a full cultural shift towards democratized user-defined data workflows, supported by technology.

Why is Data Mesh Gaining Attention?

Today’s massive data volumes come with an overload of context that might not be relevant to every business question. For many enterprises, business users are the best decision makers to identify relevant data — but require support from technical operations staff to configure their own data management.

Data mesh proposes that business teams directly build their metadata-driven workflows while technical staff route, govern, and secure the business user “data sandbox” (i.e. the data infrastructure).

Data mesh sets itself apart as a framework by rethinking how business units interact with data and how the technical teams support them. As a result, enterprises develop a decentralized, dynamic environment for their data. By focusing on specific needs and leveraging specific data, both technical and business teams serve the organization with their best skills.

What Data Mesh Does to Solve Bottlenecks

Data mesh is driven by four principles that feed a communal, iterative approach to democratizing data context.

• Domain ownership to manage pockets of data around shared business questions.
• Data “products” enable business users to curate and share data to answer cross-domain questions.
• Federal computational governance unites all business groups under automated guardrails for data safety, compliance, communal sharing, and autonomy.

In practice, data mesh tasks each business unit with packaging their data within the relevant context of their work. Infrastructure admins build and operate the systems that enable non-technical domain users.

Can Data Mesh Work?

Data mesh leans on enterprises to combine a blend of social and technology components to tackle data-driven work. However, this is an intentional strategy that requires everyone to rethink how they approach data management.

Even with familiar components, you can’t simply force old mindsets into this new model. Consider whether your organization is prepared for the constant iterative change that balanced governance requires.

If you’ve already offloaded your data into functional silos, data mesh might be a viable option. But do consider that defining data products at the user level should accompany an emphasis on expanded data literacy. User-friendliness cannot completely prevent data misuse and misanalysis, making proper data literacy essential.

Rather than use data mesh as a standalone solution, you might meld it with other concepts like data fabric and real-time cloud connectivity.

Hybrid data architecture brings multiple concepts together to complement each other. When you consider comparing data mesh against data fabric and other services, view from the lens of combining rather than choosing. 

Consider a Real-Time Data Connectivity Solution

To successfully implement data mesh, your data integration strategy must be able to support its distributed model. Traditional ETL processes are not designed with to support data movement into multiple data repositories.

Consider data virtualization as an alternative. Data virtualization does not require organizations to compile and store data into a single storage solution, but rather provides direct, real-time access to data from multiple sources. Real-time connectivity gives business users the ability to own their data in a self-service model, while IT administrators gain the ability to maintain user permissions and data security controls to ensure complete governance and compliance.

It is not a small undertaking to move into a data mesh model, but if you are considering data mesh for your organization, modern data connectivity tools can help bring it to fruition.

To explore how CData can help your organization become more data-driven, start your free trial of CData Connect Cloud today.