This article is the second of a serie on the Data Mesh architecture. It focuses on discussing what is a Data Mesh, explaining the architecture with a worked example.

What is a Data Mesh?

The signature characteristic of a Data Mesh architecture is being decentralized, split by domains that usually align with the organization boundaries, both technological (source systems) and business (internal customers).

Each domain is managed by a domain team that must be self sufficient and works according to DataOps best practices. The core domain team can be supported by central platform or specialty teams for non-core needs.

The domain team organizes their deliverables as data products, each guided by a product owner, responsible to maintain the product vision and apply a product mindset in the scoping and prioritization of work.

The following image illustrates the high level idea of the data mesh architecture.

The data mesh is born when data products from different domains start collaborating.

The overall data platform becomes the composition of these discrete data products, from multiple domains, available through a self service data infrastructure.

The data products collaborate through explicit, well curated interfaces, enforced by data contracts.

The discovery and collaboration is guaranteed by the adoption of a federated governance model and simplified by a layer of shared platform services, rules and protocols.

In the next article we will look in more detail at data products, for now we can just consider them components that are part of a domain.

Types of data domains

According to the Domain Driven Design definition: “A domain is a sphere of knowledge, influence, or activity. The subject area to which the user applies a program is the domain of the software.” from DDD Reference by Eric Evans, the inventor of the Domain Driven Design (DDD).

The following image shows how three domains with a few data products could be connected.

Domains are classified based on their role in the mesh:

• source aligned domains
  These are the domains that are close to the sources, usually operational systems, and have the role to collect and serve that data. The same data could be exposed in multiple ways, according to the different needs of the other domains.
• consumer oriented domains
  These are the domains that provide the desired information to power consumer facing applications and reports. They are the ones that apply use case specific business rules and are the consumer oriented interface of the data platform.
• aggregation domains
  These are the domains that remain “in-between”, the intermediate layer that builds on source data and delivers reusable information. They role is often to apply complex business rules or algorithms, adapt the grain or codes of multiple sources or to calculate values needed in multiple domains.

The classification is not rigid, is somewhat arbitrary and it is mostly to identify the major role of a domain. Don’t lose sleep over it.

Identifying the data domains

Dividing the data needs in domains is not always so straightforward, but organizations have worked out how to divide responsibilities and can learn to assign also these new responsibilities.

A good starting point are the existing responsibilities, so who owns the data sources will have the responsibility to make them available in source aligned domains and who is in need or responsible of delivering some insights or metrics will probably own the relevant consumer oriented domains.

The social aspect of relations and dependencies inside and across parts of an organization is a key element to consider to organize the domains in a data mesh.

Adoption of a data mesh architecture is advisable only when there is a shared desire to bring and accept more responsibility in the periphery of an organization.

Technical complexity of a Data Mesh

On the technical side, a data mesh based platform resembles in many ways a software built by object oriented components, when deployed on an uniform technological infrastructure, or by the composition of micro-services, when deployed across multiple different technologies.

This brings into a data mesh platform some potential issues that a monolithic platform does not have.

You now have to account for the many ways that interaction between components can go wrong. Examples are components not being available, having stale data or having changed in incompatible ways.

Also on the delivery side some pros and cons must be must weighted. On one side there are the improved understanding and greater simplicity of building smaller, domain scale, data products. On the other side there are the need to provide the technical abilities to the domain teams and the increased complexity of operating a distributed system with many moving parts.

An example of a (fictional) Data Mesh

Now that we have described what a Data Mesh is, let’s make a small example that should be easy to follow by anyone.

The setting is an organization with bot a website and an app with significant traffic.

Given the different skills and technologies involved in their creation, and maybe also the different targets, we assume that each is handled by a different team.

The organization has reporting for both the web and the app, but also wants to calculate the cross device sessions and add them to their customer 360 platform.

The image below illustrates the possible initial setup.
To keep the image readable, I have drawn the domain boundary disjointly, but with the same background color.

Starting from the bottom we see two domains, one for the web traffic and one for the app traffic. They contain the source aligned data product that loads and serves the traffic data and the data product that use such data to calculate and publish the sessionized visits.

The image just shows the connection that we are going to discuss, but most probably in a real setting there will be other data products reading the traffic data, including the web reporting data products that we show on top.

In the center we show a “Cross device sessions” data products, whose goal is to take the app and web sessions and calculate if and how the customers use both services and what is the dynamic of such uses.

On the top we see three customer aligned data products that calculate the metrics and values to serve the end user applications web reports, customer 360 and app reports.

It should be quite clear that these customer aligned data products belong to the same domain of the team that own the end user applications served by the data products.

What about the cross-device sessions data product?

By definition it belongs to a domain owned by who is responsible of calculating the cross-device sessions.
A good candidate would probably be the team owning the domain with the customer behaviour data product, that is the major dependency of our data product.

As a teaser I am showing an alternative where all the session calculations are clustered in a single data product.

When an organization fully embraces the data mesh approach it becomes common that the technical teams running the website and the application are tasked to just collect and made available the traffic data from their domain and the calculation of all sessions is done by the team owning the definition of session.

They are the best possible team to know of upcoming changes in the definition, alert other teams about it in due time, code the changes and allow some transition period when the old and new definition are still available.

In this article we have discussed in more detail the idea of data mesh, looked at an example and shown that no one size fits all.

In the next article we will discuss in more detail the characteristics of data products.

Originally published on 4/02/2025 at https://pragmatic-data.org/

Articles in the Data Mesh Series

• Why adopt a Data Mesh architecture
• The Data Mesh architecture (this article)
• Data Mesh and data products
• Building a Data Mesh with dbt and the PDP
• Domain Driven Design basics