How to achieve Data Mesh teams and culture?

This is the third part of our data mesh blog post series and will focus on the toughest part of the whole data mesh paradigm: the people and culture. We’ll consider the following:

• How to build a data mesh domain team
• How to avoid organisational silos and achieve mutual benefits
• How to get started with data mesh implementation

If you haven’t already read the earlier posts covering the basics of the data mesh paradigm and its technological aspects, you might want to first read:

When should organisations consider data mesh

The role of technology in data mesh architecture

How to build a data mesh domain team

According to Zhamak Dehghani, the creator of the data mesh paradigm, businesses handle their complexity by breaking the organisation into smaller domains that are independently managed and have their own clear goal. This removes dependencies within the organisation and allows more agile and lean ways of working. In the same way as business is restructured into domains, Dehghani argues that technology and data should be too. This leads to the creation of domain teams that are responsible for owning, developing and serving data products related to their domain.

Domain teams should be small enough to maintain agility and shared targets. The optimal size of a Scrum team is often considered to be between three and nine people, and similar team sizes should be seen in data mesh domain teams as well. These teams should be cross functional, including data engineers, scientists, designers and developers that can fulfil the domain’s needs. Having independent teams doesn’t mean that they can’t share talent, but it should be clearly defined where individual responsibilities lie. The goal of team independence is to ensure an individual domain team doesn’t need to wait for other teams’ input to progress their goals and deliver new iterations of their products.

Having small teams focusing on narrow targets and visions enables horizontal scaling as teams can advance different aspects of the same product simultaneously. More cross-functional operators such as a marketing team or COO can feed ideas to different domain teams to advance interests that cover multiple domains. These actors working across domain borders have a critical role in ensuring the end product advances in a desirable direction.

How to avoid organisational silos and achieve mutual benefit

Even though the domains are independent of each other, communication should happen across the mesh network. This means that product teams should have open and shared demos, retrospectives and knowledge-sharing ceremonies. This ensures that domain teams don’t become too fixed in their own opinions when it comes to things like technological decisions and architectural design patterns. Open discussion ensures that the technology validation enabled by the data mesh paradigm has a positive outcome for the company as the results of hypothesis testing are communicated between experts.

By shifting from a monolithic, centralised approach to the data mesh paradigm, cultural change and business process renewal is inevitable. Substance and data professionals will need to change not only their existing ways of working but also often their whole outlook on how data is viewed. One of the key differences is to start thinking of data as the main product rather than a mere by-product or consequence. At the same time, teams and individuals utilising the data should be treated as customers.

In a monolithic, centralised setup accountability remains with one centralised function and responsibilities are often given to individual people. This results in hyper-specialised roles like data protection officer or data governance manager. In the world of data mesh, organisations must remain comfortable with trust and responsibility being shared across multiple domains. The ideology must shift from trusting a single, centralised function to trusting multiple smaller teams.

In the big picture, organisations need to find the optimal state between domain autonomy and organisational-level governance. This state will differ greatly across industries and even between individual companies in the same sector. At first, autonomous domains may find local decision making difficult and require more organisational-level governance, but in the long run these capabilities will develop and more decisions can be made on a team level.

How to get started with data mesh implementation

Data mesh is not an approach to implement thoughtlessly based on top management decisions or without a clearly defined need or limitation in the current way of operating. The need should rise internally, starting on a small scale and testing hypotheses in small iterations. One team should start experimenting due to acknowledged need and internal motivation towards the data mesh ideology. This team should develop a strictly defined data product, own its data and serve it for others to use.

The rest of the organisation can continue with their existing approach until the results of the experiment show positive outcomes that justify expanding the experiment further. At no point should the organisation leap to holistically replacing all the existing pieces at once, but instead continue to gradually add more teams to the experiment. Implementing data mesh architecture all at once based solely on a management decision to replace a company’s current way of organising would be doomed to failure.

In the early stages of the data mesh journey, the self-service infrastructure platform and other tools related to the architecture shouldn’t be the main concern. These will be developed and iterated throughout the journey, and placing too much emphasis on them early on will make the financial return of the experiment tougher to justify.

As with any other type of reorganisation and development work, the first initiatives should focus on quick wins that have significant business outcomes. Therefore, selecting the first domain teams to experiment with the paradigm is critical to its overall success.

Conclusion

As discussed in our technology-focused data mesh blog post, the data mesh paradigm is not a technological challenge. However, it’s a major cultural step that requires organisations to adapt to completely new ways of working and organising, as well as needing them to provide trust and freedom to their teams. The desire for change needs to originate from the business teams and the journey must consist of experimenting, learning and iterating organically.

There is no one-size-fits all blueprint for adapting data mesh. With the lack of real-life implementation examples, the adaptation builds strongly on small-scale experimentation culture and on figuring out effective ways of working within the existing landscape. To start with these new processes and forms of trust will face change resistance. Therefore, demonstrating positive results early on and getting commitment from top management is crucial.

Read more about the Futurice approach to data and AI, and our competences around data platforms. If you’re interested in the work we’ve done with our clients, take a look at our case studies.