An organizational guide to machine learning business cases: Sanoma

Machine learning (ML) is a core data value creation technology. Sanoma is a large European media and learning company. In this article, we focus on how machine learning has been put to work at Sanoma to transform the core customer experience and deliver business growth.

We are thankful to Timo Rinne, VP Digital, at Sanoma for collaborating on this in-depth article with us. You can read through Timo’s key takeaways on the Sanoma ML journey at the end of this article.

Sanoma advanced analytics strategy

Sanoma is one of the companies leading the way in the use of advanced analytics in the Nordics.

For years Sanoma has demonstrated its strategic commitment, sharp business focus and bold experimentation within advanced analytics. In particular, the company has gained significant experience in using ML to create a loyal customer base through personalized, contextualised and timely content. Sanoma’s machine learning solution portfolio is extensive, with the capability to experiment rapidly with new ideas and further improve existing production models and solutions.

In business terms, Sanoma has been able to deliver an impressive return on investment (ROI) thanks to these efforts. The question “Can machine learning create business value?” has already been proven multiple times. Sanoma has been able to move forward using advanced analytics as a core part of their day-to-day work that is now built into all media operations. The plan is to continue to identify advanced analytics opportunities and deliver solutions.

Sanoma data maturity

Based on a generalized framework of data maturity Sanoma can be identified as an advanced user of machine learning in operations.

Strategy and direction

Data and advanced analytics have been recognised as a source of competitive advantage for the future. This recognition has made it possible to integrate analytics workflows in core business value creation and day-to-day operations.

The business value of data

With multiple successful point-solution production use cases the value of data and advanced analytics have been clearly demonstrated. Now there's a shift towards holistic ML portfolio building which enables optimisation of entire value streams instead of point-solution optimisation.

Data and technology

Sanoma has been able to build a strong technological base for moving forward with advanced analytics use cases. A common data enablement layer provides shared capabilities for data collection and analytics, respecting and complying with data privacy legislation. With more business unit centric data business cases Sanoma has been shifting towards a data mesh approach where business units self-manage data for their use cases.

When should organisations consider data mesh?

Processes and ways of working

As Sanoma’s various business units have quite different value drivers, Sanoma has built data teams close to the units’ value creation core. In all units there are a few different ways of working that suit the business units’ operations overall. There is a clear shift towards more aligned ways of working and the inclusion of business unit-specific multidisciplinary teams in order to cut the time to market of advanced analytics use cases with end-to-end implementation capabilities.

Organisation and competences

In order to prove its value, advanced analytics has historically been developed using the help of centralised teams. This has enabled shared capabilities that could develop core data enablers and prove the value. Currently there is a team of highly skilled advanced analytics professionals from which business units can source dedicated skilled individuals for their business cases. This has enabled a deeper understanding of the business unit-specific value creation logic and better alignment with business goals.

Machine learning value creation in practice

High-quality journalism and customer-centricity are the themes that define the strategy of the Sanoma news business. All of the advanced analytics projects are built to support these two themes. Furthermore, ‘privacy first’ is a core guideline for all solutions.

This is how high-impact ML opportunities are identified in Sanoma’s news business.

The diagram contextualises the opportunity identification, prioritisation and planning process and how it consists of thematic layers which together form the opportunity space of potential ML business cases. Let’s go through each layer in turn.

Vision and depth of adoption

This is where high-level guidelines related to the main strategic business objectives are set. It consists of the top-level strategy, the ambition level within data and advanced analytics, and the current data maturity of the organisation.

Sanoma’s top-level strategy, as well as its data strategy and data maturity, have already been discussed.

Sanoma has adopted a strong privacy-first approach for developing new ML solutions, with data minimisation as one of the core tenets. This demands that all solutions should minimise the amount of data collected and minimise the length of time the data is stored. From a legal point of view, GDPR and the data protection act create hard limits on what can be collected. Another guiding factor is journalistic integrity, which states that for all news sites the final decision is always made by journalists. The explainability of ML solutions is also seen as a mandatory requirement.

Value creation

This layer focuses on operational value creation in which the organisational structure, the business unit and its resources, the value streams and the limitations are taken into consideration.

An example value stream at Sanoma boils down to the essential mission of journalism: how to provide important information to customers in a timely and efficient manner.

This can then be divided into five bigger themes:

Editorial planning: Understanding what is important and interesting to customers.

Article production: The core value-creating part of the journalistic process.

Distribution: After the content has been produced we need to make sure that users find the most interesting content on the right service, at the right time and in the right context.

Paywall: The commercial side of journalistic services. Presenting customers with content that could give them additional value, but they don’t yet have access to.

Subscription: Providing customers with a plan that best suits their needs.

Performance targets

Sanoma uses key performance indicators (KPIs) to direct the alignment and efforts of business units. These can often be high-level metrics, but as operational value creation teams often feel they cannot effectively influence high-level metrics, they often need to be broken down into smaller and more manageable targets.

KPIs can either be mapped to objectives and key results (OKRs), or helpful tools such as value-stream mapping can be used to make things easier to handle. This linking makes sure that the concrete objectives of the business unit directly guide ideation by linking top-level strategic targets to actual development ideas.

Example of a KPI: Increase active Sanoma accounts

Example of an OKR objective: Create a usage habit for a new user

Example of an OKR key result: Increase total time spent across user devices by x%

OKRs are again broken down into specific and more granular drivers that are close to the day-to-day business operations. This allows us to measure the impact and gives a starting point for the impact estimation and ROI calculations as we can estimate how big a change we would require to recoup the estimated investment. At this stage it should be noted that not all drivers can be mapped directly to monetary impact; business consideration should be applied to balancing monetary drivers and, for example, quality drivers.

Example of a driver: Increase number of sessions per user

Concrete opportunity space

This is the layer where the identified drivers are finally mapped to concrete ML (and non-ML) development ideas.

Development ideas are created in, for example, a workshop using the established mapping as a starting point. At this stage, plenty of development ideas should be added even though they might be technically or otherwise limited as in the next layer ideas are pruned to just a small prioritised subset. Existing solutions can also be mapped here in order to carry out a gap analysis and see where previous efforts have been directed.

Example of an ML development idea: send personalised and contextual notifications at the right time to the right channel/device.

In the picture above a partial mapping chain is shown for the content distribution process.

Prioritisation and ROI calculations

The main goal of the prioritisation phase in the mapping process is to cut down the list of possibilities by carrying out light impact estimations, with heavier ROI calculations for a select few. There are multiple frameworks available, for example, some Sanoma teams are applying Impact, Urgency and Effort analyses.

Impact analysis: Every ML development idea contains some elements of uncertainty, but if there is evidence available that something can be improved, or that the results fluctuate, there is a good chance that it could be improved with ML.

Because ML projects have an element of uncertainty, one should take into account the different routes that the project might take; this could be that some portions or sprints might need to be redone in fast iterations in order to get results. The project team should also be ready to kill their ML darlings if all accessible avenues are tested and nothing seems to stick.

Effort analysis: Effort should not only include the developer time required but also all the stakeholders, training, infrastructure and maintenance related to the ML development idea.

Urgency analysis: This is to assess the business criticality of the use cases. This boils down to assessing clear functionality gaps in the current portfolio. For example, an incremental model update could bring some uplift to the current model, but in terms of urgency it would most likely be low as there is already a solution in place which yields results. Examples of high-urgency cases could be launch blocking, a core ML feature or something concerning a mandatory privacy regulation.

https://www.atlassian.com/team-playbook/plays/prioritization-matrix

These two matrices help to map ML development ideas in relation to each other. From the matrices, you can start selecting ML development ideas in the following order for more detailed investment calculations.

Easy wins – High impact, high urgency and low effort

Must do – High urgency

Big bets – High impact, high effort (building longer road maps)

Incremental – Low effort, low impact (day-to-day continuous improvement)

Avoid/Later – High effort, low impact

Readiness to take into use and continuously develop ML solutions

After the calculations are done, the readiness to take the solution into use and the continuous improvement potential is assessed.

Organisational aspects: The most important thing to consider is the organisational readiness to take advantage of the developed solutions. Most ML projects only generate value if they are fully integrated into the process for which they are developed. This often requires training for end-users and a plan for how the solution is rolled out, and an understanding of how these things influence the current process.

This is especially important in business cases where the solution heavily modifies current processes. A good example of this in the Sanoma case would be how the news front page is structured. Frontpage layout has been done in pretty much the same way for hundreds of years: editors manually select content for a certain position and it is shown to all users in that specific position. Then the personalised frontpage was introduced, which was a huge process disruption where every user got a personal frontpage and the editorial selection process of the frontpage was transformed completely.

This was critical to take into consideration in the solution planning phase when the editorial workflow was analysed in-depth and necessary tooling was created to help journalists manage change.

Technical aspects: In general, ML solutions are never fully finished – some will absolutely require, and others can greatly benefit from, further development and design iterations. All ML solutions have some maintenance needs attached to them.

This means when assessing the effort and investment – but also the impact – one should calculate the workload associated with a few additional sprints, as there is a good possibility that the information collected from the solution’s performance leads to new improvement ideas that have the potential to multiply the total effectiveness of the solution.

Lastly, maintenance should not be overlooked as often it is time taken out of the team's future development potential, so one should try to understand the maintenance needs as accurately as possible.

In the Sanoma case where there were only small, dedicated teams, the trade-off between maintenance work, new development and continuous improvement was extensively weighted as more maintenance work would cut significant chunks out of the time that could be dedicated to continuous improvement. Especially for solutions on the verge of being taken into use, emphasis needs to be placed on the continuous improvement needs coming from journalists and business.

In hindsight, keeping the focus on continuous improvement has paid off at Sanoma. For example, the continuous improvement work related to news personalisation has increased the outcomes dramatically.

To decrease the maintenance and continuous development workloads, Sanoma solutions lean heavily on managed services from cloud providers.

Success metrics and improvements

At Sanoma, in order to justify significant investments in data, analytics and AI, all ML development projects are directly linked to top-level strategy and business KPIs, with results translated into impact on the KPIs. This end-to-end linking has worked as a great communication tool for business stakeholders and other teams. It is much easier to justify development when you can show the direct strategic effect it has produced. This is especially important with continuous incremental improvement.

Continuous improvement was the key element that made, for example, news personalisation work. As Sanoma is one of the earliest adopters of personalised frontpage functionality in a digital news service globally, there was no clear blueprint of how it should be done. The lack of blueprint was very clear from the significant impact it had on the journalistic process and from the modest results it achieved in the early piloting phase. Because of the uniqueness of the solution, a heavy, iterative AB-testing approach was taken into use where fast simplistic solutions were continuously deployed to assess their overall potential. This allowed Sanoma to improve the results significantly.

As Sanoma is quite far along in its data maturity journey, it has become clear that ML solutions need careful monitoring. Dashboards have been developed with the purpose of understanding how personalisation solutions work and how they impact content distribution. Operational dashboards were mostly monitored by the team with notifications and recommended actions to relevant stakeholders if anything out of the ordinary was detected. On top of this, weekly stakeholder meetings were held in which the weekly performance and AB tests were discussed and action plans for any deviations were created and followed. This helped all key stakeholders to understand the situation, which in turn helped the full-scale production use.

The benefits of how Sanoma identified ML business cases

How has this kind of approach helped Sanoma?

A clear focus: Deriving ML development projects directly from top-level strategy and business KPIs ensures clear focus and that scarce resources are allocated where they will make the most meaningful impact for customers and business.

Organisational alignment: It gives all teams clear points on which they can attach their work and see which items are interlinked between multiple teams and their objectives.

Impact measurement: With a full-mapping strategy-KPI-OKR-driver-ML development project, all use cases have clear success metrics attached to them with a link to processes they contribute to in the value stream of the business.

Portfolio view: End-to-end mapping gives an overview of all developed and possible solutions in the right business context. This can be used to effectively assess where there is coverage and where there are apparent gaps that need to be addressed.

Team impact: Contextual mapping allows team members to understand better the business impact they are trying to achieve. This is important as it also gives the team members tools for forming a deeper understanding of why they are doing something and helps them question whether the selected use cases are the best way to achieve the desired results.

Communication and transparency: With complete mapping from strategy to an actual ML development item, teams can clearly communicate what is being done and why. It provides quick contextual information within and across business units as top-level strategy and KPIs are linked to ML development projects. Especially as ML development projects can still have quite obscure and hard-to-understand value, placing ML development projects in the context of direct business impact provides all stakeholders with a better understanding.

Key takeaways – Sanoma’s perspective

News has always been about providing information to the mass with a touch of serendipity to help you discover new things. With the ever increasing amount of visits, and increasing depth and breadth of the content we have in the digital age, we came five years ago into a conclusion that we need to be highly relevant to our users in every single one of those visits.

This meant that we would need to treat someone who visits us the first time today differently to someone who has visited us already three times. Same goes for our special interests we have as users, someone is into Premier League football and someone else into cooking.

To give all the users an optimal experience we believed we have to go personal, to leverage ML and AI and the cloud technologies to deliver a personalized experience to all our users that adapts continuously to their usage. In our transformation from print to digital there had been two transformational waves - introduction of desktop computing and then mobile. We believed that personalization would be the next transformational wave. It really turned out to be just that, big time.

News and journalism are special. Our mission is not about optimizing economic drivers or user KPIs. It is about journalism. When we started the personalization journey, the markers of what harm can be done with AI and data were all there - the intelligence was used to purposefully impact election results in different countries and the algorithms of the facebooks let the users find people thinking the same way, leading to extremism and increasing hate speech, along with other problems.

For these reasons the mission and ethics were at the core of the project since day one. The journalists and editor in chiefs were involved in all steps, we were careful not to give algorithms data that could lead to polarization, we used explainable models only, and took time to test and see that we did not narrow, but rather widen, the perspective with personalization. AI is a core part of Sanoma environmental, social and governance (ESG) guidelines.

The results of this five years journey are historic and groundbreaking and I believe we have just touched the surface of what can be done. The next five years will definitely be just as groundbreaking as the past five have been as we continue to push intelligence and insight into more of our core processes and value streams.

Timo Rinne, VP Digital, Sanoma Media Finland, News & Feature Media

About Sanoma

Sanoma is the leading learning and media company in Finland with operations in eleven countries across Europe. The Sanoma News and Feature Media portfolio consists of several subscription and ad-funded news sites such as Helsingin Sanomat and Ilta-Sanomat that together reach 97% of Finns every week.

Sanoma’s mission

“Our Finnish media provide independent journalism and engaging entertainment also for generations to come. Our unique cross-media position offers the widest reach and tailored marketing solutions for our business partners.

Our customers are at the heart of everything we do. We aim to serve them in the best way. We connect media content, services, brands and people – everywhere and every day. For teaching and learning we develop new, personalised solutions both for digital and print. For advertisers we provide powerful marketing opportunities through our reach and consumer insight.”

Written by Teemu Toivonen (Ex-Sanoma, Head of AI & Personalization, News and Feature unit), presently a Senior AI & Data consultant at Futurice, with the permission of Sanoma Media Finland.