StrategyPerformance marketingHow TUI is perfecting performance marketing through incrementality testing

How TUI is perfecting performance marketing through incrementality testing

"Our process isn’t complicated. It helps us to test and learn. We use incrementality tests to measure the impact of a given tactical or strategic shift."

As a travel company, TUI is faced with a double edge sword. On the one hand, it benefits from vast swathes of customer search data. On the other, attribution modeling can lack impression-level data that give a limited view of campaign performance. Perfecting performance marketing is an ongoing priority for TUI.

Nicolas Elshout, Manager of Digital Media, brings us on the journey TUI has been on to create a sustainable performance marketing strategy:

  • Adding a layer of incrementality testing to augment traditional attribution models
  • Build vs buy: Bringing attribution in-house
  • Creating the right performance marketing team

Elshout shares how these learnings have helped TUI’s digital media team measure its impact, including a sales uplift of 7X following a Facebook video campaign.

Incrementality testing to augment attribution

Attribution is one way that helps marketers optimize their campaign algorithms. However, it does not give marketers the full picture. TUI’s attribution model, for example, is limited by a lack of impression-level data.

Accordingly, TUI has adopted incrementality testing to augment its picture of marketing performance. It runs cyclical A/B testing comparing a group exposed to ads to a non-exposed group. These tests run across different ad types and different channels.

“Our process isn’t complicated. It helps us to test and learn. We use incrementality tests to measure the impact of a given tactical or strategic shift. Long term we measure the impact on sales behavior by tracking the sales uplift or brand lift in the exposed group. For the short term, we track small tests looking at media KPIs like cost per completion or return on ad spend (ROAS).”

“Knowing how media pressure impacts a certain audience then helps define our retargeting strategy. How much of our budget should we spend on retargeting? If we double the budget, would we get double the incremental return? Or are we running into a wall?”

Incrementality tests can answer these questions but do require careful consideration. Vast amounts of data are needed to get meaningful results, and important sales peaks may have a sudden impact on results. Tests are carefully scheduled, and the process is communicated to the team.

Build vs Buy: Bringing performance marketing in-house

The build versus buy debate is always part of the conversation when it comes to any digital transformation journey.

Buying off-the-shelf can offer scale, security, and technical support, but can leave digital marketing teams struggling with vendor lock-in. Building, although expensive and time-intensive is personalized and tailor-made to fit a company’s pre-existing technical infrastructure and future requirements.

There is of course no universal answer as each use-case has diverse needs – but TUI, after a period of testing with vendors, opted to build its attribution solution in-house. Elshout elaborates.

“The attribution question has always been there. We started testing with external partners to answer the question, what value are we driving? Their systems were amazingly sophisticated, but our situation required systems capable of processing greater complexity.”

“Secondly, the same technologies were being leveraged by our competitors. We weren’t differentiated enough. We had to build something that played into our strengths, namely the quality of our customer data.”

The quality of TUI’s customer data is high. To squeeze the most value from this competitive advantage, it uses performance models built on click data and behavioral data. This helps TUI understand where traffic comes from and what incremental value it drives. Crucially, the model helps define what channels or impressions are impacting this behavior. Bringing this in-house has also meant developing the right team.

“We have built our data science team including a digital marketing technology lead to automate and optimize our attribution algorithms. They also improve the pipelines feeding data to these models.”

Creating the right performance marketing team

A comprehensive performance marketing strategy also needs a structured and skilled workforce to plan, execute, and interpret findings.

TUI allocates a specialist to each channel. Specialization gives TUI an in-depth understanding of its digital channels including within walled gardens and other platforms or partners it operates with.

However, a healthy test and learn model requires learnings to be disseminated across teams. TUI, therefore, runs digital centers of excellence. Specialists come together to discuss what they (and their technical partners) have learned.

“Typically, these meetings occur every two weeks, so the specialists get a tight feedback cycle. Successful or not, test results are shared. Challenges are discussed and future tests are scrutinized. It helps us benefit from the scale that we have as a group because we can get in-depth insights and learn from each other.”

“We also internalize externalities by running roundtables with our agencies and other advertisers, usually quarterly.”

7X sales uplift from Facebook video retargeting

Combining insights from attribution models with incrementality test results is particularly important for video marketing. Not all incremental impact on brand or sales will be due to clicks, so TUI utilizes A/B testing to account for impression data from videos.

TUI created a generic video that became automatically personalized for each destination it serves. It aimed to include these videos in dynamic retargeting campaigns on Facebook for hotels.

“We wanted to see the impact of bringing these videos into our retargeting campaign on sales uplift. We recorded one group with the old messaging and one group with the old messaging plus a video.”

“The result in incremental sales was astonishing. We saw a 7X increase in transactions for the exposed group.”

Incremental testing augmented TUI’s traditional attribution models giving the digital media teams the insights they needed about the performance of their video marketing.

Reflections and looking ahead

TUI is still in the continuous improvement phase of its performance marketing. Incremental tests are an added layer of complexity to attribution modeling, and as such cannot be run every week. As it tests and learns, Elshout reflects on the lessons learned thus far.

“There are multiple factors that come into play to make sure that it’s a success. Having the right team in place is one of the most important parts. Make sure they are trained, and all share the right vision and roadmap. That’s one of the biggest challenges for brands and even agencies in the current climate.”

“Having the right technical infrastructure is another major factor. Not just to support the roadmap, but to make sure it’s flexible enough to be modified as you learn, and change is required.”

“When we are testing a new platform and it’s been validated, the data pipelines must be flexible to cope with new sources and platforms where the format may change. Even a dot on one could be a comma on another.”

As new platforms surface that TUI wishes to test, flexibility in its pipelines will allow it to ingest data necessary to run its attribution models and incrementality tests. Its performance marketing practices demonstrate a clear impact on sales and can help it test the waters with the likes of TikTok and other emerging platforms.

Subscribe to the ClickZ newsletter for insights on the evolving marketing landscape, performance marketing, customer experience, thought leadership, videos, podcasts, and more.

The ClickZ Marketing Masters Podcast Season 2 is now live! Tune in wherever you get your podcasts, including AppleSpotifyGoogle, and Amazon. Don’t forget to subscribe!

Join the conversation with us on LinkedIn and Twitter.


US Mobile Streaming Behavior

Whitepaper | Mobile US Mobile Streaming Behavior


US Mobile Streaming Behavior

Streaming has become a staple of US media-viewing habits. Streaming video, however, still comes with a variety of pesky frustrations that viewers are ...

View resource
Winning the Data Game: Digital Analytics Tactics for Media Groups

Whitepaper | Analyzing Customer Data Winning the Data Game: Digital Analytics Tactics for Media Groups


Winning the Data Game: Digital Analytics Tactics f...

Data is the lifeblood of so many companies today. You need more of it, all of which at higher quality, and all the meanwhile being compliant with data...

View resource
Learning to win the talent war: how digital marketing can develop its people

Whitepaper | Digital Marketing Learning to win the talent war: how digital marketing can develop its people


Learning to win the talent war: how digital market...

This report documents the findings of a Fireside chat held by ClickZ in the first quarter of 2022. It provides expert insight on how companies can ret...

View resource
The 2023 B2B Superpowers Index

Whitepaper | Digital Transformation The 2023 B2B Superpowers Index


The 2023 B2B Superpowers Index

The Merkle B2B 2023 Superpowers Index outlines what drives competitive advantage within the business culture and subcultures that are critical to succ...

View resource