# Data for Creatives: Why the Math Isn’t the Hard Part

- Link: https://phiture.com/mobilegrowthstack/data-for-creatives-mobile-apps/
- Published: 2026-10-27T16:08:25+01:00
- Author: Cole Dennis

###     Table of Contents

 * [Trust the platform, with some nuance](https://phiture.com/mobilegrowthstack/data-for-creatives-mobile-apps/#trust_the_platform_with_some_nuance)
 * [The real bottleneck is ecosystem literacy, not statistics](https://phiture.com/mobilegrowthstack/data-for-creatives-mobile-apps/#the_real_bottleneck_is_ecosystem_literacy_not_statistics)
 * [Map creative tactics to the metrics they move](https://phiture.com/mobilegrowthstack/data-for-creatives-mobile-apps/#map_creative_tactics_to_the_metrics_they_move)
 * [Use the scientific method, and isolate one variable](https://phiture.com/mobilegrowthstack/data-for-creatives-mobile-apps/#use_the_scientific_method_and_isolate_one_variable)
 * [Don’t engineer creative expertise out of the test](https://phiture.com/mobilegrowthstack/data-for-creatives-mobile-apps/#dont_engineer_creative_expertise_out_of_the_test)
 * [Report business-relevant deltas, not raw metrics](https://phiture.com/mobilegrowthstack/data-for-creatives-mobile-apps/#report_business-relevant_deltas_not_raw_metrics)
 * [Build a real creative testing roadmap](https://phiture.com/mobilegrowthstack/data-for-creatives-mobile-apps/#build_a_real_creative_testing_roadmap)
 * [The takeaway](https://phiture.com/mobilegrowthstack/data-for-creatives-mobile-apps/#the_takeaway)

 

At a talk aimed mostly at growth teams, we opened by asking the room for a quick
show of hands: who here works on a creative team or has a creative role? Only a 
couple of hands went up, with some hesitation, which fit a presentation built mainly
for growth marketers, though we noted there was plenty in it for creative professionals
too.

I’ve spent 13 years as a creative digital marketer. I originally studied biochemistry
and math in school before switching to writing, so I think of myself as a nerd in
addition to a creative. I like to illustrate that combination with a picture of 
Fibonacci, another math nerd who loved beauty: Fibonacci’s spiral is often placed
over famous artworks, like Vermeer’s Girl with a Pearl Earring and the Mona Lisa,
as a supposed ideal ratio, though it never quite lines up perfectly. I also hold
a US patent for a system that analyzes the performance of UA creatives and then 
iterates on them using machine learning, so I’ve spent a long time working from 
both sides of the data and creative divide.

One of the first questions growth leaders ask me is how to improve the data literacy
on their creative team. My answer starts by pointing out that math comes in orders
of magnitude of difficulty. The most difficult is chaos theory. Second is UA campaigns
and campaign data. After that comes dividing the bill at dinner. Then come ASO test
results, which aren’t particularly complicated: once a test reaches statistical 
significance, the platform gives you a red bar and a green bar, and if the green
bar is larger, you can generally assume it did better than the control.

If the data isn’t complicated for ASO test results, that means the real difficulty
lies somewhere else in the process.

Table of Contents

 * [Trust the platform, with some nuance](https://phiture.com/mobilegrowthstack/data-for-creatives-mobile-apps/#trust_the_platform_with_some_nuance)
 * [The real bottleneck is ecosystem literacy, not statistics](https://phiture.com/mobilegrowthstack/data-for-creatives-mobile-apps/#the_real_bottleneck_is_ecosystem_literacy_not_statistics)
 * [Map creative tactics to the metrics they move](https://phiture.com/mobilegrowthstack/data-for-creatives-mobile-apps/#map_creative_tactics_to_the_metrics_they_move)
 * [Use the scientific method, and isolate one variable](https://phiture.com/mobilegrowthstack/data-for-creatives-mobile-apps/#use_the_scientific_method_and_isolate_one_variable)
 * [Don’t engineer creative expertise out of the test](https://phiture.com/mobilegrowthstack/data-for-creatives-mobile-apps/#dont_engineer_creative_expertise_out_of_the_test)
 * [Report business-relevant deltas, not raw metrics](https://phiture.com/mobilegrowthstack/data-for-creatives-mobile-apps/#report_business-relevant_deltas_not_raw_metrics)
 * [Build a real creative testing roadmap](https://phiture.com/mobilegrowthstack/data-for-creatives-mobile-apps/#build_a_real_creative_testing_roadmap)
 * [The takeaway](https://phiture.com/mobilegrowthstack/data-for-creatives-mobile-apps/#the_takeaway)

## **Trust the platform, with some nuance**

I said this to someone earlier the same day, and she pushed back: you can’t actually
trust the platforms; they lie to you all the time. That deserved a more nuanced 
answer. Yes, you look back at your tests and adjust for noise, and you make sure
you have enough volume over a certain amount of time. But when it comes to individual
tests, if you trust Apple and Google, you can trust the red and green bars.

UA campaigns are a different story; it’s harder to make sure every creative is getting
the same scale, and even when it’s possible, it’s always more costly.

## **The real bottleneck is ecosystem literacy, not statistics**

I’ve been consistently shocked, over the years, as the industry has grown and the
tools have become more user-friendly, that very few companies take the time to educate
their creatives about what the actual digital ecosystem for ads or ASO looks like.
I’ve worked with designers where we would mention a specific ad network, or talk
about DSPs, and a year into working there, they would ask me what a DSP actually
was. No one had ever taken the time to explain it.

Most of the heavy lifting that has to happen, and most of the education that needs
to happen, is logistical, and it needs to happen before creatives ever go live. 
Creative teams need to understand the ecosystem: the many different ways a company
is doing its advertising and marketing, and all of the logistical concerns that 
come with that. I think it’s important to walk creative teams through the whole 
marketing funnel and to train them up as if they were growth marketers, so they 
understand where their creatives are actually being seen. That one step does about
half the work of explaining results to a creative team later, because once people
understand the ins and outs, or even just the gist, of different ad networks, it’s
rare that anyone has a straightforward answer to “what’s your best ad,” and understanding
the ecosystem is what makes that answer make sense.

For organizations that work on only one or two platforms, I spend time teaching 
the algorithm specifically. On Meta, for example, I teach creative teams about ad
groups and how the platform optimizes groups of creatives, because otherwise teams
start to think small: what’s the point of trying all these different versions if
half of them never get seen? The answer is that the algorithm is going to pick a
champion, and you have to deal with that unless you’re willing to force spend.

## **Map creative tactics to the metrics they move**

The next most valuable thing, in my view, is to map creative to metrics. If a company
is struggling with a particular part of the funnel, there are different creative
tactics suited to that problem. This mapping is loose, but it gives teams a place
to start.

  |  Funnel stage |  Creative lever |  Metric to watch |  
   |  Awareness |  Design, the visual itself |  Views, impressions |  
 |  Interest |  Messaging,  the hook that captures attention |  CTR |  
 |  Consideration |  The content of the message, separate from how it’s delivered |  Installs, CTI |  
 |  Conversion |  Integrity of the full creative experience, not one isolated asset |  Conversion rate |

If a team is struggling to get views, or has an awareness problem, design is the
first thing to tackle, since it’s what most people respond to in a visual ad. Messaging
is where interest is captured, and that loosely ties to clicks and CTR. Consideration
is the content of the message, which we separate from messaging itself: what you’re
saying is different from how you’re saying it, and that correlates to installs and
CTI. Conversion is where things get more complicated, because creatives aren’t as
straightforward by that point, and it comes down to the integrity of the whole funnel
and everything the user sees along the way.

## **Use the scientific method, and isolate one variable**

I teach the scientific method to a lot of the creative teams I work with, because
I think understanding logic matters more than understanding statistics when it comes
to getting creatives to work with data. Since ASO data, especially, can be very 
simple, the real responsibility sits with the logic used to design the creative,
so that whatever hypothesis is being tested, the results actually hold up.

That starts with being clear about what kind of test you’re running. I like to use
movie posters as an example, because studios change the design a lot between releases.
I point to the Pete’s Dragon remake poster against the one from the original film
from the ’80s: the content is the same, but the design is completely different. 
That’s a design test. Nothing about the messaging or the value proposition changes,
and the audience isn’t different either. You’re only trying to capture attention
in a different way. Being this specific about what you’re testing is what helps 
a creative team interpret the results in a way that makes sense, because the point
of any creative test is to decide what to do next, not just to find a performant
creative.

A messaging test looks different, and my favorite example is Campbell’s Soup. Years
ago, while building the machine-learning ad-analysis platform tied to my patent,
my team made Campbell’s Soup ads to test it. The theory was that you can have as
many heartwarming family scenes as you want, but not everyone likes their family,
so you can’t use family imagery to sell to every individual person. The content 
and the value proposition stayed the same; you’ll want to drink soup because of 
your home life, but the messaging changed. Again, that’s isolating the independent
variable and making sure that whatever data comes back can lead to an actionable
decision.

None of this involves statistical analysis or any math at all. It’s about making
sure the creative team has done everything it can before a creative goes live, to
make sure the results will actually be useful.

## **Don’t engineer creative expertise out of the test**

This part is a little trickier to talk about, because it’s hard to oversell the 
importance of creative expertise. I often work with clients who want very rigid,
laboratory-safe tests, whether in the App Store or in UA campaigns, and as a result
they start thinking smaller and smaller, trying to limit their variables as much
as possible so that when they see a half a percent difference in IPM, they can point
to exactly why it happened. Taking a broader view of creative and the creative process
means you can analyze creatives that are pretty wildly different from each other
and still get actionable results.

I use two posters for Evil Dead 2 to make this point: one released in the United
States, one in Japan. There isn’t much that’s similar between them. They’re both
marketing the same movie, but the design is wildly different, the tone is wildly
different, and even how the main characters are portrayed is very different. Taking
a broader view of how creative gets made, you can think of this less as a test of
one specific execution and more as a test of whether you’re using local culture 
experts in your marketing funnel if you’re advertising in a lot of different locations.
You could, in theory, translate the Japanese poster into English and the American
one into Japanese and run an A/B test between them, but that isn’t really how movie
posters work. It’s more worth highlighting as a test of how the creative gets produced
in the first place.

A more specific version of this came up in my first job at an agency, where every
new client’s first question was what the perfect button color should be. It drove
me a little crazy. So I ran a fairly large test: ten ads, three button variations
on each. One used a red button, because repeated past testing had shown red generally
performed best. One had a button the designers made specifically for that ad, with
no guidelines beyond “make a button that fits this ad.” One ad ran with no button
at all. Every single time, the button designed by the designer for that specific
ad won.

I think that’s a good way to show that creative expertise isn’t something you should
engineer out of your creative tests. I know it can sound a little self-serving coming
from a creative marketer, but these kinds of tests hold up again and again.

## **Report business-relevant deltas, not raw metrics**

When it comes to UA campaigns specifically, there’s more nuance than with ASO, and
it’s harder, or at least more costly, to get every creative the same scale. So the
question becomes how to show a creative team which ad is actually doing best, or
how to build a dashboard or report that makes that clear.

My approach is to talk about almost every metric as a delta over the average performance
of that campaign, rather than comparing raw CTR, CTI, or IPM numbers side by side.
That creates a dynamic benchmark for whatever time period and network you’re looking
at, and from there you set a threshold above which something counts as successful;
in my example, 150% over the average.

  |  Example ad |  Delta vs. average IPM |  Status |  
   |  Creative A |  +210% |  Above 150% threshold |  
 |  Creative B |  +40% |  Below threshold |  
 |  Creative C |  −15% |  Underperforming |

Looking at the creative production pipeline as part of the creative process also
means bringing in relevant business metrics. I worked with a client whose creative
team was split roughly in half between 2D designers and video designers, for a children’s
education app. The video team kept getting despondent because their ads were consistently
showing up below the 2D ads in reports, underperforming on raw installs and subscriptions.
Once I started showing them cost per paying install and cost per subscription by
platform, something the growth team hadn’t done before I got there, the two ad types
turned out to be comparable. Creative teams aren’t always brought into this kind
of data, since they’re usually assembled around a specific production expertise,
and they benefit a great deal from seeing it.

The 150% benchmark itself gets reverse-engineered from the team’s own capacity. 
If a creative team can produce a new video every two weeks, and the account loses
roughly 30% of creative performance every two weeks on its main networks, then hitting
150% of the average is what it takes to keep introducing ads that stay ahead of 
that decay. This gives a creative team something to work toward that isn’t just 
a feeling, and I think it’s good not only for the business and the performance of
the creatives, but for morale. It’s nice for a design team to have targets to hit,
like the rest of the marketing organization, and to be able to point to something
on paper that shows they’re achieving them.

## **Build a real creative testing roadmap**

When creative teams are thinking through new creative, I think it’s important to
keep the scientific method built into everything. My team uses brainstorm boards
where current screenshots sit at the top, so every iteration and every test gets
discussed as an iteration against a control. That keeps the hypothesis and the testing
mindset in view, and it lets a creative team start thinking of every test as part
of a larger decision tree, rather than taking stabs in the dark.

Without that structure, I like to compare what happens to “like treats like,” an
idea from medical history associated with Pliny the Elder, an ancient Roman physician.
Pliny believed things like crushed bees could treat stinging pain, or that onions
could treat colds, runny noses, and irritated eyes, on the logic that whatever caused
a symptom must be the key to solving it. A broken creative pipeline that isn’t using
data to inform itself can end up working the same way. When an ad gets too prescriptive-
download game, buy coin, feel good about it- and a creative team isn’t making its
own decisions, ingesting data, and acting on it, the ads can end up feeling flat,
especially if hypotheses get narrowed down just to produce actionable data without
keeping the bigger picture in mind.

I lay out three stages of advancement for a creative team.

  |  Stage |  What the creative team does |  
   |  Logical |  Helps form hypotheses, sits in on strategy conversations, assists with research (give them the same research tools, like an AppTweak-style license, that the rest of the team already has) |  
 |  Strategic |  Builds its own testing roadmap, separate from the ASO or UA roadmap, usually starting with evergreen creative so it doesn’t disrupt scheduled launches |  
 |  Driver |  Manages its own testing campaigns, including a slice of real ad spend |

 

That third stage, handing some amount of money to the creative team to run its own
testing campaigns, makes some people nervous. My view is that the best creative 
teams I’ve been part of have been behind the wheel on at least some creative spend.
That doesn’t mean handing over login credentials and walking away, but if the creative
team is actually launching its own creatives, even without help, it’s a strong learning
experience, and it cuts down the amount of information that has to be driven by 
someone else. No one likes endless meetings or endless documentation, and an actively
involved creative team means a lot less of both.

## **The takeaway**

The core point I want to make is that none of this is really about math. ASO data
is simple once a test reaches significance. What takes work is everything around
it: making sure creative teams understand the ecosystem they’re designing for, mapping
creative tactics to the metrics they actually move, running tests that isolate one
variable at a time, protecting creative expertise instead of testing it out of existence,
translating results into business-relevant numbers, and building a real testing 
roadmap that gives creative teams more ownership over time.

For growth leaders, that suggests a different starting point than asking how to 
improve a creative team’s data literacy. It’s worth asking instead how much of the
ecosystem, and how much of the business logic behind the numbers, the creative team
has actually been given. My experience, across 13 years and a long run of these 
tests, is that creative teams given this kind of structure don’t just produce better
results. They become part of deciding what to test next.

 

### Read also:

 * [ASO Monthly #122: App Store’s new creative assets, Larger Screenshots on Store Listings, Broader CSL Capabilities and more…](https://phiture.com/asostack/aso-monthly-122/)
 * [Acquiring Better Users — Eric Seufert & Thomas Petit on UA Tactics](https://phiture.com/mobilegrowthstack/acquiring-better-users-eric-seufert-and-thomas-petit-on-ua-strategies-1fe34b01c010/)
 * [Google PlayTime 2025 Recap: New AI, Discovery, and Developer Updates](https://phiture.com/asostack/google-playtime-2025-recap-new-ai-discovery-and-developer-updates/)