# Spotting ASO “Keyword Dances”

- Link: https://phiture.com/asostack/spotting-aso-keyword-dances-c2e82c9e0f3b/
- Published: 2017-12-15T14:37:56+01:00
- Author: Luca Giacomel

#### Like true nerds 🤘

![Spotting ASO “Keyword Dances” -min](https://phiture.com/wp-content/uploads/2017/
12/Spotting-ASO-“Keyword-Dances”-min.jpeg)

A quick note about me: I am [Luca](https://www.linkedin.com/in/lgiacomel/) and I
work as a Data Scientist at [Bending Spoons](http://www.bendingspoons.com/) (yes,
we are hiring in ASO/Marketing!) where I also coordinate the App Store Optimization
team.

This story starts like many other stories, with me on my desk, in the morning, before
the first cup of coffee of the day, and with my brain still in “autopilot mode”.
One of our project managers comes at me, freaking out because an app suddenly lost
20–30% of its downloads overnight, without any particular reason (middle of the 
week, no ASO updates, no nothing — at least, apparently).

After quickly calming my colleague, with a fake sense of tranquillity that was mainly
given by the fact that I was still caffeine sober, I decided to follow my usual 
ASO problem-solving scheme to understand what could have happened:

 1. **Check if it is a reporting issue** (timeframes, partial data, wrong integrations).
    _Nope, everything was fine._
 2. **Exclude that it is a change in paid UA traffic**. _Nope, we didn’t have ads on
    that app. Only organic downloads, mostly coming from searches in the App Store._
 3. **Understand which country has the sharpest decrease in downloads**. Is it maybe
    bots in some remote, low ARPD country favoring our competitors? _It didn’t look
    like, and US was the country suffering the most._
 4. **Check if there are signs of seasonality trends** in the search ads popularity
    of the keywords for which we were ranking in high positions. _Nope, nada, nichts,
    all as flat as flat can be. Heck, I even checked at the bank holidays of the various
    countries where our downloads were decreasing. Nothing even there._
 5. **If everything else fails, it’s finally time to get a coffee (😙) and go back 
    to the basis by analyzing rankings**. The total downloads you make after all are
    the sum all the downloads make in the various searches, so in theory if we are 
    tracking all the keywords for which our app ranks we should be able to quickly 
    identify the issue.

Luckily, at BendingSpoons, we have developed [**Crystal** and **Pico**](http://bendingspoons.com/technologies.html),
two amazing internal platforms designed by our wonderful data engineers that track**
a huge quantity** of App Store data, including search rankings. And you might think
that this is how the story ends: I find a few searches that caused the download 
drop and quickly send an ASO update to apple to promptly correct the problem. “_Ah
you fool! Not so easily.”_

Sometimes the more data you have, the more it is difficult to make sense out of 
it. Out of the thousands of searches for which our app was ranking on that day hundreds
changed their value, some improving, other worsening. Moved by my big curiosity 
I then opened a Jupiter notebook, pulled some ranking data from our Big Query database
and plotted it. This is a graph of just a few, handpicked rankings I tried to analyze.

⌊app rankings-min⌉⌊app rankings-min⌉

_Our very understandable app rankings. Ah! This is really so very very helpful._

Headache. It was the right time for my second coffee.

While going back from the coffee machine to my desk, I [opened a statistics/machine learning book](https://web.stanford.edu/~hastie/ElemStatLearn/)
that always gives me inspiration in these cases. I like to call it _“The Bible”_
because it is a 750 pages manual that covers mostly every method of statistic or
machine learning ever devised by the great statistical minds that populated our 
world — and, even though I probably fully understand just below 10% of it, it always
gives me great hints.

This time I decided to use a very famous technique called Principal Component Analysis,
to see if there was any way to get a better representation of the above graph. Basically
what PCA does is to try to find linear combinations of the starting variables to
generate a much smaller set of factors that are uncorrelated between each other.
In this case what we hope is that the app search rankings are highly correlated 
one with the other, and that there is a way of summarizing them effectively with
just a “few lines”.

⌊Total % of variance explained by factor. High variance explained by few factors
= good. -min⌉⌊Total % of variance explained by factor. High variance explained by
few factors = good. -min⌉

_Total % of variance explained by factor. High variance explained by few factors
= good._

And luckily for us, I found that it’s enough to have 3 factors to account for 90%
of the total variance of the original variables. In other words, 3 lines can explain
most of what is going on in the messy graph of the ranks above.

⌊Factor time series with switch-points -min⌉⌊Factor time series with switch-points-
min⌉

_Factor time series with switch-points_

Every line represents a group of searches (provided that you find the correct rotation
for your PCA loadings 🤓) and as you can see there are some interesting patterns,
as searches “loading” to the same factors moved together in interesting ways.

### **Solved a problem, but found a bigger one.**

I was finally able to make sense of this. Thanks to this principle component analysis,
by analyzing the correlation between the original rankings and the factors I plotted
above, I managed to detect that the issue was in a particular word that was giving
us a lot of downloads but that was only present in our keywords field. I then quickly
sent an update to correct the problem, adding this word also in the subtitle. This
luckily solved our issue 🙃.

Nevertheless, I was still very curious as it seemed like our app had many moments
of unexpected changes in rankings, some in which we lost positions for some keywords,
some for which we gained. While we might not have noticed them before because the
effect on downloads were negligible (e.g. rankings that changed were too high to
bring our app downloads anyways) some others might have had.. Do you have anything
for us [ThomasBCN](https://medium.com/u/991bcd095ddf?source=post_page-----c2e82c9e0f3b----------------------)?
Oh yes of course you do! 😉

⌊thomasbcn on twitter-min⌉⌊thomasbcn on twitter-min⌉

_Our favorite ASO expert tweeting. Yes, Thomas BCN ❤️ ASO._

It looked like I came across something well known by the ASO community, the so-called“
Keyword Dances”.

### **Keyword dances timing**

Another interesting discovery is that “Keyword dances” seem to be too well distributed
in time to be simple coincidences.. They seem to be of different types and they 
seem to happen at “almost-fixed” (and therefore possibly predictable?) intervals
in time:

⌊keyword dances timing-min⌉⌊keyword dances timing-min⌉

_Random or.. not?_

And to confirm it we can see this, with more or less precision (our rankings might
not precisely attributed to the day) on a totally different app:

⌊Factor analysis for a completely different app -min⌉⌊Factor analysis for a completely
different app -min⌉

_Factor analysis for a completely different app_

I will continue working on these things to try figure out what actually happens 
during the keyword dances, but in the meanwhile I hope to have shed some light on
the matter, allowing you, ASO friends, to give explanations a bit quicker to yourselves/
your bosses/your customers.

#### **The end**

Whoa, you made it to the end 🙌. Cool, one last thing, remember to clap if you found
this article useful! 👏👏
