# How to Build a Kick-Ass Notifications System

- Link: https://phiture.com/mobilegrowthstack/fear-and-loathing-of-notifications-31dd117bb66b/
- Published: 2017-10-09T12:45:08+01:00
- Author: Andy Carvell

#### Mitigating Risk and Maximizing Impact

## ⌊How to Build a Kick-Ass Notifications System ⌉⌊How to Build a Kick-Ass Notifications System ⌉

### **“The only thing we have to fear is fear itself” — Franklin D Roosevelt**

Notifications can deliver significant engagement uplift, but, if abused, may result
in negative user sentiment. What we routinely see at [Phiture](https://phiture.com/),
is that teams are far more likely to _under-deliver_ on notifications impact due
to _fear of annoying users _than they are to drive users away through over-communicating.

This is a follow-up to my recent post where I outlined a model for considering the
impact of notifications; the [RRF Framework.](https://phiture.com/mobilegrowthstack/rrf-a-framework-for-building-impactful-notifications-73c7b91c45a7/)

In this post, I’ll expand on the topic of impact. The first part of the post deals
with how to conquer fears and unlock greater potential from notifications:

 * **Testing ideas for notifications while limiting risk**
 * **Dealing with fallout and mitigating negative effects**
 * **Putting users in control**

In the second part, I’ll also outline some useful concepts for building a notification
system that auto-optimizes towards maximum impact and minimum user annoyance. At
the heart of this smart system is a Notification Service that is informed by:

 * **User profiles, based on past behavior and predictive analytics**
 * **External and internal events to trigger notifications**
 * **A Prioritization Engine**

I’ll lay out what exactly an adaptive, scalable and essentially _stateless system
_for notifications could look like.

### **Notifications deliver impact**

Relevant, timely notifications demonstrably improve retention; I’ve seen this for
myself when my team built and deployed a real-time activity notifications system
at SoundCloud. There are also plenty of [reports](https://www.airship.com/blog/predict-and-prevent-customer-churn-and-increase-app-retention/)
out there that support this hypothesis (albeit produced by vendors of marketing 
automation tools, so some bias there).

Notifications won’t fix a poor product; in fact, they might amplify the flaws by
bringing users back more often to a dismal experience. However, if the product is
already engaging, notifications can usually help keep users around.

⌊Push demonstrably works⌉⌊Push demonstrably works⌉

_Push demonstrably works (source: Localytics)_

_The fear is often overblown_

It’s certainly possible to increase user churn and opt-out rates for email and push
notifications with poorly-designed notifications, but fear of these negative effects
is routinely blown out of proportion, leading to a significant under-utilization
of these powerful channels and leaving valuable impact undelivered**.**

Fear of notifications being perceived by users as spammy is endemic in teams building
mobile products. What typically fails to elicit anywhere near the same level of 
panic is the huge chunk of users who abandon the product within the first 24 hours,
with many more churning over the next few days. Given this scenario is the status
quo for the majority of apps out there, you’d think teams would be reaching for 
the big guns in an effort to fight back this crushing tide of abandonment threatening
to wash their products into obscurity. Notifications, when used wisely, can significantly
improve the situation, but their potential is often muted due to fear of causing
offense.

In actual fact, relevant notifications in a multi-user ecosystem can create an engagement‘
snowball effect’. This is best observed in social networks, where social interactions(
likes, comments, etc.) generate notifications, which in turn generate new interactions,
and so on.

⌊The virtuous cycle of social notifications ⌉⌊The virtuous cycle of social notifications⌉

_The virtuous cycle of social notifications_

### **How to test ‘risky’ ideas with minimal downside**

If there are concerns about possible negative effects stemming from notifications,
ensure such effects are properly accounted for when instrumenting the analytics 
around these notifications.

A data-driven approach, a robust testing framework, and judicious use of control
groups are all that is needed to establish whether these fears are justified and
provide the team the much-needed freedom to experiment and learn from mistakes.

The formula for ‘safe’ testing of new messaging strategies and notification types
is simple enough:

 1. **Test with a small (but significant) sample of users,** ideally_ new users _(many
    of whom will, as we have established, churn anyway, and hence have the most upside)
 2. **Compare the results versus a control group**
 3. ** Establish what impact was achieved**
 4. ** Check if fears were justified **(review negative signals such as opt-outs and
    churn vs the control group)
 5. ** Adjust course **accordingly

When one considers that an average app [loses 77% of its Daily Active Users in the first 3 days](https://andrewchen.co/new-data-shows-why-losing-80-of-your-mobile-users-is-normal-and-that-the-best-apps-do-much-better/),
testing bold new experiments on this group has very little downside: _these users
will churn_ if you don’t find a way to keep them around a little longer and give
them another chance to have a positive interaction with your product.

⌊There is often more upside than downside when it comes to reducing new user churn⌉⌊
There is often more upside than downside when it comes to reducing new user churn⌉

_There is often more upside than downside when it comes to reducing new user churn_

If positive impact is achieved, consider broadening the test to all users, or examine
the results on older cohorts (i.e. existing, engaged users), again vs a control 
group and with a minimum-viable sample size. If they behave more negatively, you
can still roll it out to new users only and increase that critical early-stage engagement.
In my experience, most decent, informative notifications drive impact across new,
engaged, and lapsed users, albeit to differing extents.

### **Mitigating Fallout from Negative Effects**

Testing and measuring for negative signals such as opt-outs uninstalls and drops
in engagement will let you know what proportion of users are turning their backs
on notifications, or the product itself. The critical optimization problem to solve
is how to mitigate such negative effects among users who dislike your notifications
without slashing the overall impact by dialing back notifications in general, even
to those who would happily engage with more of them. Most companies do not solve
this problem; those that do are able to unlock additional growth.

First and foremost, notifications should be designed with the user in mind and should
be timely, relevant (ideally personalized), etc. rather than generic broadcast messages
to the entire user base. These considerations were covered in my [original RRF post](https://phiture.com/mobilegrowthstack/rrf-a-framework-for-building-impactful-notifications-73c7b91c45a7/).

For now, let’s assume that the notifications are at least ‘decent’, as evidenced
by the significant positive impact on engagement KPIs among users receiving notifications.

In the remainder of this article, I’ll outline three approaches that will mitigate
negative effects associated with notifications in ways that don’t sacrifice impact
with the general population of users:

 * granular notifications preferences
 * explicit and implicit subscription
 * dynamic user-level optimization

Table of Contents

 * [Putting users in control](https://phiture.com/mobilegrowthstack/fear-and-loathing-of-notifications-31dd117bb66b/#putting_users_in_control)
 * [Blueprint for a smarter notification system](https://phiture.com/mobilegrowthstack/fear-and-loathing-of-notifications-31dd117bb66b/#blueprint_for_a_smarter_notification_system)

## **Putting users in control**

Writing for Wired, personal technology expert David Pearce recently made a [hyperbolic case](https://www.wired.com/author/david-pierce/)
for turning off all push notifications. Pearce argues that it’s time for users to
_take back control _of their attention and fight back against what he refers to 
as ‘commercials’ being inserted into users’ lives. I would argue that only poorly-
designed notifications with very low relevance seem like commercial interruptions,
but I do agree with Pearce’s core argument that users cherish a feeling of control
over their digital lives.

#### **󠀁[This is what 8 million Instagram followers does to your notifications ](https://www.youtube.com/watch?v=zGl796352RI&feature=youtu.be&t=4)󠁿**

**_Smart systems mitigate risk, rather than avoid it and lose the upside_**

A fearful approach to the risk of annoying users would be to adopt a very cautious
approach, for example, a strict rate limit which caps the maximum number of notifications
that a user will ever receive within a certain period. Such methods are crude and
leave a lot of potential impact untapped.

In the next section, I’ll outline some nuanced approaches that can be taken to mitigate
the risks of opt-outs and negative sentiment, then go on to describe a system that
can both mitigate risks and capitalize on the — frequently untapped — upside from
highly-engaged users.

**In-House Notifications Preferences**

Providing users a more fine-grained way of controlling which notifications they 
receive serves two purposes. Firstly, it allows users to opt-out of specific categories
of notifications without opting out at the system level (thus keeping the channel
available for other types of messaging). Secondly, opt-outs from inside the app 
are more easily tracked by analytics and hence provide a valuable feedback loop;
if opt-outs for a specific notification category suddenly spike, it’s a valuable
and timely indication that something is wrong and affords the developer a chance
to fix it. When users opt-out at the system level, it can be tougher to track this
in a timely fashion.

The Pinterest app (shown below) provides the user extensive control over the kind
of notifications they receive, with preferences for both email and push, from the
settings menu inside the app.

⌊Pinterest gives users a comprehensive list of notification categories to control⌉⌊
Pinterest gives users a comprehensive list of notification categories to control⌉

_Pinterest gives users a comprehensive list of notification categories to control_

With the advent of Android 8.0 (Oreo), similar functionality is provided at OS-level,
through a new feature called [Notification Channels](https://developer.android.com/guide/topics/ui/notifiers/notifications.html#ManageChannels),
which should obviate the need for a more custom UI. However, Android’s notoriously
slow upgrade cycles mean that Channels won’t be ubiquitous for a long time and iOS
notifications preferences at the system level are still binary ON/OFF, so it’s probably
still prudent to build and maintain an in-house notification preferences system 
for the time being.

⌊Android 8.0 introduces ‘Notification Channels’, allowing users to customize preferences
for specific categories (‘channels’) of notifications ⌉⌊Android 8.0 introduces ‘
Notification Channels’, allowing users to customize preferences for specific categories(‘
channels’) of notifications ⌉

_Android 8.0 introduces ‘Notification Channels’, allowing users to customize preferences
for specific categories (‘channels’) of notifications_

#### **Explicit Subscription**

Another way to put users in control of the notifications they receive is to give
them features that allow them to request to be notified when things happen regarding
specific comment threads, newsfeed items, content, or users.

This is next-level granularity since it allows users to be ultra-specific about 
specific content that they would like to be notified about, as opposed to broad 
categories, or a single global opt-in for all notifications. Consider the die-hard
_deadmau5_ fan, who wants to be notified whenever he drops a new track or someone
who wants to ensure they don’t miss a single update on a conversational thread… 
explicit subscription mechanisms allow them to do just that.

⌊Explicit Subscription example on Youtube⌉⌊Explicit Subscription example on Youtube⌉

#### **Implicit Subscription**

The problem with an explicit subscription is that it often remains something of 
a power-user feature; it can be tough to drive awareness and mass-adoption of such
controls, even though users might have appreciated receiving updates about certain
things they are interested in at a higher frequency had they discovered the subscription
feature.

A smarter notification system might begin implicitly subscribing users to certain
content based on their behavior history, obviating the need for them to explicitly
hit the subscribe button but making it clear that they can unsubscribe.

Such auto-subscriptions could be processed periodically after e.g. a user has listened/
watched content from a particular artist/read posts from a particular contributor
enough times for the system to have a high degree of confidence that this is one
of the users’ favorite content creators or content types.

If YouTube would auto-subscribe users to channels after they had watched 10 of its
videos, I predict few users would complain (since they have implicitly expressed
a clear preference for the content) and subscription rates would be significantly
higher. Users could still unsubscribe using the button. The upside for YouTube and
the creators on the platform would be huge in terms of additional engagement generated
from fans who would never have been bothered to press the subscribe button but love
the content nonetheless.

## **Blueprint for a smarter notification system**

Notification preferences and subscriptions are great for giving the user more control,
but wouldn’t it be great if users _never felt the need_ to adjust their notifications
settings? Imagine building a system that is smart enough to adjust each user’s mix
of notifications according to their changing tastes and their engagement with notifications.

⌊A smart notification system adjusts to users’ engagement with previous messages
and uses behavior history to estimate the relevance of notifications before deciding
whether to send them⌉⌊A smart notification system adjusts to users’ engagement with
previous messages and uses behavior history to estimate the relevance of notifications
before deciding whether to send them⌉

_A smart notification system adjusts to users’ engagement with previous messages
and uses behavior history to estimate the relevance of notifications before deciding
whether to send them. Events shown are just examples; these would be specific to
the app/platform._

The system described takes inputs in the form of events occurring on a platform.
The example used is a content platform: this could be a music streaming platform
like SoundCloud, a video streaming service such as Netflix or HBO, a blogging platform
like Tumblr or Medium. Any platform where there is a lot of content being produced/
released and consumed would be a good fit for such a notification system.

The system has the following basic properties:

 * **Each notification type has a rate limit (aka Frequency Cap): a frequency which
   under normal circumstances will not be exceeded e.g. one notification per 24hrs.**

 

 * If more than one channel is supported (e.g. Email as well as Push Notifications),**
   rate limits are also channel-specific**; Push rate limits can be different to
   Email rate limits for the same notification type.

 

 * Rate limits are calculated and adjusted **on a per-user level **by maintaining
   a user Engagement Score (more on this later)

 

 * **Rate limits can be exceeded (more notifications of this type are sent within
   the rate limit window) in exceptional circumstances. Exceptions are decided by
   the Prioritization Engine, which estimates the relevance of each notification
   to a specific user; if the relevance estimate is high enough, it‘s likely worth
   exceeding the user’s daily maximum.**

_Stateless system = cheaper and faster to deploy_

One advantage of the system outlined above is that it is essentially stateless; 
it processes each event in real-time, delivers notifications more or less instantly,
and does not incur the significant computational and storage overhead required for
buffering/caching events and notifications. This means that such a system can be
built and deployed faster… in the retention team at SoundCloud, we deployed an MVP
version of such a system within three months of kicking off the project. While we
made numerous enhancements over time, the system delivered very significant impact
from launch day onwards.

_Aggregation of similar notifications requires a more expensive approach_

While the system described is one that can be built and deployed relatively quickly
and cheaply, a more sophisticated system would cache events as they come in and 
maintain queues of notifications at a per-user level. Such a system would be able
to choose between competing notifications (selection) and aggregate notifications
of a similar type (e.g. “Your post received 5 likes”).

Aggregations are undeniably powerful, but this functionality has been omitted from
the above system since they introduce considerable complexity, both in terms of 
system design and implementation. When buffering and aggregating notifications, 
the designer of the system needs to consider such issues as timeliness, batch sizes
for aggregations, how long to wait before sending the next notification to a user
if no new events are received (but qualified events are buffered), etc. To avoid
this additional complexity, the remainder of this article will focus on the simpler,
real-time, stateless system outlined above.

### **User Engagement Scoring: enabling self-balancing, personalized notification limits**

Imagine an app that could dial up or down the frequency of notifications based on
your personal engagement with them. Ignore your notifications, and it sends fewer,
less often. Click on a bunch of them and it starts sending you more of the things
you’re interested in.

By employing a Machine Learning approach, or simpler algorithmic method of adding
and subtracting points based on interactions (or lack thereof) with past notifications,
the system dynamically adjusts the volume of notifications each user receives (i.
e. adjusts their individual rate limit).

_Why bother with Engagement Scores if the app already includes granular notification
preferences?_

There are a couple of reasons why maintaining an engagement score might be a good
idea, even if a comprehensive set of preferences are implemented in the app settings
menu:

 1. Engagement Score is automatic and based on inference rather than explicit user 
    action: users do not need to explicitly find this feature and make manual changes.
    Thus, all users benefit from it, not just the users who find and use the manual
    settings.
 2. Even fine-grained preferences typically only allow the toggling of specific notification
    categories on/off. An Engagement Score allows every _on_ the possibility to be 
    _more on_ and reduces the likelihood that anything will be turned _off_ since it
    will be dialed down to a lower frequency by the system.
 3. While, in an ideal world, the engagement score system works so perfectly that it
    obviates the need for explicit preferences entirely, there will still be some users
    who enjoy the feeling of control that a full set of preferences provides, much 
    as some people would prefer a steering wheel and pedals in their self-driving car,
    just in case they feel the need to take over.

### **The Prioritization Engine: exceptions to the rate limit for top-priority messages**

Think of a Prioritization Engine as a sorting machine that tags message-user pairs
with a level of importance, and thus influences the decision on whether to notify
them with this specific message, or leave them undisturbed.

The Prioritization Engine’s job is to guess — as best it can — the relevance of 
a specific notification to a specific user. A really smart system would employ machine
learning to become progressively smarter, based on a feedback loop in the form of
open rates and conversion data from previous notifications. The aim here is to discard
notifications that are unlikely to resonate with the user before they are sent while
flagging those that have a great chance of being engaging, so they can be prioritized
and possibly even override the standard rate limit established for that user, to
send a notification that would otherwise have been prevented by the rate limit.

While it sounds fancy and complex, an MVP prioritization engine could employ simple
heuristics (basic rules of thumb/logic rules) to try to identify the ‘best’ or ‘
worst’ notifications for a specific user. As long as it succeeds in improving, on
average, the ratio of open to unopened notifications (i.e. increasing the average
_relevance _of notifications) that the system delivers, it’s adding value to the
system.

Improving on simple logic rules, the Prioritization Engine could leverage analytics
data about the user’s historical engagement. For example, pre-calculated ‘favorites’
lists (favorite content, favorite creators or publishers of content) based on the
usage history of each user and prioritizing notifications about content or creators
the user has expressed a clear predilection for.

The logic goes that notifications about content/creators/product categories that
the user has interacted with a lot in the past will _be welcome at a higher frequency_
than of those that had received little prior attention from the user.

The danger of this kind of prioritization logic is that it can create an echo-chamber
effect, where new content is not surfaced due to a bias toward the already-discovered.
This effect could be reduced by only applying prioritization to notifications once
the rate limit has been reached, to decide if a new one should be permitted — if
predicted to be sufficiently relevant — to bypass the rate limit.

### **Conclusion**

 * Notifications usually fall short of their true potential **due to fear of overusing**
   them
 * Careful **experimentation with control groups **is necessary to establish ‘how
   much is too much’. Test first with new users, where feasible
 * **Giving users explicit** **control** over the kind of notifications is good 
   practice and also provides valuable information
 * **Inferring user tastes** from their behavior may unlock further potential for
   relevant notifications
 * **Dynamically adapting** the volume of notifications sent on a per-user level,
   based on prior engagement and/or predicted relevance, can unlock further impact

I would love to hear from growth practitioners what is working well for them in 
terms of optimizing notifications impact. If you are willing to share your experience,
I’d appreciate hearing from you. Likewise, if you have comments or feedback on the
concepts outlined above, please get in touch or leave a comment below.

Finally, if you’d like help with building such a system for your business, feel 
free to reach out to my consultancy firm, [Phiture](https://phiture.com/).