Email onboarding: Improving the welcome email for registered users
TLDR: I proactively collaborated with marketing and design to create a new welcome email using content design methods that led to a 1200% increase in article reads and impacted positively on the approach to email company-wide. This project formed part of my systems thinking approach to improve register user engagement.
Background
At The Economist, registered users are given 3 free articles to read each month. They represent a large, warm audience to convert into paying subscribers. We often focus on improving our registration walls, registration forms or onboarding in product without also updating the registration onboarding emails, essentially creating a leaky bucket.
The Registration ecosystem at The Economist is split between two teams: Product and Marketing
In September 2022, Product held a workshop and ideated with stakeholders from product, marketing, data, research and engineering on how we might encourage these users to become paying subscribers. We know that the more engaged the user is, and more articles they read, the more likely they are to subscribe.
We came up with lots of ideas throughout the day: some around personalisation, giving value to the user and building habit. One of my ideas that focused on the welcome email and surrounding user journeys was selected as a project to focus on developing with marketing and added to the roadmap.
The Welcome Email is something that all registered users receive and have the chance to interact with, and therefore had the largest reach. However, the email is currently below industry standards with regards to both the open and click through rate. This idea wasn't chosen as a project to focus on by Product leads, however, I negotiated to get time allocated on both mine and a designer's roadmap so that we could collaborate with Marketing to improve the email.
Creating a content strategy statement
During the sprint we created a content strategy statement to guide ideation and this was also used when working on the email project. The strategy statement is a little framework I frequently use on projects to help identify and get alignment across stakeholders the user needs, business goals and the overarching solution. It can help reframe a problem from a user focused point of view. And this was really important for this email project.
The emails were created with a strong business need and with less focus on the audience receiving them, so this was a way to put some focus back onto the users.
Discovery
The email journey consists of seven emails (if you’re opted into marketing). Each of those messages has a specific objective ranging from welcoming the user, presenting our content to selling a subscription.
The Product Designer and I took a zoomed out approach and mapped all the emails the users might receive in their first two weeks after registering to understand their experience with The Economist. It's easy to forget that these aren’t the only comms from TE that the user is receiving. It had the potential to be overwhelming.
The emails all fall below industry standards with regards to both the open and click through rate. We hypothesised that these emails weren't easy to read, were not engaging and were diving into selling the subscription before letting readers engage with the content properly.
The Product Designer and I ran some user testing on the existing welcome email to get some feedback.
“It's a lot of information on an email. I’m a big bullet point person. Instead of paragraphs, I’d like to see what I’m getting “we offer this this this.”
“Plain email, doesn’t really catch the eye. Not immediately engaging. Not really intrigued to read it.”
“Find intelligent life online” - kill it. I don’t understand why that’s important. I also don't know why it's pushing me to subscribe before I've read anything!"
“I don’t think that (the sign off from Zanny) is a personal touch as we all know it isn’t really coming from her… You could just write ‘from The Economist’ ”
Guiding marketing with a workshop
Wanting to dig deeper, we ran a workshop with marketing to do a competitor analysis exercise. We asked everyone to bring a good email example and one they thought wasn't as good. This was a good exercise to get everyone to think about what good and not so good looks like and see things from a user's perspective. We also reviewed the performance of the emails in this session and shared the direct user feedback.
In a second workshop, we wanted to compare the emails to the user needs and business goals in the content strategy statement we created for the sprint. We audited the emails against this statement to see which emails were delivering, which weren’t and how we could improve them and this journey.
We decided that we would focus on updating the welcome email first as a litmus test for improvements for the rest of the journey. Rather than just changing the whole journey, we decided to take an iterative approach so we could understand what worked and didn’t, and then apply those learnings to other emails.
The Product Designer and I then worked as a pair to write user stories to identify what users need to know across the journey and prioritised certain needs for the welcome email.
The new welcome email would get straight to the point with: a welcome and introducing user entitlement, and also make content recommendations to show the breadth of our content (rather than tell) and get people reading.
Prototyping an Editorial-led option & a wildcard
We created two variations of the welcome email.
This one was heavily influenced by our editorial content, with the goal of driving engagement, by showing the breadth of our content and formats.
We also wanted to try an alternative design, something completely different to our previous emails. Instead of a text heavy, editorial email, we wanted to show the content in alternative formats, and to focus on what you get as a registered user - including 3 free articles a month, selected newsletters and (at the time) podcasts. This led to the creation of The Wildcard Design. It's very different to any emails The Economist creates.
Remote user testing: The Wildcard wins
To get some initial feedback on the new designs, we ran some remote user testing. We ran these as ‘10 second tests’ where the user only gets 10 seconds to first view the design. This helps imitate the real environment that the emails might be viewed in - and we could identify what the user noticed first and what stands out when skimming the emails.
After the initial questions, users were given another opportunity to go back and look at the emails in more detail to compare them side by side.
The results came out quite clear, with the majority finding Option C ‘The Wildcard’ as the easiest to understand, easiest to read and most engaging, with the current email coming in as the least engaging of the 3.
In market testing: The Editorial-focused option wins
The Product Designer and marketing team were keen to go to market with the Wildcard design, however, I insisted that we also ABC test in market. This is because I felt with the Wildcard we were simply reverting back to showing and not telling and we were not aligning with the content strategy statement. We made some adaptations to the Wildcard design to include links to articles to make it a fair test.
The results of the ABC testing proved my hypothesis correct. While the Wildcard performed better than the original, it was the more dense Editorial-focused version that performed the best. This variation produced the following results in comparison to the original email:
-
1,212% increase in article sessions
-
65% increase in site sessions
-
45% increase in click rate
-
16% decrease in unsubscribe rates
Our main goal was to increase content engagement, we wanted to get people reading, articles sessions is the prime metric and that’s what we achieved here.
What we learned
While incredibly important, qualitative testing should be viewed as a great guide but may not always reflect audience needs accurately. When we use remote user testing sites, the scenario is fabricated and we can't always recruit Economist readers. User testing and interviews are great for guiding and refining designs, but we needed to take the feedback with a pinch of salt and ensure we are aligning with the overarching project goals and user needs.
Our focus with this project was all about showing people what content they can access, sparking their curiosity to start reading, leading to increased engagement. The wild card also falls back into the trap of telling and not really showing.
The collaboration with marketing has continued
We have been working with Marketing on optimising the subject lines for the Welcome email, shortening the journey and introducing more trigger emails when readers hit the paywall having maxed out their free article limit. I've secured a sprint again for both myself and the designer to partner up with Marketing to assist on this project.
I've continued to work with Head of Lead Management and Conversion on other projects and have been contacted to help assist on other projects within the company. Additionally our approach to emails has made waves within marketing and now all emails are taking this stripped back approach. While the content on these emails from other areas of marketing still need optimising (I don't have capacity to pitch in), it positive to see our learnings here have been taken onboard.