Letter is an email design tool I’ve been making throughout 2019 and into 2020. One of the largest milestones recently was releasing the product demo, along with 50+ custom newsletter components. The demo also allows anyone to use the app without having to create an account:

Since sharing, loads of people tried the app, and there was a trickle of accounts created – around 30 in total. Despite there being nearly 1,400 people on the Product Hunt waitlist, I decided to announced the demo app only to my Twitter friends so that fewer accounts would be created.

The idea is onboard a few people at a time, so I can see what breaks, iterate, and then invite more. A bit like this Slack quote:

“We started inviting teams in batches and watched what happened. Then we made some changes, watched what happened, made some more changes…”
Slack people on firstround.com

Reducing vanity metrics by preventing signups

By making the demo version available without having to sign up, I hope to save time for everyone. I only want people to create an account if they find the app useful.

It would be easy to gather signups by forcing people to register before trying Letter, but in the long run, it creates another vanity metric. It can result in 100s of users who don’t actually use the app – they only signed up to test it out.

‘Vanity metrics: good for feeling awesome, bad for action.’
Tim Ferris, as quoted on crazyegg.com

The landing page explains the main features of the app, and the demo has an interactive walkthrough to show how to use those features:

If people like it, they’ll sign up, and upon doing so, they get added to the waitlist:

Using a waitlist to control the growth

The demo version of the product worked well enough, but I was aware that other areas still needed to be improved. To prevent any delay, I made the demo version available to anyone right away, but kept full access invite only.

When people sign up, they’d go onto a waitlist that I can see in the Letter admin dashboard:

From here, I manually grant full access to people I’ve spoken to before, who I know are willing to highlight issues and give feedback.

Even at this stage, I noticed some broken experiences:

Product as prototype

In an ideal world, it’s possible to design and prototype every detail with those ‘best practices’ you see in textbooks. However since I have very limited resources and live in the real world, the end product often becomes the prototype.

I noticed some issues quite early on, just from the first handful of people:

It’s unclear you go onto a waitlist after signing up
Nobody updates their profile
It’s not clear what a ‘publication’ is

I realised a simple onboarding could fix a lot of this. Here’s the next iteration of the prototype, showing the before and after changes:

Onboarding Improvements

Before

People would sign up via a form (1st first image below), and upon signing up, there was nothing more. No email confirmation or instructions for what’s next. The only thing they’d see next was a modal (2nd image below) when they tried to create a new project:

Nobody would customise their profile or publications either, as they don’t know what publications are, or that an account settings page even exists. It resulted in lots of dummy avatar icons:

Now I’m fixing this by adding a user onboarding flow, that guides them through setting up their profile, adding a publication, and also explains the early access program:

After

The final screen also explains what happens next, and prompts people with how they can get access right away. The Twitter DM link goes straight to the Letter twitter account, and the ‘Email Us’ link is pre-filled with subject line and message.

A further improvement could be sending a welcome email upon signup. I did try sending an email after 20 members signed up (see it here), but this message is now replaced in the onboarding. This is something I’ll revisit later.

Other improvements

Other notable improvements this week are private and public mode for letters:

Open product roadmap

Now that those improvements have been made, it’s time to invite the next batch of users, and identify the next areas to improve. If you’re interested in what’s next, you can check out the product roadmap on Trello: