From gut instinct to data driven decisions

Design is ever-evolving, and there’s value in taking small, insulated steps to reach the grand product vision. When we launched Microsoft Flow last year, most of the design decisions were made using our collective intuition and market research. We wanted to create something powerful, something useful, and then send it into the wild. To raise it up right and hope we’d instilled good survival skills.

Months later, we now have data pouring in from telemetry and user studies. It’s challenging our initial thinking in the best way; making us more aware of the opportunities to design better. For us, this is one of the most exciting phases of product development. Moving from gut instinct to informed, learning-based moments to make decisions that craft better user experiences.

Data gives designers a new perspective to better understand the problem, making every decision intentional.

On the Microsoft Flow team, we work in a two-week design sprint cycle. In this timeframe, we intentionally keep room for experimentation to iterate on new design ideas; to fail fast and learn from those mistakes. This process helps us uncover and address many of the disguised unknowns before releasing the feature.

In this post, we want to share this process with you, and the reasons behind our design decisions.

Listening — and finding the problem — through telemetry

Really understanding how people react throughout the experience is a great way to uncover problems. And when you combine that with solid telemetry information, it becomes more impactful. You give weight to a problem; you see many sides of the same coin.

At Flow, we gather data from various listening channels — including from our research group that spends time understanding patterns and articulating the data. Together we find different connection points among the data to clearly find the right problem to solve.

Let’s take the basic example of ‘Create a flow’. In usability studies, participants were able to create a flow, but had some difficulties. Observations and comments included ’too much configuration required’, ‘discoverability issues with commands’, and ‘high number of clicks’.

Unsurprisingly, the telemetry info showed a large percentage of our users were dropping off. Some of the specific observations provided through the data included:

· Success rates are lower at creator than from a template
· Most people leave immediately after landing at the creator
· Most save failures were due to forgetting the Flow name
· Templates follow a Power Law — the top 4 templates are 50%+ of running Flows

After several iterations, we designed a new experience which included: automatic Flow name generation, reduced occurrences of creator, improved discoverability of commands, and a more streamlined workflow.

Let’s take another example: the creation experience from a template. Each template requires connections to individual services and then filling out required fields. The old experience presented this information in three screens, and with a high number of clicks. This jumping-of-screens led our users to feel a bit confused.

The new experience occurs on a single screen. Instead of spreading out the information, we brought the most important pieces into one place and showed them progressively with needed visual cues.

We have 60 seconds to wow the user

At the launch of Flow, we hypothesized that we’d have five minutes to wow the user, and have them stick around. But telemetry showed us that most people who dropped out spent less than a minute on the website. Alright then. Maximum value in sixty seconds.

Time constraints are awesome opportunities for designers. The opportunity for strict optimization. So, we came up with ‘zero-input’ templates for new users. Instant value from Flow in just one click. We designed custom templates with a combination of services. No input required from the users; just options presented to them plainly. Some examples:

· Email me popular flow templates right now (Just email)
· Get the daily weather forecast for Seattle (Notification + Weather)
· Get a daily digest of the top CNN news (RSS + Email)
· Get daily inspiration from Brain Pickings (RSS + Email)

As an added example, we’re capable of user-friendly contextual design. Say a user comes from a Microsoft Office product — we can easily detect that and provide more relevant templates, reducing decision-making stress and providing instant value. Ta-da!

Updated empty page experience

Becoming an integral part of the Microsoft system

As a Microsoft product, there’s great potential for expanding our experience throughout the ecosystem. Microsoft Flow plays a unique role in connecting services and simplifying workflow at various levels to empower our users.

In the last three months, Office 365 and SharePoint integration has driven a lot of valuable growth. And we’re working on the making the integration experience as seamless as possible. As designers, we love thinking in terms of cohesiveness and patterns. For this, we defined a set of principles:

· Make it part of partners ecosystem
· Keep the experience minimal to reduce cognitive overload
· Consistency over uniqueness

See some of our integration experiences below, and be on the lookout for more interconnectivity with your favorite Microsoft products.

Embeded Flow experience in SharePoint

Embeded Flow experience in PowerApps

Flow’s first run experience for Office 365 users

The journey continues

Every digital experience is a series of steps and actions. Successful designs make those experiences feel simple, intuitive, and enjoyable. And that journey is a continuous dialogue between the user and the designer. It’s a designer’s responsibility to understand user expectations and sticking points, respond accordingly, and deliver the best experience in the end.

This is just the start of a long journey for us. We’re enjoying the mountain of data coming our way; embracing the power of real-world insight to make the best design decisions. Data goes way beyond new opportunities and better observations. It’s about the evolution of behavior over time, and remaining curious and studious to all those signals to improve our potential for impact.

Drop us a note with any feedback or suggestions. We look forward to seeing what you automate! 🙂

Written by: Nitish, Unnati, and Danielle