Published on | Prototyping: From UX to Front End — Medium Akhil Dakinedi

This post is a documentation of the “messy middle” of the design process. The goal is to expose the ambiguity, uncertainty, and craziness of it all. It will show how we went about making design decisions in a major feature of our core product by prototyping, testing, and iterating with real users over the course of four months.

The product context here is actually extremely important, so let’s start with a brief background of the company objectives and the business goals which are crucial in understanding the design decisions made down the road.

The context

JANA makes products for emerging markets in order to make the internet more accessible as the next billion comes online.

When I joined JANA as a Product Designer earlier this year, the company had just committed to building a great web browser for Android. The twist was that the app actually rewarded its users with real money for using it, which the users could then use towards recharging their prepaid phone’s data and/or talktime plans right from the app. The audience was users in emerging markets (India, Malaysia, Indonesia, Brazil, and more) who were just coming online and needed a solution to offset the rising data costs in their respective economies.

Advertising spend was moving to digital very quickly and we had an incredible opportunity ahead of us.

The model worked because digital ad spending has skyrocketed in emerging markets lately, and brands were willing to pay for users’ prepaid recharge in exchange for real estate in our web browser. JANA had already established a strong network of partnerships with these advertisers. The web browser was simply the platform that connected the advertisers to the users. Users could recharge their mobile phones for free by viewing a few ads every now and then. JANA’s mission was to make internet free for the next billion, and leveraging their advertising partners to monetize an increasingly growing digital audience in emerging markets was a crucial stepping stone to achieving that larger vision.

India was JANA’s largest and most lucrative emerging market, and it was the country selected where the web browser would first launch.

The company had focused their efforts on their largest and most valuable emerging market — India. Smartphone growth and digital ad penetration were projected to be at their highest in India, so it offered the highest chance of success. For us, this meant operating in India’s currency (the rupee) and working with the telecom operators in India (Vodafone, Airtel, Idea, etc.) to offer users recharge packs.

The product

This was an Android-only web browser, as the market share of Android is significantly higher than iOS in emerging markets than it is in the United States. There was already a beta version of it (called mCent Browser) being worked on when I joined the team. It was sponsored, so there were ads. There were all kinds of ad units in it — the banner ad, affiliate links, interstitial ads, pulldown ads, and more. Although they impacted the user experience, users in emerging markets didn’t seem to mind ads that much. In fact, they seemed so used to it (from what we saw during our user tests) that they expect it from all digital products that they used.

The browser rewarded the users with points for using it, which could then be converted to actual currency to buy recharge packs. The concept is similar to how credit cards and airline miles reward their users with points.

Why a web browser?

Great question. Simple answers:

Intent-based search advertising was an incredibly effective ad unit
Web browsers accounted for a significant fraction of screen time for Android users in emerging markets
The idea of free internet played very well with the concept of a browser as a portal to the open web

Without going too much into the details of how the ad tech industry works, I’ll say that a web browser is extremely attractive to advertisers as a platform to advertise in and is also a very high-usage app category for our users. It was the perfect blend of leveraging search advertising with display advertising in one product, so it made a lot of sense to play in this space.

There was also explosive growth in emerging markets for web browsers that offered free and unrestricted browsing. UC Browser and Opera Mini are only two examples of third-party web browsers that are positioning themselves to take advantage of increasingly fast Android penetration and the nearly two billion users that will be coming online by 2020 in emerging markets. Needless to say, there was a very strong product-market fit here.

There was already a lot of work done around the branding and presentation of the product to consumers and advertisers. The name, the messaging, the copywriting, and the content strategy around the product’s marketing was all backed up by market research as well as consumer research.

As far as a mobile web browser goes, it was pretty standard in terms of feature set and functionality. You could search, type in URLs, bookmark pages, view your history, browse in private mode, and all the other fun stuff. The only real differentiator of this app from other Android web browsers was that it offered recharge packs for free, given the user used it enough. It was definitely a volume business.

Users would earn points for simply navigating to websites, loading webpages, viewing videos, or reading articles. All normal browsing behavior would reward the user with a certain number of points over time. Once the user had accrued enough points, they could convert it to their local currency and actually purchase recharge packs right from the app.

Why points?

This is worth talking about. Why did we even have the concept of points in the first place? Why not just directly display how much currency the user was earning? There were benefits to abstracting the actual money earned through the points mechanism:

For one, we could control the conversion rate of points to currency easily on our end, tweaking it to whatever made the most sense.
Secondly, we could use one unit of earning globally in different countries instead of having to localize monetary denominations throughout the app with that specific economy’s currency.
And finally, using points allowed us to provide the illusion of constantly earning something in large denominations. We’ve all seen credit cards offer first-time usage rewards of 50,000 points which actually equate to as little as $10.00 USD, but 50,000 points still feels like a lot when you’re about to sign up for a card and don’t have a good sense of what the points are actually worth. We wanted to do something similar.

There were business constraints on how much money we could give out per day, per user. Operating at a potential scale of thirty million users meant that we could not simply let users rack up large amounts of cash by having them constantly use the app. Our data insights team had experimented with varying methods of payouts, and settled on giving out Rs. 1.00 per day per user (in India) as the most cost-efficient one. For the sake of testing, we settled on 100 points equating to Rs. 1.00 in the app, which meant that we would be giving out a maximum of 100 points per day to each user. Now the problem was figuring out what the user experience of this points earning mechanism would be.

Screenshots of mCent Browser in January 2017. A fairly basic UI with all the browser options nested in the overflow menu. The banner ad at the bottom bought in a lot of revenue and is what enabled users to receive recharge packs for free, as advertisers were very willing to pay for consistently displayed prominent real estate on the screen. The progress bar on top would slowly fill up as the user accrued points and would highlight when filled up to indicate that the user could now purchase recharge packs.

The problem

The team had already run some basic variants of experiments around displaying progress of points earned per day to the user, and had implemented a very obvious and straightforward one for now — a linear progress bar. It was displayed at the very top of the app and filled up slowly in blue as the user earned points. Once the user had accumulated enough points, they could go to their account and purchase a recharge pack.

The real problem was brought up by our data scientists. Our retention metrics said that users were using the browser to simply earn their points for the day and were then abandoning the app in favor of other browsers. We had to keep the earnings capped per day due to financial constraints of operating at such a large scale, so we needed to change something else here in order to break this “check-in” behavior exhibited by our users.

It was high time to revisit the progress bar to see if there was a better interaction model that we could use to get users to keep using the browser even after they had hit their daily cap.

High-fidelity prototyping

This was an exciting challenge. We were all sold on the value of the product, and we just now had to work on the user experience of earning points in a way that encouraged normal browsing behavior and discouraged the “checking-in” every day behavior.

Ideation

This was the perfect task for the design team. We decided to test a few varying ways of displaying the points earning mechanism as the user is browsing the web. The engineers and data scientists in the company were still toying with some experiments around payout amounts and making the browser work more efficiently. Meanwhile, the design team would be testing new earning mechanics. The plan was to do this remotely with actual users in-market.

Some early ideation sketches on various methods of displaying points accrual. Many of them involved persistent floating buttons or variants of progress displays that highlighted with a momentary micro-animation to signify a change to an active state. My sketches tend to be very loose and not detailed. The goal of these is to simply get lots of ideas on paper quickly to see if two disparate ideas can be combined in some unique way, and to throw out the bad ones fast before pursuing them too far.

I sketched out plenty of rough concepts to see if any of them are worth pursuing. This phase was just lots of exploration to try and narrow down the solution space and nail down two or three ideas that we could build out in a prototype and feasibly test within a quick two-week timeframe.

I filled up half of a sketchbook with all the different ways we could do this. Initially, I had many different, divergent ideas. As I did it more, questions kept coming up around how certain aspects of it would work with our technical capabilities. I got these answered by asking around and it helped constrain the ideation by narrowing it down and helping the concepts converge onto a few solid ones that we felt very strongly about.

The concepts

We narrowed it down to two ideas that we decided would be worth testing:

A new navigation change where users access their account through a FAB, which also updates with temporal animations whenever the user earns points.
A different navigation update with a bottom nav, where the account icon animates and highlights momentarily with a notification dot whenever the user earns points.

A screen flow for the FAB concept. The user sees a welcome message on the home page that they can dismiss, after which the message condenses into a FAB indicating to the user that all the account information is nested in the FAB and can be accessed from it whenever the user wished to. Every time a new page was loaded, the FAB would display a points balance update for 1.5 seconds and then collapse it back down. The intention was for the FAB to disappear when the user scrolls down and re-appear when they scrolled up. Tapping the FAB would bring up a partial view of the user’s account where they could peek at their balance and pull it up full-screen if they wished to do so.Screen flow for the bottom navigation concept. Again, users see the welcome message which can be dismissed. The user continues to browse and after enough points have been accumulated, a notification dot appears on the wallet icon, which opens a condensed version of the partial view to peek at the current points balance. Tapping “Explore” opens up the full wallet (interchangeably referred to as “My Account”) where the user can purchase recharge packs in points.

Due to certain business goals and for the sake of new market evaluation, we had to run these tests with users in Malaysia (not India). The market was similar enough to India that it didn’t have any major UI implications aside from updating the currency denominations from the Indian Rupee (Rs.) to the Malaysian Ringgit (RM), as shown in these flows. We also wanted to test in both these versions whether keeping the wallet section (the “My Account”) portion of the app in points made sense to users or not.

We quickly realized that we would need to somehow fake a browsing flow in the prototype so that users feel like they are using the app like a normal web browser. Additionally, these ideas required really fine tuned micro-animations in order to work properly. I went to my prototyping tool of choice that I had used in situations like this in the past — Framer Studio.

The Floating Action Button prototype

We remotely video-called users in Malaysia and had them open these prototypes on their phones. With some funky JavaScript, I tried my best to ensure that the prototypes would scale properly on any Android device and that the tap zones would behave as expected. Here’s the live link to the Framer FAB prototype if you want to just view it in a web browser on desktop and click around in it.

We scripted the test in a way where we asked the users specific questions on every screen as they spoke their thoughts out loud on every screen. The main dialog on the home screen could be collapsed by tapping “Got it”, which then animated into the FAB. Most users understood this right away and knew where to access their account (or their “wallet”, as it was referred to in the mockups at this time; this was another dimension we were testing). Scrolling would hide the URL bar and the FAB to draw focus to the content, and reverse-scrolling would peek them back in. This kind of dynamic content manipulation was the by far the largest benefit of using Framer.

Since this was a simulated browsing flow, we had to instruct users on the one quick link that we prototyped out a full flow for — the Mudah.my link, a popular online shopping marketplace in Malaysia. As the user taps on the link, their points balance would update with a quick animation and the page would load. I had faked the page-load in sequence with a progress bar as it would look like on a slow 2G network connection to simulate it as close to reality as possible. We tried to test as many dimensions as we could. I had even faked a banner ad infinitely animating at the bottom of the screen to gain insights into whether users would find this annoying or not.

The Bottom Nav prototype

<a href="http://ift.tt/2tORfiV">http://ift.tt/2tORfiV</a>

The bottom nav prototype was similar, but again, tested for different variables whenever possible. Here’s the link to that final Framer prototype that we tested. For this one, the wallet could be accessed directly from the bottom nav, along with the tab switcher, an Offers page, and a link to go back home. Note that there is no banner ad at this bottom here. This is primarily for the ergonomic benefits of not overcrowding the bottom space with too much stuff leading to accidental taps. Additionally, this prototype was updated midway during the tests to display the actual Malaysian currency values in the wallet instead of the points. We wanted to test whether there was a significant difference in user perception and expectations when associating the word “wallet” to currency denominations.

Just like the other prototype, users could collapse the welcome dialog by tapping “Got it”, they could open up a partial view of their wallet balance from whatever site they were browsing, and they could even open up the full wallet by tapping “Explore” in the partial view. The full wallet screen has a slightly different layout here as well to test a newer version of it.

The biggest driver for having the partial view in both prototypes was so that users would not have to constantly keep going back to the home screen to check their wallet balance. They could simply open it up, peek at it, and continue browsing whatever page they were on.

Again in this prototype, the user can tap on the Mudah.my link to open up the faked slow-loading site. The plan was to see if users noticed the subtle animation in the wallet icon in the bottom nav and whether it was prominent enough to draw attention to itself whenever it updated. If the user tapped the wallet icon at this point, the total balance would update and the full wallet would also update with new messaging. Being able to rapidly prototype these conditional logic flows is another huge benefit of using Framer in this situation.

Note that all of these designs were in medium “prototype fidelity”. The welcome message is simply a giant green dialog, the content is all generic, and there’s little to no branding in the experience. We were not trying to A/B test illustrations or the UI look & feel. We simply wanted to validate the earnings mechanisms and navigation structures.

The results

We ran remote tests with twelve users in Malaysia, each lasting for sixty to ninety minutes. We asked users about their general thoughts on the app, what they would like to do, and any opinions about improving the app. Users were able to successfully download and open the prototype in a windowless web browser and showed their phone screens to us over video-calls as they were using it.

So what did we actually find? Our hypothesis for the FAB was that users in emerging markets wouldn’t understand a relatively new design convention, but nearly everyone got it right away. They saw it appear and disappear, and tapped it to access their account too. We had asked users what sites they frequently used, and when we visited some of those webpages on our devices, we noticed that all those sites were all using FABs in some form or another (frequently incorrectly and completely against the Material Design Guidelines).

An example of popular news site in Malaysia using FABs to promote actions on their own mobile website. If we were to use our own FAB on top of this, we would be covering web content and buttons on mobile sites.

This caused some huge complications. If we used our own FAB in the app, it would cover the webpage FAB, which we had no control over. Moreover, webpages had their own banner ads or bottom-sticky footers on the screen, which frequently interfered with the placement of the FAB itself. As a user, being unable to interact with a button on the site that I’m visiting because the browser’s own button is covering it can be enough reason for me to abandon the app and try a different browser instead. Needless to say, we scrapped the FAB concept immediately.

The bottom nav fared better. Users commented that they generally liked seeing all the available options highlighted in sight and not tucked away in the overflow menu. However, not everyone noticed the micro-animation upon earning points. The web content was so distracting that the users’ attention was constantly on the site itself, not the icon in the bottom nav. The results on this were mixed, and we didn’t have enough conclusive evidence to declare the bottom nav as the winner. We did agree that with some tweaking, the bottom nav had a lot of potential and should be explored further with a different earning mechanism down the line.

The biggest learning from this high-fidelity prototype experiment was that users are in a very different mindset when they are in the “browsing” state. Although we faked a website to load and made some hotspots tappable, users found it very easy to immerse themselves in the webpage’s experience. Regardless of how hard the FAB or the bottom nav icons tried to call for the user’s attention through animation, the web content was king.

Next steps

After a quick retrospective, we realized that we tested for way too many variables at once. We were trying to figure out user opinions on the navigation structure, the way they access their account, whether their balance is displayed in points or in the actual currency, and if displaying the exact amount of points earned in the micro-animation made a difference. This is all obvious in hindsight, but it wasn’t at the time — we just wanted to maximize the findings we got. It was difficult to compare the two concepts side-by-side because of these vastly differing elements.

There was a silver lining to the results. If we could get users immersed enough in the browsing experience, they would not abandon the app after they had earned their points for the day. The linear progress bar was so salient that it seemed to scream “Alright, you’re done with me for the day! Go back to your old web browser now.” If we could cleverly disguise the points earning mechanism and allow the user to keep using the app as they would use their normal web browser, then we have a real shot at breaking the check-in behavior problem that we initially had set out to solve.

A change in strategy

The more we thought about the task ahead of us as a product team, the more challenging it seemed. We were, of course, always testing the beta build of the browser with the regular linear progress bar with users all the time. The most consistently revealing finding to come out of these was that there’s always a lot of things screaming for the user’s attention:

The ad units in the web browser
The ads on the webpage they choose to visit
The flashy colors and animations of the webpage’s UI and navigation controls.

Adding even more complex animations and interactions on top of this already cluttered experience to try and tell the user that they are earning points didn’t seem like the right way forward. We needed to redo the entire model of earning points itself.

We were approaching a major deadline for the beta launch of the product and needed something quick. We settled on doing a quick two-week design sprint to burn through all the bad ideas quickly. The first week would be ideation and iteration on ideas to figure out what to prototype. The plan was to then build out two or three quick prototypes to test them out in-person in India by physically traveling there and putting them in front of real users in the second week.

Planning

Reference UI treatments of gamified reward mechanisms as seen in Pokémon Go (left) and Google Maps (right)

In a product like this, the temptation of gamifying the earning mechanism is very strong. We knew the solution space was massive, but needed to narrow down the problem space first so that we were thinking of the right solutions for the product. Specifically, we wanted to test certain dimensions to guide our ideation in the solution space:

Triggers: What causes the user to earn points?
Modifiers: What changes the rate at which users earn points?
Earning indicators: How does the user know that they have just earned points?
Progress measures: How does the user know what they have earned?

Ideation

We used the dimensions to guide our ideation during the brainstorming process. We put engineers, product managers, and designers in a room to get the design sprint going. We did silent brainstorming on post-its, and had people put up a wide range of diverse ideas on the walls while they explained them (without any negative critique). We voted on the ones we liked, and dug deeper into them to explore them further.

Engineers, project managers, and designers all collaborated on ideation sessions to get as many ideas up as we possibly could.

Throughout the entire process, we kept stressing that there were no constraints — anything and everything goes as long as they related to the dimensions. We repeated this process one more time with a second group of people from similar teams at the company until we felt confident that we had a good range of ideas to pull from.

I was, of course, doing my own ideation during this whole time with the way I’ve found works best for me — sketching out lots of ideas and thoughts and putting them down on paper. I tend to vigorously do this for 2–3 days and then look back on all of them to pick out the best ones. Eighty percent of the stuff I draw out or write down never sees the light of day.

I sketched out lots of wildly divergent ideas over the course of two days. Many of the ideas borrowed reward mechanisms and gamification conventions from video games, loyalty programs, cashback offer programs, and airline/hotel points accrual systems. This involved a lot of reading on proven psychology concepts as well.

It’s a process of:

Getting thoughts down on paper to see how vastly different ideas can be combined in ways that we may not have thought of before.
Doing lots and lots of it to maximize the chances of a groundbreaking insight that only comes with thinking about the problem a lot.
Discarding the high-risk, low-reward ideas quickly before chasing them too far down a rabbit hole.

Once I had a set of solid 5–6 concepts in my mind, I re-grouped with the design team to share and discuss them all. We drew them out on whiteboards to visually explain them and see if they made sense, discussed the merits of the psychological foundations the ideas were based on, and sketched out user flows to see if they would tell a coherent story. We wanted to pick 2–3 ideas to see if we could build some testable prototypes out of them in two days.

Some of the very early loose ideation sketches around how we would incorporate the potential candidates for a concept into a UI and screen flow to feasibly and quickly prototype it out. It had to be immediately obvious to the user what was happening and how they were earning. If we couldn’t convert the concept to a UI, we threw it out.

Our selection criteria was:

Concepts that were feasible to test with users
The ones that were likely to elicit helpful feedback — not necessarily the ones closest to reality or easiest to prototype
A set of concepts that tested different dimensions, ideas, and hypotheses
Concepts that seemed most promising or surprising

After some exhaustive debate on what’s most feasible to build and test in the extremely condensed timeline that we were looking at, we picked three concepts and we had a plan.

Low-fidelity rapid prototyping

We had two days and two designers to create prototypes before flying out to India. Deciding on the fidelity of the prototype wasn’t a hard choice — it had to be extremely low fidelity. We were only looking to get feedback about the overall concept, not how it’s executed in the UI. We briefly debated doing paper prototyping in the user tests or printing out screenshots of the mocks to put in front of users, but we realized that in order to get the most out of the tests, we would need users to see it on a mobile phone and have the users physically tap through things.

We settled on using InVision for the tests. We would make static mockups of different states of the screens, link them via hotspots, and have users simply click-through the prototype. Only one or two UI elements per screen would actually be tappable. The efficiency gains of making everything interactive were very low for the amount of time it required, so we all agreed that going with a limited functionality click-through prototype in InVision was the right course forward.

Prototype 01: Anticipation of Reward

Nir Eyal’s acclaimed psychology book Hooked talks at length about how dopamine in the human brain is released not when we receive a reward, but is instead released during the anticipation of receiving it. This is a well-proven psychological concept that slot machines, video games, and randomized card packs exploit to the fullest extent. Through our brainstorming, we tried to come up with ways of applying this notion to the mechanism of earning points.

For this concept to work, we had to make the switch from a continuous points accrual model to a discrete one. i.e., the user would not constantly be earning points for browsing but would instead be making progress towards the next milestone where they would earn a whole bunch of points at once. We wanted to test whether the anticipation of leading up to the moment of earning a large reward was strong enough to warrant a switch to the discrete earnings model.

The screen flow for the prototype. The “My Account” icon on the top left is replaced with a circular loader that slowly fills up towards the next points milestone. The goal was to test whether the anticipation of this reward changed user behavior in any way. Once the user had reached the points milestone, they could tap into it, redeem their points (41 in this case), and view the conversation rate from points to their local currency on the Activity tab of their account.

For the prototype, we created a series of screens where we killed the linear progress bar up top and replaced the “My account” link to the left of the URL bar with a circular icon that filled up as the user got closer to their next points milestone. When the circle fully filled up, the user could tap it to open it up and receive a certain amount of points at once. They could keep doing this over and over again until they had hit their points limit for the day. We even made it so that the user would receive a varying number of points each time they filled up the circle, gamifying the variable reward aspect of it even more and potentially leading to greater engagement.

The simple onboarding flow that we had mocked up to get users familiarized with the discrete earnings model. We knew that most users would simply skip through all of this without reading it (based on user research), but we added it in anyway in order to highlight the importance of the circle and let users form a mental connection between the circle’s visual states, the accrual of points, and purchasing recharge packs, even if it was mostly subconscious.

We had to make a quick simplified onboarding experience to explain to users how this worked and what they needed to do. We knew most users would not read this and simply skip over to the main home screen, but we wanted to reinforce the notion of the circle at the upper left being essential to the points mechanism. Our hypothesis was that even if users skipped the entire thing, they’d recall seeing the circular icon in the onboarding and would instinctively tap it to see what would happen.

The dimension we wanted to test here was to see whether progress towards subgoals would have any impact on user engagement with the product. The subgoals, in our case, were the discrete milestones where the user earned a bunch of points at once.

<a href="http://ift.tt/2tOIcyq">http://ift.tt/2tOIcyq</a>

The live InVision prototype can be viewed here. Ignore the extra space at the bottom of the screen (I left it empty intentionally so that we could use the Android device’s native navbar functionality — which would show up on top of this when viewed on a real device — to navigate backwards in the flow). We faked a browsing flow in Flipkart, the most popular shopping site in India, to simulate the experience of actually surfing the web. Hotspot hinting was disabled for the actual user test, but are active in the link above to demonstrate where the hotspots actually are.

Note that the circle actually fills up extremely quickly in the prototype above. Each page load increments the progress by one-fourth, which is not realistic. If we wanted it to be realistic, we’d have to simulate somewhere between 28–32 page loads in order for the circle to fill up completely, and we did not have that kind of time. Prototype bloat was another issue we were keenly aware of, as load times on the slower network connections in India can be problematic during a user test. So we went with the condensed flow, fully expecting users to comment about the fast pace at which they are making progress and knowing to set aside that kind of feedback to only focus on the overall anticipation towards the subgoal.

We also had lots of ideas for how to make the “My Account” section feel more like an account and less like a screen where the user could purchase recharge packs, but again, that’s not what we were testing for. We learned our lesson from testing too many things at once with the high-fidelity Framer prototypes earlier, so this time around, we were laser-focused on testing one thing at a time. It was this kind of devotion to sticking with one dimension to test per prototype that allowed us to build out three wildly different versions of the points earning mechanism in less than two days.

Prototype 02: Conversational UI

One of our selection criteria was to pick concepts that seemed most promising and surprising. While the first prototype may have been the promising pick, this second one filled the role of the surprising one. A big problem that we were constantly facing with our onboarding efforts in teaching users how to use the app was that most users don’t read any of the content. They simply keep tapping to the next thing.

We knew the problem but didn’t have a very good solution until during one of the ideation sessions, one of our engineers sketched out an idea for having a “virtual assistant” within the app, which would guide the user through the process of earning points and would even assist the user with the flow of purchasing a recharge pack. This seemed to solve quite a few problems we had in a very unique way. My brain started going to town with all sorts of ideas for conversational interfaces. And just like that, the chatbot idea was born.

The messaging UI of Quartz. Users are frequently presented with two options as a choice for an input response. Both options usually trigger the same actions from the app. Having multiple options allows users to project a tone on to the experience and provides them with the illusion of the experience being a back-and-forth conversation, just like texting.

Quartz served as a big piece of inspiration here. The app changed how users consumed news by simply presenting it in a familiar interface. The SMS back-and-forth chat bubble interface is such a prevalent UI convention that users tend to read all the copy in each bubble, even if it is just lightly skimming it. All the text presented in the interface immediately rises to a higher level of priority in the user’s mental hierarchy because it creates the illusion of a person existing on the other end, awaiting a response from the user.

The first-time user experience screen flow for the chatbot prototype. Users are prompted about the existence of the chatbot and are guided through a series of welcome messages explaining the value proposition of the app entirely through text message bubbles.

We called him Rechargebot (placeholder, don’t judge). It lived in the same spot as the circular fill icon was in the previous prototype, and that’s where the user could access it from. A quick onboarding taught users what its purpose was. Upon tapping into it, users would be guided through a series of sequential messages informing them how to interact with it.

The dimension we were testing here was that of abstraction. There is no progress indication of how many points the user currently has. It’s all invisible. The user is simply notified when they’ve accrued enough points to purchase the lowest-denomination recharge pack. We really wanted to test if hiding the mechanism of earning entirely would cause any confusion or not.

Click here to view the live InVision prototype. There’s some funky things going on in the prototype due to device viewport restrictions (including the extra space to save room for the native Android navbar), but it’s enough to get the point across. All the sequentially loading messages are on timed hotspots that lead to new screens, faking them appearing into view (our low-budget solution to making this into a working thing in under two hours). We would’ve loved to do the “…” typing animation before a message appeared, but we deemed it to not be worth the effort required to put it in.

Example screens of the chatbot guiding the user through the process of purchasing a recharge pack. We wanted to see how many users would instinctively tap on the first available option and how many users would try to swipe across to search for more options to input as a response.

We also integrated the process of purchasing a recharge pack right into the chatbot itself. The user wouldn’t even need to leave the conversation to finish the most crucial action of the entire flow — purchasing the recharge pack. We really wanted to see how users felt about being guided through the entire process without ever seeing how many points they actually had. The idea was that once the user had browsed enough, the chatbot would go into a highlighted state, which when tapped would inform users about the newly available recharge they they could purchase. If the user simply wanted to go their account and purchase it manually, they could still do that by tapping the icon in the top-right of the Rechargebot screens.

In Quartz, multiple choices are frequently presented to the user to respond with, only for both of them to do the same thing. It does a great job at providing the illusion of choice by switching up the tone of one of the options to allow users to dictate the mood of the conversation. We did something similar here where often, we’d display two options for the user to pick and yet they would both trigger the same response from the system.

More example screens of the chatbot reinforcing the back-and-forth conversational link between the user and the system. Even if there was only one option to choose from, the user physically tapping it to indicate intent actually goes a long way in making it feel like a conversation and less like the user constantly being bombarded with messages without a chance to respond or change the topic of discussion.

We even had user options for little things like saying “thanks!” or saying “got it!”, even if it was the only action the user could take. We did this to reinforce the conversational link between the user and the system, with the added benefit of getting users to acknowledge the fact that it was a two-way conversation that seemingly reacted to their input instead of it just being a stream of preset messages bombarding the user with instructions. The copy was extremely informal and casual, with the goal of putting the user in a more relaxed state of mind.

The entire team was very curious to see how this would perform in India. We had all read the infinite blog posts about how conversational UI’s are the future, but we had no idea if it would work in an emerging market or even if the users there were accustomed to seeing something like this. This was the one wild idea that seemed a bit too “out there” for us, but we stuck with it. We all believed that there were a ton of valuable insights to be gained from testing it, regardless of whether it succeeded or failed.

Prototype 03: Continuous accrual with feedback

For our final prototype, we decided to go with something that would test something which the other two are not, namely, the dimension of transparency. The former two prototypes hid the actual numeric value of points being earned during the browsing flow, but we wanted to have at least one where users were seeing an actual number tick up as they were browsing (similar to how the current linear progress bar did).

We also wanted to swap the navigation of the UI in this one to draw more attention to a specific animation that we had going on. We went with a bottom navigation and removed the bottom banner for it due to ergonomic concerns of overcrowding at the bottom of the screen. We were very cautious to not overtest too many things at once by changing too many variables, but we felt like this specific prototype necessitated this change in order to properly get feedback on it.

Screen flow of the Continuous Accrual prototype. The UI here was changed to contain a bottom navigation with the link to “My Account” at the center. Users can open and close the partial view at will. The button at the center would animate and update with the users’ point balance every time it changed.

In the center of the bottom navigation, we placed a branded button that opened and closed a partial view which displayed the user’s points balance. The concept of a partial view where users could peek at their points balance during the browsing flow had fared well in the high-fidelity Framer prototypes earlier, so we decided to bring it back with a small tweak here.

As the user continued to browse, the contents of the button at the bottom would update momentarily with a quick animation displaying the numeric value of the points the user had earned front and center. Opening the partial view from the button after this updates the contents of the partial view to display the cumulative points earned thus far. At this point, the user can physically add the points they’ve earned to their account or simply view their account. The points are not automatically deposited to their account because we wanted users to transfer them manually at set intervals, not just let them automatically accrue to a high balance without redeeming any of it (there were some financial and operational constraints around breakage driving this decision as well).

Click here to see this prototype. Try not to click around too fast on this one, there’s a few animations that were hacked together very quickly to fake some page loads. The navbar space at the bottom acts strange during the animation as well. Follow the hotspots to watch the points balance increment numerically in real-time in the bottom button. We wanted to test whether full transparency for points earned still had any potential relative to the other two ideas we were testing.

Between the three prototypes, this was the closest to the current version of the progress bar. The mechanism was still the same, only the design treatment in the UI is different here. We also changed up the browsing flow in this one to see if users’ expectations of points earned is skewed by the type of content they consume.

Ready to roll

A quick word on the design fidelity of the prototypes here. We were not testing for which design treatment worked best. All assets used in the prototypes were made off of existing brand assets that our brand designer had been working on. We didn’t go too crazy with brand exploration here, since the goal of these user tests was to gain insight and feedback on the mechanism of one very specific core feature of the product, not to play with alternative branding or design treatments for the UI elements. We also didn’t make any changes whatsoever to the “My Account” portion of the app. We simply kept it as a separate entity that links to the browser with the points mechanism. All the design and functionality updates for this was being saved for prototyping and testing later in the year.

So after some intense days of ideation, brainstorming, concepting, sketching, and prototyping, we had three solid concepts that we wanted to go with. All three of these prototypes were built in about a day and a half, after which we spent a few hours spent with our user research team to discuss how to build a script around the technical limitations of the prototypes. We wanted to get the most out of the testing sessions, so we had to set expectations correctly for all our participants before the testing began. How much do we tell the users? What order do we show the prototypes in? What questions do we ask them? How frequently do we interrupt their browsing to prime them? We had to figure all this out in the next couple of days as we were leaving for India.

Meanwhile, our user research team was co-ordinating with our participants in India while we put the final touches on our prototypes. As expected, we did not have enough time to fine-tune them to the slightly higher level of fidelity that we would’ve liked them to have. And now that the script was nearly ready, we couldn’t tweak the prototypes any further. The team truly believed that they had prepared the best they possibly could have in the timeframe we had to get this done. And then it was off to India for live user-testing.

In-market testing

A team that conducted the testing in India consisted of five people:

Product Owner
Product Manager
Product Designer (me)
User Researcher
User Testing Facilitator

We got our visas, packed our bags, and headed out to New Delhi. The plan was to do some live-iteration and co-creation with our participants. We knew that we’d get some actionable feedback about the prototypes immediately on Day 1, and we wanted to make all those changes for the next day. So I would take all the feedback and update the prototypes that night in order to maximize the value we got out of them the following day. We agreed to not make any major changes to the prototypes and not create new ones entirely (I still needed to sleep). We would only make minor updates for each one all for testing purposes.

Iteration plan

We still needed a way to regroup at the end of each day and figure out what the most sensible changes to make for the following day were. We would all have to achieve consensus on what changes would elicit the most valuable feedback for the next day. On the night before the first user tests, in a packed restaurant in New Delhi over delicious Indian food, we came up with a list of five questions that we’d want to try and answer at the end of every day:

Does the initial value proposition of earning rewards for normal usage make sense across all the prototypes?
Is the link between points to currency to recharge being clearly communicated across all the prototypes?
What triggers for points earning are working best in each prototype?
How should the mechanism of earning points be indicated in each prototype?
How should progress towards the next available recharge be indicated to the user in the prototypes that it’s applicable?

I also walked the team through all the prototypes and their limitations at this point. There were many technical constraints that we had to work around. I briefed them on the browsing flow that I had simulated, what hotspots and links worked, and which ones didn’t work. I told the team what types of feedback from the users to ignore and which ones to pay attention to. For instance, users were definitely going to comment at the rapid speed at which they were earning points. They would say that they preferred one prototype over the other because they liked the content of the simulated browsing flow better in one over the other. We had to all be actively be aware of what the user is saying and whether we should follow-up on it or not.

Our gameplan for debriefing after every day of the week.

At the end of each day, we would refer to our notes to see what we had observed, discuss whether we could draw any meaningful conclusions out of the learnings, go over the five questions to see how widely varying the answers to them were across the prototypes, and then figure out what to change in the prototypes for the next day. I would then change them that night and have them ready for the next morning.

The Script

The participants for the test were recruited from Upwork. These were Android users in New Delhi who had signed up to participate in this research study for a cash prize of Rs. 1,500 (roughly $23.00 USD). Each test was scheduled for 60–90 minutes, and we were scheduled to run all of them in a private room at a café in a popular shopping plaza in the heart of New Delhi.. The demographics of the users varied every day as well. We wanted to cover as large of a subset of users as possible, so we had scheduled:

Day one: Blue collar employees
Day two: Working professionals
Day three: High school students
Day four: College-aged students/seeking employment
Day five: Housewives

We worked with our user researcher and user testing facilitator to come up with the script that we would use for the testing process. The rough outline was as follows:

Warm-up, introduce ourselves, tell them why they’re here, and make them feel comfortable and welcomed. Offer them free coffee and snacks.
Inform the participants that they’re about to see three different “beta” versions of an app that aren’t fully functional yet. Tell the participants to give us genuine and honest feedback about their experience and let them know not to feel compelled to be nice to us.
Ask them general questions about how they used their phones: how many devices they had, what apps they used the most, how many SIM cards they had, how frequently they purchased recharge packs, etc.
Show them one of the three prototypes at random. Have them tap through it and ask them to speak out loud whatever they were thinking of as they were going through it.
Lead the user through the scripted experience, answering any questions they had along the way.
We would take notes on anything relevant that the user commented on. Things like what app it reminded them of, any confusion around what the points are for, and any general remarks about the design of the browser.
Redirect or probe users further on a specific thing they said in reference to the points mechanism or browser features.
Take lots and lots of notes. Write all the relevant things down so that we have a well-documented list of everything the user is telling us about how they perceive the product.

Our two test devices pre-loaded with all three prototypes.

We had two Android devices preloaded with all three prototypes on each one and labeled appropriately. We would reinforce to the user after using each prototype to pretend as if the next one they were seeing was an “alternative design” to the one they had just seen. This way, we’re getting feedback on each one independently instead of receiving comparative feedback between two prototypes.

Although most of our participants could read, write, and speak in English, a significant number of them preferred to communicate in Hindi. Our user testing facilitator was fluent in Hindi and provided translation services whenever necessary (it also helped that I understood and spoke Hindi well enough to converse with the participants and step in as needed).

We had a general format for the script, but we also knew we would need to be flexible through the tests to adapt to certain user behaviors and patterns. If we get someone too talkative, we’d have to speed up the process. If we get someone who isn’t speaking out loud enough, we’d have to get them to open up a bit more. If we get someone who’s actually providing great user feedback, we’d try to get them to sketch out their preferred experience on a piece of paper. It was supposed to be an iterative and dynamic process.

Live iteration with user feedback

And so it began. We ran the first two days of tests without changing too much, since we knew we’d need a decent enough sample size to make any meaningful improvements. We talked to six participants over the course of two days, after which we re-grouped, looked at our five questions, followed our iteration plan, and decided on what to change.

The typical scene at one of our user testing sessions, where the team members observed the participant go through the prototypes, took all relevant notes, and documented what was to be changed for the next day. We did this every day for 8–9 hours for five days in a row at a café in downtown New Delhi, India.

The feedback

For the reward anticipation prototype, users commented during the onboarding that it was some sort of “game” where they had to fill up a circle. Most users did not read the text on the onboarding screens (as expected), but they all looked at the accompanying conceptual imagery that came with the text. Once they were into the browsing flow, it became clear to them that the circle was filling up. However, four of the six users didn’t think that the icon was tappable in its filled state. They said they would not tap it and would keep browsing instead. We took note of this and simply asked them to tap into it in order to proceed with the rest of the test.

Users did not seem to understand the highlighted affordance offered by the circle’s active state. They were so engaged and so in the mode of “flow” with the web content that they didn’t pay any attention to the circle whatsoever.

In the prototype, the user is taken through the circle filling up twice, and the prototype rewarded the user with different amounts both times (testing variable rewards). But instead, his led to user confusion and users commented that the sites they visited during the second loop earned them more points. This was clearly incentivizing gaming the earnings mechanism. Users would get too engrossed in figuring out what was giving them more points instead of just continuing to browse normally, which we did not want.

Our dear friend Rechargebot, however, was not faring so well. Users were left confused when recharge packs suddenly became available for no apparent reason. Abstraction of the earnings didn’t seem to be working, because users didn’t know what was happening. They seemed to understand the conversational UI interaction model just fine and well. They interacted with the message prompts and found it engaging. They actually read all of the content, which we expected and were satisfied with. But overall, the whole concept of hiding the points being earned entirely wasn’t working.

The dimension of abstraction that the Chatbot prototype was trying to test (left) did not seem to resonate with users at all. It led to lots of confusion and ambiguity. The Continuous Accrual prototype (right) seemed to draw the user’s attention to the button whenever it updated with points but didn’t change the users’ behavior in any significant manner.

As for the Continuous Accrual prototype, users unanimously commented that they liked the bottom navigation. They liked seeing all the options available to them up front. But again, users commented that the button here didn’t seem tappable. It was animating with the points the users were earning every time they navigated to a new page, which some users noticed but others completely missed. Overall, users still didn’t think it was tappable. Some users thought that it was just the app’s logo and didn’t expect it to be interactive. So it was mixed feedback on this one, but we felt like with a couple minor tweaks, this had potential.

As for the browsing flow, it surprised us how engaged users got with the content on the webpages, given it was all heavily scripted. We literally told the participants what to do next, for example: “Now let’s pretend you’re trying to search for shoes…” or “Say you want to purchase a pair of headphones,” and even then, users commented on the options available to them, they critiqued the design of Flipkart’s mobile experience, and wished to continue to the checkout flow. In the prototype with the politics webpages, users commented on their political preferences and what headlines they agreed or disagreed with.

The extremely strong engagement with the web content may have been a big contributing factor to the users not noticing the visual triggers of the circle filling up and the buttons going into their active states, as the web content quickly and effectively steals the users’ attention as soon as a page loads.

The updates

Based on all this user feedback, we decided to make a few changes. The major one was to stop testing the chatbot (R.I.P #killyourdarlings). Although the conversational UI was working and was miles ahead of the other two for getting users to read the content on the screen, the dimension of abstraction that it tested for was not performing well at all. It led to more confusion, if anything. So we gave up on it. Arguably, we could’ve blended portions of it with one of the other prototypes, but we had agreed to not make any major changes, only incremental small updates to the prototypes.

Our revised version of the Reward Anticipation prototype, now featuring a bottom nav and an icon update to look more like a profile icon. I made the highlighted state affordance as prominent as possible, just to see if users would still miss an icon like this that really pops 200% more. Everything else in this prototype was tweaked to work with the bottom nav. We now had this and the Continuous Accrual prototype both feature a bottom navigation, so they were easily comparable to each other.

As for the Reward Anticipation prototype, we switched it up to use a bottom navigation in order to better match the continuous accrual prototype. We put the circular filler in the middle of it, just like in the other prototype. Since we were now down to two prototypes, we wanted them to visually look as similar possible so that the only difference would be the actual earnings mechanism. The way it worked was still the same, with the circle filling up as towards the next points milestone. We also made the the filled-in state of the button more visually prominent in order to provide a stronger tap affordance.

The other change we made here was to update the circle’s visual itself to include a different icon, now more representative of a profile icon indicating that it’s tappable. The goal was to make it look more like it belonged in the bottom navigation and less like it was an intentional branding of the app.

The Continuous Accrual prototype only got minor updates in that the button at the bottom now looked less like an app icon. I updated it to look more like a profile icon inside so that user’s didn’t mistake it for some brand element.

The Continuous Accrual prototype only had the minor visual update of changing the app branding to be more like a profile icon. Functionally, everything else was the same. There were also lots of minor copy updates throughout both the prototypes to clarify things better, as well as modifications to our script in order to get better feedback from the users. Additionally, we updated the browsing flow in both the prototypes to be the exact same. After making these changes, we tested the two prototypes with participants again on the third and fourth day.

More feedback

The feedback this time around felt more focused. People were responding directly to the button in the bottom navigation and were commenting on how it was changing over time. As testers, we also started asking better questions and felt like our understanding of the users was getting stronger by the day as we spent more time with them and in their environment.

More user testing scenarios where we were going through the prototype. This user was showing us how many different devices and SIMs he had as part of our preliminary demographic questions.

This time around, it was a lot clearer to users what was happening. For the Reward Anticipation prototype, users said that the circle at the bottom was “filling up” as they progressed through the browsing flow. When we asked them what it meant, they tried tapping on it to see what happens, which displayed how many points they had just earned. When we asked them if this was obvious, they said no, to which we followed-up with “Where would you expect to see it?” to which the unanimous response was “On the home screen”.

In the Continuous Accrual prototype, the visual trigger of displaying the points earned turned out to be a distraction more than anything else. It was great to see the points being earned to the user, but the information wasn’t useful as they were browsing. Again, it was the web content that the users were focusing on. They only really needed to know their points once they had accrued enough of them to purchase recharge packs. Once again, when we asked users where they’d prefer to see their points balance, most of the participants instinctively responded that they would like to see it on the home screen.

Final updates & testing

So after four days of testing, we had hit what felt like a breakthrough. Users preferred to see their earned points somewhere on the home screen of the browser. We suspected that this was in response to other popular browsers on the market (UC Browser, Opera Mini, Chrome) and how they display relevant content on the home screen. A browser homescreen was kind of a landing page to launch the user into their browsing experience. Of course, this is only what the users said they would like. We’d still need to test if it would actually work.

So on the penultimate evening, I took the two prototypes and merged them into one master prototype. This was not a massive redesign, it was actually lesser work since we were taking away the functionality of a persistent visual progress indication throughout the entire browsing flow.

Our screen flow for displaying the user’s points balance on the homescreen of the browser as a card. We determined that 100 points would be the threshold at which users could add points to their account, and that the button would be disabled until then. As the user was browsing, the total points balance would tick up.

We kept the browsing flow and the points mechanism the same as it was in the Reward Anticipation prototype, but added one slight twist. On the homescreen, we displayed a card at the top which updated as the user accrued points. Every time the user hit a points milestone, the homescreen card would update with the new points balance. There was also a redesigned “Add to My Account” button, which would remain inactive until the user hit the 100 point threshold, after which it would become enabled. Some helper text was also added to inform the user about how the points mechanic worked, and a “View My Account” button was also added to the top right portion of the card.

We did not do anything in the UI to indicate to the user that there was a card on the homescreen that was updating at the same time as the full-screen points milestone. We simply wanted to see whether users would instinctively go to the homescreen to view it.

<a href="http://ift.tt/2uMNbNn">http://ift.tt/2uMNbNn</a>

On the final day, we tested this prototype. As expected, users commented on the web content a lot and engaged with it heavily. The circle at the bottom was updating towards the user’s next points milestone, but most users didn’t even notice it. They noticed the extremely salient active state, and mentioned that they thought they had just earned points. When we asked them what they would do next to check it, they all instinctively tapped on the Home button in the bottom tab bar to check their balance.

All users understood the points card at the top of the homescreen. They were able to explain to us how it worked, where they would go to see their account, and at what point they could add points to their account. This prototype was testing very well. Users also mentioned that they wanted to check their points balance in the middle of a browsing flow, instead of being forced to go back to the homescreen every time they wished to check their points balance. We felt that the partial view would work really well here.

The final homescreen card redesign, with a partial view popping up from the “My Account” icon in the middle whenever it was activated. We toned down the bottom nav to be less distracting and more streamlined, as the colorful bar and icons were turning out to be very distracting. This felt a lot more cleaner and accessible not just to us, but also to our users.

I managed to make these changes during the final set of user tests, and we simply showed the mockups to the users, asking them what their thoughts were. The partial view was just the homescreen card displaying as a bottom sheet from the bottom tab bar. User feedback was extremely positive and their immediate next question was “When can I use this app?”

Retrospective

So after five intense days of testing, iterating, and prototyping, we had something conclusive. The prototypes naturally converged into one final prototype, but this was never the intent. We went in with three vastly different ideas to collect insights and learnings about various dimensions that we were testing with users. It was just a happy coincidence that we ended up landing on something that would work in accordance with user expectations on the final day.

A co-creation session with one of our user testing participants who is seen here sketching out an example of how he would like to see his points balance displayed in the app. A lot of our revisions and tweaks came from sessions like this.

We then looked back at our five questions one last time to see what we had learned about how our product worked with user expectations:

1. Does the initial value proposition of earning rewards for normal usage make sense across all the prototypes?

Absolutely. In our introductory questions, we had asked users whether they would use this browser over their current default browser. Nearly all of them said yes because this one offered rewards for usage whereas others didn’t, which is strongly indicative of a good product-market fit.

2. Is the link between points to currency to recharge being clearly communicated across all the prototypes?

Yep. After users tapped the “Add to My Account” button in the applicable prototypes, they were taken to the Activity section of their account, where they saw the raw conversion of points to rupees. All users understood what had happened and what they could now do with the currency.

3. What triggers for points earning are working best in each prototype?

We found that this actually doesn’t matter. The visual triggers ended up being distracting more than anything else. The homescreen card worked the best, and the mechanism here was to actually hide the visual trigger until the user decided to manually check it on their own.

4. How should the mechanism of earning points be indicated in each prototype?

It should be indicated in the background. Not during the browsing flow. There was no significant different whether we had a circle filling up informing the user of their next milestone. The dimension of subgoals was not working and the best way to indicate points being earned was to simply let it accrue in the background.

5. How should progress towards the next available recharge be indicated to the user in the prototypes that it’s applicable?

We did not get any conclusive evidence to make a statement on this. There was no indication from either the prototypes or the users about whether seeing their progress towards the next available recharge pack would have made a difference in their browsing behavior.

Back in action

So we came back to Boston with a solid idea, shared our findings with the rest of the team at JANA, and got cranking on actually building this out. The larger team was very pleased to see the results of the testing and seemed optimistic about the new experience. We allotted two weeks for any design iterations and tweaks and to figure out any back-end changes that would need to be made in order to get this new UX of points earnings to work.

More ideation and sketching on how to simplify the visual design of the homescreen card while also making it feel fun and interesting. It had to strike the right balance between usability and aesthetics, which we were striving for.

One big decision made almost immediately was that we would still have continuous background accrual of points, not the milestone-based discrete earnings that we tested for the dimension of subgoals in the Reward Anticipation prototype. There wasn’t enough evidence to make a case for users being motivated by subgoals, so we decided to keep continuous background earnings for now. It was now time to see if we could update the visuals of the points card itself to make it look a little nicer.

Various mockups of iterations that we had done with the homescreen card. All of them had pros and cons and were weighed against the implementation effort required. The larger cards felt like they were too dominant and were taking over the homescreen, but we liked them for their visual appeal. The points balance was more immediately visible in the smaller card versions. I used our existing brand guidelines for this to see what the simplest assets I could use were while still conveying the brand voice properly.

Working with our brand designer, I played around with some visual refinements to the points card. The goal was to explore if there was a better way of organizing the information in there to be more efficiently presentable to the user. We incorporated some of the original branding for the browser in here, as it was the only spot in the homescreen that we owned and provided the largest opportunity for branding.

We got lots of feedback around the prominence given to certain buttons and the overuse of certain colors. We played around with the idea of users being able to boost their points by undertaking certain actions, and also greeting them by name on the card itself. Note that the entire right side of the card was intentionally being left blank to accommodate future ad units.

An example mockup of a swipable ad unit displayed within the points card done by our brand designer. The points card was essentially a design system that had to scale to accommodate any future add-ons inside of it, which is why we left the right one-third of it empty in all our iterations and revisions. Ad units fit in very well here.

Now that we had validated the user experience and product direction of the points card, we felt that it could serve a dual purpose for monetization by holding a swipable gallery of ads from advertisers, similar to how Facebook and Instagram do it. We all felt like we could do better than a generic banner ad at the bottom of the screen, so we were desperate to explore some more interactive, playful ideas to display more engaging ads to our users.

One piece of feedback we kept receiving from both users and other members of the team was that the card simply felt too large. Especially on devices with smaller viewports and lower resolutions, it would take up almost two-thirds of the screen, pushing the popular quick links to almost below the fold. Additionally, it didn’t make a lot of sense to use the same level of button prominence for a navigational action and a transfer control (in reference to the “View My Account” & “Add to My Account” buttons). All of this was great and totally valid feedback.

We ourselves had felt like we were going a bit too far with the changes on the card, whereas we only really a simple change or two. We focused on the branding of it and decided to tone it down. There was a lot going on in the background of the points card that we tested in the prototype, so we experimented with using a minimal version of the waves at the bottom.

The final design of the points card. A full-width card with the focus being on the number of points earned. I tweaked the visual hierarchy of this quite a bit to nail down the right relationship between the elements in the card. Even when squinting, the points balance should be the most prominent thing in there. It was around this time also that we updated the points threshold to be 1,000 points instead of 100 points, so the mockups from here on out will show 1,000 points being shown as the threshold.

We went with a full-width points card that stretched to the edges of the viewport, and was attached to the URL bar on top. We shrunk it vertically so that it wasn’t taking up so much real-estate, and we fine-tuned some of the margins and padding around it. We knew that this would have to still accommodate ad units in the future, so we built it in a way that would ensure that the ad units would still fit.

An example mockup of the shortened, smaller points card being able to accommodate the same swipable ad units that were shown in the larger Uber mockups.

We also had to work around some technical constraints around how the architecture of the recharge system worked in the “My Account” section so that it worked while providing user feedback about the system state along every step of the way. Users could choose to add points to their account from either the homescreen card or the partial view, and we had to create intermediate states for all these use cases.

The screen flow for using adding points to their account from the partial view. This worked exactly the same way as if they were adding it from the homescreen card. There’s an intermediate state where the button itself updates to let the user know what’s happening and the user sees a success message once the transfer is complete. Then, the conversion factor is displayed in the Activity tab of the user’s account, just like in the prototypes.

At this point, it was determined that we would be using 1,000 points as the threshold for when users could add points to their account. This number came out of research that our data insights team had done into what made the most sense to give out in terms of points while trying to maximize user payouts and trying to minimize our costs. This was even better, as users would see their points balance increment by a larger number each time, effectively providing the illusion of progress.

I’d also like to address the fact that the final designs actually do have a banner ad at the bottom. Throughout the entire design process, we were desperately trying to avoid the overcrowding situation at the bottom with the Android native navbar, the bottom nav, and the banner ad all asking to be fat-fingered by users. But removing the banner isn’t a simple process. It was actually generating a fair amount of revenue for the company. Additionally, this type of ad unit has been around so long that the ad tech industry (including Facebook Ad Network) has developed many automated mechanisms of filling it with different types of targeted content that works surprisingly well. Simply ditching the banner ad in favor of a swipable ad gallery isn’t as easy as one might expect. The industry simply isn’t setup that way yet and it will take a couple of years to get there. This is a perfect example of design having to temporarily compromise the user experience a little in order to first meet the product’s monetization objectives to sustain growth and stay afloat.

Testing, Releasing, and Shipping

So we had a final design for the first version of this. We built out all the screens, spec’d them in Zeplin, walked everyone through the functionality of it, and handed it off to our engineers. But before we simply launched it to all users, we had to back-up the strength of the user experience with quantitative data as well. So we decided to run four different variants of this new points experience in an experiment:

Variant 1: 25% of users still see the old linear progress bar experience
Variant 2: 25% of users saw this new experience with the manual “Add to My Account” button
Variant 3: 25% of users saw this new experience with the manual “Add to My Account” button and a slightly faster earnings rate at the beginning which slowed down as they approached the threshold (as opposed to linearly scaling it)
Variant 4: 25% of users saw this new experience with an automatic deposit function, where the points were automatically added to the user’s account

We already had qualitative data for all of this, we were simply looking for validation with real numbers as well. Variant 1 served as the control, Variant 2 was the new experience as designed, Variant 3 attempted to see if tweaking the earnings rate affected retention or time-in-app in any significant manner, and Variant 4 tried out a version where points were automatically added to the user’s account.

The version where points were automatically added to the user’s account. There is no “+ Add to My Account” button here since the points get automatically transferred when the user accrues 1,000 points. The card simply displays the balance.

This was the first time I had worked on such a major redesign of an app operating at such a big scale, so I was very curious to see the results. We ran this experiment for about 2–3 weeks to determine if any of our important metrics moved or not. And at the end of it, we were happy to find that Variant 3 (the one that we designed) won out by a huge margin:

Average time-in-app ticked up by 55%
Day 7 user retention jumped up by 36%
Day 7 median time-in-app jumped up by 60%

These numbers are in relation to what they were before, but this was a pretty significant uptick from before. The new experience with a modulated earnings curve seemed to increase the amount of time users spent in the app and how long they kept using the app. The Day 7 metrics are only displaying the users who used the app for at least seven consecutive days. These metrics — especially the time-in-app jump — are strongly indicative of the fact that users are no longer using our browser just to “check-in”, but are actually beginning to form regular browsing habits by simply browsing to earn points.

If you download mCent Browser from the Play Store right now (as of July 2017), you can try out the new experience for yourself. The product is still technically in beta. If you’re outside of India, you may not be able to go through the entire flow of purchasing recharging packs, but you should be able to sign up, browse around, and warn points. We’ll be iterating and updating the points experience as well as other core components of the app over time, so if you’re reading this a few months from now, you may see something that we found works even better.

The total timeline for all of this, from when we decided to experiment with high-fidelity prototypes, to changing our strategy and testing low-fidelity prototypes in-market, and to building and shipping it was approximately four months. We were able to move quickly to churn through the bad ideas fast by designing, iterating, and testing at the appropriate levels of fidelity. The goal was to learn and improve the product from real user feedback, which I think we accomplished with a relative degree of success.

The messy middle

It wasn’t a perfect process. We made rushed decisions at key points during testing without fully validating them. We did not do a great job at eliminating our own bias in selecting one concept over another. We didn’t fully randomize the user demographics through the week in order to get uniformly distributed feedback.

But at the end of the day, we were mostly right and our redesign moved the key metrics as we had hoped. We talked to users, we probed them on insightful things they were saying, we iterated and updated our prototypes based on their feedback, we tested a set of ideas to validate different dimensions, and we confirmed our qualitative findings with actual quantitative data at scale. And that was good enough. The truth is, it’s messy and uncomfortable. But the more you do it, the more you get accustomed to the process and you learn to thrive in the chaos of it all.

We’ve all seen design talks and presentations about how a team went from a formulating a problem and following a seemingly perfect plan on a very precise timeline to come up with a solution that did well. But the truth is, the design process isn’t a clear-cut path that tells you exactly what to do next. There are decisions made with uncertainty, ideas that don’t provide the right feedback due to the way they were executed or tested, and unexpected groundbreaking insights that could happen at any time during the process. You simply need to repeat the process of creating divergent ideas at the beginning, eliminating the bad ones, and converging onto the ones that seem to be working. It’s rapid iteration done right.

Me, hard at work sketching out some of the prototyping ideas discussed earlier.

As a rising designer, I’m always on the hunt for documentation about the actual process followed by tech startups and design firms, not the one they made-up after the fact. I want to look at the overall path taken to see where a crucial decision was made to turn the tide of the outcome. I want to see what kinds of things work and what designers shouldn’t waste their time doing. I want to learn from mistakes already made so that there’s no need to repeat them.

And there’s a severe lack of this documentation anywhere online. So I hope this serves as a useful reference for anyone reading who’s also curious about the same things. Due to the subjective nature of design, it’s incredibly difficult to make the right decisions at the right time. But with the right team, it’s definitely possible to get better at the process of taking loose user feedback, turning it into meaningful insights, and creating actionable design prototypes out of them.

If there’s one big takeaway I can give to people after all this, it’s that you need to actually talk to your users. If your team keeps making decisions based on personal preferences and what they think will work without having actual user data to back it up, it simply won’t succeed. This is even more true when designing for an audience outside of the country and culture that you yourself live in.

Most non-designers have some understanding of the design process. They can comprehend the concept of divergent ideation and convergent thinking. But that’s an extremely theoretical, oversimplified way of looking at it; it almost never works out that way. In reality, the variation in ideas is a lot wilder, the peaks and troughs go higher and lower, and the whole process is stretched very thin on the timeline. It takes a lot of tough decisions and some luck with applying the right insights gained to truly converge on the right design solution for the product.

Finally, I hope that this gives some perspective in what a designer’s role in the design process actually is. The end product is but a small sliver of all the scrapped sketches, concepts, and prototypes. Up to eighty-five percent of the work I did here will never see the light of day, and this is true of any design project. The only way the world gets to know about it is through outlets like this. This is why I’m not the biggest fan of dribbble shots that only show the prettified final look & feel of the product and never talk about the design thought process that led to it.

So thanks for reading if you made it all the way down here. I hope this has informed you of something about the design process that you didn’t know before. And I hope that it has demonstrated that design can’t work in isolation. It is simply one of the many pillars that hold the product up. The best way to move forward and make meaningful changes is to acknowledge that there’s always room for improvement and actively get everyone on-board to pursue that goal as a team. Good luck!

Gamifying a web browser — The Design Process was originally published in prototypr on Medium, where people are continuing the conversation by highlighting and responding to this story.

Gamifying a web browser — The Design Process

Cool stuff, every week

More Articles