Published on | Prototyping: From UX to Front End — Medium Rain Lieberman

Throughout me career I had the opportunity to conduct several types of user testings. From guerrilla testing on the streets, to one one testings in an office to remote testing online. Each has it’s propose and benefits. In this article I’d like to focus on the benefits of remote user testing and the steps of how to create one.

The Benefits of Remote User Testing

Confirmability: Remote user testing allows you to test your prototype at the user’s natural environment. Whether it be their home or office, users are more relaxed than they would be in a testing lab where there are other people (the tester) watching them.

Familiarity: Users test on their own device. Many time when conducting a face to face test users are unfamiliar with the device the test was running on. This adds a level of nervousness and they have to “learn” how to use a new device in addition to doing the task. (for example, they own an iPhone and the test is on an Android device. Basic navigation between pages is very different on these devices).

Speed: Testing your prototype remotely allows you to reach a higher number of available users faster. When comparing remote testing to face to face, it can take up to two weeks to find, filter and invite users to come in for a test. conducting a test online will take a maximum of two days, depending on the platform you use and their user database.

Quantity: Most remote testing platforms have a large data base of users for you to filter through. This makes it easier to test your prototype on large numbers that it would having to invite individuals to come in for a test.

Quality: Again, most remote testing platforms allow you to filter users to your required demographics such as: gender, age, location and device. Some even give you user ratings to which you can choose from. (For instance if you want an experienced tester with 5 star ratings).

Design Your User Journey

Start by clearly defining your goal. What is the end goal you want the users to achieve? Where would they start their journey? What would be a natural flow from the starting point to the end point? Are there any other possible scenarios and if so, should you include them in your test?

Creating a Clickable Prototype

Now that you have your user journey it will be much easier to create your wireframes. Mihir Pathare wrote a good article on how to decide the level of fidelity for your prototype. I have tested prototypes of all levels, from paper sketches to high fidelity designs. To test an early idea I would recommend low investment, low fidelity wireframes whereas if you want to test the user’s reaction to a new icon I would recommend a high fidelity design.

Paper prototype for a train booking app

Once your your prototype is ready, I recommend you share the link with team members and ask them to test it. Sometimes there are scenarios you did not think about that will flush out in this process.

How to write questions that give you the best results

Look back at your testing goal and run through the prototype to start forming the questions and tasks you want to give your users. Think what is the information you want to gather from this test? Do you just want to see if they can use a new feature easily or are you interested in their other experiences and opinions?

A Draft of questions prepared for a test

To gain more insight, I also ask at key steps how difficult did they find the task and what were their thoughts. As the test records not just their screen but also audio, you will be able to hear their answers. From my experience it is more effective to ask these questions during the task rather than at the end as the experience is fresh in their mind.

For more help on how to formulate good questions I highly recommend Fabricio Teixeira’s article.

Setting the User Mindset

With every test I run, I like to set the user’s expectations. Framing their mindset tends to make them feel more at ease and therefore the results are more truthful and accurate. Questions I tend to ask at this stage are more general and open. For example, when testing a Wishlist feature for Topshop I asked my users in advance how often do they shop online, do they always buy what they like instantly or do they save items for later.

If you are testing a low fidelity prototype this is a good place to remind them that this isn’t a finished product and not all functionalities will be working. If they end up trying to click on an area that isn’t part of the test they then understand why the prototype is not reacting.

Summary Questions

This is where you can ask your users any additional questions you may have. It’s always good to ask if there is anything you have missed or if they have anything to add. Often times users may have something to contribute or add but unless asked specifically they will not mention it. I ask summary type questions such as: How did they find the overall experience? Do they have any recommendations on how to make it better? Have they seem or used a similar feature in a different place and if so, how do they use it?

Running Your Test

So your test is up and ready to go. You probably worked on it for a while now and are eager to release it to the world. Stop! Before you release your test to all users, I suggest you limit it to a single user and test it once. Even if the questions you wrote are great and your prototype is running smoothly, you may have missed a step or, there might be a confusion caused in the process of building your test. This is not your wrongdoing nor the platform you are testing on. It just happens sometimes and the best way to catch errors like that is by running the test once or twice before releasing it to more users.

Wishlist feature prototype for Topshop

For example, while working at Arcadia Group, we designed a test for a new Wishlist feature for Topshop. We ran over the test with the entire team at various stages before we were sure it was good to go. We released the test for two users initially. When the results came back we realised that although the test was not “bad”, we were not getting the insight we were hoping for. We went back to the drawing board, changes some of the designs and flows and tweaked the questions to be more clear. We then release the test once more for only just a single user. This time the results we got were far more insightful and we were happy to then release it to the rest of our users.

Getting the Results

You have ran your test and you received a multitude of answers, videos and perhaps even a transcript. Now is time to go through all the data and gather the key findings.

Screenshot of a user testing session

An added benefit to the remote testing platforms, such as usertesting.com is that all the data you collected is in one place. It is easy to watch the videos, leave comments, highlight sections and share your findings with team members.

I hope this article has helped you to choose in favour of remote testings and has helped you in creating your own test. If you feel like I missed anything or have anything to add, I’d be happy to read your comments.

Remote User Testing: Why and How to run a test was originally published in Prototypr on Medium, where people are continuing the conversation by highlighting and responding to this story.

Remote User Testing: Why and How to run a test

Cool stuff, every week

More Articles