Understanding spatial cognition makes VR better

Hi! I’m Chris, a researcher on the VR team at Shopify with a background in Cognitive Psychology, specifically perception and visual spatial working memory.

Perception (the experience of what we see, hear, and feel) is discussed at length in the VR community, and rightfully so, but one topic that gets less attention is cognition. I’ll grossly oversimplify cognition as:

Attention, memory, and processing — basically all the gears and cogs grinding away in our brains.

(Image via UBC)

Understanding human cognition can be extremely valuable when developing VR experiences. If you’re not a psychologist or cognitive scientist this may seem daunting, but you don’t need a PhD to make good VR experiences. You need only be mindful of the power and limitations of human cognition.

Everyday we’re discovering more ways to improve VR by understanding how humans process and attend to the world around them. We’re also discovering ways in which the unique qualities of VR can aid cognitive performance. Have you ever wanted to have a super-powered memory?

Spatial memory

Humans are visual and spatial learners. In most contexts, vision is our dominant sense. For example, for many of us the names of high school friends and teachers, locker codes and 10th grade chemistry have faded with time. However, I’d be willing to bet you could still draw a relatively accurate layout of your high school. Information gathered visually, specifically spatial information, just seems to stick better. This is all to say vision has a special status in our brain and presenting information visually and spatially can be the key to enhancing memory.

As author Joshua Foer details in his book Moonwalking with Einstein, the world’s most competitive memory champions don’t rely on innate talents to perform incredible memory feats. Instead, these champions have rigorously practiced simple techniques to anchor difficult to remember information to familiar people & places. Imagining these “anchors” of people and places in absurd situations makes them even more memorable. Psychologists and Neuroscientists have reinforced that these memory champions are not outliers in regards to intellect or brain structure. They have merely honed their craft at leveraging their spatial memory.

(Image by Jeremy Dawson)

If I give you a list of ten objects and their locations in a hypothetical home, for example, “The blue bowl is in the cupboard under the kitchen sink,” then quiz you on their locations ten minutes later you may get half correct. However, if I give you the same task in a virtual environment your retention rate is likely going to increase. In the virtual environment you can explore each room and see each object in context, surrounded with rich spatial information. Cheryl Grady and colleagues suggest that pictures induce a more elaborate or associative encoding than occurs with words.

This all makes sense from an evolutionary perspective — vision came first. As a species, we had to remember where we stashed food or where the best berries grew long before we had the ability to use language or think in words. The visual centres of our brain are much more deeply routed with many more synaptic connections.

Why is VR special?

Remembering things visually and in the context of the space around us just comes more naturally. And this is where VR can shine. VR enriches the layers of vision, space, and user driven exploration. All of these layers help to strengthen the memories and feelings associated with them.

Where traditional 2D displays have you manipulate a disembodied view point or avatar, in VR you directly control the view with your head. In VR, a view point doesn’t move into a new room, you step into a new room.

(Image via youtube.com/valve)

In reality, we constantly make small head movements that go unnoticed but provide our brains with a wealth of information. Those head movements are lost on a 2D screen. In VR, the head tracking is precise enough that all those sub-centimetre head movements are translated to our visual system in the same way they would be in reality. This engagement of what’s called proprioceptive feedback allows us to feel like we are directly experiencing these spaces making them richer and more memorable.

My Advice

How can you start leveraging this information right now to improve your VR experiences?

• Give information an anchor. Add visual and spatial cues to abstract information or functionality. A great example is how Fantastic Contraption’s menu uses objects instead of buttons and text.

You put a space helmet on your head to access the menu. This takes you to a new environment where physical objects serve as the actions you can take.

• Think ‘Odd Ball.’ If you want people to remember the location of an object in your environment without being explicit, place something odd around it. When people are presented something they didn’t necessarily expect, that visual information can stick in memory better.
• Group similar objects. Be it items in a virtual menu, tool box, inventory, etc. People may forget the location of specific items but if items are grouped together by higher level features they may be easier to find.
• Show people rather than tell them. Spatially organizing the steps in a workflow will support recognition rather than recall. This leaves the users’ working memory free for completing the task rather than remembering the steps.
• Use depth and distance. A unique feature of VR is depth, or the ability to place objects in or out of peoples’ reachable space. Things that are closer or more easily accessible will be viewed as more important.
• Don’t forget about visual design. Use visual cues like colour, lighting, architectural style, etc. to spatially segregate parts of the environment that have specialized functions. Also, provide distinctive transitioning elements, like doorways, between these environments.

These simple techniques should help you craft more usable experiences in VR. Rich spatial cues are one of VR’s unique features and should be leveraged to their full advantage. A basic understanding of spatial memory and other cognitive mechanisms is an extremely valuable consideration in VR development.

If you like these types of posts, be sure to hit the ?. Also, let me know in the comments if you’ve encountered good examples of leveraging spatial cognition in VR.