large_Crossover Series NFTs.png

medium_Crossover Series NFTs.png

small_Crossover Series NFTs.png

thumbnail_Crossover Series NFTs.png

From Photography to NFT Art: Twisted Digital Landscapes with Nate Hill

From Design System to ‘NFT Design System’: Creating TinyFaces

Neurodiversity and Inclusion: Choosing Kinder Design

medium_652ccf22c8853584bde5553986a4eed0.png

small_652ccf22c8853584bde5553986a4eed0.png

thumbnail_652ccf22c8853584bde5553986a4eed0.png

Web5 is here. Goodbye Web3? TBD.

<p>Published on <a href="https://blog.prototypr.io/https-medium-com-aetherpoint-augmented-reality-design-crossing-the-screen-space-divide-f1d3a0004c69?source=rss----eb297ea1161a---4">| Prototyping: From UX to Front End &mdash; Medium</a> Andrew Johnson </p>
<h3><strong>Augmented Reality Design — Crossing the Screen Space Divide</strong></h3>
<p><em>(Originally posted on </em><a href="http://www.aetherpoint.com/blogpost/augmented-reality-design-crossing-the-screen-space-divide/"><em>aetherpoint.com</em></a><em> where you can find footnotes.)</em></p>
<p>Currently, we have plenty of great digital flat screen interfaces and fun Augmented Reality experiments, but there are few examples of user interfaces that traverse 2D and 3D spaces when efficient for a user. Typically, flat UI is constrained to flat planes and most AR objects float independently within a scene. Interfaces that can switch between 2D and 3D give us the opportunity to take advantage of both spaces.</p>
<h3>Tied to Space</h3>
<p>Our design mediums are grounded and tied to the world around us. Video games have been using a <a href="https://www.gamasutra.com/view/feature/132674/game_ui_discoveries_what_players_.php">variety of representations</a> to communicate within a world and it’s space. These range from <a href="http://tvtropes.org/pmwiki/pmwiki.php/Main/DiegeticInterface">diegetic</a> <a href="https://en.wikipedia.org/wiki/Head-up_display">heads up displays</a> and in game UI that both the player and character can see, to non-diegetic UI that only the player can see.</p>
<p><img alt="" src="https://cdn-images-1.medium.com/max/700/0*mGdTONYKi88GYtgt.jpg">This is a Diegetic heads up display from the <a href="https://en.wikipedia.org/wiki/GameCube">GameCube</a> game <a href="https://en.wikipedia.org/wiki/Metroid_Prime">Metroid Prime</a> released in 2002. Both the character and player experience the same set of information through a <a href="https://en.wikipedia.org/wiki/First-person_shooter">first person view</a>. It would be confusing for any of the HUD information to move independently because it’s fixed to the character’s helmet view and by extension the player’s perspective.</p>
<p>While games bend and break the laws of nature for game mechanics and storytelling, general concepts like depth, perspective and volume are still rooted in our real world. In the same way, film carries us through 3D space with consideration to composition and <a href="http://www.op-art.co.uk/history/perspective/">perspective</a>. ‘Graphical space’ has been explored in design with things like <a href="https://en.wikipedia.org/wiki/El_Lissitzky">El Lissitzky’s</a> <a href="https://www.moma.org/interactives/exhibitions/2010/online/#works/01/19">Proun Room</a>, designed in the 1920’s. Graphic, User Interface and Interaction Design often employ <a href="https://material.io/">space and depth</a> (along with <a href="https://medium.com/@mwcrowley/in-defense-of-skeuomorphism-26801713915">more representational types of skeuomorphism</a>) to communicate emotion and behavior. Even if interfaces try to avoid skeuomorphism, UI interactions and <a href="http://the12principles.tumblr.com/">transitions</a> are still based on <a href="https://blog.prototypr.io/6-animation-guidelines-for-ux-design-74c90eb5e47a">behaviors</a> grounded in physical realism.</p>
<h3>Medium-Sized Spaces</h3>
<p>Each medium shapes how things behave in it’s ‘world’ which in turn gives people expectations of how it should behave. <a href="https://en.wikipedia.org/wiki/William_Morris">William Morris</a>, a contributor to the <a href="https://en.wikipedia.org/wiki/Arts_and_Crafts_movement">Arts and Crafts movement</a> is attributed to the quote:</p>
<blockquote><p><em>“You can’t have art without resistance in the material.”</em></p></blockquote>
<p>Sidestepping the question of what art is, it’s clear that our <a href="http://nowviskie.org/2013/resistance-in-the-materials/">mediums invoke strong forces</a> on how we create things. <a href="https://en.wikipedia.org/wiki/Marshall_McLuhan">Marshall McLuhan’s</a> coining of the phrase <a href="https://en.wikipedia.org/wiki/The_medium_is_the_message">“The medium is the message”</a> is the classical line of thinking applied here. Mediums shape our content/form/function based both on how things fit into people’s lives<a href="http://www.aetherpoint.com/blogpost/augmented-reality-design-crossing-the-screen-space-divide#fn:1">(1)</a> — as well as technical and physical constraints around how the information itself is delivered<a href="http://www.aetherpoint.com/blogpost/augmented-reality-design-crossing-the-screen-space-divide#fn:2">(2)</a>. As each one of these mediums (applied, we can think of them as disciplines) develop on their own they create new things and methods (sometimes radically different, beautiful things) and push against their medium.</p>
<h3>Knocking Down Walls</h3>
<p>As mediums develop and overlap however, <a href="https://en.wikipedia.org/wiki/Hyperreality">distinctions between disciplines get more fuzzy</a><a href="http://www.aetherpoint.com/blogpost/augmented-reality-design-crossing-the-screen-space-divide/#fn:3">(3)</a>. Augmented / Mixed Reality <a href="https://jiaxinwen.wordpress.com/2017/03/30/non-diegetic-ui-design-for-vr-games/">blur the lines</a>, allowing objects and information to be ‘placed’ in a personal or <a href="https://medium.com/super-ventures-blog/multiplayer-ar-why-its-quite-hard-43efdb378418">multi user space</a>.</p>
<p>AR benefits from borrowing techniques (similar to video games combining interaction, audio and visuals to tell stories) from film, UI design and game development. Naturally, design conventions and patterns<a href="http://www.aetherpoint.com/blogpost/augmented-reality-design-crossing-the-screen-space-divide#fn:4">(4)</a> develop as a combination of the previous mediums in a new unique space. These disciplines increasingly overlap with AR and other spacial, context aware approaches. Here, we can continue to look at what current approaches have to offer while learning from some of the amazing work done before us.</p>
<h3>AR: 3D Visualization</h3>
<p>The idea of using three dimensions to provide spacial context while visualizing information is a topic that’s already been explored. <a href="https://www.aiga.org/aiga/content/inspiration/aiga-medalist/muriel-cooper/">Muriel Cooper</a><a href="http://www.aetherpoint.com/blogpost/augmented-reality-design-crossing-the-screen-space-divide#fn:5">(5)</a> and her students pioneered a <a href="https://walkerart.org/magazine/muriel-cooper-turning-time-into-space">wide range of this work</a> at the <a href="http://museum.mit.edu/150/115">Visible Language Workshop</a> in 1975 (and after it merged into the <a href="https://www.media.mit.edu/">MIT Media Lab</a> in 1985). <a href="http://www.inventinginteractive.com/2010/02/01/information-landscapes/">Information Landscapes</a> was far ahead of its time, employing new modes of interaction in 3 dimensions with data visualization and typography.</p>
<p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FQn9zCrIJzLs%3Ffeature%3Doembed&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DQn9zCrIJzLs&#038;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FQn9zCrIJzLs%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="640" height="480" frameborder="0"><a href="https://medium.com/media/c77b3a9c0fafb03343a05b2b77b03238/href">https://medium.com/media/c77b3a9c0fafb03343a05b2b77b03238/href</a></iframe></p>
<p>This set of work found new ways to use space and time to show relationships between different pieces of information. <a href="https://en.wikipedia.org/wiki/Lisa_Strausfeld">Lisa Strausfeld’s</a> <a href="http://prior.sigchi.org/chi95/Electronic/documnts/shortppr/lss_bdy.htm">Financial Viewpoints</a> project used these strategies for visualizing large sets of data in a 3D space.</p>
<p><img alt="" src="https://cdn-images-1.medium.com/max/640/0*J-TCB8p2xeNCehu4.jpg">Lisa Strausfeld’s Financial Viewpoints visualizing 7 mutual funds.</p>
<p>This project allowed people to explore an information space in a way that was both reminiscent with how they explore the real world but also without physical restraints, like the effort involved with walking. Lisa writes:</p>
<p><em>“3D virtual information spaces, because they are abstract worlds of suspended text and images rather than simulations of the physical world, have a somewhat paradoxical relation to the body. As explorers of these spaces we become (at some level of consciousness) both embodied and disembodied. We rely on bodily intuition to navigate through and understand the structure and contents of the space, but we also have the ability to do what our bodies do not allow us to do in the physical world: we can fly (even through time), change our size (e.g. zoom through multiple scales of information) and see through objects (via transparency).”</em></p>
<p>Financial Viewpoints opened additional layers of information for viewing and gave people the ability to traverse that space. In an analogous way, AR allows people move in unrestricted ways through space while remaining grounded in the surroundings around them.</p>
<h3>AR: Mapped to the Real World</h3>
<p>Many AR apps and experiments place and create objects within a space based on their world’s representation. At the same time, being in a real space imposes boundaries. People must still avoid running into physical objects and walls, and obscuring too many of these reference points with digital elements can be disorienting. Users sometimes need to walk over to something they dropped within an AR space to manipulate or remove it (contrast this with a video game world, where movement is only as limited as the developers make it). While this encourages exploration of space and direct manipulation of things, AR objects can still be isolated without any efficient means of controlling them.</p>
<h3>AR: Sensors</h3>
<p>In addition to being transparent about surroundings, AR (or context based sensors in general<a href="http://www.aetherpoint.com/blogpost/augmented-reality-design-crossing-the-screen-space-divide#fn:6">(6)</a>) can make use of additional signals and inputs. This opens up a range of <a href="https://experiments.withgoogle.com/ar">creative applications</a> that can utilize things like <a href="https://twitter.com/keiichiban/status/976850288949809154">computer vision</a>, geolocation, ambient light, user position and <a href="http://annafuste.com/portfolio/inertia/">tangible interfaces</a>. A number of artists, designers and programmers have explored these</p>
<p><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*FcXWEFS_1MY5Fy_O.png">A collection of <a href="https://experiments.withgoogle.com/ar">AR Experiments</a> featured by Google.</p>
<p>These experiments combine features that open up new interfaces. Approaches like these align information and controls with how we might expect them to behave in the real world — depth / perspective, collision, and movement in space. <a href="https://www.youtube.com/watch?v=arOEUVO1rIw">Paper Cubes</a> by <a href="http://annafuste.com/">Anna Fusté</a> and <a href="http://judithamores.com/">Judith Amores</a> allow the placement of real world cubes to create and redirect virtual objects.</p>
<p><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*BDxv6PdwVOhZbEZC.png"></p>
<p>These block markers (using image recognition) allow for more overlap between AR space and real space. People can adjust the blocks intuitively like they would any other physical block and see how it effects the stick figures in digital space.</p>
<h3>Closing the Gap</h3>
<p>Instead of separating the AR world and flatscreen UI, we can create things that move fluidly between both spaces and take advantage of both places. This may mean doing more then enabling tap/touch/point events using <a href="https://en.wikipedia.org/wiki/Ray_casting">ray casting</a> on 3D objects in space. Generally, objects in world space hope to be controlled / labeled. Local flat UI (which people are already familiar with) often wishes it could better reach out into worldspace to do the controlling. This creates questions around how UI layers should interact with simulated depth (on screens the representation is still ironically flat<a href="http://www.aetherpoint.com/blogpost/augmented-reality-design-crossing-the-screen-space-divide#fn:8">(8)</a>). Andrew Hart’s <a href="https://twitter.com/AndrewProjDent/status/981635577488519170">retail</a> and <a href="https://twitter.com/AndrewProjDent/status/966740568708714498">locational</a> concepts demonstrate a combination of knowing when and where to position 2D and 3D UI. 2D isn’t inherently better or worse than 3D, there’s a time and place for both.</p>
<p>Interactions like <a href="https://www.youtube.com/watch?v=ONOZRiZmH_c">throwing a Poké Ball</a> in <a href="https://www.pokemongo.com/">Pokemon Go</a> shift from the mode of the local flat screen to the AR world space fluidly. These sorts of patterns allow us to stay grounded in the utility of existing UI conventions while also taking advantage of what AR has to offer.</p>
<p>To illustrate this in a different way, I designed a ‘push to world’ UI that switches between 2D and 3D modes of showing <a href="https://earthquake.usgs.gov/earthquakes/">live se</a><a href="https://earthquake.usgs.gov/earthquakes/">i</a><a href="https://earthquake.usgs.gov/earthquakes/">smic earthquake data</a> from the <a href="https://www.usgs.gov/">United States Geological Survey</a>. The ‘zooming out’ of the panel aims to communicate that the UI window has shifted over to a new space and behaves differently. <a href="https://jonikorpi.com/">Joni Korpi’s</a> <a href="https://jonikorpi.com/zoomable-ui-for-the-web/">modeless zoomable UI</a> concepts and patterns function in a similar way here — space and transitions are used give people more context. Joni writes about this in his <a href="https://jonikorpi.com/zoomable-ui-for-the-web">Zoomable UI for the Web article</a>:</p>
<blockquote><p>“Every article feels like it has a spatial location on the site. The zooming animation helps me understand where elements come from and where they’re going. I’m trying to aim for a system that makes me think ‘the article with yellow bars somewhere in the bottom left” instead of “the article named Golden Grid System’.”</p></blockquote>
<p>Animation (and having 3D space in general) can help people with mental organization. A X/Y/Z space offers the opportunity to use real ‘depth’ in our 3D UI.</p>
<p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fplayer.vimeo.com%2Fvideo%2F268470613%3Fapp_id%3D122963&amp;dntp=1&amp;url=https%3A%2F%2Fvimeo.com%2F268470613&#038;image=https%3A%2F%2Fi.vimeocdn.com%2Fvideo%2F699207046_1280.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=vimeo" width="1394" height="1080" frameborder="0"><a href="https://medium.com/media/6a0662722ae3627a558873c6b432bd56/href">https://medium.com/media/6a0662722ae3627a558873c6b432bd56/href</a></iframe></p>
<p>In the same spirit as Muriel Cooper’s Information Landscapes, this depth gives us a convenient option to look at information from another angle. In this case, showing data in 3D can give us a better sense of scale and proportion (<a href="https://en.wikipedia.org/wiki/Moment_magnitude_scale">seismic waves</a> are measured on a <a href="https://en.wikipedia.org/wiki/Logarithmic_scale">logarithmic scale</a>).</p>
<p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fplayer.vimeo.com%2Fvideo%2F268470658%3Fapp_id%3D122963&amp;dntp=1&amp;url=https%3A%2F%2Fvimeo.com%2F268470658&#038;image=https%3A%2F%2Fi.vimeocdn.com%2Fvideo%2F699207079_640.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=vimeo" width="720" height="1282" frameborder="0"><a href="https://medium.com/media/ddb20363bb732989d46447d677c43caa/href">https://medium.com/media/ddb20363bb732989d46447d677c43caa/href</a></iframe></p>
<p>Being able to see depth and perspective helps us gain an appreciation for differences in magnitude relative to other earthquake sites. While the same set of information could be visualized in <a href="https://www.engineeringbigdata.com/global-earthquake-magnitude-data-analysis-r/">other ways</a>, a 3D representation lets us see details that might otherwise be lost or take up too much space in 2D. Conversely, flat graphics are a more efficient way to communicate and traverse simple sets of information. The game <a href="https://unknownworlds.com/subnautica/">Subnautica</a> allows the player to <a href="https://www.youtube.com/watch?v=Dn1wCancvrY">manipulate</a> and hold onto (or fire not so gently) objects in space, partially bypassing problems of distance.</p>
<p><img alt="" src="https://cdn-images-1.medium.com/max/320/0*gb86Ev21V3DTqd4r.gif">The Propulsion Cannon in the video game <a href="https://unknownworlds.com/subnautica/">Subnautica</a></p>
<p>Maybe we can apply <a href="http://blog.leapmotion.com/summoning-superpowers-designing-vr-interactions-distance/">similar techniques</a> like these to manipulating UI windows<a href="http://www.aetherpoint.com/blogpost/augmented-reality-design-crossing-the-screen-space-divide#fn:9">(9)</a>, allowing them to be quickly tethered and retrieved to local flat UI. There are benefits to both<a href="http://www.aetherpoint.com/blogpost/augmented-reality-design-crossing-the-screen-space-divide#fn:10">(10)</a> 3D and flat space, and methods for fluidly transitioning in and outside of those modes.</p>
<h3>Staying Rooted</h3>
<p>Digital mediums will continue to become more in sync with our world, giving us the opportunity to help define when and where information shows up. As we do this we can explore how interfaces shift and adapt to people and their space’s needs, virtual or otherwise (the medium). There’s a lot to learn from the progression of both 2D and 3D disciplines. As we string together flat rectangles with spacial boxes we should make sure we’re anchored to the right places along the way.</p>
<p>Questions or comments? DM me on <a href="https://twitter.com/Aetherpoint">Twitter</a> or drop me an email!</p>
<p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fupscri.be%2Ff51076%3Fas_embed%3Dtrue&amp;dntp=1&amp;url=https%3A%2F%2Fupscri.be%2Ff51076%2F&#038;image=https%3A%2F%2Fe.enpose.co%2F%3Fkey%3DdRXnS9Gplk%26w%3D700%26h%3D425%26url%3Dhttps%253A%252F%252Fupscri.be%252Ff51076%252F%253Fenpose&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=upscri" width="800" height="400" frameborder="0"><a href="https://medium.com/media/b85dfbb5286d8a25cf2e754b9462cf45/href">https://medium.com/media/b85dfbb5286d8a25cf2e754b9462cf45/href</a></iframe><img src="https://medium.com/_/stat?event=post.clientViewed&amp;referrerSource=full_rss&amp;postId=f1d3a0004c69" width="1" height="1" alt="stat?event=post.clientViewed&amp;referrerSou"></p>
<hr>
<p><a href="https://blog.prototypr.io/https-medium-com-aetherpoint-augmented-reality-design-crossing-the-screen-space-divide-f1d3a0004c69">Augmented Reality Design — Crossing the Screen Space Divide</a> was originally published in <a href="https://blog.prototypr.io">Prototypr</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>


Augmented Reality Design — Crossing the Screen Space Divide

Augmented Reality Design — Crossing the Screen Space Divide

Augmented Reality Design — Crossing the Screen Space Divide

Top Stories