Build Design Systems With Penpot Components
Jul 21, 2:15 AM
Penpot's new component system for building scalable design systems, emphasizing designer-developer collaboration.
smashingmagazine.com
medium bookmark / Raindrop.io |
Apps can use Apple’s augmented reality (AR) technology, ARKit, to deliver immersive, engaging experiences that seamlessly blend realistic virtual objects with the real world. In AR apps, the device’s camera is used to present a live, onscreen view of the physical world. Three-dimensional virtual objects are superimposed over this view, creating the illusion that they actually exist. The user can reorient their device to explore the objects from different angles and, if appropriate for the experience, interact with them using gestures and movement.
Designing an Engaging Experience
Use the entire display to engage people. Devote as much of the screen as possible to viewing and exploring the physical world and your app’s virtual objects. Avoid cluttering the screen with controls and information that diminish the immersive experience.
Create convincing illusions when placing realistic objects. Not all AR experiences require realistic virtual objects. Those that do, however, should include objects that appear to inhabit the physical environment in which they’re placed. For best results, design detailed 3D assets with lifelike textures and use the information ARKit provides to position objects on detected real-world surfaces, scale objects properly, reflect environmental lighting conditions on virtual objects, cast virtual object shadows on real-world surfaces, and update visuals as the camera’s position changes.
Consider physical constraints. Bear in mind that people may attempt to use your app in an environment that’s not conducive to an optimal AR experience. For example, they may open your app in a location where there isn’t much room to move around or there aren’t large, flat surface areas. Try to anticipate scenarios that might present challenges, and clearly communicate requirements or expectations to people up front. Consider offering varying sets of features for use in different environments.
Be mindful of the user’s comfort. Holding a device at a certain distance or angle for a prolonged period of time can be fatiguing. Consider how people must hold their device when using your app, and strive for an enjoyable experience that doesn’t cause discomfort. For example, by default, you could place objects at distance that reduces the need to move closer. A game could keep levels short and intermixed with brief periods of downtime.
If your app encourages user motion, introduce it gradually. In a game, for example, the user shouldn’t need to move out of the way to avoid a virtual projectile as soon as they enter AR. Give them time to adapt to the experience first. Then, progressively encourage movement.
Be mindful of the user’s safety. Moving around too much can potentially be dangerous if there are other people or objects nearby. Consider ways of making your app safe to operate. For example, a game could avoid encouraging large or sudden movements.
Use audio and haptic feedback to enhance the immersive experience. A sound effect or bump sensation is a great way to provide confirmation that a virtual object has come into contact with a physical surface or other virtual object. In an immersive game, background music can help envelop the user in the virtual world. For related guidance, see Audio and Haptic Feedback.
To the extent possible, provide hints in context. Placing a three-dimensional rotation indicator around an object, for example, is more intuitive than presenting text-based instructions in an overlay. Textual overlay hints may be warranted, however, prior to surface detection or if the user isn’t responding to contextual hints.
If you must display instructional text, use approachable terminology. AR is an advanced concept that may be intimidating to some users. To help make it approachable, avoid referring to technical, developer-oriented terms like ARKit, world detection, and tracking. Instead, use friendly, conversational terms that most people will understand.
| Do | Don’t |
|---|---|
| Unable to find a surface. Try moving to the side or repositioning your phone. | Unable to find a plane. Adjust tracking. |
| Tap a location to place the [name of object to be placed]. | Tap a plane to anchor an object. |
| Try turning on more lights and moving around. | Insufficient features. |
| Try moving your phone slower. | Excessive motion detected. |
Avoid unnecessary interruptions to the AR experience. The environment is analyzed and surfaces are detected each time the user exits and reenters AR. In addition, the phone and camera position has probably changed. As a result, previously placed objects are likely to be repositioned—they may even no longer appear to rest on real-world surfaces. One way to avoid interruptions is to let people make changes to objects and settings without leaving AR. For example, if a user has placed a chair they’re considering purchasing into their living room, they may want to try out other fabric options.
Entering Augmented Reality
Indicate when initialization is occurring and involve the user. An initialization process, during which the surroundings are evaluated, occurs each time your app enters AR. This can take a couple of seconds. To reduce possible confusion and speed up the process, show an indication of activity during initialization and encourage people to explore their surroundings, or to actively look for a surface.
Placing Virtual Objects
Surface detection indicator
Object placement indicator
App-specific indicator
Help people understand when to locate a surface and place an object. A visual indicator is a great way to communicate that surface targeting mode is active. A trapezoidal reticle in the center of the screen, for example, helps people infer that they should find a horizontal, flat surface. Once a surface is targeted, an indicator should change in appearance to suggest that object placement is now possible. Design visual indicators that feel like part of your app experience.
Respond appropriately when the user places an object. Accuracy is progressively refined (over a very short period of time) during surface detection. If the user taps the screen to place an object, place it immediately using the information that’s currently available. Then, once surface detection is complete, subtly refine the object’s position. If an object is placed beyond the bounds of the detected surface, gently nudge the object back onto the surface.
Avoid trying to precisely align objects with the edges of detected surfaces. In AR, surface boundaries are approximations that may change as the user’s surroundings are further analyzed.
User Interaction with Virtual Objects
Favor direct manipulation over separate onscreen controls. It’s more immersive and intuitive when a user can touch an object onscreen and interact with it directly, rather than interact with separate controls on a different part of the screen. Bear in mind, however, that direct manipulation can sometimes be confusing or difficult when the user is moving around.
Allow people to directly interact with virtual objects using standard, familiar gestures. For example, consider supporting a single-finger drag gesture for moving objects, and a two-finger rotation gesture for spinning objects. For related guidance, see Gestures.
In general, keep interactions simple. Touch gestures are inherently two-dimensional, but an AR experience involves the three dimensions of the real world. Consider the following approaches to simplifying user interactions with virtual objects.
Limit movement to the two-dimensional surface on which the object rests.
Limit object rotation to a single axis.
Respond to gestures within reasonable proximity of interactive virtual objects. It may be difficult for people to precisely touch specific points on objects that are small, thin, or placed at a distance. When your app detects a gesture near an interactive object, it’s usually best to assume the user wants to affect the object.
Consider whether user-initiated object scaling is necessary. Scaling is generally appropriate when an object, like a toy or game character, doesn’t have an intrinsic size and the user may want to see it larger or smaller. For an object with a finite size relative to the real world, like a piece of furniture, scaling is irrelevant if the item is placed at an accurate size. Scaling is not a remedy for adjusting the distance of an object—making an object larger to make it appear closer, for example, just results in a larger object that’s still far away.
Be wary of potentially conflicting gestures. A two-finger pinch gesture, for example, is quite similar to a two-finger rotation gesture. If you implement two similar gestures like this, be sure to test your app and make sure they’re interpreted properly.
Make sure virtual object movements are smooth. Objects shouldn’t appear to jump when the user resizes them, rotates them, or moves them to a new location.
Explore even more engaging methods of interaction. Gestures aren’t the only way for people to interact with virtual objects in AR. Your app can use other factors, like motion and proximity, to bring content to life. A game character, for example, could turn its head to look at the user as they walk toward it.
Handling Problems
Allow people to reset the experience if it doesn’t meet their expectations. Don’t force people to wait for conditions to improve or struggle with object placement. Give them a way to start over again and see if they have better results.
Suggest possible fixes if problems occur. Analysis of the user’s environment and surface detection can fail for a variety of reasons—there’s not enough light, a surface is too reflective, a surface doesn’t have enough detail, or there’s too much camera motion. If your app is notified of insufficient detail or too much motion, or if surface detection takes too long, offer suggestions for resolving the problem.
| Problem | Possible suggestion |
|---|---|
| Insufficient features detected | Try turning on more lights and moving around. |
| Excessive motion detected | Try moving your phone slower. |
| Surface detection takes too long | Try moving around, turning on more lights, and making sure your phone is pointed at a sufficiently textured surface. |
Offer AR features only on capable devices. If your app’s primary purpose is AR, make your app available only to devices that support ARKit. If your app offers AR as a secondary feature—like a furniture catalog that includes product photos and allows some products to be viewed in AR—avoid displaying an error if the user tries to enter AR on an unsupported device. If the device doesn’t support ARKit, don’t present optional AR features in the first place. For developer guidance, see the arkit key in the UIRequiredDeviceCapabilities section of Information Property List Key Reference, and the isSupported property of ARConfiguration.
Learn More
For developer guidance, see ARKit.
