Published on <a href="https://blog.prototypr.io/face-controlled-apps-are-here-b1c6eb4a3e27?source=rss----eb297ea1161a---4">| Prototyping: From UX to Front End &mdash; Medium</a> Brad Dwyer 
By tapping into the expressiveness of our faces, face-controlled apps and games will soon reshape and expand our smartphone experience. The applications include gaming, accessibility, social media, and more.
<img alt="" src="https://cdn-images-1.medium.com/max/853/1*sXfVopSDnWaPlQtSsofVRQ.gif">By using your face as the controller, Animoji enables experiences never before possible.
When we made the jump from computers to smartphones a decade ago the number of digits we used for input decreased from 10 fingers to 2 thumbs. The pros outweigh the cons in many situations but we lost a lot of expressiveness in the process.
<blockquote>Using our face as an input device restores the dexterity we lost when when we ditched the physical keyboard.</blockquote>
The iPhone X contains a “TrueDepth” 3D sensor that, when combined with Apple’s ARKit APIs, can interpret facial expressions. After you grant them permission (<a href="https://washingtonpost.com/news/the-switch/wp/2017/11/30/apple-is-sharing-your-face-with-apps-thats-a-new-privacy-worry/">and they undergo an extensive Apple review process to ensure they respect rules regarding data usage and user privacy</a>), apps can use this data to access things like your “left eye blink percentage” or “mouth close percentage”.
<img alt="" src="https://cdn-images-1.medium.com/max/800/1*Ads_S_SLw6sYHu_4OU5ZGA.jpeg"><a href="https://itunes.apple.com/app/measurekit/id1258270451?mt=8">MeasureKit</a> provides a window into the face data available to developers from iPhone X.
This data was used by Apple to create Animoji puppets that mimic your facial expressions. But the potential uses go far beyond animated poop emoji and, with some creativity, app developers can use facial motions and gestures as rich input devices that allow apps to be controlled in novel ways.
This week, two new games were released that do just that.
<img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hrYzk-py0LFPVTQBBgf-CA.gif"><a href="https://www.google.com/url?sa=t&amp;rct=j&amp;q=&amp;esrc=s&amp;source=web&amp;cd=1&amp;cad=rja&amp;uact=8&amp;ved=0ahUKEwjSiJjO9pHYAhUm1oMKHSKhCJEQFggrMAA&amp;url=https%3A%2F%2Fitunes.apple.com%2Fapp%2Fid1312458558%3Fsource%3Dtechstories.org&amp;usg=AOvVaw3Mq2RrfqqORFwrvMdFZDWv">Rainbrow</a> by <a href="https://twitter.com/nathangitter">Nathan Gitter</a> is controlled by raising or furrowing your brow.
<a href="https://www.google.com/url?sa=t&amp;rct=j&amp;q=&amp;esrc=s&amp;source=web&amp;cd=1&amp;cad=rja&amp;uact=8&amp;ved=0ahUKEwjSiJjO9pHYAhUm1oMKHSKhCJEQFggrMAA&amp;url=https%3A%2F%2Fitunes.apple.com%2Fapp%2Fid1312458558%3Fsource%3Dtechstories.org&amp;usg=AOvVaw3Mq2RrfqqORFwrvMdFZDWv">Rainbrow</a> by <a href="https://twitter.com/nathangitter">Nathan Gitter</a> is a frogger-like game completely controlled by raising and furrowing your eyebrows.
You can imagine other games controlled by gestures like winking or opening and closing your mouth.
Taking things one step further is an app released by my company called <a href="https://nose.zone/download">Nose Zone</a> (<a href="https://nose.zone/download">free download</a> in the App Store, iPhone X only).
Nose Zone is another face-controlled app but, instead of using facial expressions for control, it uses the direction you point your face to control a laser projected from your nose.
<img alt="" src="https://cdn-images-1.medium.com/max/768/1*DHCzaBLPyKL36hpvuNC7GA.gif">In <a href="https://nose.zone/download">Nose Zone</a>, your face is the controller and your nose shoots lasers.
<h3>The future (games are just the start)</h3>
These two games are pioneering the face-based gaming genre. They are fun and the category has a bright future ahead of it (if you haven’t tried them out go find an iPhone X; you’ll be happy you did). But the more interesting aspects will come when developers take these concepts and apply them to UX design in general.
To start, there are accessibility-related possibilities that could help users with a disability interact with smartphones in more seamless ways. For example, imagine apps adding a Nose Zone-style pointer for paralyzed people as a substitute for touch input.
Beyond accessibility and gaming, the potential uses are endless. We will all be surprised by the creative uses developers come up with. To illustrate, here are just a few ideas:
<ul>
<li>A social media app that lets you “react” to a post by making a face</li>
<li>An eBook reader that lets you switch the page hands-free by winking (right eye for forwards, left eye for backwards)</li>
<li>A dating app that assesses your reactions to prospective matches based on your facial expressions</li>
<li>A help menu tooltip that appears when you look confused</li>
<li>A video app that can automatically exit when you fall asleep</li>
</ul>
What other face-controlled app ideas can you come up with? Will you use them when they’re available? Let me know in the comments or on twitter <a href="https://twitter.com/braddwyer">@braddwyer</a>.
<blockquote>Brad Dwyer is the founder of <a href="http://about.hatchlings.com">Hatchlings</a>, a startup that makes games and apps in Des Moines, Iowa. Their new face-controlled game, <a href="https://nose.zone">Nose Zone</a>, is available to <a href="https://nose.zone/download">download for free in the App Store</a> (iPhone X required).</blockquote>
<iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fupscri.be%2Ff51076%3Fas_embed%3Dtrue&amp;url=https%3A%2F%2Fupscri.be%2Ff51076%2F&#038;image=https%3A%2F%2Fe.enpose.co%2F%3Fkey%3DdRXnS9Gplk%26w%3D700%26h%3D425%26url%3Dhttps%253A%252F%252Fupscri.be%252Ff51076%252F%253Fenpose&amp;key=d04bfffea46d4aeda930ec88cc64b87c&amp;type=text%2Fhtml&amp;schema=upscri" width="800" height="400" frameborder="0"><a href="https://medium.com/media/b85dfbb5286d8a25cf2e754b9462cf45/href">https://medium.com/media/b85dfbb5286d8a25cf2e754b9462cf45/href</a></iframe><img src="https://medium.com/_/stat?event=post.clientViewed&amp;referrerSource=full_rss&amp;postId=b1c6eb4a3e27" width="1" height="1" alt="stat?event=post.clientViewed&amp;referrerSou">
<hr>
<a href="https://blog.prototypr.io/face-controlled-apps-are-here-b1c6eb4a3e27">Face-controlled apps are here — and they are about to transform how we interact with our devices</a> was originally published in <a href="https://blog.prototypr.io">Prototypr</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.

Face-controlled apps are here — and they are about to transform how we interact with our devices

Top Stories