medium bookmark / Raindrop.io |

New technologies come and go, but not since the internet have we seen such a dramatic game changer as AI. AI promises to disrupt just about every industry you can think of. In other words, just about every one of our clients have been, or will be, affected by AI — and much like the internet and our shift into digital design, we must also start to augment our design processes to be better able to meet these needs.

However, due to the exponential growth of this technology, we currently lack a shared understanding of the principles of AI and common vocabulary is often misattributed. For example, AI does not have one unified definition and should be a term used to encompass many concepts associated with AI: Neural Networks, Deep Learning, Machine Learning, Natural Language Processing — just to name a few — but the term AI can sometimes be used to describe things like ‘chatbot’ which doesn’t necessarily sit under AI explicitly.

It’s important though, that we begin to define and agree upon what we understand to be the fundamentals of AI, as the process in which to build successful products or services is dependent on the interconnectedness and alignment across our disciplines.

So the impetus for this article is two-fold;

1. Create a framework for thinking about designing with AI, and
2. Create a reference to better understand the fundamentals of AI and how the components fit together.

The Framework

Input and Output

It’s no surprise that input/output can come in a variety of different form factors. Here we essentially need to figure out:

1. What data we are trying to collect,
2. The method in which we are collecting the data, and
3. How we are communicating to the user.

To help figure this out I use the following equation:
Brand + Data + UI = Input/Output

Brand

Branding is an incredibly important component when designing with AI, particularly when we are using derivatives of the same algorithms/API’s. How will a user feel after an interaction with your product and is this in line with the overarching brand strategy? The type of brand personality you infuse play a huge role in the differentiation of your product or service, the level of engagement you achieve, and the way your users interact with it.

“Our approach on personality includes defining a voice with an actual personality. This included writing a detailed personality, and laying out how we wanted Cortana to be perceived. We used words like witty, confident and loyal to describe how Cortana responds through voice, text and animated character.” — Marcus Ash, Group Program Director, Cortana

There are many ways your brand can filter through your product — the obvious being voice user interface (VUI), but it could also be as simple as a series of beeps or the way you write your conversation dialog.

Data

Thanks to the IoT, there is now more data available than ever before. In fact, by 2020 there is expected to be more than 44 zetabytes of data in the world — that’s 15x every grain of sand on earth! So your data input could be anything from a simple text input, to NFC, to your smartwatch.

Depending on how you are using the data you collect, you may require structured data or unstructured data. Structured data basically means data that is already organized into a structure like a database, csv or json file and has labels associated with it. Unstructured data on the other hand, is data that has not been classed or categorized. For example a set of random images or a call center recording.

AI Components
Structured Data, Unstructured Data

User Interface

Given that you may be dealing with a connected ecosystem — you may have different form factors to communicate through. Some may have screens or other ways of communicating such as flashing LED lights or some may not have the ability to communicate at all. What is the most appropriate communication interface?

VUI is a hot topic at the moment. Voice recognition accuracy has significantly improved over the last 12 months and it’s expected that at least 50% of all searches will be done via voice or image by 2020. However, we still need to think about whether the methods you are using to collect the data fit in with existing heuristic patterns for your product. And, how will you account for fall-back or secondary options?

“No one wants to wait 10 seconds for a response. Accuracy, followed by latency, are the two key metrics for a production speech system…” Andrew Ng, former Chief Scientist at Baidu

Another common example is free form text through a conversational interface such as Facebook messenger or SMS. This allows the user to converse in a way that is natural to them. Not only does this reduce friction, it also allows us to see if the product we have created is actually meeting the real needs of its users!

AI Components
Voice User Interface, Conversational Interface

Recognition

The recognition phase, is where we begin to identify this anonymous input. The term ‘understanding’ in the human world is very different from the definition in the computer world. In the simplest terms, whatever we put into a machine invariably gets broken down into a data point that, on its own, has no meaning. However, once we look at that data point in the context of what we call a ‘class,’ meaning or understanding is attributed.

“Speech recognition systems that map audio signals to words are the canonical example of assessment. The input is a raw audio signal and the output is a set of words that can then be used to determine meaning.” — Kris Hammond, IBM Watson AI XPrize

There are 2 ways we can assign classes to the data we have collected — supervised or unsupervised. Supervised means we manually assign a class or label to a data point while unsupervised means using one of a variety of algorithms, such as TSNE, to automatically cluster data based on how it perceives the relationship between the data.

Here is where it starts to get complicated; data points don’t have to be purely 2 dimensional. A data point could be multidimensional; it could have several variables attached to it.

A good example is if we think about a person. You could use many different features to describe a person — their height, eye color, favorite food, hobbies. Imagine you did this for all of your work colleagues and then had to try and figure out the commonalities between them all. This is where AI comes in handy — it doesn’t see the meaning of these features, it just sees them as numbers. These numbers can then be plotted in a three dimensional space where ‘people’ that it thinks are related are clustered more closely together. We can then use this structure to begin to infer meaning.

AI Components
Speech Recognition, Audio Recognition, Facial Recognition, Image Recognition, General Recognition, Text Extraction, Concept Extraction, Supervised Learning, Unsupervised Learning

Reasoning

Reasoning is intrinsically linked to the recognition of the input, and where the term ‘understanding’ is again used in the context of the machine. Once we have mapped the data in whatever way is appropriate, we can begin the interpretation. Moving from categorizing a series of words, to being able to understand the meaning of those words, derive sentiment, and extract entities.

Here is an example using Text Razor:

In this case, the Text Razor model can understand that ‘House’ in this context, actually represents a company or organization rather than a building.

As you can see, the interpretation of the input can be heavily influenced by the context — this is where things start to get interesting! Previous conversations can help narrow down your user’s intent.

In essence, this is facilitated by machine learning. According to Kris Hammond, there are three types of learning:

1. High volume data learning (being able to learn by analyzing huge amounts of structured or unstructured data)
2. Correlation learning (understanding the relationships between data) and
3. Incremental learning (learning from the execution of a previous task).

The reasoning phase should provide a synthesis of data based on confidence scores or a prediction of future states.

Confidence scores are a rating that may be provided with the output of the reasoning, based on how well a machine thinks it has performed. The most well-known use of a confidence score was in the infamous Watson Jeopardy scene where the AI replied ‘What is Toronto?????’ in response to “Its largest airport is named for a World War II hero; its second largest, for a World War II battle” in the category of “U.S. Cities.”

After dissecting the code, Watson developers saw that it had come to two conclusions — one being ‘Toronto’ and the other being ‘Chicago’ (the correct answer)—however both had a very low confidence score of about 30%. Because it was unable to make a clear decision, it expressed its low confidence with question marks.

AI Components
Speech Identification, Audio Identification, Facial Identification, Image Identification, Data Analytics, Synthesis, Predictive Analytics, Language Understanding, Machine Learning, Concept Expansion, Sentiment Analysis

Response

And finally, here is where we start making decisions of what is to happen as a result of the interpretation of our data. We could respond by communicating with the user in the form of dialog, or an automated email. Or we could take action, like turning the lights a different color or ordering a book off Amazon.

You could also determine your response based on the confidence score of your interpretation. If you have a high degree of confidence you could use an implicit confirmation such as saying “ok, setting additional timer for 3 minutes” and also completing the action of setting the timer. However, if you only have a small degree of confidence, you could use the response to illicit further confirmation by asking “You would like to set a timer for 3 minutes, is that correct?” before completing the action.

AI Components
Language Generation, Decision Making, Relationship Learning, Knowledge Refinement

Putting it all together

Now that you understand each phase, here is a dissection of a fictional product based on the framework.

A final thought

The internet did not disrupt the commerce industry. The applications we developed leveraging the internet’s many capabilities, such as encryption and globalization, were the defining factors in transforming an entire sector. AI will prove to be the same. The winners will be the people who truly understand the capabilities of the technology and are able to develop new and creative ways of applying it.

Hopefully this framework is a good starting point on your discovery and learning of the amazing potential of AI.