Published on | Prototyping: From UX to Front End — Medium Timotheus Gmeiner

5 Steps Towards Designing with Machines and Living Data Streams

The greatest achievement of our technology may well be the creation of tools that allow us to go beyond engineering — that allow us to create more than we understand — Danny Hills, The Pattern on the Stone

In every era, the most effective technical creators have always been those who understand the practical applications and changing limitations of technology. People with the skills needed to excel in partnership with machines have been the creators with the most insightful and most effective solutions. Today is no different. Leading solution-finders who are adept at collaborating with AI tools and the data at their disposal will have more time to focus on the human aspects of design. They will create new unimaginable experiences, services, widgets, artifacts and simply magical creations that will bring us into the future.
To become the idea ‘design centaur’, you will augment the design engineering workflow with various specialist AI’s and function as an AI-human hybrid intelligence. Just like the centaur model brought new levels of performance to chess-play, so too will it transform design engineering to enable creators new unprecedented levels of capability.
Are you are a designer, engineer, or entrepreneur who doesn’t want want to be left in the dust? This article we will go over how you can start to transcend traditional engineering practice by practically incorporating AI into the product design engineering development process.

Step 1: Attitude Adjustment

Augmentation versus Replacement

It is narrow-sighted to only consider what aspects of your work advancing technologies will be able to replace. Instead, put your mind towards thinking about how your creative potential will expand as you learn to leverage intelligent agents. How can you creatively collaborate with AI tools to extend and enhance your talents? Think of these technologies as tools for augmenting your intelligence as opposed to threats that will replace human capability.

Working together is better forever.

Intelligence is a spectrum

Intelligence is not a single dimension. It is a complex of many types and modes of cognition, each one a continuum. …. Intelligence is a combinatorial continuum.— Kevin Kelly, The Myth of a Superhuman AI

Machines are not more or less intelligent. They are just different (Source: The Myth of a Superhuman AI)

Recognize that there are spectrums of intelligence – in people and in machines. Humans display many types of thinking: symbolic reasoning, memory, emotional intelligence, deduction, induction, spatial intelligence, etc. As designers and engineers you should not look to machines as discrete machines.

Visual depiction of human thought. 3D visualization of EEG and MRI data.Visual depiction of machine thought. Graphcore’s AI ‘thoughts’.

Instead, look to them as true collaborators with strengths where you are weak. AI tools are, and will continue to be, better than you at certain tasks. That’s a good thing. Start thinking of AI agents as tools that will enable further thought diversity in your creative process.

Play to your strengths

For now, machine intelligence is not able to design solutions that take into account the subtleties of human experience. Humans have a conscious experience that makes us uniquely capable of developing intuitively ‘human’ insights (at least for now). Recognize that your empathy and ability to uncover uniquely complex linkages between disparate nodes of human experience is your human advantage. You are not a data parser! You are a complex and emotional animal. Own it! It is super more important — now more than ever.

There are many types of potential types of artificial minds that could eventually exist. They may all have different levels of ‘human-likeness’ and different capacities for consciousness. The key thing to remember though is that human consciousness is a unique within this spectrum of potential minds. (Source: Conscious exotica: From algorithms to aliens, could humans ever understand minds that are radically unlike our own?)

To thrive, you need to play to your strengths rather than focus on your limitations. You must learn to augment your methodology where machine processes can improve it. Does it make sense for you to spend three months making hundreds of iterations of a component design when a machine could do it in seconds? No. Does it make sense for you to spend three months personally interacting with and developing an empathetic understanding of the experience of your customer? Yes.

Step 2: Make friends with data (friends with benefits)

Design has always been data science.

Fundamentally, product design engineering is a data processing activity. The whole process involves teams of creative and skilled people from multiple domains (e.g. ethnographers, UX designers, scientists, engineers, manufacturers, business developers, etc) working together to identify and refine configurations of materials into forms that address unmet, poorly met, or previously unknown needs of consumers. This entire process generates and requires a significant amount of unstructured quantitative and qualitative data.

Great products are the result of collection, wrangling, analyses and organization of various forms of quantitative and qualitative into a structure piece of information. (Source: Data driven vs. data informed design in enterprise products.)

From this perspective, product design processes from ‘waterfall’ to ‘agile’ to ‘Tycho’ to ‘lean’ to ‘design thinking’ can all be viewed as processes for establishing some structure to the data being generated- let’s call it design metadata. This design metadata could be related to end-user preferences for product material surface texture, or differing manufacturing limitations by different vendors for the same sub-component, or varying performance standards for regulators across the globe.
Existing product design engineering process requires that people organize this information. This requires entire teams of highly-trained experts. People who have spent decades training themselves through their experience (supervised), and the experience of others (semi-supervised and reinforced) to become knowledge experts in various domains and subdomains related to various phases of the product development process.
To train an engineer to be contextually aware of all relevant best practices, constraints, appropriate regulations related to medical devices, cleantech, automotive, aerospace, etc can literally take decades.
However, once fully-trained, they can easily use their knowledge to classify problems and generate solutions that can be effective within their narrow domain space.
For example, a regulatory expert in the medical device space does not have the understanding required to develop effective solutions for aerospace regulatory problems (although there is no doubt some transferable knowledge).
The point is, all complex product systems (jet engines, MRI machines, nuclear reactors, automobiles, airplanes, etc) contain more design metadata than a single person — even the most talented, most genius creative minds could take into account when creating a new design. Humans do not have enough data processing bandwidth or parallel computing ability to maintain a balanced perspective of all the relevant data for any design solution.
To get around this, designers and engineers create rules of thumb, design guidelines, best practices, etc (generalized algorithms) for classifying and categorizing design problems in order to generate initially approximate and then optimized design solutions for any given problem. In nerdspeak, product design, engineering is a multivariate optimization problem. Designers of all stripes must struggle with often conflicting variables to tune and optimize the success of a given solution.
The trouble is, as the world becomes more complex, designing solutions using generalized rules may no longer be suitable. As a society, we are both creating and solving problems at a faster rate than at any point in history. To be effective, I believe that the development process of future systems must become significantly more data-driven that it is today. We need the assistance of machines to help us manage, analyze, and act upon the vast swarms of design metadata that we create.

The power of the digital twin

As the Internet of Things is built out, a ‘digital twin’ of our civilization’s designed physical reality will be developed. Layers of information will be collected about every level of society, industry, and enterprise. This information will not only inform new opportunities for real-time insight into complex systems but it will also enable decision makers to develop predictive and generative models of the future. Predictive models will be able to generalize likely outcomes based on previous events and real-time data feeds. Generative models will be able to generate 1000’s of potential scenarios given real-time and often incomplete historical information.

A cyber-physical product being used by a customer collects data (e.g. living data streams) that can be analyzed and visualized by its developers to gain unique insights about its performance and operation. (Source: How do digital twins work?)

This will be akin to a global nervous system that gives designers real-time insight into the real-world use and performance of the product solutions they create; exposing options for iteration and improvement, all while creating unique design metadata to help feed into generative systems.
This will allow solution finders to look farther into the future to understand the potential impacts of their designs. A designer with seamless access to appropriately visualized information about her end-end user, the systems supply-chain, global trends, and various other nth-order interdependencies will have significantly more opportunity to create solutions with less negative externalities. Through effective collaboration with AI tools and living datastreams, more people are going to be able to create more intelligent and sustainable solutions. As a solution finder, you must learn how to leverage this living data to develop improved solutions. One interesting strategy for product designers to currently do this is through generative design.

Transcending Engineering through Generative Design

The broad power of iterative systems is the biggest advance in engineering since the scientific method” — Steve Jurvetson

New tools using iterative and evolutionary algorithms are already enabling innovators to engineer hardware from living data streams (this has already been done with software for some time). This technology allows a designer to mimic nature’s evolutionary design process. The designer must feed various success criteria and constraints into a software tool that leverages the computing power of the cloud to rapidly generate hundreds or thousands of design options while testing each iteration against the success criteria and constraints and learning from them to develop a more optimized solution.

Designing for complex systems is a multivariate problem. A human designer can hold five or six design variables in their head and therefore must stick to controlled processes, rules of thumb, and industry best practice to reach suitable designs. A machine learning system can handle a nearly unlimited amount of design variables. In this way generative design tools enable designers to speak in the language of goals and constraints by inputting many design variables (you can add data feeds that refine the weighting on those variables) and then iterate through thousands of possible design iterations to find the most optimal geometric configuration.

The evolutionary design process is iterative and data-driven (Source)

Generative design tools empower designers and engineers to:
1. Explore a wider range of design options: Generate thousands of design variations in the time it takes you to generate one- in and provide evidence for or against the validity of each design.

Generative tools can develop multitudes of concepts to meet defined constraints.

2. Optimize for new and hybrid fabrication methods: Generative design lets you create optimized complex shapes and internal lattices. Some of these forms are impossible to make with traditional manufacturing methods. Instead, they’re built using new additive manufacturing methods.

Generative tools can develop optimal complex internal geometries tune to meet specific requirements.Parts auto-generate and adapt like plants and bones to find the best form for the environment and function you define.Generative design tools allow you to simulate static and dynamic loads in real time.

3. Optimize for materials, manufacturing methods, and cost: Set goals and parameters, and the software will create high-performing design options based on those constraints. The software resolves conflicting design constraints so you can focus on innovating.

With generative design you can rapidly develop and explore hundreds of potential solutions to a design problem, many more than a human engineer could economically conceptualize. With generative design, there is no single solution; instead, there are potentially thousands of great solutions. (Source)

These are tools that enable evolutionary designs to problems wherein the solutions improve as more refined representative ‘digital twin’ knowledge models of the problem space are developed. As opposed to single-shot, prescriptive and discrete solutions. You need to learn to work collaboratively with the machine to develop the best solution for the constraints you have established.

Step 3: Shift your strategy towards a future of living data streams

The ability to see the signal through the noise is the asset you want. In ancient times having power meant having access to data. Today having power means knowing what to ignore. — Yuval Harari, Homo Deus: A Brief History of Tomorrow.

Understand your changing role

The role of designers will become a role in creatively defining and breaking down problems into requirements and constraints for sub-solutions to be generated (at least in part) by AI-assisted tools. The role of engineering executives and managers of the future will be that of community managers, team facilitators, and creative enablers (e.g. see Chief Enabling Officers) to distributed networks of creative solution finders who use their unique insights and contextual understanding to design tailored solution constraints for generative AI systems (for more on this see my previous article Collaborative Making in Overdrive).
The real gap will be in product development process management. As prediction becomes cheap the value of decision-making increases. What decisions should be handled be automated and what will the impacts be? What processes should change as a result? In order to make these decisions, process and product managers will have to really understand their domain, the data ethnography they are leveraging and the capabilities and biases of their models. You need to have an understanding of thinking that went into the collection of the data you are using.
Uncover new digital service layers
Through the development of cyber-physical products, many traditional hardware companies (e.g. Renault, Philips, GE, etc) have shifted towards hardware-as-a-service (Haas) business models.

This shift enables them to remain competitive via agile, data-driven, digital services layers delivered through hardware that collect vast swarms of context-specific, high-value data streams.For example, GE makes more money selling analytics ‘advanced asset performance management services’ to airliners that buy their jet engines than they do on the sale of their physical engines. The value delivered by intelligent agents improves over time through access to continuous streams of context-specific real-time data (eg.‘living data’) coming off these devices.
Tesla vehicles provide an excellent example of this — the more people that drive their vehicles, the more sensor and performance data they get, the better their self-driving agents become, and the more useful their product. As companies further leverage these data flows, they may be able to expand services while maintaining ownership of a niche through developing both the hardware systems that collect the data and the intelligence that processes it. For example, as Tesla’s fleet of vehicles expands, and their full-autonomy capabilities of their self-driving agents improve they may create a rideshare service that could steal market share from other ride-sharing services.

Cyber-physical systems learning loop.(Source)

Design for the Circular Economy

Because more value is delivered through the digital service layer than through the physical artifact, companies developing cyber-physical devices are economically incentivized to redesign their hardware with sustainable resource management in mind. These companies maintain ownership of their material assets used to build the product and design their products to be easily disassembled and reconfigured into new or upgraded products. Therefore designers and engineers must select materials and design the product such that the materials can be recovered and up-cycled (see Philip’s light-as-a-service example).
As the demand for raw materials increases (more people with more money want more stuff) designing for the circular economy will be a critical concept for designers, engineers, and entrepreneurs to understand. Get started learning about this with IDEO’s Circular Design Guide.

The future of hardware is cyber-physical products designed for the circular economy.

New opportunities create new challenges.

The fluid and evolutionary nature of intelligent cyber-physical products provides an exciting design challenge for design engineers, UX designers, and AI developers but creates a massive headache for industry regulators. Regulatory bodies exist to protect the public by ensuring that developers maintain repeatable levels of quality and performance for the complex devices and systems they develop. Because of this, hardware product design has been traditionally deterministic due to regulation. You design a car or a surgical tool, you make thousands of them, and in an ideal world — they each perform exactly the same.
If your AI service layer for your medical diagnostics device gets better over time, the product becomes difficult to regulate because the value it provides is continuously changing. This evolutionary nature raises questions about the trustworthiness and reliability of AI-assisted systems.
Similarly, manufacturing physical products at scale requires significant investment in expensive tooling and fabrication process development. If the data coming from your machine keeps giving your generative design tools new and improved component designs — your production systems will need to allow for flexible design configurations and production lines. The increasing customizability of designs via new data-driven, generative processes as well as the configurability of manufacturing processes introduce new complexities when drafting a design control model for regulatory consideration for products requiring market approval.

The changing tide

Regulatory bodies like the FDA are adapting and changing regulatory conditions to be more flexible to the high-value potential for new AI-assisted hardware products (e.g. semi-autonomous diagnostics systems and customizable medical implants). Further, additive manufacturing tools and advanced collaborative robotics significantly reducing the prohibitive costs associated with tooling and process development. As this continues, designers will be able to create products that are more flexible to modification and/or reconfiguration and manufacturing facilities that are more flexible and reconfigurable to greater production varieties.

Collaborative robotics can be easily trained to perform precision tasks with relative flexibility for reconfiguration.

Step 4: Understand your tools and their limitations

To work productively with any team you need to have an idea of how the unique knowledge and talents of your co-workers complement your own. You must start to plan your creations with an awareness of your expanding capability in mind.

For design engineering to become a team sport between humans and machines you need to understand the current capabilities and limitations of machine intelligence tools. Below is a rough outline of how some of the tools in the AI landscape are can be used across the design engineering landscape today and potentially be used in the future. The tools listed below give some insight into what’s possible, some examples of application and some imagination into where it could be used in the future.

There are places throughout the product development cycle where AI-tools can be applied to augment your capability. (source)

This is an introductory list of AI-assisted tools that you can use to help augment your potential at various stages of product development:

Management and Strategy

Faethm — A decision making tool for predicting the impacts of the implementation of exponential technology.

Research

Iris — The AI that reads science papers and extract all the key concepts for you.
Wolfram Alpha — Tool for computing expert-level answers to questions.
Google — You should know what this is!

Concept Development

Pinterest Visual Search — Search the web for specific features in images to find design inspiration.
AutoDraw — A drawing tool that pairs the magic of machine learning with drawings from talented artists to help everyone create anything visual, fast.

Concept Validation

Affectiva — An AI that analyzes how people and interact with concepts and products via computer vision.

Design & Development

Autodesk Dreamcatcher; Desktop Metal Live Parts; Autodesk Generative Design — Generative design tools.
Fusion 360 — Topology optimization to reduce the weight and material usage while maintaining or exceeding performance criteria.
Netfabb; nTopology; SolidThinking Inspire; CogniCAD — Apply internal lattices and optimized surface structures to lightweight and improve strength.
Design Graph — Designers and engineers simply search across all of their files for a part type, such as a bolt or a bike seat, and Design Graph returns dozens or hundreds of ideal options — think Google search for 3D CAD data.

Quality Control

Instrumental — Computer vision automatic inspection of components on production lines.

Assembly and Packaging

Rethink Robotics — Use reinforcement learning to enable human-in-the loop training of collaborative robots.

Counterfeit Detection

Entrupy — System inspecting for determining counterfeit goods.

Cloud Manufacturing

MakeTime — Cloud manufacturing, a service oriented, customer centric, demand driven manufacturing.

Distribution

Amazon — Recommendation systems improve your sales and planning algorithms help shipping companies direct products and across the globe. Fulfillment robotics/drones/logistics centres and sorting and deliver goods.

Step 5: Understand the machine!

If we want to live with the machine, we must understand the machine, we must not worship the machine. — Norbert Wiener

AI-assisted tools for designers and engineers will continue to advance. You should develop some understanding of how these tools are going change in the future. Furthermore, you should develop some level of understanding of the fundamental theory behind how they work. Below is a curated list of links to get you started.

These are some great video resources from Autodesk University for you to start learning more about the future of AI assisted design:

Understand how AI works

Start at from high-level concepts and work your way down to low-level nitty-gritty details:

a.High-level — Understand some background and its implications to your work generally.

Machine Learning for Designers
UX of AI — Google
An Executive’s Guide to AI — Don’t be intimidated, ‘executive’ just means no-frills and clear explanation in this case.
The Andreessen-Horowitz AI Playbook
Practical Artificial Intelligence for Dummies
Beginners Guide to Machine Learning, AR, and cognitive computing

b. Mid-level — Understand enough about different algorithms/processes in order understand their potential application.

c. Low-level — Understand the details of what it takes to build even basic intelligent systems.

Get started!

Go try and build something new! Start applying what you learn!

Want more insights and ideas for building a happy engineering career? Please subscribe. You can also follow me on twitter at @timosaurous

http://eepurl.com/dm18gz

Augmented Engineering Intelligence was originally published in Prototypr on Medium, where people are continuing the conversation by highlighting and responding to this story.

Augmented Engineering Intelligence

5 Steps Towards Designing with Machines and Living Data Streams

Top Stories