What are the odds a new product will still be around in two years' time?

If you ask the Product Design & Marketing Association, they're not good: between 25 and 45 percent of consumer goods in its survey eventually fail in the market. The U.S. Department of Commerce is even more pessimistic, suggesting that 95 percent of new products miss their sales and performance goals. Regardless of whom you believe (or how reliable their methods are), there's no doubt that product innovation is fraught with risk and uncertainty. To a professional designer, these statistics aren't just interesting; they're tragic. The dollars and hours wasted creating the next Zune, PCjr, or Edsel measure in the trillions, and besides clogging stores and landfills, failed products like these represent lost opportunities to make things right.

Those in the innovation business have grudgingly accepted this situation (if it was easy, everyone would do it, right?) but it's beginning to look as though the odds might change. The arrival of improved sensing, processing and prototyping technologies helped usher in the smartphone era and the new DIY economy, but I'd argue that we're just beginning to see how technology might disrupt product innovation. Three technologies in particular could transform not just how we design but how we make the decisions that tell us what to design. Properly applied, they could spell the end of failed consumer products.

Bresslergroup's Chris Murray

How We Decide What To Design Today

New products have a bad track record because they're usually solving the wrong problem.

Successful product innovation hinges on uncovering an unmet consumer or social need, then meeting it. Sounds simple, but in practice there's nothing harder than grasping the true needs of others, or knowing with any kind of certainty whether your new approach is better than the old one. If your product is genuinely innovative to the point where it creates a new category or asks users to adopt a new behavior, the task is even tougher.

So the researchers who work with designers have a dizzying array of techniques for finding out, most of them far more advanced than the focus groups and phone surveys of years past. What we do at Bresslergroup is representative of what happens industry-wide, with researchers working in small teams, spending hours at a time in target users' homes, or following them as they go about their routine, alternately observing (like an ethnographer), measuring (like a scientist) and interviewing (like a journalist). Once the data is gathered, researchers and designers spend time to extract the most compelling user needs and where current products fall short—but that requires a lot of interpretation.

How We'll Decide What To Design in the Future

The hybrid approach we use now does yield valuable insights, but in my experience it's difficult to get an accurate picture of habits and needs in a less than natural setting. I can ask a subject to be honest when I ask personal questions, but they're notoriously bad at it. And I can ask them to pretend I'm not there during observation, but my presence undeniably skews the outcome.

The Internet of Things

It's this gap that I'd like to see bridged with smart, connected devices. The Internet of Things (IoT) has been hyped relentlessly for more than a decade now, with varying results, mostly because gathering a lot of data isn't useful or interesting unless you know what to do with it. Still, Gartner has forecasted that 4.9 billion IoT devices will be in use by 2015—and 25 billion by 2020, with consumer products in the lead.

What if we used all of those smart thermostats to help us to design the next-generation smart thermostat? Going further, what if we could employ those wearable devices to learn how people really use, for example, their kitchen appliances? What if we enlisted that car's in-dash navigation system to discover not just transportation habits but shopping and dining patterns too?

Ethnographic research typically stretches over an hour or two. With IoT we could monitor a product's use over days, weeks and months for a much more accurate picture. For design researchers the world of quantitative big data will merge with qualitative research. This kind of benevolent surveillance would require explicit permission from subjects (and probably payment too), but this is already standard practice in most user research.

Far from replacing in-person inquiry, the data collected by connected devices could make it more useful by adding reliable context and freeing researchers to focus on observing rather than having to simultaneously interrogate participants with streams of questions.

An image from Affectiva, whose software tracks subjects' facial movements

Facial Expression Recognition Software

"The face is the window to the soul." We've all probably heard this at some point, but increasingly we're learning just how clear that window is. Beginning with Paul Ekman's work in the 1960s, research psychologists have come to realize that human facial expressions are not only surprisingly universal but surprisingly honest. It's now generally accepted that subtle facial cues offer a more accurate depiction of someone's reactions and emotional state than their words—a phenomenon Malcolm Gladwell popularized in his 2007 book Blink and that startups like Affectiva have recently used to design some potentially game-changing software.

By tracking subjects' facial movements, and comparing them with a library of expressions, Affectiva's software (along with competing products by companies like Emotient and Eyeris) can infer whether subjects are delighted, frustrated, engaged or confused—often more accurately than a skilled human observer. And unlike human observers, they can scale. Properly applied with the full consent of participants, these tools offer nothing short of an automated, reliable way to know how thousands of people feel when using a new product, digital or otherwise.

One of the great frustrations of product design is that the longer you spend perfecting something, the less you're able to accurately evaluate how it will be received. An objective, digital "observer" could let us look with eyes unclouded by familiarity at thousands of target users.

For designers, this could offer a way around the "curse of knowledge"—our inability to imagine what it's like to use something for the first time, after working on it for months. One of the great frustrations of product design is that the longer you spend perfecting something, the less you're able to accurately evaluate how it will be received. An objective, digital "observer" could let us look with eyes unclouded by familiarity at thousands of target users. When asking "Is this interface intuitive?" or "Is this aesthetic intimidating?" about a prototype product, we'd get a better shot at an answer approaching impartial truth.

Rapid Prototyping

How we create that prototype, though, presents a sticking point.

Gearing up to produce even a small consumer gadget can cost hundreds of thousands or millions of dollars, with an outcome that's far from certain. Short of actual production parts, the best we can usually do is several high-fidelity models or mockups that simulate the look, feel and function of the finished product at significant cost.

In all the discussion of 3D printing and other rapid prototyping (RP) technologies over the past few years, we've tended to focus on the DIY economy and short-run, high-value manufacturing like medical and aerospace. But its biggest impact may be felt in more mundane categories.

Rapid-prototyped versions of a tabletop candle display

It's already common practice to 3D print early versions (prototypes) of new products, so they can be evaluated for function, aesthetics and feel. This has become standard practice at Bresslergroup and other product design consultancies and incubators. As the cost comes down and the fidelity goes up, we're brushing up against a much more exciting possibility, of printing products that people can actually buy.

It's unlikely the economics of RP will make this approach competitive with traditional mass production at any real volume, but that's not the point. By putting a few thousand early versions of a new product on the market, designers stand to learn a tremendous amount about consumer interest and demand, and to get real-world feedback that can be applied in subsequent runs. It's a way of testing the salability of a product, in addition to its usability.

This strategy is equally valuable for startups as it is for large corporations who are starting to employ the same "lean startup" techniques as entrepreneurs to reduce risk—to build, measure and learn in a modest manner before scaling up the volume of a consumer validated product idea. For early adopters, it's a chance to evaluate innovative technology and to walk around with a limited-release edition in their pocket.

Putting Them Together

All three of these technologies have been around for years now, and have proven themselves to have significant value. What I'm proposing is a new use for familiar tools, to solve an old problem—high-risk product innovation—that was once intractable.

The early Internet was full of terrible websites and digital applications that were practically unusable. We owe today's relatively useful, user-friendly online world to a rapid expansion in the skills of digital designers, and to elevated user expectations—but also to some incredibly powerful analytic tools, and entire fields of expertise around digital testing. This kind of iterative, knowledge-based improvement exists for physical products too, but it's far more hit-and-miss; atoms are harder to manipulate than bits.

There are two main reasons why products fail: lack of user need, or the user need isn't as mass-volume as previously thought.

Used in concert, these technologies could close that gap. I think there are two main reasons why products fail: lack of user need, or the user need isn't as mass-volume as previously thought. Nearly every product failure can be traced back to a decision based on this misinformation, so imagine the impact on product innovation if we knew our mistake before committing ourselves.

So when it's time to decide which problem to solve, let the Internet of Things help us do the digging. Let observant software tell us when things are frustrating or perplexing. Unleash a short run of 3D-printed products on the market, and let them tell us whether we got it right. It's at odds with the romantic notion of product design as an instinctive, magical process, but as someone who's designed dozens of products (some of them successful!), I can assure you it takes a lot more iteration than inspiration, and we could use all the help we can get. And couldn't we all do without another Zune?

Note: Bresslergroup advertises through Core77's Design Directory. This essay was independently evaluated by Core77's editorial team.