Detail of an XBee module in a prototype for the Tweet-a-Watt, a hacked wireless home-power monitoring system by Ladyada. Source: Flickr

One weekend this past July, an invitation-only group of 40 artists, designers, and researchers from design hotspots and leading institutions such as IDEO, Microsoft Research, NYU, Stanford, Umeå Institute of Design, Wired Magazine UK, and Yamaha, among others (complete list here), gathered in an (almost) secret location in London. During the fourth-annual Sketching In Hardware Conference (SH09), three big-impact themes emerged: tools to support the design of better, more complex experiences; the challenges of open innovation; and the basic conditions for open devices to become a reality.

Jan Borchers, head of the Media Computing Group at RWTH Aachen University , demonstrates the LumiNet.

What is Sketching in Hardware? The napkin sketch is the lingua franca of all design. We all do it because—hundreds of years since we started doing it—it's still the best way to get inspired, to get unstuck, to get real.

Until recently, electronic-device design has been sprinting up the steep incline of Moore's Law. Our ability to conceptualize early ideas is tripping on its shoelaces. It's hard to simplify the inherent dynamism of an electronic device—no matter how elaborate the margin doodle; it often confuses more than clarifies. And how could it not? Electronic devices are alive and interactive. They gather information about their environment or user, process values, and respond accordingly. Even the most well-intentioned sketch quickly reaches the limitations of the medium.

In a project for the Switched On London lighting festival in 2008, the Jason Bruges Studio, represented at SH09 by Daniel Hirschmann, created a series of lighting experiments on the London Bridge called "North-South over East-West." Hollow cathode lamps, controlled via sensors attached to the bridge, behaved differently given different inputs, allowing passersby to leave behind unique "colored light shadows" instead of footprints. The studio prototyped several versions of the experience and had the chance to gauge the involvement of the public, see their real-time reactions, and modify the design on the spot. If a picture is worth a thousand words, a hardware sketch—a living picture—is worth a thousand movie frames.

For "North-South over East-West," the studio members had an existing electronic module to build on—the DMX—but in most cases, designers have to create almost entirely new ones from scratch. Camille Moussette, a PhD student at the Umeå Institute of Design in Sweden, produced a series of hardware sketches that explore haptics on mobile devices for "HAPI," his collaborative project with Nokia. Moussette started with low-fidelity props to simulate the conditions of movement and vibration, and then built hardware sketches with different purposes, followed by a final, higher-fidelity prototype.

(1) Camille Moussette, "HAPI" low-fidelity exploration, a UID Interaction Design Degree Project exploring haptics on mobile devices in collaboration with Nokia. (2) Hardware sketch to get a feeling for the haptics. (3) A fully operational mock-up.

Although most of the time some sort of software is required, sketching in hardware is more about interactions in the physical world than simply on-screen representations. Hardware sketches are the building blocks of technology design. They allow the designer to explore experiences mediated by products or staged in spaces without requiring engineering support during creative phases.

The Design of Technology Early TV sets and radios were closer to furniture than to the sophisticated electronic objects that sit in our living rooms and pockets today. With the introduction of the microchip, and the subsequent miniaturization of electronic components, almost to the point of disappearance, came a new set of challenges for designers.

First of all, their designs needed to communicate the functions all those little chips could perform and indicate how a device should be used and understood. In the past, product-development processes would only include designers in the final stages. Designers would take engineering specifications and try to create the outer shell. This product-development strategy only works up to a certain point, and therefore, many opportunities to create more compelling and complex experiences were lost.

The more recent Nike+ line—a result of the partnership with Apple—delivered a more holistic experience for the consumer. Source: Flickr

A few years later, 'content' became the favorite subject of many design conversations. More powerful software and the Graphic User Interface (GUI) meant that content had to also be designed. Finally, the combination of how users would access information through physical inputs was important for good design.

Radio in a Bag, 1982, Daniel Weil. Source: http://digitalwellbeinglabs.com

In the early 80s, Pentagram partner Daniel Weil, designed a new radio (in a bag). That 'product' marked a shift. Industrial designers should no longer concern themselves with just packaging technology, but actually designing it.

Fifteen years before, Nam June Paik's art installations were great examples of how hardware and software could be designed in tandem to create complex, expressive results. Paik had full control of both the physical aspects of his work and its content.

Nam June Paik, "Electronic Superhighway", 1995. Source: Flickr

In 2001, Gillian Crampton-Smith, building on her own experience running the Computer-Related Design Department at the RCA, founded the Interaction Design Institute Ivrea (IDII) in Italy.

The IDII had visiting staff that included John Maeda, Joy Mountford, John Thackara, Bill Verplank, Nathan Shedroff, Bill Moggridge (co-founder of IDEO), and David Liddle (co-founder of Interval Research). During its short life, the IDII graduated some of the best Interaction Designers in the market. From its studios and rooms one of the main components of hardware sketching was born: the Arduino board. The Institute also collaborated in the development of Processing, a simplified programming language aimed at designers, artists, and non-programmers created by Casey Reas and Ben Fry at MIT.

At this point, a new discipline—interaction design—was created, and the first tools were in place to support the goal of "designing technology."

Map showing the influence of Ivrea on the prototyping tools we have available today (download here).

The original Arduino Board (left) and its best-selling version the Duemilanove (right).

Designing Better And More Complex Experiences SICS researcher and a member of the Sketching in Hardware community, Ylva Farnaeus, is featured on this news segment about emotional relationships with robots.

In leading companies and consultancies, designers are asked to come up with the next breakthrough in mobile technology, gaming or a health care delivery system for the future. The challenges are complex and systemic.

For professionals to tackle those difficult challenges, we need more specialized tools that would allow us to focus on the experiences and not in building prototypes. For beginners to engage in physical computing, we need to shorten the steps to the first 'Hello World' moment and also give a hand to climb the steep learning curve of Sketching in Hardware.

Tools To Improve The Design Process Designers should be spending more time creating variations of experiences and running those through users instead of reassembling the building blocks. I'm not saying tinkering is bad; don't get me wrong. But reinventing the wheel is not exactly how we should spend our precious time. Based on this assumption, I presented three new prototyping toolkit concepts (download here) during this year's conference: the Nomaduino, the Bioduino and the Gameduino.

The Nomaduino is used to sketch experiences using mobile devices. It would allow sensors and different types of screens to be attached to a main cell phone board. The designer wouldn't have to limit the experiments because of phone's technological limitations, like processor speed, or the difficulties of scaling it to a few dozen testers.

The Bioduino focuses on biometric data gathering, but comes with the same simplicity to connect sensors and send information through the Internet. On projects in the area of telemedicine, the whole physical prototype build could be spent trying out how patients would respond to service offerings.

The Gameduino would help designers quickly put together controller types using sensors (accelerometers, e.g.) and actuators (vibration motors, e.g.). It sends values through Bluetooth to a computer running the game prototype. Scenarios and characters could be downloaded from the Gameduino Community website to kick-start the process.

The Nomaduino community, the Bioduino kit and Gameduino, in this order.

Andre Knörig presented Fritzing, developed by the Interaction Design Lab at the University of Applied Sciences Potsdam in Germany. It's an open sharing platform that can be used to document and share Arduino based prototypes. Once the assembly is built virtually, the designer can order custom made printed circuit boards or PCBs.

Fritzing screenshot showing an Arduino based prototyped assembly.

Tools For Beginners and Education Shigeru Kobayashi and team, creators of the Gainer prototyping board, presented their new development in collaboration with Kayac: the physical x wonderfl. From the website one can connect a board to different inputs/outputs and try out Adobe Actionscript code in real time. It doesn't require any software installation at first and is a great platform for practitioners from all over the world to share code and different assemblies. It significantly lowers the barriers of entry to physical computing.

Ed Baafi from the SETC, presented the development of ModKit and gave a live demonstration of the new tool for beginners inspired by the MIT Scratch software. The latter was created to stimulate children to get interested in programming. It makes simple programming commands visual. Controlling hardware using ModKit follows the same rationale.

Open Innovation On one side we have companies eager to learn from users or to ask them questions to enable the design of more innovative products and services. On the other side there is a legion of DIYers building the next big thing, one weekend project at a time.

According to Lawrence Lessig, Stanford professor and a founding board member of the Creative Commons, Technology is responsible for bringing us back from a Read Only (RO) culture to a Read-Write (RW) one. RO and RW are computer terms that describe how discs can either be read or if they can also support being rewritten. Human beings have been RW during most of our history. But for some time we stopped creating things and started consuming them instead.

During TED, Lessig presented the story of John Philip Sousa. In 1906, Sousa went to the senators to complain about a device he called "talking machines" (the gramophone). Showing all his disregard and dissatisfaction with the emerging recording industry he reminded the senators: "When I was a boy...in front of every house in the summer evenings, you would find young people together singing the songs of the day or old songs." And he continued saying that when one person learns to sing a song and teaches it to someone else, days or years after, creative changes and additions can be made. Therefore, a recording industry would only encapsulate and allow for repetition. As a consequence, according to Sousa, that would slow down innovation.

These points were brought up on Mike Kuniavsky's presentation about RW Material Culture (download here). The connection with Material Culture extends Sousa's and Lessig's ideas to artifacts, not just music or poetry. But when it comes to Open Innovation what seems to be a simple case of connecting companies to people and vice versa requires a bit more than setting up an online forum.

Bug Labs GUI redesign was done as an open project at IDEO. Source: Bug Labs client page

Dave Vondle, an electrical engineer at IDEO, presented an open project in collaboration with Bug Labs. The company approached IDEO asking for a redesign of its device's graphic user interface. The Bug is a Linux based prototyping kit that offers a range of inputs and outputs that can be easily connected to create custom or prototypical devices. The lessons shared by Vondle should inspire many companies on how to bring open innovation from their internal PowerPoint presentations to real life. Some highlights were:

• Communities work when the topics are clearly articulated and the response request is similarly clear.
• Know your audience; communicate in their language.
• Community doesn't come easy or free.
• Profiting from a community is delicate.
• Honest feedback improves quality.
• The more you put into it, the more you get out of it.

The processing power in many mobiles phones, GPS navigation systems or digital cameras is far greater than that of the computers used up to the early 90s. However the PCBs on any of those are designed so that no one can tamper with or add software to the microprocessor. That's unfortunate for you and for the manufacturer. You could be having lots of fun tinkering and creating new uses and misuses. And those could feed back as sources of inspiration for the company that designed the camera in the first place.

Tom Igoe's proposal for what he called Flexible Warranty, presented during his talk at SH09, has as 'real examples' many Canon digital cameras. They are famous for being quite easy to hack and a whole community was created around it. Surprisingly, when asked about how that would affect the camera's warranty, Canon's reply was that as long as the hack doesn't make any actual changes to the camera, the warranty is not affected.

In this case, the reprogramming is done with the use of an SD card, but Tom was suggesting that companies should leave open doors for users to get to the chips by exposing the firmware programming port. That's what he calls extensible firmware. Ultimately it would allow anybody to take any electronic device with whatever capabilities it offers and use it as a prototyping platform.

A much simpler version of that is having open APIs. For instance, Application Programming Interfaces allow one to connect a computer program with Twitter or Google Maps. Through APIs, the program designer can use variables to display a graph or simply read data, like your 20 most recent Tweets. Both the iPod and the Wiimote have open communication protocols (you can't reprogram them, but you can speak to them). The latter has been widely used by the sketching in hardware community to build prototypes where accelerometer and buttons are required as inputs.

What's next? We are coming from years of passiveness to the possibilities of designing and modifying pretty much everything around us. In the coming years, we will see open innovation happen because somebody, somewhere around the globe used an open device to create a new application of an electronic product that nobody could predict. Open doesn't mean that everything will be free but, we will all benefit, from companies to individuals.

Wanna know more? A comprehensive list of prototyping toolkits. A great collection of references and links from theory to components. Interaction Design on Wikipedia. The Sketching in Hardware conference is organized by ThingM. All the presentations can be found here and the notes here.

Fabricio Dore is an Interaction Designer at IDEO Munich. With a background in Industrial Design, he works in the intersection between hardware and software on the development of products, services, environments and systems. His experience spans across communication design, furniture, consumer electronics, retail spaces, packaging, game design, automotive interaction and social software. Trained on the Scandinavian tradition of empathic and human centered design he delivers solutions for clients such as Ford, Chrysler, Volvo, Microsoft, at&t, Massachusetts General Hospital, Vodafone, Nokia and YelloStrom.

Thanks to the Sketching in Hardware community for offering constructive feedback and suggestions on accuracy.