The Interface Is the Message
By Ben Delaney © 2001

This article first appeared in IEEE MultiMedia, January-March 2002

The Ars Electronica Festival, a leading confluence of artists, computer geeks, and scientists, took place for the fifth time in Linz, Austria, this past September. People from around the world came to explore the concept of Takeover: Who is Making the Art of Tomorrow. While the implied question wasn’t decided in any substantial way, there’s no question that the art of tomorrow will often be based on the interfaces between humans and computers. And if only one irrefutable truth emerged from the often tangled ideas presented during the week of the festival, it’s that the interface of tomorrow will be substantially different from what we’re accustomed to today.

Alternatives to Xerox PARC’s WIMP (windows, icons, menus, pointing device) interface, popularized by Apple’s Macintosh, have been tried for years, and some are making their way into the mainstream. Voice recognition is finally almost here—as it has been for the last 10 years. Bar coding is now commonplace, and force feedback is used in many devices. Various sensors and transceivers make it easy to track goods and people as they move through environments. Add to all that a variety of emerging biometric systems that can recognize individuals with ever-increasing accuracy. Most of these systems, however, are extensions to WIMP and, for the most part, still rely on that grizzled standard.

As the mouse grows gray around the whiskers, artists and computer scientists are showing new interface designs at Ars Electronica and other venues. Many of will never become mainstream, but some could point the way to novel modes of interaction between people, computer systems, and the rest of the physical world.

Something fishy

Artist and computer geek John Klima makes art that explores how the interface affects our interactions with the real world.. His pieces, which have been shown in New York’s Whitney Museum among other venues, force us to take a new look at how we interact with computers and what computers are for.

Klima recently showed an art piece called Fish, which is a Rube Goldberg contraption of wires, hoses, pumps, and both live and virtual fish. Presented as a game, the concept is fairly simple. The player’s task is to guide a virtual fish through a virtual underwater maze to safety. In one section of the environment lurks virtual carnivorous fish, which try to eat the innocent goldfish the player is guiding. When played on the "aggressive" level, it’s almost impossible to survive the maze. Klima runs the system in a modified video arcade game box, using the original joystick and buttons.

There’s more to Fish, however, than just a 3D video game. The game cabinet sat off to the side of a fairly large room, which was dominated by five spherical plastic fish tanks. Four tanks contained live goldfish. In a large central tank, a Jack Dempsy and Oscar hung out—both hungry, carnivorous chiclids many times larger than the goldfish. If the player won the game by guiding the virtual goldfish through the maze, a real goldfish was pumped into a safe holding tank. If the player lost the game, and the virtual fish got eaten, a live gold fish was pumped from the safety of the holding tank to the peril and sure death of the carnivore’s tank.

Figure 1. Artist and technician John Klima shows some visitors around the Fish installation at the Postmasters Gallery in New York City. Photo: Ben Delaney

Klima has turned the interface to a life and death decision into a game, and in the game the ultimate reality isn’t immediately obvious. He sees the intentional obscurity of the interface as part of the artistic equation. The interface requires discovery, which makes the art more interesting. Furthermore, this work asks if a simple interface can be adequate for serious and consequential work. It makes us wonder if the bewildering interfaces to complex processes, such as power plants, might be easier to understand and use if they were a bit more artistic, game-like, and immediate. What’s more, playing a game with real consequences (even if they are only important to a goldfish) forces us to addresses the meanings of our actions. This is an interface with a conscience, an idea that makes sense in these days of escalating, mindless, consequence-free computer game violence and highly consequential computerized weaponry.

Sophisticated parties

While Kilma’s work uses a fairly conventional interface to connect the user to a decidedly unconventional application, Ars Electronica gave visitors the opportunity to test novel interfaces to novel applications, but with more mundane goals.

Imagine a dance club where the DJ does her best to match the music to the crowd’s mood. If the activity level seems low, she might want to liven things up, but if the patrons are getting too rowdy, she might slow down the pace a bit. A good DJ takes a crowd’s pulse and adjusts her music and lighting to achieve the desired results.

Now, imagine a similar club where the DJ is replaced by a computerized system with thousands of MP3 recordings on its massive hard disks and a broadband Internet connection through which it downloads hot hits on a moment’s notice. Ars Electronica offered one solution for how this Robo-DJ might gauge a crowd’s mood. A Sophisticated Soirée was an after-hours lounge where the visitors wore bio-monitors that modified the music and projected imagery via a computer interface, providing a variable ambiance based on an unconscious consensus of mood.

Developed by an Austrian team calling themselves 91v2.0, A Sophisticated Soirée included 64 pre-registered participants at once. Each was fitted with pulse monitors—a pair of small, self-adhesive, disposable electrodes attached to the skin. The monitors connected to a small wireless transmitter, and all 64 broadcast simultaneously. The data stream, 64 bits wide, is then used by a PC to control the music and projected visual images in the room.

This system provided no direct feedback to the individual participants. Their input was limited to their pulse rate. A change in the consensus pulse rate from all 64 participants, averaged up or down, caused changes in the room’s ambiance. While no one person could consciously change the environment, the group’s involuntary consensus could. Of course, the reverse is also true; the environment affected the participants, creating a closed loop system. Inspired by Neal Stephenson, it reminded one of William Gibson’s definitions of cyberspace in his book Neuromancer: "A consensual hallucination experienced daily by billions."

How about a drink?

How many times have you been in a bar and had your mind go blank when a server arrives to take your order? Wouldn’t it be great if a bartender could read your mind and deliver exactly the right drink, even if you didn’t know what that drink would be? Another project shown at Ars Electronica offered just that option.

Brain Bar was an unimposing installation consisting of a chest-high pedestal on which a circle of vials containing various fluids surrounded a small set of pumps and valves. All this in turn is in the midst of an open-centered turntable, which moves a cocktail glass under each vial’s pouring tube. A robot bartender mixed cocktails based on a brainwave reading from the customer, who wore a light, sensor-studded headband.

Figure 2. Step right up to the BrainBar. Instead of telling the bartender what you want, don a brainwave sensing headband and see what your brain says you need. Photo: Sabine Starmayr

The system was built by Smart Studio (http://smart.interactiveinstitute.se/smart/smart_sv/brainbar_sv/ ny_brainbar.htm) around off-the-shelf amateur electroencephalography equipment (the Interactive Brainwave Visual Analyzer [http://www.ibva.com]), connected to a Mac G4 cube. A second computer controlled the machinery that mixes the drinks. The systems measured alpha and beta waves over 30 seconds. Then, using a custom algorithm, the system made a rough estimate of how relaxed or agitated the customer is. The bar chose from a menu of 10 traditional drinks, mixed from eight ordinary bar ingredients: vodka, gin, rum, tequila, sour mix, Coca Cola, milk, and Kahlua.

Fredrik Petersson, creative engineer at Smart Studio, described how people react to the system:

We were surprised by how well this is accepted by the people watching it. It’s interesting to see that people don’t hesitate to accept a drink from a machine. Watching the reactions of people approaching the bar is really what makes it interesting. Two people who would otherwise seem extremely different might get the same drink from the bar. Being a machine, it uses a different set of metrics to select a suitable drink for a person than most other bartenders might have. It makes people wonder, asking themselves, What do we have in common?

IPaq as advanced interface device

The Ars Electronica Center includes its own experimental center, Futurelab, that investigates innovative ways to combine computers, society, industry, and art. The Futurelab took a stab at addressing two major drawbacks of CAVE display systems, the high cost of their image generators, and the typically obscure control and navigation systems they use. They built a CAVE-like system, the PC CAVE, using standard PCs and commercial off-the-shelf graphics cards. That solved the cost problem.

A new navigation/control system addresses the second problem. Called Palmist, it is a Compaq IPaq hand-held computer, running Windows CE. Hardware modifications and clever programming make it both a controller and a navigation device for the PC CAVE. An Ascension tracker is attached to the IPaq, which has wireless access to the local area network. A simple, touch-operated control and navigation system is displayed on the IPaq’s color screen. The interface lets the operator choose what simulation to run, control the CAVE hardware, choose objects seen in the simulation, and navigate through the virtual environment. The system can also display and facilitate modifications of the scene graph. A small, inexpensive package, the Palmist is an elegant solution to the nagging problems of navigation and control in CAVEs.

Figure 3. Participants can pick objects in a complex environment and display information about them with the Palmist control and navigation device. Photo: Ars Electronic Futurelab

The Futurelab’s PC CAVE was a single-screen system that can be built for less than $10,000 per screen. The Palmist controller is similarly low cost. More information is on their Web site at http:// futurelab.aec.at/homepage/ show_pro.asp?PID=305.

While none of these interface devices are ready for mainstream computing, they present a wealth of alternatives to the WIMPy interface that nearly all of us use, and complain about, every day. Expect to see some of these innovations at your local computer store soon.

Ben Delaney is president of CyberEdge Information Services. He writes frequently about advanced technology and interfaces. Reach him at ben at cyberedge.com.

Questions? More Info? Email the Webmaster

407 M. L. King Jr. Way, Oakland, CA 94607 USA
+1 510 419-0800

Our privacy policy may be read here.