Toshio Iwai's interactive art

“This game is different. It's designed to produce harmony, not adrenaline. So far, we are seeing different kinds of reactions to [Electroplankton]. Some test players are confused; they keep looking for their score or the next enemy.”
_{— Satoru Iwata, 2005}

if you grew up playing games, picking them out based on preview coverage and preorders could be considered a luxury when you also likely had parents or well-meaning relatives that gave you games as gifts. this is usually how one would end up with at least one piece of shovelware that got blown out at clearance, which promptly got swapped around with friends because the stakes of ownership were about as low as cafeteria table trades. in my case, my dad was the parent that did his homework where I expect most adults didn’t care to bother, and he routinely picked out winners for christmases and birthdays that are still spilling over into my preferences today, introducing me to games like katamari damacy, warioware, and tetrisphere. some disappointments still wormed their way into my collection, which you have to expect and learn to smile through when it comes to gifts, but it was a surprisingly eclectic taste that I can only attribute to someone being able to appreciate games as a creative medium rather than a simple time sink. that being said, there’s really only one game that flags in my mind as being truly eclectic from that time, and my dad is maybe the only way I could have also ended up with electroplankton, a DS game that in the americas only ever saw a release online and on shelves at nintendo world in new york city. I was likely too young to have any truly nuanced ideas about what made a game “fun” back then, but I still remember having to ask myself even at that age when I was poking at it: is this thing really even a game?

games are forms of play at their core, and electroplankton itself probably falls somewhere in the lineage of toys that includes fidget spinners and stress balls. it’s usually thought of as a music performance tool, and has ten different modes to play around with, but it’s a tool that lacks even basic trimmings you would expect out of something that had serious intentions of being one like the ability to save your work. other music tools in the DS library like korg DS-10 did eventually come along to be more practical options for production; for electroplankton, however, it was presented foremost as an ephemeral experience, and your performance was never expected to have much permanence. this limitation somehow manages to end up feeling appropriate being conceived by toshio iwai, an interactive media artist that has flirted with making games for almost their entire career. most of his work has only found acclaim being exhibited in museums, which unfortunately is always a limit to reaching a wider audience. in japan, his name commands starpower in certain contexts, and allowed him the distinction of being displayed proudly as electroplankton’s sole creator right on the box, which elevated him as an equal among labeled game auteurs like kojima, sid meier, and michael jackson¹.

online and in magazines of 2005, electroplankton had an understood but limited appeal among the majority of players that kept close to well-crystalized genres. published by nintendo, the company was riding the tail end of their attitude era at this point, as advertising campaigns imagined in lockstep with sega were having their last heyday with “touching is good” as the DS’ taunt, focus began shifting to the expansion of new audiences, and reggie was now one year past being the fresh face that was a fan favorite for punchy soundbites. their E3 2005 press conference is maybe the most split-brained the company has ever looked, where the wii was unveiled for the first time beating the drum of a “revolution” championing backwards compatability alongside a zelda that was ready to bleed all the color out of wind waker, but it was electroplankton instead that earned top billing that day as a creative demo. it’s jarring today to see an E3 press conference feature statistics and charts before corporate suits realized such things could go out live on spike TV or get chopped up into clips online. it’s really jarring to see something so niche touted immediately after boasting about software sellthrough rates.

reggie presented electroplankton as a leap from the left brain to the right brain, giving us a quote said with his usual certainty that feels almost offered to critics: it may not technically be a game at all. this is a loaded modifier today, and is still likely to be used as a putdown for the wrong reasons. for a nintendo that was eyeing new audiences, though, it’s a perfectly forward characterization — there is perhaps no more of an honest capture of iwai’s work in games when you examine his output. unless you knew who that was, though, you’ve probably never had a chance to understand what electroplankton was going for or in what sort of context it existed.

Interactive art outside of games

toshio iwai has been professionally active now for over fourty years. when you hear that someone is an “interactive artist” or makes “media art”, it’s tempting to conclude that they work mostly in digital mediums. fourty years ago, this wasn’t such a foregone conclusion, and iwai instead got his creative start in the analog medium of flipbooks, made during his downtime in junior high. these were crudely done, if we’re to judge the examples we can find online, and many people are probably happy to develop their talent up to this level or to instead translate this kind of skill more literally into animation (with flipbooks often being cited as an influence for an ample number of traditional and webgen animators). for iwai, the fascination wasn’t so much in the illusion of movement, but instead the mechanics in how it could be created, interacted with, or modulated. that curiosity is what has informed effectively all of his work, and it’s an attitude worth keeping in mind as you explore his portfolio.

zoetropes and phenakistoscopes often are described as “pre-film” devices, stepping stones on the way to film’s development, when in reality they are distinct and tactile experiences that deserve better classification. iwai took a natural fascination with them for this reason, as their movement was not always controlled on a fixed cycle of frames with precise timing. some variants of zoetrope do make passive observers out of the audience — linear zoetropes were introduced as a novelty on tokyo’s subways in the 90s — but iwai at this point already demonstrated a focus on making his art participatory. his early pieces operated largely on fixed motors, but it was not long after that his more complex 3D zoetropes offered crankshafts that allowed changing their speed. audiences had already became players and participants in his experiences, as he rejected the notion that they were only allowed to be simple observers.

iwai also quickly adopted music prominently in his installations, a thematic element that later would become his trademark. while they tend to lack overall melodic structure, saying he employs “sound” as some critics do diminishes the role they have augmenting his pieces. in most cases, iwai uses audio as direct feedback for his interactive visual elements, which gives them similar purpose as sound effects might when landing a punch or a jump in a game. iwai diverges from this simple modality when pressing a button or twisting a knob by giving his sounds a visual permanence that make them inseparable from the scenes themselves, adding another singing plankton to an ecosystem or making a record glow brighter with a hum the faster it’s spun. for many of his works, this gives the audio life that would better qualify them as sequencers or synthesizers. one action can radically alter your experience and your performance, yet you won’t feel pressured to achieve a perfect tonality or musicality — the thrill of the interaction is nurtured entirely by the exploration of the medium becoming a full physical extension of your touch.

“What I want to do is create media. When a medium first arises, a whole world of new potentials also comes into being. There is no such thing as ‘correct use’ at the beginning. The only thing that people have to go on is the elation — the sense that this new thing is really interesting,”
_{— Toshio Iwai, 2002}

iwai’s later works became bridges to digital interaction in more exotic ways that would avoid physical touch altogether. subjects in another time, another space (1993) learned to move around as their bodies were manipulated with effects in realtime on monitors reflected back to them. while apple’s photobooth probably impresses more at home today, iwai was able to augment participants passively here by observing them perform movements (a prelude to ideas like the kinect, maybe). this same principle informed other projects like sound-lens (2001), where sonar was used as a musical instrument to generate sounds and assign personality to lighted objects. iwai reportedly led participants around harajuku equipped with headphones and focused receiver discs, like space-age metal detectorists, stopping to focus them on vending machines or street lamps so they could appreciate their unique sound signatures. iwai here brings the magic of discovery back to a world that qualifies electricity as a utility, engaging our unseen senses and perceptions, by personifying its use in silent performers and demonstrating its enduring presence. the senses are stimulated not by grand coordinated symphonies or an assault on them, but by unlocking a simple creativity from probing the environment.

iwai has plenty more projects with an invisible depth, particularly the experimentally mad children’s show ugo ugo luhga that was produced live with amigas and video toasters, which alone did plenty to raise his profile². as I alluded to earlier, however, iwai’s application of sound became a defining trait for people that knew his name. a lot of that is credited to his work in games, where he imagined controllers as instruments more than simple input devices.

Quantization of sound and unique control

“I became enthralled with playing new games [growing up]. When I first got a chance to view a prototype of the Nintendo DS and Nintendo invited me to create something with it, I was just as excited. I thought, "What could I do with it?" I then wanted to combine all the things that had captivated me—observing plankton through the microscope; recording my voice and sounds around me with the tape recorder; making all kinds of sounds with the synthesizer; and the fun of controlling images and sounds with the NES. The end result was Electroplankton.”
_{— Toshio Iwai, 2005}

iwai’s approach to games has as much to do with his high-minded art sense as his technical acumen, with most of his installations being made possible by his own programming work. ugo ugo luhga is an indelible example here, where iwai wired NES controllers directly to the amigas generating the show’s CG characters to have their limbs controlled and mouths flapped in realtime. the show was groundbreaking in many senses, not only for its on-the-fly compositing that mixed digital worlds and bluescreened child actors, but also for its viewer participation that reimagined the world of call-in shows away from talking heads. sumo matches using scanned art sent in by avid viewers had callers shouting into their phones to push their fighter forward, with whoever shouted loudest pushing their opponent out of the ring.

the fact that iwai mapped expressive movement to relatively simple controllers should not be considered an accident, as it reveals a core approach of iwai’s design of input with games: quantization. without going too deep into specifics, quantization is the method by which a sequence of continuous possibilities (say, every frame) is mapped to a smaller set of possibilities, like each beat of a song. in the context of music games, this allows for the effect where when a button is pressed to trigger an action, it can “snap” to a defined beat that makes it sound like part of a cohesive whole, even when it’s pressed slightly out of time. in tandem with ideas about redefining input and interaction, iwai was able to leverage these concentrations in ways that brought true innovation to his games.

Otocky (オトッキー) (1987)

otocky is a simple shmup for the FDS that takes the directional pad and assigns each firing direction a musical note, quantized to a triangle wave that tirelessly marches along. to see iwai designing for the famicom is not much of a surprise, as the famicom was already well established by nintendo as a sort of philosophical blending of a game console and general computer that encouraged divergent software, and the controller offered tremendous familiar flexibility. there is a limited freedom to what’s possible in otocky, but you can freely spam your fire button in multiple directions to establish chords and also collect items that change the sound of your fire, encouraging you to adapt your play to establish your own musical style. most sources regard this now as the first example of generative music in games, making it likely the first example of media art in games as well. rez makes similar explorations of musical evolution that are hard to ignore in comparison, and otocky obviously deserves credit for being first in that lineage, though at least one UGA interview indicates it was only passively explored there during development. nonetheless, while it’s probably not going to rank on any listicle of unearthed NES gems, it’s an undoubtedly unique approach that shares consequence with other NES titles that established conventions on the platform and lends early prestiege to an iwai attempt at games.

Sound Fantasy (1994) / SimTunes (1996)

sound fantasy is unqualifiably one of the most consequential unreleased games that never came to pass. produced in conjunction with gunpei yokoi, while it was fully completed for the SNES and received coverage in magazines, it now only lives on as a prototype cartridge that was dumped online twenty years after the fact. the game was slated to retail in a big box, like those that championed mario paint and earthbound, and also was expected to come bundled with a SNES mouse. some evidence pins the game at such a late stage in development that it was already released to manufacturing, with iwai having a complete in-box copy of the game.

the game offers four different modes (only three in the dumped prototype) that range from inspired to yet another copy of arkanoid. “pix quartet” is undoubtedly the game’s featured mode, as it was seeded from iwai’s music insects (1992) where participants used a trackball to place musical pixels to a canvas that virtual bugs ran across to vocalize. this creates a split mode of play, where the player can focus on either authoring an image, creating a sophisticated composition, or for proper effect, both. you’re perfectly free to drag your mouse across the screen to create a snowy storm of pixels, and the bugs are also perfectly happy to trot along whatever path you give them to make it sound beautiful. it’s a balanced approach to the creation of audio and visuals in cooperation where other approaches like black midi instead exist only at extremes.

a world where sound fantasy was released is likely the story of one where it would have been a market flop. while iwai claims he doesn’t know why the game was canned, the inclusion of the mouse would have certainly meant an inflated price at retail, and the eccentricity of it being an exploration tool makes it an already hard sell at the outset. figuring out how to market this thing was probably a dilemma, even if it could have been repackaged with mario characters adorning it as a counterpart to mario paint. should it have gone to retail, yokoi would have been attributed to its success or failure, which could have impacted the fate of the virtual boy or its eventual (incorrect) perception as being responsible for his departure from nintendo.

for iwai, he eventually recovered the time spent on sound fantasy by developing simtunes in conjunction with maxis after that team took an interest in his work during a field trip to his art residency in san francisco. unlike most of the other titles in the pre-EA maxis library bearing the “sim” name, simtunes didn’t find itself trying to simulate any real world experience and never strayed too far from pix quartet’s core. it’s a testament to iwai, however, that his vision earned recognition from the studio that brought life to some of the games that have best realized the exploration and interaction between systems while still maintaining an overarching quirkiness.

Tenori-on (テノリオン) (2001)

“In days gone by, a musical instrument had to have a beauty, of shape as well as of sound, and had to fit the player almost organically…. Modern electronic instruments don’t have this inevitable relationship between the shape, the sound, and the player. What I have done is to try to bring back these…elements and build them in to a true musical instrument for the digital age.”
_{— Toshio Iwai}

perhaps the most “non-game” of any of the examples we’ll look at, tenori-on began life as a basic sequencer toy on the wonderswan that was produced and sold by iwai, with most reports quantifying this limited release at around 100 copies. this is no doubt a follow-on from iwai’s appreciation for yokoi, where iwai in retrospectives has exalted that the genes of yokoi were inside of him. recognizing the potential to graduate tenori-on from a trinket to a piece of equipment, yamaha years later contracted iwai to produce the concept as an instrument. looking at its final form brings to mind more heralded examples of grid-based sequencers that followed, like the novation launchpad or the polyend seq, but iwai envisioned his device more directly for performances rather than production. while the aforementioned tools sit on a table and may instead rely entirely on being enhanced by additional devices, the tenori-on’s lights were visible from both sides when it was held up, allowing the audience to peer in on the action, and could also be run on AA batteries that gave it portable flexibility like a game boy. none of iwai’s desire for accessibility is lost in this jump to the professional world, and anything you sequence to test what’s possible will probably come out sounding, at the very worst, not unpleasant.

yamaha’s tenori-on can reasonably be considered iwai’s climax, enjoying two years of breathing room after electroplankton and was well received as a cult hit. as we move back to examine electroplankton after inspecting the rest of iwai’s output, it’s revealed to us that many of the modes here are really just repackagings of his prior works with a new theming. luminaria is lifted directly from composition on the table (1999), with seemingly all of the samples there being used in the game. lumiloop is just time stratum III (1989). while iwai did make plenty of live appearances to demonstrate electroplankton as a creative tool, often spinning multiple DSes at the same time to audiences, the tenori-on saw heightened success as a performance tool that captured the attention of names like hans zimmer and bjork³. electroplankton may have failed to capture that same attention, but so much of it still informs the tenori-on’s character, from the way patterns bounce being illuminated to the samples imbued with a slight echo like they were also recorded deep on the ocean floor. neither deserve to be appreciated without the other: electroplankton in some ways is the totality of iwai’s experience creating media art, making full use of the microphone and touch screen, while the tenori-on demonstrates how that knowledge looks when exercised from end to end at the hardware level, as yokoi would have.

iwai’s impact is invisible to most of the world, which is expected for artists that traffic in the experimental. today, it’s quaint to examine the interaction we have with digital devices, when the process for creating interfaces and cheap plastic technology is so easy that many interfaces are created without really any care at all. iwai serves as a reminder that interactive art has the power not just to discover new possibilities, but also to act as a release for them when they’re too eccentric or impractical to garner attention any other way. iwai committed himself to creating interfaces that celebrated play, and while he was eventually able to see true commercial success, a lot of his work opened up paths of interaction that took years to be realized. as we settle into an era of computing that embraces the sterile, the same yet again as before, iwai’s works continue to recenter what I think we would be wise to stay mindful of when we perform design today: from the point of creation to completion, embrace the quirky, and make things that are fun not just for people that you consider to be your users, but your players.

---