As a collaborative project between an American graphic designer and Japanese music composers, the interactive webpage “Patatap” does not immediately reveal its literary possibilities. Its opening blank screen offers only a gray-colored background and one message: “Press any key, A to Z or spacebar, and turn up speakers.” Upon following orders, the experience suddenly takes off. Typed words turn into explosions of sound and color, and their sense-mixing is reminiscent of the muse for their project: Synesthesia, a sensory phenomenon in which letters (Grapheme Synesthesia) or sounds (Chromesthesia) appear to have their own inherent colors.
The graphic designer behind the piece, Jono Brandel, additionally specializes in computer programming. After studying Media Arts at UCLA, he began working in Google’s Creative Lab as part of their Data Arts team. Channeling those acquired skills, he makes the wheels of Patatap turn with a lengthy scroll of coding. The project utilizes an HTML-based webpage, complete with an embed code so players can place Patatap on the webpage of their choosing, or share it on social media.
The composers handle the corresponding notes to Brandel’s designs. Nagoya-based duo, Lullatone (also known as musicians Shawn Seymour and Yoshimi Tomida), lend their dreamy, whimsical sounds to the piece, expanding the sensory experience to an auditory level. Together, Lullatone and Brandel offer people a chance to see and hear what letters, words and sentences look and sound like.
In withholding a ready-made story, the creators of Patatap encourage players to experiment. They can type the alphabet, their name, an invented narrative, a classic text…the literary possibilities are endless. In my own experimenting, I started off by grabbing a dictionary, flipping through the pages and typing words I happened to fall upon. For example, the word “Breath” becomes a series of visuals: a circle made out of little white triangles, a shooting white bar, a waning orange circle, a bright flash on the screen, a gray loop that disappears as it moves and a zipping black diagonal line. It starts off sounding like clapping and drum beats before ending on a shimmery, high-pitched finish.
Hitting the spacebar changes the color palette of the page, allowing for some variation in how words look and sound in different dimensions. Pressing the space bar several times took me from the gray background screen to pale lilac, tomato red, deep blue, eggshell white and dark purple before returning to the original gray screen.
In line with the site’s surreal atmosphere, the next book I paired with Patatap was Alice’s Adventures in Wonderland by Lewis Carroll. I combined the two by simply typing in text from the edition I had on hand (Signet Classic, 1960). Before typing chapter five’s title, “Advice from a Caterpillar” (49), I set the screen to the deep blue palette. The title soon mutates from a squiggle of sea green and yellow, sounding hiccups and echoes for the word “Advice.” Its color scheme changes as it spatially progresses to other words, landing “Caterpillar” in a zooming hum of pink vibrations and black shards in the gray scheme once more.
When put through Patatap, the entire chapter sounds like a symphony. It rings, beeps, clicks, pops, hushes, sings. Meanwhile the screen seizes through confetti dynamite, rotating beige hexagons, oozing splotches of violet. The sensory experience is evocative of childhood, a time period where media focuses heavily on colors, shapes and sounds to convey word meanings. The overall effect is engaging and somehow calming, as the translated text becomes bodily perceptions.
However, not every text fits well with Patatap’s bright and upbeat interpreter. When a more somber novel like Jane Eyre is applied to the site, its mood doesn’t quite match the atmosphere of the scenes. As Helen Burns dies, her “fit of coughing” (118) sounds like jazzy elevator music mixed with animation from a children’s show. This imperfection is to be expected, since Patatap isn’t designed to understand word meanings. “Death” yields neon colors and movements just as much as “kiss” does.
Still, Patatap’s virtual union with words is an exciting and absorbing experiment. In letting literature express itself through visuals and sound, Patatap offers a redefinition of the reading process. It takes the imaginative process formerly confined to the mind and reflects it through the body. Words receive their own movements and noises, revealing these individual quirks as the reader progresses through a text. As a whole, Patatap shows viewers how to read all over again, in a new way.
Bridget Sweet was a student of Dr. Lisa Swanstrom for a course in Science Fiction & Simulation taught at Florida Atlantic University in the Spring term of 2014.
