The computational gaze

Image: Tim Ellis. Flickr

I’ve written and spoken before about what I call mechanomorphism — a word that I developed to describe the concept of machine intelligence as a companion species. This framing of AI is distinct from anthropomorphism, where we try (and inevitably fail) to make machines approximate human behavior. Instead, I envision a future where we appreciate computers for the ways in which they’re innately “other”.

Another way to put it is that I’m fascinated by the computational gaze — how machines see, know, and articulate the world in a totally alien manner. I’ve been talking a lot with my boss, John Maeda, about computational literacy and how to help people understand foundational concepts of computing. But computational literacy posits the machine as a tool (which it often is!). The computational gaze, on the other hand, suggests the machine as a collaborator or companion intelligence.

Collaborating with machine intelligence means being able to leverage that particular, idiosyncratic way of seeing and incorporate it into creative processes. This is why we universally love the “I trained a neural net on [x] and here’s what it came up with” memes. It has this delightful “almost-but-not-quite-ness” to it that lets us delight in the strangeness of that unfamiliar gaze, but also can help us see hidden patterns and truths in our human artifacts.

The increasing accessibility of tools for working with machine learning means that I’m seeing more examples of artists, writers and others treating the machine as collaborator — working with the computational gaze to create work that is beautiful, funny, and strange. Here are some folks who are doing particularly interesting work in this arena:

Visual feedback loops

In the visual arts, Ronan Barrot and Robbie Barrat have a show in Paris where they collaborate with a GAN to paint skulls. “It’s about having a neural network in a feedback loop with a painter, influencing each other’s work repeatedly — and the infinitude of generative systems.

Mario Klingemann has also been playing with GANs in his “Neural Glitch” series:

“Neural Glitch” is a technique I started exploring in April 2018 in which I manipulate fully trained GANs by randomly altering, deleting or exchanging their trained weights. Due to the complex structure of the neural architectures the glitches introduced this way occur on texture as well as on semantic levels which causes the models to misinterpret the input data in interesting ways, some of which could be interpreted as glimpses of autonomous creativity. 

—Mario Klingemann
Mario Klingemann, Neural Glitch

Writing with machines

Alison Parrish does wonderful creative writing work in collaboration with generative systems. Some of her highlighted work is here, and many projects have open-source code or tutorials. Here’s an example of Alison’s Semantic Similarity Chatbot, which she describes as “uncannily faithful to whatever source material you give it while still being amusingly bizarre”.

Alison Parrish, Semantic Similarity Chatbot

I also often come back to Robin Sloan’s “Writing with the Machine” project from a couple of years ago, where he trained an RNN on a corpus of old sci-fi stories and used it to auto-suggest sentence completions in his text editor.

Robin Sloan, Writing with the Machine

Enjoying the weirdness

From a more playful perspective, I particularly love the work that Janelle Shane has been doing, documented on her site AI Weirdness:

I train neural networks, a type of machine learning algorithm, to write unintentional humor as they struggle to imitate human datasets. Well, I intend the humor. The neural networks are just doing their best to understand what’s going on. 

— Janelle Shane

Here’s her illustration of some of the cookies her neural net came up with when trained on cookie recipes:

Janelle Shane’s neural net-generated cookies

Machines cheat in bizarre ways

One of my favorite things is seeing how machine learning systems will find bizarre ways to “cheat” in order to fulfill the goals that are set for them. Recently, there was a lot of discussion around this AI that steganographically encoded invisible data into maps in order to achieve the stated goal of recreating aerial imagery from said map. There’s also a fantastic Google sheet that describes all the ways various AI systems have found unexpected and strange workarounds!

Indolent cannibals

In an artificial life simulation where survival required energy but giving birth had no energy cost, one species evolved a sedentary lifestyle that consisted mostly of mating in order to produce new children which could be eaten (or used as mates to produce more edible children).

The literal computational gaze

This last piece is not about machines as collaborators, but is still one of my favorite pieces in that it so powerfully evokes the sense of the machine’s alien gaze. This is from 2012, and is a video by Timo Arnall called Robot Readable World.

I find this kind of work delightful and meaty, and I hope to see more of it. As soon as I learned to code, I started making generative things — fake ad generators, chatbots, etc. I loved making work that, even though I had shaped it, continued to surprise me. I felt warmth and curiosity towards my strange mechanical collaborators. In a moment where the computational gaze is being used in so many exploitative and questionable ways, I hope that there is also space for work that allows us to explore all that is delightful and creative about our computational companions.

  • Category: