Category: Uncategorized

I’ve written and spoken before about what I call mechanomorphism — a word that I developed to describe the concept of machine intelligence as a companion species. This framing of AI is distinct from anthropomorphism, where we try (and inevitably fail) to make machines approximate human behavior. Instead, I envision a future where we appreciate computers for the ways in which they’re innately “other”.

Another way to put it is that I’m fascinated by the computational gaze — how machines see, know, and articulate the world in a totally alien manner. I’ve been talking a lot with my boss, John Maeda, about computational literacy and how to help people understand foundational concepts of computing. But computational literacy posits the machine as a tool (which it often is!). The computational gaze, on the other hand, suggests the machine as a collaborator or companion intelligence.

Collaborating with machine intelligence means being able to leverage that particular, idiosyncratic way of seeing and incorporate it into creative processes. This is why we universally love the “I trained a neural net on [x] and here’s what it came up with” memes. It has this delightful “almost-but-not-quite-ness” to it that lets us delight in the strangeness of that unfamiliar gaze, but also can help us see hidden patterns and truths in our human artifacts.

The increasing accessibility of tools for working with machine learning means that I’m seeing more examples of artists, writers and others treating the machine as collaborator — working with the computational gaze to create work that is beautiful, funny, and strange. Here are some folks who are doing particularly interesting work in this arena:

Visual feedback loops

In the visual arts, Ronan Barrot and Robbie Barrat have a show in Paris where they collaborate with a GAN to paint skulls. “It’s about having a neural network in a feedback loop with a painter, influencing each other’s work repeatedly — and the infinitude of generative systems.

Mario Klingemann has also been playing with GANs in his “Neural Glitch” series:

“Neural Glitch” is a technique I started exploring in April 2018 in which I manipulate fully trained GANs by randomly altering, deleting or exchanging their trained weights. Due to the complex structure of the neural architectures the glitches introduced this way occur on texture as well as on semantic levels which causes the models to misinterpret the input data in interesting ways, some of which could be interpreted as glimpses of autonomous creativity. 

—Mario Klingemann

Writing with machines

Alison Parrish does wonderful creative writing work in collaboration with generative systems. Some of her highlighted work is here, and many projects have open-source code or tutorials. Here’s an example of Alison’s Semantic Similarity Chatbot, which she describes as “uncannily faithful to whatever source material you give it while still being amusingly bizarre”.

I also often come back to Robin Sloan’s “Writing with the Machine” project from a couple of years ago, where he trained an RNN on a corpus of old sci-fi stories and used it to auto-suggest sentence completions in his text editor.

Enjoying the weirdness

From a more playful perspective, I particularly love the work that Janelle Shane has been doing, documented on her site AI Weirdness:

I train neural networks, a type of machine learning algorithm, to write unintentional humor as they struggle to imitate human datasets. Well, I intend the humor. The neural networks are just doing their best to understand what’s going on. 

— Janelle Shane

Here’s her illustration of some of the cookies her neural net came up with when trained on cookie recipes:

Machines cheat in bizarre ways

One of my favorite things is seeing how machine learning systems will find bizarre ways to “cheat” in order to fulfill the goals that are set for them. Recently, there was a lot of discussion around this AI that steganographically encoded invisible data into maps in order to achieve the stated goal of recreating aerial imagery from said map. There’s also a fantastic Google sheet that describes all the ways various AI systems have found unexpected and strange workarounds!

Indolent cannibals

In an artificial life simulation where survival required energy but giving birth had no energy cost, one species evolved a sedentary lifestyle that consisted mostly of mating in order to produce new children which could be eaten (or used as mates to produce more edible children).

The literal computational gaze

This last piece is not about machines as collaborators, but is still one of my favorite pieces in that it so powerfully evokes the sense of the machine’s alien gaze. This is from 2012, and is a video by Timo Arnall called Robot Readable World.

I find this kind of work delightful and meaty, and I hope to see more of it. As soon as I learned to code, I started making generative things — fake ad generators, chatbots, etc. I loved making work that, even though I had shaped it, continued to surprise me. I felt warmth and curiosity towards my strange mechanical collaborators. In a moment where the computational gaze is being used in so many exploitative and questionable ways, I hope that there is also space for work that allows us to explore all that is delightful and creative about our computational companions.

I tweeted this last week, and figured I should put my keyboard where my mouth is and take a stab at talking about design process for the real world. First, a caveat: I do think it’s valuable to frame ideal processes so that we know what we’re aspiring to. But often writing about design process has an all-or-nothing tone to it: It makes you feel that if you’re not doing it the “right” way, then you’re not doing good work, and won’t end up with a good product.

So first: there’s no one “right” way to do things. But there are a set of approaches that are generally good practice for user experience and product design: things like talking to your users, making sure to do divergent exploration, getting feedback, and continual iteration. However, it’s rare that I see a designer in a situation where they can execute a design process exactly as they would like to.

Instead, we all end up working in the messy in-between — a place where we need to make trade offs in our process due to real-world constraints. Those constraints tend to be things like:

• Limited time: Deadlines won’t always accommodate a perfect process.
• Skeptical stakeholders: People with authority over the project may not believe in the value of a thorough design process and see it as something that slows down the project or adds to cost.
• The way things have been done before: If you’re trying to grow a design practice in an organization that hasn’t had a strong design or product culture, change doesn’t happen overnight. 
• Personnel constraints: Sometimes you don’t have enough people or the right people to execute on all the pieces of the design process thoroughly.
• Budget: This one is self-explanatory 🙂
• And much more…

So, given those constraints, how do you decide where to cut corners and where to push for more? What’s a good design process for your design process? 

In my experience, here are a few rubrics for making these decisions:

1. Know your strengths and focus resources on your weaknesses.

What are your core abilities as an individual or a team? If you’re really familiar with your intended users, perhaps you don’t need to go as deep on user research, and instead you focus intensively on design exploration. On the other hand, if you have strong UX/UI design experience and instincts on your team, you might be able to spend less time exploring and iterating and more time talking to users.

This piece of the puzzle requires the ability to accurately self-assess. Be honest with yourself about your strengths and weaknesses, and design your process to support you where you need it most. If you have deep experience in one area, don’t be afraid to trust your instincts.

It can feel like sacrilege to say “we don’t need [x] because we’re really good at [y]”, but remember that ideal design processes are designed to check you — to make sure you’re considering options and needs that you might not immediately think of. Deep experience and skill can also help provide some of those checks and balances.

2. Learn to identify the immovable objects

In looking at your constraints, know which ones are fixed and which ones can’t be budged. This is a bit easier with things like budget, time, and people — for example, if you don’t have a budget for extensive user research, it’s clear you will have to work with some guerrilla research tools and approaches. But it’s more challenging to know which cultural pieces are immovable.

For example, you may have a stakeholder who just doesn’t buy the value of a strong design process. Most designers will find themselves in this position at some point, especially if you’re working in-house. Know when not to waste your time on unwinnable arguments. In those situations, there are one of two paths forward. One is that you can find small ways to inject better process and show how those approaches led to better outcomes. Seeing tangible proof of the utility of a good design process can lead to more investment and trust in that process for future projects. The second path is — unfortunately — that some stakeholders just won’t be convinced and it will prove to be a serious constraint on your ability to do deep design work.

It takes time to figure out which situation you are in, but in either case, knowing how fixed your constraints are helps you identify where to focus your efforts.

3. What has to be perfect now and what can be fixed later?

As designers, it’s always crucial to understand the overall product and business strategy for the experiences we’re designing. One of the reasons for this is that it can help to prioritize where to focus resources in our “messy in-between” processes. What features or users are most critical to the success of the product?

Constraints mean that we almost always have to pick things that aren’t going to get as much love and attention as we would ideally like. Can a feature be removed for launch, or is there a scaled-down MVP of that feature that will suffice for now? Which user group has to have their needs deeply met for success? Can other groups’ needs come later? It’s hard not to want everything to be perfect, but knowing what truly has to be perfect can help in focusing limited resources on the right things.

These are by no means exhaustive, but they are a few key rubrics that I frequently use. Most importantly, I hope that we can all share more about how we navigate design in situations that rarely meet the platonic ideal. In doing so, I believe we can alleviate a lot of the guilt and impostor syndrome that seems to be common amongst designers who are worried that they aren’t “doing it right”. Let’s embrace the imperfections of design process in real organizations and projects, and share tools for creating the best work within the constraints of those situations.