To Prevent Harm, Look To Ethical Design Tools

From ChatGPT to AI generated images and videos and self-driving cars, technology has seen unparalleled growth with little oversight.

While there has been an increased spotlight on design and tech ethics, even the term “ethics” can be overwhelming, vague, inaccessible, and impractical. Right now, there’s a giant gap between the safeguards we should be taking and the technology we're designing.

As designers working in tech, we rarely have time to dive deep in ethical theory and scholarly journals. There is a massive gap about how to integrate ethics into daily work, and design ethics needs to be scoped to something that is practical and goal driven.

Jump to…

An influx of ethical design tools
My methodology for selecting the tools
6 ethical design tools to try out

1. Black Mirror Brainstorm
2. Inverted Behavioral Model
3. Provocatype
4. Dichotomy Mapping
5. Consequences
6. Ethics Assessment

Conclusion

Sifting through an influx of ethical design tools

In an article for WIRED, Rachel Botsman argues that one of the leading causes of harm in tech is the failure to mitigate unconsidered consequences of a product at scale. As designers, being human-centered and empathetic is not enough—we have to account for any harmful outcomes that could come out of our designers during the design process.

It’s naive to expect tech’s move fast and break things culture to die a quick death, or for companies to start prioritizing people over profit. We need to work within this system. The past several years have seen a huge rise in ethical design tools to help accomplish this.

I’ve previously written about the importance of ethical design and discussed ideas to help you start being a more ethical-centered designer. For context, I like to frame ethics in terms of preventing harm before products go to market, but I’ve found that harm is typically not often discussed.

This is where tools can become helpful, as they can provide a structured framework and language for product teams to surface harm. It’s important to keep in mind that harm and ethics cannot and should not be treated as a checklist. Tools are just one resource that we can utilize to start creating safer products.

Ranging in complexity and purpose, these tools are being published by everyone from large companies such as IDEO and Spotify to lesser-known independent designers. Many of the tools are published with step-by-step instructions, templates, and even examples of scenarios they can be used in, but they are not widely known across the broader design industry.

In part because there is no central repository for knowledge among designers, more and more tools are then being developed, flooding the market with information overload. Unsurprisingly, these tools are not well integrated into the design process and have a low adoption rate among designers.

Seeing a need for collecting and synthesizing some of these ethical tools, a group of researchers identified, mapped, and analyzed a collection of 63 ethical design methods and then published a microsite based on their findings. They found that a surprising 44% were published in scholarly journals frequently inaccessible due to paywalls—but it’s worth noting that only some of the tools were surveyed, and there are countless others that are public and open source.

While their work is groundbreaking in providing a synthesis of ethical design methodologies, it is complex and academic as opposed to practice-oriented, and still leaves a ton of options for designers to choose from. Frustrated at trying to wade through all these tools, I decided to create a short list of grab-and-go tools that could be quickly used while designing.

For me, it was important to ensure that any tools selected were created specifically for designers, could be easily used for a wide range of educators and designers, and focused specifically on helping surface unconsidered consequences.

I came up with three goals to help narrow all the tools down:

1. Provide tools to help designers center ethics at the beginning of the design process
2. Provide tools that are quick and easy to use without prior knowledge or much setup
3. Provide tools that are collaborative and created specifically for design teams

My methodology for selecting the tools

As the creator of Ethical Design Resources and an organizer for the Ethical Design Network, I had a very healthy list of ethical design tools at my disposal. I began by pulling together resources from the above paper and resources posted on the Ethical Design Network and Ethical Design Resources.

After removing the duplicates, I was left with 87 tools. I set criteria to filter all of the tools in order to meet the goals above using the language and instructions from the tools. Some interpretation was required and I relied on my expertise as a designer.

Team driven (n = 82)

Because design is often successful when it is collaborative, the first criteria involved selecting those that were designed for use in teams or group settings.

Open source (n = 57)

As a believer in open access, ensuring the resources were publicly available was key. Filtering out any private or pay-walled tools excluded nearly a third.

Design specific (n = 41)

The next criteria excluded any resources that did not apply directly to design, including tools for development, general technology, and data analytics.

Recently created (n = 39)

Wanting the tools to be as relevant as possible, I filtered them by date, removing any tools that were created over five years ago.

Includes template (n = 22)

Since templates help speed up the time it takes onboard and use, I included only the tools that had ready-to-use templates.

Surface unintended consequences (n = 11)

I excluded any tools that were used for other purposes, including measuring existing products or developing ethical codes of conduct.

Easy to use (n = 6)

To meet a wide range of designer expertise, it was important to select tools that didn’t require pre-existing knowledge, so I filtered out tools with high complexity.

Six ethical design tools to try out

These six tools are the result of this synthesis. I created the information below using a comparative analysis of each tool based on its description and instructions. I tried to be as objective as possible, but had to make some assumptions and inference based on the information that was publicly available.

1. Black Mirror Brainstorm

Creator: Joshua Mauldin
Date: 2018
Published: UX Collective, Medium.com
Goals: Understand how a product could be misused
Outcome: List of anti-goals to refer back to for a product
Time required: 30-45 mins

Example: We want to create personalized content so people spend more time on our product. To do that, we’re building a targeting engine that sifts through millions of data points to spot trends. Our goal is to figure out how it can go wrong for the people whose data we’re collecting.

Process:

• Brainstorm ideas for what might go wrong
• Create plot points or a story for the bad situation
• Communicate the story and idea on the poster (template included)
• Present each poster and collect and synthesize common ideas into a list of anti-goals

2. Inverted Behavioral Model

Creator: Kat Zhao
Date: 2018
Published: Design Ethically Toolkit
Goals: Predict harmful user behaviors and consequences
Outcome: List of features and consequential behaviors to consider when designing
Time required: None listed, estimating around 30 mins

Example: We are trying to determine the harmful consequences of several features within Facebook before we start designing. Let’s explore the newsfeed, marketplace, and groups.

Process:

• Write out the prompts and motivating factors that might come into play
• Try predicting some behaviors that might result
• Analyze and see what consequences might result from those behaviors
• Rank each consequence from most likely to occur to least likely to occur

3. Provocatype

Creator: Danish Design Center
Date: 2018
Published: Digital Ethics Compass Toolkit
Goals: Provide a new perspective for your product or idea
Outcome: An unethical and irresponsible version of your product
Time required: 30-60 minutes

Example: We are working on a centralized application for storing healthcare records across doctor’s offices in order to reduce medical errors. Our goal is to figure out how this might harm patients.

Process:

• Describe the unethical solution and optionally draw it
• Explain why the solution is unethical and who it is harming
• Determine what part of your existing idea or solution inspired this unethical solution
• Brainstorm how your existing idea or solution could evolve to become unethical

4. Dichotomy Mapping

Creator: Kat Zhao
Date: 2018
Published: Design Ethically Toolkit
Goals: Determine what beneficial or harmful results may occur from a specific feature
Outcome: List of harmful and beneficial outcomes from implementing a feature
Time required: None listed, estimating 15-20 mins

Example: We intended for our social media newsfeed to show the most relevant content based on friend recommendation. Instead, it ended up showing the most outlandish and “clickbait-y” content, fueling the fake news problem on the platform.

Process:

• List the product features you want to examine
• Brainstorm all of the positive aspects in the beneficial column
• Brainstorm what can happen if the feature is implemented to the extreme
• Consider how those results might affect a single user as well as a group of users

5. Consequences

Creator: Ziqq Rafit
Date: 2021
Published: Design Ethicquette
Goals: Surface negative consequences through several different lenses
Outcome: An action plan to how to address and mitigate the negative outcomes
Time Required: None listed, estimating 30 mins

Example: N/A

Process:

• List as many positive/negative and intended/unintended outcomes as possible of your design solution. Think about how your design will impact the environment, people, and business.
• List down all the negative outcomes (from Step 1) and assign who will take ownership and be accountable in addressing them moving forwards.

6. Ethics Assessment

Creator: Spotify’s Ethics Guild
Date: 2021
Published: Spotify Design
Goals: Recognize and avoid potential harm when designing a product or service
Outcome: List of outcomes in ranked order based on concern
Time required: None listed, estimating 45 mins

Example: Spotify wants to ensure that their mobile app isn’t practicing deceptive design when it comes to ad targeting. For this exercise, they want to explore the question: what’s the worst headline about your product you can imagine?

Process:

• List potential negative effects into three categories: physical, emotional, and societal
• Create examples of how your product might encourage or cause this effect
• Rate from 1-5 the chance that your product produces a certain effect
• Rate from 1-5 the level of concern your team has about this particular harm
• Write out next steps for each harm in a separate document based on these ratings

Conclusion

These six are just some of the amazing tools out there to help you start to identify harmful consequences within your work. More work is needed to determine if these tools are successful at mitigating harm and what specific use cases they are best used for, such as for education, a service, a B2B/B2C product, or a startup/enterprise solution.

The right tools can improve efficiency and safeguard against mistakes, but just as with any other tool, it’s important that these aren’t treated as an end-all-be-all solution. Lennart Overkamp argues that before we can successfully use any tool, we need to define success metrics that are ethical, achievable, and inspirational. Ethical design is not something that can be quickly checked off during the design process and as designers—we need to increase our ethical awareness throughout all stages of the design process.

If you have used any of these or other ethical design tools in your work, I’d love to hear about it!