Introduction
简介
Before selecting data and training models, it is important to carefully consider the human needs an AI system should serve - and if it should be built at all.
在选择数据和训练模型之前，重要的是要仔细考虑 AI 系统应该满足的人类需求 - 以及是否应该构建它。
Human-centered design (HCD) is an approach to designing systems that serve people’s needs.
以人为本的设计 (HCD) 是一种设计满足人们需求的系统的方法。
In this tutorial, you will learn how to apply HCD to AI systems. Then, you will test your knowledge in an exercise, by applying HCD to design issues in interesting real-world scenarios.
在本教程中，您将学习如何将 HCD 应用于 AI 系统。然后，您将在 练习 中测试您的知识，方法是将 HCD 应用于有趣的现实场景中的设计问题。
Approach
方法
HCD involves people in every step of the design process. Your team should adopt an HCD approach to AI as early as possible - ideally, from when you begin to entertain the possibility of building an AI system.
HCD 让人员参与到设计过程的每一步。您的团队应尽早采用 HCD 方法处理 AI - 理想情况下，从您开始考虑构建 AI 系统的可能性时开始。
The following six steps are intended to help you get started with applying HCD to the design of AI systems. That said, what HCD means for you will depend on your industry, your resources, your organization and the people you seek to serve.
以下六个步骤旨在帮助您开始将 HCD 应用于 AI 系统的设计。也就是说，HCD 对您的意义将取决于您的行业、您的资源、您的组织以及您想要服务的人。
1. Understand people’s needs to define the problem
1. 了解人们的需求以定义问题
Working with people to understand the pain points in their current journeys can help find unaddressed needs. This can be done by observing people as they navigate existing tools, conducting interviews, assembling focus groups, reading user feedback and other methods. Your entire team – including data scientists and engineers – should be involved in this step, so that every team member gains an understanding of the people they hope to serve. Your team should include and involve people with diverse perspectives and backgrounds, along race, gender, and other characteristics. Sharpen your problem definition and brainstorm creative and inclusive solutions together.
与人们合作以了解他们当前旅程中的痛点可以帮助找到未解决的需求。这可以通过观察人们使用现有工具、进行访谈、组建焦点小组、阅读用户反馈和其他方法来实现。您的整个团队（包括数据科学家和工程师）都应该参与这一步，以便每个团队成员都能了解他们希望服务的人。您的团队应该包括并让具有不同观点和背景的人参与进来，包括种族、性别和其他特征。明确问题定义，共同集思广益，提出富有创意和包容性的解决方案。
A company wants to address the problem of dosage errors for immunosuppressant drugs given to patients after liver transplants. The company starts by observing physicians, nurses and other hospital staff throughout the liver transplant process. It also interviews them about the current dosage determination process - which relies on published guidelines and human judgment - and shares video clips from the interviews with the entire development team. The company also reviews research studies and assembles focus groups of former patients and their families. All team members participate in a freewheeling brainstorming session for potential solutions.
一家公司希望解决肝移植后患者服用免疫抑制药物剂量错误的问题。该公司首先在整个肝移植过程中观察医生、护士和其他医院工作人员。它还采访他们关于当前剂量确定过程的信息 - 该过程依赖于已发布的指南和人为判断 - 并与整个开发团队分享采访视频片段。该公司还审查研究报告并召集前患者及其家属的焦点小组。所有团队成员都参与了一场自由的集思广益会议，以寻找可能的解决方案。
2. Ask if AI adds value to any potential solution
2. 询问 AI 是否为任何潜在解决方案增加了价值
Once you are clear about which need you are addressing and how, consider whether AI adds value.
一旦您清楚自己要解决哪些需求以及如何解决，请考虑 AI 是否增加了价值。
- Would people generally agree that what you are trying to achieve is a good outcome?
- 人们是否普遍同意您试图实现的结果是一个好的结果？
- Would non-AI systems - such as rule-based solutions, which are easier to create, audit and maintain - be significantly less effective than an AI system?
- 非 AI 系统（例如基于规则的解决方案，它们更容易创建、审核和维护）的效率是否会大大低于 AI 系统？
- Is the task that you are using AI for one that people would find boring, repetitive or otherwise difficult to concentrate on?
- 您使用 AI 执行的任务是否是人们会觉得无聊、重复或难以集中注意力的任务？
- Have AI solutions proven to be better than other solutions for similar use cases in the past?
- AI 解决方案是否已被证明比过去类似用例的其他解决方案更好？
If you answered no to any of these questions, an AI solution may not be necessary or appropriate.
如果您对其中任何一个问题的回答是否定的，则 AI 解决方案可能不是必需的或不合适的。
A disaster response agency is working with first responders to reduce the time it takes to rescue people from disasters, like floods. The time- and labor-intensive human review of drone and satellite photos to find stranded people increases rescue time. Everybody agrees that speeding up photo review would be a good outcome, since faster rescues could save more lives. The agency determines that an AI image recognition system would likely be more effective than a non-AI automated system for this task. It is also aware that AI-based image recognition tools have been applied successfully to review aerial footage in other industries, like agriculture. The agency therefore decides to further explore the possibility of an AI-based solution.
一家灾难响应机构正在与急救人员合作，以减少从洪水等灾难中救出人员所需的时间。人工审查无人机和卫星照片以寻找被困人员需要耗费大量时间和劳动力，这增加了救援时间。每个人都同意加快照片审查将是一个好的结果，因为更快的救援可以挽救更多生命。该机构认为，人工智能图像识别系统可能比非人工智能自动化系统更适合完成这项任务。该机构还意识到，基于人工智能的图像识别工具已成功应用于审查农业等其他行业的航拍镜头。因此，该机构决定进一步探索基于人工智能的解决方案的可能性。
3. Consider the potential harms that the AI system could cause
3. 考虑人工智能系统可能造成的潜在危害
Weigh the benefits of using AI against the potential harms, throughout the design pipeline: from collecting and labeling data, to training a model, to deploying the AI system. Consider the impact on users and on society. Your privacy team can help uncover hidden privacy issues and determine whether privacy-preserving techniques like differential privacy or federated learning may be appropriate. Take steps to reduce harms, including by embedding people - and therefore human judgment - more effectively in data selection, in model training and in the operation of the system. If you estimate that the harms are likely to outweigh the benefits, do not build the system.
在整个设计流程中，权衡使用人工智能的好处与潜在危害：从收集和标记数据，到训练模型，再到部署人工智能系统。考虑对用户和社会的影响。您的隐私团队可以帮助发现隐藏的隐私问题，并确定隐私保护技术（如差异隐私或联邦学习）是否合适。采取措施减少危害，包括更有效地将人（以及人类判断）嵌入数据选择、模型训练和系统运行中。如果您估计危害可能超过好处，请不要构建系统。
An online education company wants to use an AI system to ‘read’ and automatically assign scores to student essays, while redirecting company staff to double-check random essays and to review essays that the AI system has trouble with. The system would enable the company to quickly get scores back to students. The company creates a harms review committee, which recommends that the system not be built. Some of the major harms flagged by the committee include: the potential for the AI system to pick up bias against certain patterns of language from training data and amplify it (harming people in the groups that use those patterns of language), to encourage students to ‘game’ the algorithm rather than improve their essays and to reduce the classroom role of education experts while increasing the role of technology experts.
一家在线教育公司希望使用人工智能系统“阅读”学生作文并自动为其评分，同时让公司员工重新检查随机作文并审查人工智能系统有问题的作文。该系统将使公司能够快速将分数反馈给学生。该公司成立了一个危害审查委员会，建议不要建立该系统。委员会指出的一些主要危害包括：人工智能系统可能会从训练数据中获取对某些语言模式的偏见并将其放大（伤害使用这些语言模式的群体中的人们），鼓励学生“玩弄”算法而不是改进他们的作文，并减少教育专家在课堂上的作用，同时增加技术专家的作用。
4. Prototype, starting with non-AI solutions
4. 原型，从非 AI 解决方案开始
Develop a non-AI prototype of your AI system quickly to see how people interact with it. This makes prototyping easier, faster and less expensive. It also gives you early information about what users expect from your system and how to make their interactions more rewarding and meaningful.
快速开发 AI 系统的非 AI 原型，以了解人们如何与其交互。这使得原型设计更容易、更快捷、更便宜。它还为您提供了有关用户对您的系统的期望以及如何使他们的交互更有价值和更有意义的早期信息。
Design your prototype’s user interface to make it easy for people to learn how your system works, to toggle settings and to provide feedback.
设计原型的用户界面，让人们轻松了解您的系统如何工作、切换设置和提供反馈。
The people giving feedback should have diverse backgrounds – including along race, gender, expertise and other characteristics. They should also understand and consent to what they are helping with and how.
提供反馈的人应该具有不同的背景——包括种族、性别、专业知识和其他特征。他们还应该了解并同意他们正在帮助什么以及如何提供帮助。
A movie streaming startup wants to use AI to recommend movies to users, based on their stated preferences and viewing history. The team first invites a diverse group of users to share their stated preferences and viewing history with a movie enthusiast, who then recommends movies that the users might like. Based on these conversations and on feedback about which recommended movies users enjoyed, the team changes its approach to how movies are categorized. Getting feedback from a diverse group of users early and iterating often allows the team to improve its product early, rather than making expensive corrections later.
一家电影流媒体初创公司希望使用 AI 根据用户的偏好和观看历史向用户推荐电影。该团队首先邀请一群不同的用户与电影爱好者分享他们所说的偏好和观看历史，然后电影爱好者推荐用户可能喜欢的电影。根据这些对话以及用户喜欢哪些推荐电影的反馈，团队改变了电影分类方法。尽早从多样化的用户群体获得反馈并经常进行迭代，使得团队能够尽早改进其产品，而不是在以后进行昂贵的修改。
5. Provide ways for people to challenge the system
5. 为人们提供挑战系统的方法
People who use your AI system once it is live should be able to challenge its recommendations or easily opt out of using it. Put systems and tools in place to accept, monitor and address challenges.
一旦您的人工智能系统上线，使用该系统的人应该能够挑战其建议或轻松选择退出使用它。建立系统和工具来接受、监控和应对挑战。
Talk to users and think from the perspective of a user: if you are curious or dissatisfied with the system’s recommendations, would you want to challenge it by:
与用户交谈并从用户的角度思考：如果您对系统的建议感到好奇或不满意，您是否想通过以下方式挑战它：
- Requesting an explanation of how it arrived at its recommendation?
- 要求解释它是如何得出建议的？
- Requesting a change in the information you input?
- 要求更改您输入的信息？
- Turning off certain features?
- 关闭某些功能？
- Reaching out to the product team on social media?
- 在社交媒体上联系产品团队？
- Taking some other action?
- 采取其他行动？
An online video conferencing company uses AI to automatically blur the background during video calls. The company has successfully tested its product with a diverse group of people from different ethnicities. Still, it knows that there could be instances in which the video may not properly focus on a person’s face. So, it makes the background blurring feature optional and adds a button for customers to report issues. The company also creates a customer service team to monitor social media and other online forums for user complaints.
一家在线视频会议公司使用人工智能在视频通话期间自动模糊背景。该公司已成功与来自不同种族的多元化人群测试了其产品。尽管如此，它知道在某些情况下，视频可能无法正确聚焦于人的脸部。因此，它使背景模糊功能成为可选项，并添加了一个按钮供客户报告问题。该公司还成立了客户服务团队，监控社交媒体和其他在线论坛的用户投诉。
6. Build in safety measures
6. 建立安全措施
Safety measures protect users against harm. They seek to limit unintended behavior and accidents, by ensuring that a system reliably delivers high-quality outcomes. This can only be achieved through extensive and continuous evaluation and testing. Design processes around your AI system to continuously monitor performance, delivery of intended benefits, reduction of harms, fairness metrics and any changes in how people are actually using it.
安全措施保护用户免受伤害。他们力求通过确保系统可靠地提供高质量的结果来限制意外行为和事故。这只能通过广泛而持续的评估和测试来实现。围绕您的人工智能系统设计流程，以持续监控性能、预期效益的实现、危害的减少、公平指标以及人们实际使用方式的任何变化。
The kind of safety measures your system needs depends on its purpose and on the types of harms it could cause. Start by reviewing the list of safety measures built into similar non-AI products or services. Then, review your earlier analysis of the potential harms of using AI in your system (see Step 3).
您的系统需要的安全措施类型取决于其用途以及可能造成的危害类型。首先查看内置于类似非人工智能产品或服务中的安全措施列表。然后，查看您之前对在您的系统中使用人工智能的潜在危害的分析（参见步骤 3）。
Human oversight of your AI system is crucial:
对您的人工智能系统的人工监督至关重要：
- Create a human ‘red team’ to play the role of a person trying to manipulate your system into unintended behavior. Then, strengthen your system against any such manipulation.
- 创建一个人类“红队”，扮演试图操纵您的系统使其发生意外行为的人的角色。然后，加强您的系统以防止任何此类操纵。
- Determine how people in your organization can best monitor the system’s safety once it is live.
- 确定组织中的人员如何在系统上线后最好地监控系统的安全性。
- Explore ways for your AI system to quickly alert a human when it is faced with a challenging case.
- 探索让您的 AI 系统在面临棘手情况时快速向人类发出警报的方法。
- Create ways for users and others to flag potential safety issues.
- 为用户和其他人创建标记潜在安全问题的方法。
To bolster the safety of its product, a company that develops a widely-used AI-enabled voice assistant creates a permanent internal ‘red team’ to play the role of bad actors that want to manipulate the voice assistant. The red team develops adversarial inputs to fool the voice assistant. The company then uses ‘adversarial training’ to guard the product against similar adversarial inputs, improving its safety.
为了增强其产品的安全性，一家开发广泛使用的 AI 语音助手的公司创建了一个永久的内部“红队”，扮演想要操纵语音助手的坏人的角色。红队开发对抗性输入来欺骗语音助手。然后，该公司使用“对抗性训练”来保护产品免受类似的对抗性输入的影响，从而提高其安全性。
Learn more
了解更多
To dive deeper into the application of HCD to AI, check out these resources:
若要深入了解 HCD 在 AI 中的应用，请查看以下资源：
- Lex Fridman’s introductory lecture on Human-Centered Artificial Intelligence
- Lex Fridman 关于以人为本的人工智能的 入门讲座
- Google’s People + AI Research (PAIR) Guidebook
- Google 的 People + AI 研究 (PAIR) 指南
- Stanford Human-Centered Artificial Intelligence (HAI) research
- 斯坦福以人为本的人工智能 (HAI) 研究
Your turn
轮到你了
In the exercise, you will use HCD to navigate issues in AI system design scenarios.
在练习中，你将使用 HCD 解决 AI 系统设计场景中的问题。