Introduction

简介

A model card is a short document that provides key information about a machine learning model. Model cards increase transparency by communicating information about trained models to broad audiences.

模型卡 是一份简短的文档，提供有关机器学习模型的关键信息。模型卡通过向广大受众传达有关训练模型的信息来提高透明度。

In this tutorial, you will learn about which audiences to write a model card for and which sections a model card should contain. Then, in the following exercise, you will apply what you have learned to a couple of real-world scenarios.

在本教程中，您将了解为哪些受众编写模型卡以及模型卡应包含哪些部分。然后，在以下练习 中，您将把学到的知识应用到几个真实场景中。

Model cards

模型卡

Though AI systems are playing increasingly important roles in every industry, few people understand how these systems work. AI researchers are exploring many ways to communicate key information about models to inform people who use AI systems, people who are affected by AI systems and others.

尽管人工智能系统在每个行业中都扮演着越来越重要的角色，但很少有人了解这些系统的工作原理。人工智能研究人员正在探索多种方法来传达有关模型的关键信息，以告知使用人工智能系统的人、受人工智能系统影响的人和其他人。

Model cards - introduced in a 2019 paper - are one way for teams to communicate key information about their AI system to a broad audience. This information generally includes intended uses for the model, how the model works, and how the model performs in different situations.

模型卡——在2019 年的一篇论文中引入——是团队向广大受众传达有关其人工智能系统的关键信息的一种方式。这些信息通常包括模型的预期用途、模型的工作原理以及模型在不同情况下的表现。

You can think of model cards as similar to the nutritional labels that you find on packaged foods.

你可以把模型卡想象成类似于包装食品上的营养标签。

Examples of model cards

模型卡示例

Before we continue, it might be useful to briefly skim some examples of model cards.

在我们继续之前，简要浏览一些模型卡的例子可能会有所帮助。

• Salesforce's model cards
• Salesforce 的模型卡
• Open AI’s model card for GPT-3
• Open AI 的 GPT-3 模型卡
• Google Cloud's example model cards
• Google Cloud 的示例模型卡

Who is the audience of your model card?

您的模型卡的受众是谁？

A model card should strike a balance between being easy-to-understand and communicating important technical information. When writing a model card, you should consider your audience: the groups of people who are most likely to read your model card. These groups will vary according to the AI system’s purpose.

模型卡应在易于理解和传达重要技术信息之间取得平衡。在编写模型卡时，您应该考虑您的受众：最有可能阅读您的模型卡的人群。这些群体将根据 AI 系统的用途而有所不同。

For example, a model card for an AI system that helps medical professionals interpret x-rays to better diagnose musculoskeletal injuries is likely to be read by medical professionals, scientists, patients, researchers, policymakers and developers of similar AI systems. The model card may therefore assume some knowledge of health care and of AI systems.

例如，帮助医疗专业人员解读 X 光片以更好地诊断肌肉骨骼损伤的 AI 系统的模型卡很可能会被医疗专业人员、科学家、患者、研究人员、政策制定者和类似 AI 系统的开发者阅读。因此，模型卡可能假设人们对医疗保健和人工智能系统有一定的了解。

What sections should a model card contain?

模型卡应包含哪些部分？

Per the original paper, a model card should have the following nine sections. Note that different organizations may add, subtract or rearrange model card sections according to their needs (and you may have noticed this in some of the examples above).

根据原始论文，模型卡应包含以下九个部分。请注意，不同的组织可能会根据其需求添加、减少或重新排列模型卡部分（您可能已在上面的一些示例中注意到了这一点）。

As you read about the different sections, you're encouraged to review the two example model cards from the original paper. Before proceeding, open each of these model card examples in a new window:

在阅读不同部分时，我们鼓励您查看原始论文中的两个示例模型卡。在继续之前，请在新窗口中打开每个模型卡示例：

• Model Card - Smiling Detection in Images
• 模型卡 - 图像中的微笑检测
• Model Card - Toxicity in Text
• 模型卡 - 文本中的毒性

1. Model Details

1. 模型详细信息

• Include background information, such as developer and model version.
• 包括背景信息，例如开发人员和模型版本。

2. Intended Use

2. 预期用途

• What use cases are in scope?
• 哪些用例在范围内？
• Who are your intended users?
• 您的预期用户是谁？
• What use cases are out of scope?
• 哪些用例超出范围？

3. Factors

3. 因素

• What factors affect the impact of the model? For example, the smiling detection model's results vary by demographic factors like age, gender or ethnicity, environmental factors like lighting or rain and instrumentation like camera type.
• 哪些因素会影响模型的影响？例如，微笑检测模型的结果因人口统计因素（如年龄、性别或种族）、环境因素（如光照或雨水）以及仪器（如相机类型）而异。

4. Metrics

4. 指标

• What metrics are you using to measure the performance of the model? Why did you pick those metrics?
• 您使用哪些指标来衡量模型的性能？为什么选择这些指标？
  • For classification systems – in which the output is a class label – potential error types include false positive rate, false negative rate, false discovery rate, and false omission rate. The relative importance of each of these depends on the use case.
  • 对于分类系统（输出为类标签），潜在的错误类型包括误报率、误报率、错误发现率和错误遗漏率。每个指标的相对重要性取决于用例。
  • For score-based analyses – in which the output is a score or price – consider reporting model performance across groups.
  • 对于基于分数的分析（输出为分数或价格），请考虑报告各组的模型性能。

5. Evaluation Data

5. 评估数据

• Which datasets did you use to evaluate model performance? Provide the datasets if you can.
• 您使用哪些数据集来评估模型性能？如果可以，请提供数据集。
• Why did you choose these datasets for evaluation?
• 为什么选择这些数据集进行评估？
• Are the datasets representative of typical use cases, anticipated test cases and/or challenging cases?
• 这些数据集是否代表典型用例、预期测试用例和/或具有挑战性的用例？

6. Training Data

6. 训练数据

• Which data was the model trained on?
• 模型基于哪些数据进行训练？

7. Quantitative Analyses

7. 定量分析

• How did the model perform on the metrics you chose? Break down performance by important factors and their intersections. For example, in the smiling detection example, performance is broken down by age (eg, young, old), gender (eg, female, male), and then both (eg, old-female, old-male, young-female, young-male).
• 模型在您选择的指标上表现如何？按重要因素及其交集细分性能。例如，在微笑检测示例中，性能按年龄（例如年轻、年老）、性别（例如女性、男性）以及两者（例如老年女性、老年男性、年轻女性、年轻男性）细分。

8. Ethical Considerations

8. 道德考虑

• Describe ethical considerations related to the model, such as sensitive data used to train the model, whether the model has implications for human life, health, or safety, how risk was mitigated, and what harms may be present in model usage.
• 描述与模型相关的道德考虑，例如用于训练模型的敏感数据、模型是否对人类生命、健康或安全有影响、如何减轻风险以及模型使用中可能存在哪些危害。

9. Caveats and Recommendations

9. 注意事项和建议

• Add anything important that you have not covered elsewhere in the model card.
• 添加您在模型卡中其他地方未涵盖的任何重要内容。

How can you use model cards in your organization?

如何在组织中使用模型卡？

The use of detailed model cards can often be challenging because an organization may not want to reveal its processes, proprietary data or trade secrets. In such cases, the developer team should think about how model cards can be useful and empowering, without including sensitive information.

使用详细的模型卡通常具有挑战性，因为组织可能不想透露其流程、专有数据或商业机密。在这种情况下，开发团队应该考虑如何让模型卡有用且具有影响力，而不包含敏感信息。

Some teams use other formats - such as FactSheets - to collect and log ML model information.

一些团队使用其他格式（例如 FactSheets）来收集和记录 ML 模型信息。

Your turn

轮到你了

Apply what you've learned to decide how to use model cards in real-world scenarios.

应用你所学到的知识来 决定如何在现实场景中使用模型卡 。

---