331 lines
9.1 KiB
Plaintext
331 lines
9.1 KiB
Plaintext
{
|
||
"cells": [
|
||
{
|
||
"attachments": {},
|
||
"cell_type": "markdown",
|
||
"id": "63651c26",
|
||
"metadata": {},
|
||
"source": [
|
||
"第三章 评估输入——分类"
|
||
]
|
||
},
|
||
{
|
||
"attachments": {},
|
||
"cell_type": "markdown",
|
||
"id": "87d9de1d",
|
||
"metadata": {},
|
||
"source": [
|
||
"## Setup\n",
|
||
"加载 API_KEY 并封装一个调用 API 的函数"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 17,
|
||
"id": "55ee24ab",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"import os\n",
|
||
"import openai\n",
|
||
"from dotenv import load_dotenv, find_dotenv\n",
|
||
"_ = load_dotenv(find_dotenv()) # read local .env file\n",
|
||
"os.environ['OPENAI_API_KEY']"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 18,
|
||
"id": "0318b89e",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"def get_completion_from_messages(messages, \n",
|
||
" model=\"gpt-3.5-turbo\", \n",
|
||
" temperature=0, \n",
|
||
" max_tokens=500):\n",
|
||
" response = openai.ChatCompletion.create(\n",
|
||
" model=model,\n",
|
||
" messages=messages,\n",
|
||
" temperature=temperature, \n",
|
||
" max_tokens=max_tokens,\n",
|
||
" )\n",
|
||
" return response.choices[0].message[\"content\"]"
|
||
]
|
||
},
|
||
{
|
||
"attachments": {},
|
||
"cell_type": "markdown",
|
||
"id": "af0ff2b2",
|
||
"metadata": {},
|
||
"source": [
|
||
"在本节中，我们将专注于评估输入的任务，这对于确保系统的质量和安全性非常重要。\n",
|
||
"\n",
|
||
"对于需要处理不同情况下的许多独立指令集的任务，首先对查询类型进行分类，然后根据该分类确定要使用哪些指令会很有好处。\n",
|
||
"\n",
|
||
"这可以通过定义固定的类别和hard-coding与处理给定类别任务相关的指令来实现。\n",
|
||
"\n",
|
||
"例如，在构建客户服务助手时，首先对查询类型进行分类，然后根据该分类确定要使用哪些指令可能比较重要。\n",
|
||
"\n",
|
||
"因此，例如，如果用户要求关闭其帐户，您可能会给出不同的辅助指令，而如果用户询问特定产品，则可能会添加其他产品信息。\n",
|
||
"\n",
|
||
"让我们看一个例子，帮助我们理解的更清晰。"
|
||
]
|
||
},
|
||
{
|
||
"attachments": {},
|
||
"cell_type": "markdown",
|
||
"id": "f2b55807",
|
||
"metadata": {},
|
||
"source": [
|
||
"#### 对用户指令进行分类"
|
||
]
|
||
},
|
||
{
|
||
"attachments": {},
|
||
"cell_type": "markdown",
|
||
"id": "c3216166",
|
||
"metadata": {},
|
||
"source": [
|
||
"在这里，我们有我们的系统消息，它是对整个系统的指导，并且我们正在使用这个分隔符。\n",
|
||
"\n",
|
||
"分隔符只是一种分隔指令或输出不同部分的方式，并且它有助于模型确定不同的部分。\n",
|
||
"\n",
|
||
"因此，对于这个例子，我们将使用#作为分隔符。\n",
|
||
"\n",
|
||
"这是一个很好的分隔符，因为它实际上被表示为一个token。"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 19,
|
||
"id": "3b406ba8",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"delimiter = \"####\""
|
||
]
|
||
},
|
||
{
|
||
"attachments": {},
|
||
"cell_type": "markdown",
|
||
"id": "049d0d82",
|
||
"metadata": {},
|
||
"source": [
|
||
"这是我们的系统消息，我们正在以下面的方式询问模型。"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 25,
|
||
"id": "29e2d170",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"system_message = f\"\"\"\n",
|
||
"You will be provided with customer service queries. \\\n",
|
||
"The customer service query will be delimited with \\\n",
|
||
"{delimiter} characters.\n",
|
||
"Classify each query into a primary category \\\n",
|
||
"and a secondary category. \n",
|
||
"Provide your output in json format with the \\\n",
|
||
"keys: primary and secondary.\n",
|
||
"\n",
|
||
"Primary categories: Billing, Technical Support, \\\n",
|
||
"Account Management, or General Inquiry.\n",
|
||
"\n",
|
||
"Billing secondary categories:\n",
|
||
"Unsubscribe or upgrade\n",
|
||
"Add a payment method\n",
|
||
"Explanation for charge\n",
|
||
"Dispute a charge\n",
|
||
"\n",
|
||
"Technical Support secondary categories:\n",
|
||
"General troubleshooting\n",
|
||
"Device compatibility\n",
|
||
"Software updates\n",
|
||
"\n",
|
||
"Account Management secondary categories:\n",
|
||
"Password reset\n",
|
||
"Update personal information\n",
|
||
"Close account\n",
|
||
"Account security\n",
|
||
"\n",
|
||
"General Inquiry secondary categories:\n",
|
||
"Product information\n",
|
||
"Pricing\n",
|
||
"Feedback\n",
|
||
"Speak to a human\n",
|
||
"\n",
|
||
"\"\"\""
|
||
]
|
||
},
|
||
{
|
||
"attachments": {},
|
||
"cell_type": "markdown",
|
||
"id": "e6a932ce",
|
||
"metadata": {},
|
||
"source": [
|
||
"现在我们来看一个用户消息的例子，我们将使用以下内容。"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 26,
|
||
"id": "2b2df0bf",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"user_message = f\"\"\"\\ \n",
|
||
"I want you to delete my profile and all of my user data\"\"\""
|
||
]
|
||
},
|
||
{
|
||
"attachments": {},
|
||
"cell_type": "markdown",
|
||
"id": "3a2c1cf0",
|
||
"metadata": {},
|
||
"source": [
|
||
"将这个消息格式化为一个消息列表，系统消息和用户消息使用####\"进行分隔。\n",
|
||
"\n",
|
||
"让我们想一想，作为人类，这句话什么意思：\"我想让您删除我的个人资料。\"\n",
|
||
"\n",
|
||
"这句话看上去属于\"Account Management\"类别，也许是属于\"Close account\"这一项。 "
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 27,
|
||
"id": "6e2b9049",
|
||
"metadata": {},
|
||
"outputs": [],
|
||
"source": [
|
||
"messages = [ \n",
|
||
"{'role':'system', \n",
|
||
" 'content': system_message}, \n",
|
||
"{'role':'user', \n",
|
||
" 'content': f\"{delimiter}{user_message}{delimiter}\"}, \n",
|
||
"]"
|
||
]
|
||
},
|
||
{
|
||
"attachments": {},
|
||
"cell_type": "markdown",
|
||
"id": "4b295207",
|
||
"metadata": {},
|
||
"source": [
|
||
"让我们看看模型是如何思考的\n",
|
||
"\n",
|
||
"模型的分类是\"Account Management\"作为\"primary\"，\"Close account\"作为\"secondary\"。\n",
|
||
"\n",
|
||
"请求结构化输出（如JSON）的好处是，您可以轻松地将其读入某个对象中，\n",
|
||
"\n",
|
||
"例如Python中的字典，或者如果您使用其他语言，则可以使用其他对象作为输入到后续步骤中。"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 30,
|
||
"id": "77328388",
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"{\n",
|
||
" \"primary\": \"General Inquiry\",\n",
|
||
" \"secondary\": \"Product information\"\n",
|
||
"}\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"response = get_completion_from_messages(messages)\n",
|
||
"print(response)"
|
||
]
|
||
},
|
||
{
|
||
"attachments": {},
|
||
"cell_type": "markdown",
|
||
"id": "2f6b353b",
|
||
"metadata": {},
|
||
"source": [
|
||
"我将向您展示另一个示例，但您也可以随时暂停视频，自己尝试提问，并查看模型如何对其进行分类。\n",
|
||
"\n",
|
||
"这是另一个用户消息: \"告诉我更多关于你们的平板电视\"\n",
|
||
"\n",
|
||
"我们只是有相同的消息列表，模型的响应，然后我们打印它。\n",
|
||
"\n",
|
||
"结果这里是我们的第二个分类，看起来应该是正确的。"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 31,
|
||
"id": "edf8fbe9",
|
||
"metadata": {},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"{\n",
|
||
" \"primary\": \"General Inquiry\",\n",
|
||
" \"secondary\": \"Product information\"\n",
|
||
"}\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"user_message = f\"\"\"\\\n",
|
||
"Tell me more about your flat screen tvs\"\"\"\n",
|
||
"messages = [ \n",
|
||
"{'role':'system', \n",
|
||
" 'content': system_message}, \n",
|
||
"{'role':'user', \n",
|
||
" 'content': f\"{delimiter}{user_message}{delimiter}\"}, \n",
|
||
"] \n",
|
||
"response = get_completion_from_messages(messages)\n",
|
||
"print(response)"
|
||
]
|
||
},
|
||
{
|
||
"attachments": {},
|
||
"cell_type": "markdown",
|
||
"id": "8f87f68d",
|
||
"metadata": {},
|
||
"source": [
|
||
"所以总的来说，根据客户咨询的分类，我们现在可以提供一套更具体的指令来处理后续步骤。\n",
|
||
"\n",
|
||
"在这种情况下，我们可能会添加关于电视的额外信息，而不同情况下，我们可能希望提供关闭账户的链接或类似的内容。\n",
|
||
"\n",
|
||
"我们将在以后的视频中了解更多有关处理输入的不同方法。\n",
|
||
"\n",
|
||
"在下一个视频中，我们将探讨更多评估输入的方法，特别是确保用户以负责任的方式使用系统的方法。"
|
||
]
|
||
}
|
||
],
|
||
"metadata": {
|
||
"kernelspec": {
|
||
"display_name": "Python 3 (ipykernel)",
|
||
"language": "python",
|
||
"name": "python3"
|
||
},
|
||
"language_info": {
|
||
"codemirror_mode": {
|
||
"name": "ipython",
|
||
"version": 3
|
||
},
|
||
"file_extension": ".py",
|
||
"mimetype": "text/x-python",
|
||
"name": "python",
|
||
"nbconvert_exporter": "python",
|
||
"pygments_lexer": "ipython3",
|
||
"version": "3.8.16"
|
||
}
|
||
},
|
||
"nbformat": 4,
|
||
"nbformat_minor": 5
|
||
}
|