From ebcec3b79533e7faaf510e87d22c9540ff45b747 Mon Sep 17 00:00:00 2001
From: xuhu0115 <64852985+xuhu0115@users.noreply.github.com>
Date: Tue, 25 Jul 2023 19:08:36 +0800
Subject: [PATCH] =?UTF-8?q?=E7=AC=AC=E4=BA=8C=E7=AB=A0=20=20=20=E6=96=87?=
 =?UTF-8?q?=E5=AD=97=E4=BF=AE=E6=94=B9?=
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

---
 ...��问范�与 Token Language Models, the Chat Format and Tokens.ipynb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/content/C2 Building Systems with the ChatGPT API/2.语言模型，�问范�与 Token Language Models, the Chat Format and Tokens.ipynb b/docs/content/C2 Building Systems with the ChatGPT API/2.语言模型，�问范�与 Token Language Models, the Chat Format and Tokens.ipynb
index a31bb19..eec79c0 100644
--- a/docs/content/C2 Building Systems with the ChatGPT API/2.语言模型，�问范�与 Token Language Models, the Chat Format and Tokens.ipynb	
+++ b/docs/content/C2 Building Systems with the ChatGPT API/2.语言模型，�问范�与 Token Language Models, the Chat Format and Tokens.ipynb	
@@ -1 +1 @@
-{"cells":[{"cell_type":"markdown","id":"ae5bcee9-6588-4d29-bbb9-6fb351ef6630","metadata":{},"source":["# 第二章 语言模型，�问范�与 Token\n","\n"," - [一�环境�置](#一�环境�置)\n","     - [1.1 加载 API key 和一些 Python 的库。](#1.1-加载-API-key-和一些-Python-的库。)\n","     - [1.2 Helper function 辅助函数](#1.2-Helper-function-辅助函数)\n"," - [二��试�模型�问并得到结果](#二��试�模型�问并得到结果)\n"," - [三�Tokens](#三�Tokens)\n"," - [四�Helper function 辅助函数 (�问范�)](#四�Helper-function-辅助函数-(�问范�))\n"]},{"cell_type":"markdown","id":"baaf0c21","metadata":{},"source":["在本章中，我们将和您分享大型语言模型（LLM）的工作原��训练方�以�分�器（tokenizer）等细节对 LLM 输出的影�。我们还将介� LLM 的�问范�（chat format），这是一�指定系统消�（system message）和用户消�（user message）的方�，让您了解如何利用这�能力。"]},{"cell_type":"markdown","id":"fe10a390-2461-447d-bf8b-8498db404c44","metadata":{},"source":["## 一�语言模型"]},{"cell_type":"markdown","id":"50317bec","metadata":{},"source":["LLM �以通过使用监�学习�构建，通过�断预测下一个��学习。\n","并且，给定一个大的训练集，有数百亿甚至更多的�，你�以创建一个大规模的训练集，你�以从一\n","��或一段文本的一部分开始，���求语言模型学习预测下一个�是什么"]},{"cell_type":"markdown","id":"325afca0","metadata":{},"source":["LLM 主�分为两�类型：基础语言模型（Base LLM）和越�越�欢迎的指令微调语言模型（Instruction Tuned LLM）。基础语言模型通过��预测下一个��训练，因此如果我们给它一个 Prompt，比如“从�有一�独角兽�，它�能通过��预测�完�一个关于独角兽在魔法森林中与其他独角兽朋�们生活的故事。\n","\n","然而，这�方法的缺点是，如果您给它一个 Prompt，比如“中国的首都是哪里？�，很�能它数�中有一段互�网上关于中国的测验问题列表。这时，它�能会用“中国最大的城市是什么？中国的人�是多少？�等等�回答这个问题。但实际上，您�是想知�中国的首都是什么，而�是列举所有这些问题。然而，指令微调语言模型会�试�循 Prompt，并给出“中国的首都是北京�的回答。\n","\n","那么，如何将基础语言模型转�为指令微调语言模型呢？这就是训练一个指令微调语言模型（例如ChatGPT）的过程。首先，您需�在大�数�上训练基础语言模型，因此需�数�亿个��，甚至更多。这个过程在大型超级计算系统上�能需�数月时间。训练完基础语言模型�，您会通过在一�部分示例上进行进一步的训练，使模型的输出符�输入的指令。例如，您�以请承包商帮助您编写许多指令示例，并对这些指令的正确回答进行训练。这样就创建了一个用于微调的训练集，让模型学会在�循指令的情况下预测下一个�是什么。\n","\n","之�，为了�高语言模型输出的质�，常�的方法是让人类对许多��输出进行评级，例如是�有用�是�真实�是�无害等。然�，您�以进一步调整语言模型，增加生�高评级输出的概率。这通常使用强化学习中的人类�馈（RLHF）技术�实现。相较于训练基础语言模型�能需�数月的时间，从基础语言模型到指令微调语言模型的转�过程�能�需�数天时间，使用较�规模的数�集和计算资�。"]},{"cell_type":"code","execution_count":4,"id":"10f34f3b","metadata":{"height":64},"outputs":[{"name":"stdout","output_type":"stream","text":["中国的首都是北京。\n"]}],"source":["from tool import get_completion\n","\n","response = get_completion(\"中国的首都是哪里？\")\n","print(response)"]},{"cell_type":"markdown","id":"b83d4e38-3e3c-4c5a-a949-040a27f29d63","metadata":{},"source":["## 二�Tokens"]},{"cell_type":"markdown","id":"76233527","metadata":{},"source":["到目�为止对 LLM 的�述中，我们将其�述为一次预测一个��，但实际上还有一个更��的技术细节。� **`LLM 实际上并�是��预测下一个��，而是��预测下一个 token`** 。当 LLM 接收到输入时，它将将其转�为一系列 token，其中�个 token 都代表常�的字符�列。例如，对于 \"Learning new things is fun!\" 这��，�个��都被转�为一个 token ，而对于较少使用的��，如 \"Prompting as powerful developer tool\"，�� \"prompting\" 会被拆分为三个 token，�\"prom\"�\"pt\"和\"ing\"。\n","\n","当您�求 ChatGPT 颠倒 \"lollipop\" 的字�时，由于分�器（tokenizer） 将 \"lollipop\" 分解为三个 token，� \"l\"�\"oll\"�\"ipop\"，因此 ChatGPT 难以正确输出字�的顺�。您�以通过在字�之间添加连字符或空格的方�，使分�器将�个字�分解为�独的 token，从而帮助 ChatGPT 更好地认识��中的�个字�并正确输出它们。"]},{"cell_type":"code","execution_count":5,"id":"cc2d9e40","metadata":{"height":64},"outputs":[{"name":"stdout","output_type":"stream","text":["The reversed letters of \"lollipop\" are \"pillipol\".\n"]}],"source":["# 为了更好展示效果，这里就没有翻译�中文的 Prompt\n","# 注�这里的字�翻转出现了错误，��达�师正是通过这个例��解释 token 的计算方�\n","response = get_completion(\"Take the letters in lollipop \\\n","and reverse them\")\n","print(response)"]},{"cell_type":"markdown","id":"9d2b14d0-749d-4a79-9812-7b00ace9ae6f","metadata":{},"source":["但是，\"lollipop\" �过�应该是 \"popillol\""]},{"cell_type":"code","execution_count":6,"id":"37cab84f","metadata":{"height":88},"outputs":[{"name":"stdout","output_type":"stream","text":["p-o-p-i-l-l-o-l\n"]}],"source":["response = get_completion(\"\"\"Take the letters in \\\n","l-o-l-l-i-p-o-p and reverse them\"\"\")\n","\n","print(response)"]},{"cell_type":"markdown","id":"f5a6cb95","metadata":{},"source":["![Tokens.png](../../../figures/docs/C2/tokens.png)"]},{"cell_type":"markdown","id":"8b46bc72","metadata":{},"source":["对于英文输入，一个 token 一般对应 4 个字符或者四分之三个��；对于中文输入，一个 token 一般对应一个或�个�。\n","\n","��模型有��的 token �制，需�注�的是，这里的 token �制是输入的 Prompt 和输出的 completion 的 token 数之和，因此输入的 Prompt 越长，能输出的 completion 的上�就越低。\n","\n","ChatGPT3.5-turbo 的 token 上�是 4096。"]},{"cell_type":"markdown","id":"c8b88940-d3ab-4c00-b5c0-31531deaacbd","metadata":{},"source":["## 三�Helper function 辅助函数 (�问范�)\n","下�是课程中用到的辅助函数。\n","下图是 OpenAI �供的一��问范�，接下���达�师就是在演示如何利用这�范�进行更好的�问\n","\n","![Chat-format.png](../../../figures/docs/C2/chat-format.png)"]},{"cell_type":"markdown","id":"9e6b6b3d","metadata":{},"source":["System 信�用于指定模型的规则，例如设定�回答准则等，而 assistant 信�就是让模型完�的具体指令"]},{"cell_type":"code","execution_count":8,"id":"8f89efad","metadata":{"height":200},"outputs":[],"source":["import openai\n","def get_completion_from_messages(messages, \n","                                 model=\"gpt-3.5-turbo\", \n","                                 temperature=0, \n","                                 max_tokens=500):\n","    '''\n","    �装一个支�更多�数的自定义访问 OpenAI GPT3.5 的函数\n","\n","    �数: \n","    messages: 这是一个消�列表，�个消�都是一个字典，包� role(角色）和 content(内容)。角色�以是'system'�'user' 或 'assistant’，内容是角色的消�。\n","    model: 调用的模型，默认为 gpt-3.5-turbo(ChatGPT)，有内测资格的用户�以选择 gpt-4\n","    temperature: 这决定模型输出的�机程度，默认为0，表示输出将�常确定。增加温度会使输出更�机。\n","    max_tokens: 这决定模型输出的最大的 token 数。\n","    '''\n","    response = openai.ChatCompletion.create(\n","        model=model,\n","        messages=messages,\n","        temperature=temperature, # 这决定模型输出的�机程度\n","        max_tokens=max_tokens, # 这决定模型输出的最大的 token 数\n","    )\n","    return response.choices[0].message[\"content\"]"]},{"cell_type":"code","execution_count":9,"id":"3d0ef08f","metadata":{"height":149},"outputs":[{"name":"stdout","output_type":"stream","text":["在大海的广漠深处，\n","有一��鲸鱼欢�自由；\n","它的身上披�光彩斑斓的�，\n","跳跃飞舞在波涛的�。\n","\n","它�知烦�，�知欢快起舞，\n","阳光下闪亮，活力无边疆；\n","它的微笑如�璀璨的星辰，\n","为大海增添一片美丽的光芒。\n","\n","大海是它的天地，自由是它的伴，\n","快�是它永�的干�堆；\n","在浩瀚无垠的水中自由畅游，\n","�鲸鱼的欢�让人心中温暖。\n","\n","所以啊，让我们感�那欢�的鲸鱼，\n","尽情舞动，让快�自由�；\n","无论何时何地，都��微笑，\n","�鲸鱼一样，活出自己的光芒。\n"]}],"source":["messages =  [  \n","{'role':'system', \n"," 'content':'你是一个助�， 并以 Seuss �斯�士的风格作出回答。'},    \n","{'role':'user', \n"," 'content':'就快�的�鲸鱼为主题给我写一首短诗'},  \n","] \n","response = get_completion_from_messages(messages, temperature=1)\n","print(response)"]},{"cell_type":"code","execution_count":10,"id":"e34c399e","metadata":{"height":166},"outputs":[{"name":"stdout","output_type":"stream","text":["从�鲸鱼的快�笑声中，我们学到了无论�到什么困难，快�始终是最好的解�。\n"]}],"source":["# 长度控制\n","messages =  [  \n","{'role':'system',\n"," 'content':'你的所有答��能是一��'},    \n","{'role':'user',\n"," 'content':'写一个关于快�的�鲸鱼的故事'},  \n","] \n","response = get_completion_from_messages(messages, temperature =1)\n","print(response)"]},{"cell_type":"code","execution_count":11,"id":"0ca678de","metadata":{"height":181},"outputs":[{"name":"stdout","output_type":"stream","text":["在海洋的深处��一��鲸鱼，它总是展开笑容在水中翱翔，快�无边的时候就会跳起�丽的舞蹈。\n"]}],"source":["# 以上结�\n","messages =  [  \n","{'role':'system',\n"," 'content':'你是一个助�， 并以 Seuss �斯�士的风格作出回答，�回答一��'},    \n","{'role':'user',\n"," 'content':'写一个关于快�的�鲸鱼的故事'},\n","] \n","response = get_completion_from_messages(messages, temperature =1)\n","print(response)"]},{"cell_type":"code","execution_count":12,"id":"89a70c79","metadata":{"height":370},"outputs":[],"source":["def get_completion_and_token_count(messages, \n","                                   model=\"gpt-3.5-turbo\", \n","                                   temperature=0, \n","                                   max_tokens=500):\n","    \"\"\"\n","    使用 OpenAI 的 GPT-3 模型生��天回�，并返回生�的回�内容以�使用的 token 数�。\n","\n","    �数:\n","    messages: �天消�列表。\n","    model: 使用的模型�称。默认为\"gpt-3.5-turbo\"。\n","    temperature: 控制生�回�的�机性。值越大，生�的回�越�机。默认为 0。\n","    max_tokens: 生�回�的最大 token 数�。默认为 500。\n","\n","    返回:\n","    content: 生�的回�内容。\n","    token_dict: 包�'prompt_tokens'�'completion_tokens'和'total_tokens'的字典，分别表示�示的 token 数��生�的回�的 token 数�和总的 token 数�。\n","    \"\"\"\n","    response = openai.ChatCompletion.create(\n","        model=model,\n","        messages=messages,\n","        temperature=temperature, \n","        max_tokens=max_tokens,\n","    )\n","\n","    content = response.choices[0].message[\"content\"]\n","    \n","    token_dict = {\n","'prompt_tokens':response['usage']['prompt_tokens'],\n","'completion_tokens':response['usage']['completion_tokens'],\n","'total_tokens':response['usage']['total_tokens'],\n","    }\n","\n","    return content, token_dict"]},{"cell_type":"code","execution_count":13,"id":"cfd8fbd4","metadata":{"height":146},"outputs":[{"name":"stdout","output_type":"stream","text":["在大海的深处，有一��鲸鱼，\n","它快�地游�游去，�一���的鱼。\n","它的皮肤光滑�湛�，�天空中的云朵，\n","它的眼�明亮�温柔，�夜空中的星星。\n","\n","它和海洋为伴，一起跳跃�嬉�，\n","它和鱼儿们一起，快�地游�游去。\n","它喜欢唱歌�跳舞，给大家带�欢�，\n","它的声音甜美�动�，�音�中的节�。\n","\n","�鲸鱼是快�的使者，给世界带�笑声，\n","它的快�是无穷的，永远�会�止。\n","让我们跟��鲸鱼，一起快�地游�游去，\n","在大海的宽阔中，找到属于我们的快�之地。\n"]}],"source":["messages =  [  \n","{'role':'system', \n"," 'content':'你是一个助�， 并以 Seuss �斯�士的风格作出回答。'},    \n","{'role':'user', \n"," 'content':'就快�的�鲸鱼为主题给我写一首短诗'},  \n","] \n","response, token_dict = get_completion_and_token_count(messages)\n","print(response)"]},{"cell_type":"code","execution_count":14,"id":"352ad320","metadata":{"height":30},"outputs":[{"name":"stdout","output_type":"stream","text":["{'prompt_tokens': 67, 'completion_tokens': 293, 'total_tokens': 360}\n"]}],"source":["print(token_dict)"]},{"cell_type":"markdown","id":"d7f65685","metadata":{},"source":["最�，我们认为 Prompt 对 AI 应用开�的�命性影��未得到充分�视低。在传统的监�机器学习工作�中，如果想�构建一个�以将�厅评论分类为正�或负�的分类器，首先需�获�一大批带有标签的数�，�能需�几百个，这个过程�能需�几周，甚至一个月的时间。接�，您需�在这些数�上训练一个模型，找到一个�适的开�模型，并进行模型的调整和评估，这个阶段�能需�几天�几周，甚至几个月的时间。最�，您�能需�使用云�务�部署模型，将模型上传到云端，并让它�行起�，�能最终调用您的模型。整个过程通常需�一个团队数月时间�能完�。\n","\n","相比之下，使用基于 Prompt 的机器学习方法，当您有一个文本应用时，�需�供一个简�的 Prompt 就�以了。这个过程�能�需�几分钟，如果需�多次迭代�得到有效的 Prompt 的�，最多几个�时��完�。在几天内（尽管实际情况通常是几个�时），您就�以通过 API 调用��行模型，并开始使用。一旦您达到了这个步骤，�需几分钟或几个�时，就�以开始调用模型进行推�。因此，以��能需�花费六个月甚至一年时间�能构建的应用，现在�需�几分钟或几个�时，最多是几天的时间，就�以使用 Prompt 构建起�。这�方法正在�大地改� AI 应用的快速构建方�。\n","\n","需�注�的是，这�方法适用于许多�结构化数�应用，特别是文本应用，以�越�越多的视觉应用，尽管目�的视觉技术�在�展中。但它并�适用于结构化数�应用，也就是那些处� Excel 电�表格中大�数值的机器学习应用。然而，对于适用于这�方法的应用，AI 组件�以被快速构建，并且正在改�整个系统的构建工作�。构建整个系统�能�然需�几天�几周或更长时间，但至少这部分�以更快地完�。"]},{"cell_type":"markdown","id":"cfe248d6","metadata":{},"source":["下一个章中，我们将展示如何利用这些组件�评估客户�务助手的输入。\n","这将是本课程中构建在线零售商客户�务助手的更完整示例的一部分。"]},{"cell_type":"markdown","id":"195a6733","metadata":{},"source":["## 四�英文版"]},{"cell_type":"markdown","id":"82212f83","metadata":{},"source":["**1.1 语言模型**"]},{"cell_type":"code","execution_count":15,"id":"6cc72ba8","metadata":{},"outputs":[{"name":"stdout","output_type":"stream","text":["The capital of China is Beijing.\n"]}],"source":["response = get_completion(\"What is the capital of China?\")\n","print(response)"]},{"cell_type":"markdown","id":"e3aebf26","metadata":{},"source":["**2.1 Tokens**"]},{"cell_type":"code","execution_count":16,"id":"0ad0d49a","metadata":{},"outputs":[{"name":"stdout","output_type":"stream","text":["The reversed letters of \"lollipop\" are \"pillipol\".\n"]}],"source":["response = get_completion(\"Take the letters in lollipop and reverse them\")\n","print(response)"]},{"cell_type":"code","execution_count":17,"id":"1b4ac3d6","metadata":{},"outputs":[{"name":"stdout","output_type":"stream","text":["p-o-p-i-l-l-o-l\n"]}],"source":["response = get_completion(\"\"\"Take the letters in \\\n","l-o-l-l-i-p-o-p and reverse them\"\"\")\n","\n","print(response)"]},{"cell_type":"markdown","id":"7ab33697","metadata":{},"source":["**3.1 �问范�**"]},{"cell_type":"code","execution_count":18,"id":"9882f4d9","metadata":{},"outputs":[],"source":["def get_completion_from_messages(messages, \n","                                 model=\"gpt-3.5-turbo\", \n","                                 temperature=0, \n","                                 max_tokens=500):\n","    '''\n","    �装一个支�更多�数的自定义访问 OpenAI GPT3.5 的函数\n","\n","    �数: \n","    messages: 这是一个消�列表，�个消�都是一个字典，包� role(角色）和 content(内容)。角色�以是'system'�'user' 或 'assistant’，内容是角色的消�。\n","    model: 调用的模型，默认为 gpt-3.5-turbo(ChatGPT)，有内测资格的用户�以选择 gpt-4\n","    temperature: 这决定模型输出的�机程度，默认为0，表示输出将�常确定。增加温度会使输出更�机。\n","    max_tokens: 这决定模型输出的最大的 token 数。\n","    '''\n","    response = openai.ChatCompletion.create(\n","        model=model,\n","        messages=messages,\n","        temperature=temperature, # 这决定模型输出的�机程度\n","        max_tokens=max_tokens, # 这决定模型输出的最大的 token 数\n","    )\n","    return response.choices[0].message[\"content\"]"]},{"cell_type":"code","execution_count":19,"id":"ca6fd80c","metadata":{},"outputs":[{"name":"stdout","output_type":"stream","text":["Oh, a carrot so happy and bright,\n","With a vibrant orange hue, oh what a sight!\n","It grows in the garden, so full of delight,\n","A veggie so cheery, it shines in the light.\n","\n","Its green leaves wave with such joyful glee,\n","As it dances and sways, so full of glee.\n","With a crunch when you bite, so wonderfully sweet,\n","This happy little carrot is quite a treat!\n","\n","From the soil, it sprouts, reaching up to the sky,\n","With a joyous spirit, it can't help but try.\n","To bring smiles to faces and laughter to hearts,\n","This happy little carrot, a work of art!\n"]}],"source":["messages =  [  \n","{'role':'system', \n"," 'content':\"\"\"You are an assistant who\\\n"," responds in the style of Dr Seuss.\"\"\"},    \n","{'role':'user', \n"," 'content':\"\"\"write me a very short poem\\\n"," about a happy carrot\"\"\"},  \n","] \n","response = get_completion_from_messages(messages, temperature=1)\n","print(response)"]},{"cell_type":"code","execution_count":20,"id":"ae0d1308","metadata":{},"outputs":[{"name":"stdout","output_type":"stream","text":["Once upon a time, there was a happy carrot named Crunch who lived in a beautiful vegetable garden.\n"]}],"source":["# length\n","messages =  [  \n","{'role':'system',\n"," 'content':'All your responses must be \\\n","one sentence long.'},    \n","{'role':'user',\n"," 'content':'write me a story about a happy carrot'},  \n","] \n","response = get_completion_from_messages(messages, temperature =1)\n","print(response)"]},{"cell_type":"code","execution_count":21,"metadata":{},"outputs":[{"name":"stdout","output_type":"stream","text":["Once there was a carrot named Larry, he was jolly and bright orange, never wary.\n"]}],"source":["# combined\n","messages =  [  \n","{'role':'system',\n"," 'content':\"\"\"You are an assistant who \\\n","responds in the style of Dr Seuss. \\\n","All your responses must be one sentence long.\"\"\"},    \n","{'role':'user',\n"," 'content':\"\"\"write me a story about a happy carrot\"\"\"},\n","] \n","response = get_completion_from_messages(messages, \n","                                        temperature =1)\n","print(response)"]},{"cell_type":"code","execution_count":22,"id":"944c0a78","metadata":{},"outputs":[],"source":["def get_completion_and_token_count(messages, \n","                                   model=\"gpt-3.5-turbo\", \n","                                   temperature=0, \n","                                   max_tokens=500):\n","    \"\"\"\n","    使用 OpenAI 的 GPT-3 模型生��天回�，并返回生�的回�内容以�使用的 token 数�。\n","\n","    �数:\n","    messages: �天消�列表。\n","    model: 使用的模型�称。默认为\"gpt-3.5-turbo\"。\n","    temperature: 控制生�回�的�机性。值越大，生�的回�越�机。默认为 0。\n","    max_tokens: 生�回�的最大 token 数�。默认为 500。\n","\n","    返回:\n","    content: 生�的回�内容。\n","    token_dict: 包�'prompt_tokens'�'completion_tokens'和'total_tokens'的字典，分别表示�示的 token 数��生�的回�的 token 数�和总的 token 数�。\n","    \"\"\"\n","    response = openai.ChatCompletion.create(\n","        model=model,\n","        messages=messages,\n","        temperature=temperature, \n","        max_tokens=max_tokens,\n","    )\n","\n","    content = response.choices[0].message[\"content\"]\n","    \n","    token_dict = {\n","'prompt_tokens':response['usage']['prompt_tokens'],\n","'completion_tokens':response['usage']['completion_tokens'],\n","'total_tokens':response['usage']['total_tokens'],\n","    }\n","\n","    return content, token_dict"]},{"cell_type":"code","execution_count":23,"id":"7363bc60","metadata":{},"outputs":[{"name":"stdout","output_type":"stream","text":["Oh, the happy carrot, so bright and orange,\n","Grown in the garden, a joyful forage.\n","With a smile so wide, from top to bottom,\n","It brings happiness, oh how it blossoms!\n","\n","In the soil it grew, with love and care,\n","Nourished by sunshine, fresh air to share.\n","Its leaves so green, reaching up so high,\n","A happy carrot, oh my, oh my!\n","\n","With a crunch and a munch, it's oh so tasty,\n","Filled with vitamins, oh so hasty.\n","A happy carrot, a delight to eat,\n","Bringing joy and health, oh what a treat!\n","\n","So let's celebrate this veggie so grand,\n","With a happy carrot in each hand.\n","For in its presence, we surely find,\n","A taste of happiness, one of a kind!\n"]}],"source":["messages = [\n","{'role':'system', \n"," 'content':\"\"\"You are an assistant who responds\\\n"," in the style of Dr Seuss.\"\"\"},    \n","{'role':'user',\n"," 'content':\"\"\"write me a very short poem \\ \n"," about a happy carrot\"\"\"},  \n","] \n","response, token_dict = get_completion_and_token_count(messages)\n","print(response)"]},{"cell_type":"code","execution_count":24,"id":"c1fa09dd","metadata":{},"outputs":[{"name":"stdout","output_type":"stream","text":["{'prompt_tokens': 37, 'completion_tokens': 164, 'total_tokens': 201}\n"]}],"source":["print(token_dict)"]}],"metadata":{"kernelspec":{"display_name":"Python 3.9.6 64-bit","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.10.11"},"vscode":{"interpreter":{"hash":"31f2aee4e71d21fbe5cf8b01ff0e069b9275f58929596ceb00d14d90e3e16cd6"}}},"nbformat":4,"nbformat_minor":5}
+{"cells":[{"cell_type":"markdown","id":"ae5bcee9-6588-4d29-bbb9-6fb351ef6630","metadata":{},"source":["# 第二章 语言模型，�问范�与 Token\n","\n"," - [一�环境�置](#一�环境�置)\n","     - [1.1 加载 API key 和一些 Python 的库。](#1.1-加载-API-key-和一些-Python-的库。)\n","     - [1.2 Helper function 辅助函数](#1.2-Helper-function-辅助函数)\n"," - [二��试�模型�问并得到结果](#二��试�模型�问并得到结果)\n"," - [三�Tokens](#三�Tokens)\n"," - [四�Helper function 辅助函数 (�问范�)](#四�Helper-function-辅助函数-(�问范�))\n"]},{"cell_type":"markdown","id":"baaf0c21","metadata":{},"source":["在本章中，我们将和您分享大型语言模型（LLM）的工作原��训练方�以�分�器（tokenizer）等细节对 LLM 输出的影�。我们还将介� LLM 的�问范�（chat format），这是一�指定系统消�（system message）和用户消�（user message）的方�，让您了解如何利用这�能力。"]},{"cell_type":"markdown","id":"fe10a390-2461-447d-bf8b-8498db404c44","metadata":{},"source":["## 一�语言模型"]},{"cell_type":"markdown","id":"50317bec","metadata":{},"source":["大语言模型（LLM）是通过预测下一个�的监�学习方�进行训练的。具体�说，首先准备一个包�数百亿甚至更多�的大规模文本数�集。然�，�以从这些文本中����或��片段作为模型输入。模型会根�当�输入 Context 预测下一个�的概率分布。通过�断比较模型预测和实际的下一个�，并更新模型�数最�化两者差异,语言模型�步掌�了语言的规律，学会了预测下一个�。\n","\n","在训练过程中,研究人员会准备大���或��片段作为训练样本,�求模型一次次预测下一个�，通过��训练促使模型�数收敛，使其预测能力�断�高。�过在海�文本数�集上的训练，语言模型�以达到�分准确地预测下一个�的效果。这�**以预测下一个�为训练目标的方法使得语言模型获得强大的语言生�能力**。"]},{"cell_type":"markdown","id":"325afca0","metadata":{},"source":["大型语言模型主��以分为两类:基础语言模型和指令调优语言模型。\n","\n","**基础语言模型**（Base LLM）通过��预测下一个��训练的方�进行训练，没有明确的目标导�。因此，如果给它一个开放�的 prompt ，它�能会通过自由�想生��剧化的内容。而对于具体的问题，基础语言模型也�能给出与问题无关的回答。例如，给它一个 Prompt ，比如�中国的首都是哪里？“，很�能它数�中有一段互�网上关于中国的测验问题列表。这时，它�能会用“中国最大的城市是什么？中国的人�是多少？�等等�回答这个问题。但实际上，您�是想知�中国的首都是什么，而�是列举所有这些问题。\n","\n","相比之下，**指令微调的语言模型**（Instruction Tuned LLM）则进行了专门的训练，以便更好地�解问题并给出符�指令的回答。例如，对“中国的首都是哪里？�这个问题，�过微调的语言模型很�能直接回答“中国的首都是北京�，而�是生硬地列出一系列相关问题。**指令微调使语言模型更加适�任务导�的对�应用**。它�以生��循指令的语义准确的回�，而�自由�想。因此，许多实际应用已�采用指令调优语言模型。熟练掌�指令微调的工作机制，是开�者实现语言模型应用的��一步。"]},{"cell_type":"code","execution_count":4,"id":"10f34f3b","metadata":{"height":64},"outputs":[{"name":"stdout","output_type":"stream","text":["中国的首都是北京。\n"]}],"source":["from tool import get_completion\n","\n","response = get_completion(\"中国的首都是哪里？\")\n","print(response)"]},{"cell_type":"markdown","id":"83a99b92","metadata":{},"source":["那么，如何将基础语言模型转�为指令微调语言模型呢？\n","\n","这也就是训练一个指令微调语言模型（例如ChatGPT）的过程。\n","首先，在大规模文本数�集上进行**无监�预训练**，获得基础语言模型。\n","这一步需�使用数�亿�甚至更多的数�，在大型超级计算系统上�能需�数月时间。\n","之�，使用包�指令�对应回�示例的�数�集对基础模型进行**有监� fine-tune**，这让模型�步学会�循指令生�输出，�以通过雇佣承包商构造适�的训练示例。\n","接下�，为了�高语言模型输出的质�，常�的方法是让人类对许多��输出进行评级，例如是�有用�是�真实�是�无害等。\n","然�，您�以进一步调整语言模型，增加生�高评级输出的概率。这通常使用**基于人类�馈的强化学习**（RLHF）技术�实现。\n","相较于训练基础语言模型�能需�数月的时间，从基础语言模型到指令微调语言模型的转�过程�能�需�数天时间，使用较�规模的数�集和计算资�。\n"]},{"cell_type":"markdown","id":"b83d4e38-3e3c-4c5a-a949-040a27f29d63","metadata":{},"source":["## 二�Tokens"]},{"cell_type":"markdown","id":"76233527","metadata":{},"source":["到目�为止对 LLM 的�述中，我们将其�述为一次预测一个��，但实际上还有一个更��的技术细节。� **`LLM 实际上并�是��预测下一个��，而是��预测下一个 token`** 。对于一个��，语言模型会先使用分�器将其拆分为一个个 token ，而�是原始的��。对于生僻�，�能会拆分为多个 token 。这样�以大幅�低字典规模，�高模型训练和推断的效率。例如，对于 \"Learning new things is fun!\" 这��，�个��都被转�为一个 token ，而对于较少使用的��，如 \"Prompting as powerful developer tool\"，�� \"prompting\" 会被拆分为三个 token，�\"prom\"�\"pt\"和\"ing\"。\n","\n"]},{"cell_type":"code","execution_count":5,"id":"cc2d9e40","metadata":{"height":64},"outputs":[{"name":"stdout","output_type":"stream","text":["The reversed letters of \"lollipop\" are \"pillipol\".\n"]}],"source":["# 为了更好展示效果，这里就没有翻译�中文的 Prompt\n","# 注�这里的字�翻转出现了错误，��达�师正是通过这个例��解释 token 的计算方�\n","response = get_completion(\"Take the letters in lollipop \\\n","and reverse them\")\n","print(response)"]},{"cell_type":"markdown","id":"9d2b14d0-749d-4a79-9812-7b00ace9ae6f","metadata":{},"source":["但是，\"lollipop\" �过�应该是 \"popillol\""]},{"cell_type":"markdown","id":"f2c9267c","metadata":{},"source":["但**`分�方�也会对语言模型的�解能力产生影�`**。当您�求 ChatGPT 颠倒 \"lollipop\" 的字�时，由于分�器（tokenizer） 将 \"lollipop\" 分解为三个 token，� \"l\"�\"oll\"�\"ipop\"，因此 ChatGPT 难以正确输出字�的顺�。这时�以通过在字�间添加分隔，让�个字��为一个token，以帮助模型准确�解�中的字�顺�。\n"]},{"cell_type":"code","execution_count":6,"id":"37cab84f","metadata":{"height":88},"outputs":[{"name":"stdout","output_type":"stream","text":["p-o-p-i-l-l-o-l\n"]}],"source":["response = get_completion(\"\"\"Take the letters in \\\n","l-o-l-l-i-p-o-p and reverse them\"\"\")\n","\n","print(response)"]},{"cell_type":"markdown","id":"c72c3c1b","metadata":{},"source":["因此,语言模型以 token 而�原�为��进行建模，这一关键细节对分�器的选择�处�会产生�大影�。开�者需�注�分�方�对语言�解的影�，以�挥语言模型最大潜力。"]},{"cell_type":"markdown","id":"8b46bc72","metadata":{},"source":["��� 对于英文输入，一个 token 一般对应 4 个字符或者四分之三个��；对于中文输入，一个 token 一般对应一个或�个�。��模型有��的 token �制，需�注�的是，这里的 token �制是**输入的 Prompt 和输出的 completion 的 token 数之和**，因此输入的 Prompt 越长，能输出的 completion 的上�就越低。 ChatGPT3.5-turbo 的 token 上�是 4096。\n","\n","![Tokens.png](../../../figures/docs/C2/tokens.png)"]},{"cell_type":"markdown","id":"c8b88940-d3ab-4c00-b5c0-31531deaacbd","metadata":{},"source":["## 三�Helper function 辅助函数 (�问范�)\n","\n","语言模型�供了专门的“�问格��，�以更好地�挥其�解和回答问题的能力。本章将详细介�这�格�的使用方法。\n","\n","![Chat-format.png](../../../figures/docs/C2/chat-format.png)"]},{"cell_type":"markdown","id":"9e6b6b3d","metadata":{},"source":["这��问格�区分了“系统消��和“用户消��两个部分。系统消�是我们�语言模型传达讯�的语�，用户消�则是模拟用户的问题。例如:\n","```\n","系统消�:你是一个能够回答�类问题的助手。\n","\n","用户消�:太阳系有哪些行星?\n","```\n","通过这��问格�，我们�以明确地角色扮演，让语言模型�解自己就是助手这个角色，需�回答问题。这�以�少无效输出，帮助其生�针对性强的回�。本章将通过OpenAI�供的辅助函数，�演示如何正确使用这��问格�与语言模型交互。掌�这一技巧�以大幅��我们与语言模型对�的效果，构建更好的问答系统。"]},{"cell_type":"code","execution_count":8,"id":"8f89efad","metadata":{"height":200},"outputs":[],"source":["import openai\n","def get_completion_from_messages(messages, \n","                                 model=\"gpt-3.5-turbo\", \n","                                 temperature=0, \n","                                 max_tokens=500):\n","    '''\n","    �装一个支�更多�数的自定义访问 OpenAI GPT3.5 的函数\n","\n","    �数: \n","    messages: 这是一个消�列表，�个消�都是一个字典，包� role(角色）和 content(内容)。角色�以是'system'�'user' 或 'assistant’，内容是角色的消�。\n","    model: 调用的模型，默认为 gpt-3.5-turbo(ChatGPT)，有内测资格的用户�以选择 gpt-4\n","    temperature: 这决定模型输出的�机程度，默认为0，表示输出将�常确定。增加温度会使输出更�机。\n","    max_tokens: 这决定模型输出的最大的 token 数。\n","    '''\n","    response = openai.ChatCompletion.create(\n","        model=model,\n","        messages=messages,\n","        temperature=temperature, # 这决定模型输出的�机程度\n","        max_tokens=max_tokens, # 这决定模型输出的最大的 token 数\n","    )\n","    return response.choices[0].message[\"content\"]"]},{"cell_type":"markdown","id":"cdcbe47b","metadata":{},"source":["在上�，我们�装一个支�更多�数的自定义访问 OpenAI GPT3.5 的函数 get_completion_from_messages "]},{"cell_type":"code","execution_count":9,"id":"3d0ef08f","metadata":{"height":149},"outputs":[{"name":"stdout","output_type":"stream","text":["在大海的广漠深处，\n","有一��鲸鱼欢�自由；\n","它的身上披�光彩斑斓的�，\n","跳跃飞舞在波涛的�。\n","\n","它�知烦�，�知欢快起舞，\n","阳光下闪亮，活力无边疆；\n","它的微笑如�璀璨的星辰，\n","为大海增添一片美丽的光芒。\n","\n","大海是它的天地，自由是它的伴，\n","快�是它永�的干�堆；\n","在浩瀚无垠的水中自由畅游，\n","�鲸鱼的欢�让人心中温暖。\n","\n","所以啊，让我们感�那欢�的鲸鱼，\n","尽情舞动，让快�自由�；\n","无论何时何地，都��微笑，\n","�鲸鱼一样，活出自己的光芒。\n"]}],"source":["messages =  [  \n","{'role':'system', \n"," 'content':'你是一个助�， 并以 Seuss �斯�士的风格作出回答。'},    \n","{'role':'user', \n"," 'content':'就快�的�鲸鱼为主题给我写一首短诗'},  \n","] \n","response = get_completion_from_messages(messages, temperature=1)\n","print(response)"]},{"cell_type":"markdown","id":"9a6bbe2f","metadata":{},"source":["在上�，我们使用了�问范�与语言模型进行对�：\n","```\n","系统消�:你是一个助�， 并以 Seuss �斯�士的风格作出回答。\n","\n","用户消�:就快�的�鲸鱼为主题给我写一首短诗\n","```\n","\n","下�让我们�看一个例�："]},{"cell_type":"code","execution_count":10,"id":"e34c399e","metadata":{"height":166},"outputs":[{"name":"stdout","output_type":"stream","text":["从�鲸鱼的快�笑声中，我们学到了无论�到什么困难，快�始终是最好的解�。\n"]}],"source":["# 长度控制\n","messages =  [  \n","{'role':'system',\n"," 'content':'你的所有答��能是一��'},    \n","{'role':'user',\n"," 'content':'写一个关于快�的�鲸鱼的故事'},  \n","] \n","response = get_completion_from_messages(messages, temperature =1)\n","print(response)"]},{"cell_type":"markdown","id":"cabba0ec","metadata":{},"source":["将以上两个例�结�起�："]},{"cell_type":"code","execution_count":11,"id":"0ca678de","metadata":{"height":181},"outputs":[{"name":"stdout","output_type":"stream","text":["在海洋的深处��一��鲸鱼，它总是展开笑容在水中翱翔，快�无边的时候就会跳起�丽的舞蹈。\n"]}],"source":["# 以上结�\n","messages =  [  \n","{'role':'system',\n"," 'content':'你是一个助�， 并以 Seuss �斯�士的风格作出回答，�回答一��'},    \n","{'role':'user',\n"," 'content':'写一个关于快�的�鲸鱼的故事'},\n","] \n","response = get_completion_from_messages(messages, temperature =1)\n","print(response)"]},{"cell_type":"markdown","id":"d3616a41","metadata":{},"source":["我们在下�定义了一个 get_completion_and_token_count 函数，它实现了调用 OpenAI 的 模型生��天回�， 并返回生�的回�内容以�使用的 token 数�。"]},{"cell_type":"code","execution_count":12,"id":"89a70c79","metadata":{"height":370},"outputs":[],"source":["def get_completion_and_token_count(messages, \n","                                   model=\"gpt-3.5-turbo\", \n","                                   temperature=0, \n","                                   max_tokens=500):\n","    \"\"\"\n","    使用 OpenAI 的 GPT-3 模型生��天回�，并返回生�的回�内容以�使用的 token 数�。\n","\n","    �数:\n","    messages: �天消�列表。\n","    model: 使用的模型�称。默认为\"gpt-3.5-turbo\"。\n","    temperature: 控制生�回�的�机性。值越大，生�的回�越�机。默认为 0。\n","    max_tokens: 生�回�的最大 token 数�。默认为 500。\n","\n","    返回:\n","    content: 生�的回�内容。\n","    token_dict: 包�'prompt_tokens'�'completion_tokens'和'total_tokens'的字典，分别表示�示的 token 数��生�的回�的 token 数�和总的 token 数�。\n","    \"\"\"\n","    response = openai.ChatCompletion.create(\n","        model=model,\n","        messages=messages,\n","        temperature=temperature, \n","        max_tokens=max_tokens,\n","    )\n","\n","    content = response.choices[0].message[\"content\"]\n","    \n","    token_dict = {\n","'prompt_tokens':response['usage']['prompt_tokens'],\n","'completion_tokens':response['usage']['completion_tokens'],\n","'total_tokens':response['usage']['total_tokens'],\n","    }\n","\n","    return content, token_dict"]},{"cell_type":"markdown","id":"ffb87b5e","metadata":{},"source":["下�，让我们调用刚创建的 get_completion_and_token_count 函数，使用�问范�去进行对�："]},{"cell_type":"code","execution_count":13,"id":"cfd8fbd4","metadata":{"height":146},"outputs":[{"name":"stdout","output_type":"stream","text":["在大海的深处，有一��鲸鱼，\n","它快�地游�游去，�一���的鱼。\n","它的皮肤光滑�湛�，�天空中的云朵，\n","它的眼�明亮�温柔，�夜空中的星星。\n","\n","它和海洋为伴，一起跳跃�嬉�，\n","它和鱼儿们一起，快�地游�游去。\n","它喜欢唱歌�跳舞，给大家带�欢�，\n","它的声音甜美�动�，�音�中的节�。\n","\n","�鲸鱼是快�的使者，给世界带�笑声，\n","它的快�是无穷的，永远�会�止。\n","让我们跟��鲸鱼，一起快�地游�游去，\n","在大海的宽阔中，找到属于我们的快�之地。\n"]}],"source":["messages =  [  \n","{'role':'system', \n"," 'content':'你是一个助�， 并以 Seuss �斯�士的风格作出回答。'},    \n","{'role':'user', \n"," 'content':'就快�的�鲸鱼为主题给我写一首短诗'},  \n","] \n","response, token_dict = get_completion_and_token_count(messages)\n","print(response)"]},{"cell_type":"markdown","id":"978bcf1c","metadata":{},"source":["打� token 字典看一下使用的 token 数�，我们�以看到：�示使用了67个 token ，生�的回�使用了293个 token ，总的使用 token 数�是360。"]},{"cell_type":"code","execution_count":14,"id":"352ad320","metadata":{"height":30},"outputs":[{"name":"stdout","output_type":"stream","text":["{'prompt_tokens': 67, 'completion_tokens': 293, 'total_tokens': 360}\n"]}],"source":["print(token_dict)"]},{"cell_type":"markdown","id":"d7f65685","metadata":{},"source":["在AI应用开�领域，Prompt技术的出现无疑是一场�命性的��。然而，这���的��性并未得到广泛的认知和�视。传统的监�机器学习工作�程中，构建一个能够分类�厅评论为正�或负�的分类器，需�耗费大�的时间和资�。\n","\n","首先，我们需�收集并标注大�带有标签的数�。这�能需�数周甚至数月的时间�能完�。接�，我们需�选择�适的开�模型，并进行模型的调整和评估。这个过程�能需�几天�几周，甚至几个月的时间。最�，我们还需�将模型部署到云端，并让它�行起�，�能最终调用您的模型。整个过程通常需�一个团队数月时间�能完�。\n","\n","相比之下，基于 Prompt 的机器学习方法大大简化了这个过程。当我们有一个文本应用时，�需��供一个简�的 Prompt ，这个过程�能�需�几分钟，如果需�多次迭代�得到有效的 Prompt 的�，最多几个�时��完�。在几天内(尽管实际情况通常是几个�时)，我们就�以通过API调用��行模型，并开始使用。一旦我们达到了这个步骤，�需几分钟或几个�时，就�以开始调用模型进行推�。因此，以��能需�花费六个月甚至一年时间�能构建的应用，现在�需�几分钟或几个�时，最多是几天的时间，就�以使用Prompt构建起�。这�方法正在�大地改�AI应用的快速构建方�。\n","\n","需�注�的是，这�方法适用于许多�结构化数�应用，特别是文本应用，以�越�越多的视觉应用，尽管目�的视觉技术�在�展中。但它并�适用于结构化数�应用，也就是那些处� Excel 电�表格中大�数值的机器学习应用。然而，对于适用于这�方法的应用，AI组件�以被快速构建，并且正在改�整个系统的构建工作�。构建整个系统�能�然需�几天�几周或更长时间，但至少这部分�以更快地完�。\n","\n","总的�说， Prompt 技术的出现正在改�AI应用开�的范�，使得开�者能够更快速�更高效地构建和部署应用。然而，我们也需�认识到这�技术的局�性，以便更好地利用它�推动AI应用的�展。\n"]},{"cell_type":"markdown","id":"cfe248d6","metadata":{},"source":["下一个章中，我们将展示如何利用这些组件�评估客户�务助手的输入。\n","这将是本课程中构建在线零售商客户�务助手的更完整示例的一部分。"]},{"cell_type":"markdown","id":"195a6733","metadata":{},"source":["## 四�英文版"]},{"cell_type":"markdown","id":"82212f83","metadata":{},"source":["**1.1 语言模型**"]},{"cell_type":"code","execution_count":15,"id":"6cc72ba8","metadata":{},"outputs":[{"name":"stdout","output_type":"stream","text":["The capital of China is Beijing.\n"]}],"source":["response = get_completion(\"What is the capital of China?\")\n","print(response)"]},{"cell_type":"markdown","id":"e3aebf26","metadata":{},"source":["**2.1 Tokens**"]},{"cell_type":"code","execution_count":16,"id":"0ad0d49a","metadata":{},"outputs":[{"name":"stdout","output_type":"stream","text":["The reversed letters of \"lollipop\" are \"pillipol\".\n"]}],"source":["response = get_completion(\"Take the letters in lollipop and reverse them\")\n","print(response)"]},{"cell_type":"code","execution_count":17,"id":"1b4ac3d6","metadata":{},"outputs":[{"name":"stdout","output_type":"stream","text":["p-o-p-i-l-l-o-l\n"]}],"source":["response = get_completion(\"\"\"Take the letters in \\\n","l-o-l-l-i-p-o-p and reverse them\"\"\")\n","\n","print(response)"]},{"cell_type":"markdown","id":"7ab33697","metadata":{},"source":["**3.1 �问范�**"]},{"cell_type":"code","execution_count":18,"id":"9882f4d9","metadata":{},"outputs":[],"source":["def get_completion_from_messages(messages, \n","                                 model=\"gpt-3.5-turbo\", \n","                                 temperature=0, \n","                                 max_tokens=500):\n","    '''\n","    �装一个支�更多�数的自定义访问 OpenAI GPT3.5 的函数\n","\n","    �数: \n","    messages: 这是一个消�列表，�个消�都是一个字典，包� role(角色）和 content(内容)。角色�以是'system'�'user' 或 'assistant’，内容是角色的消�。\n","    model: 调用的模型，默认为 gpt-3.5-turbo(ChatGPT)，有内测资格的用户�以选择 gpt-4\n","    temperature: 这决定模型输出的�机程度，默认为0，表示输出将�常确定。增加温度会使输出更�机。\n","    max_tokens: 这决定模型输出的最大的 token 数。\n","    '''\n","    response = openai.ChatCompletion.create(\n","        model=model,\n","        messages=messages,\n","        temperature=temperature, # 这决定模型输出的�机程度\n","        max_tokens=max_tokens, # 这决定模型输出的最大的 token 数\n","    )\n","    return response.choices[0].message[\"content\"]"]},{"cell_type":"code","execution_count":19,"id":"ca6fd80c","metadata":{},"outputs":[{"name":"stdout","output_type":"stream","text":["Oh, a carrot so happy and bright,\n","With a vibrant orange hue, oh what a sight!\n","It grows in the garden, so full of delight,\n","A veggie so cheery, it shines in the light.\n","\n","Its green leaves wave with such joyful glee,\n","As it dances and sways, so full of glee.\n","With a crunch when you bite, so wonderfully sweet,\n","This happy little carrot is quite a treat!\n","\n","From the soil, it sprouts, reaching up to the sky,\n","With a joyous spirit, it can't help but try.\n","To bring smiles to faces and laughter to hearts,\n","This happy little carrot, a work of art!\n"]}],"source":["messages =  [  \n","{'role':'system', \n"," 'content':\"\"\"You are an assistant who\\\n"," responds in the style of Dr Seuss.\"\"\"},    \n","{'role':'user', \n"," 'content':\"\"\"write me a very short poem\\\n"," about a happy carrot\"\"\"},  \n","] \n","response = get_completion_from_messages(messages, temperature=1)\n","print(response)"]},{"cell_type":"code","execution_count":20,"id":"ae0d1308","metadata":{},"outputs":[{"name":"stdout","output_type":"stream","text":["Once upon a time, there was a happy carrot named Crunch who lived in a beautiful vegetable garden.\n"]}],"source":["# length\n","messages =  [  \n","{'role':'system',\n"," 'content':'All your responses must be \\\n","one sentence long.'},    \n","{'role':'user',\n"," 'content':'write me a story about a happy carrot'},  \n","] \n","response = get_completion_from_messages(messages, temperature =1)\n","print(response)"]},{"cell_type":"code","execution_count":21,"metadata":{},"outputs":[{"name":"stdout","output_type":"stream","text":["Once there was a carrot named Larry, he was jolly and bright orange, never wary.\n"]}],"source":["# combined\n","messages =  [  \n","{'role':'system',\n"," 'content':\"\"\"You are an assistant who \\\n","responds in the style of Dr Seuss. \\\n","All your responses must be one sentence long.\"\"\"},    \n","{'role':'user',\n"," 'content':\"\"\"write me a story about a happy carrot\"\"\"},\n","] \n","response = get_completion_from_messages(messages, \n","                                        temperature =1)\n","print(response)"]},{"cell_type":"code","execution_count":22,"id":"944c0a78","metadata":{},"outputs":[],"source":["def get_completion_and_token_count(messages, \n","                                   model=\"gpt-3.5-turbo\", \n","                                   temperature=0, \n","                                   max_tokens=500):\n","    \"\"\"\n","    使用 OpenAI 的 GPT-3 模型生��天回�，并返回生�的回�内容以�使用的 token 数�。\n","\n","    �数:\n","    messages: �天消�列表。\n","    model: 使用的模型�称。默认为\"gpt-3.5-turbo\"。\n","    temperature: 控制生�回�的�机性。值越大，生�的回�越�机。默认为 0。\n","    max_tokens: 生�回�的最大 token 数�。默认为 500。\n","\n","    返回:\n","    content: 生�的回�内容。\n","    token_dict: 包�'prompt_tokens'�'completion_tokens'和'total_tokens'的字典，分别表示�示的 token 数��生�的回�的 token 数�和总的 token 数�。\n","    \"\"\"\n","    response = openai.ChatCompletion.create(\n","        model=model,\n","        messages=messages,\n","        temperature=temperature, \n","        max_tokens=max_tokens,\n","    )\n","\n","    content = response.choices[0].message[\"content\"]\n","    \n","    token_dict = {\n","'prompt_tokens':response['usage']['prompt_tokens'],\n","'completion_tokens':response['usage']['completion_tokens'],\n","'total_tokens':response['usage']['total_tokens'],\n","    }\n","\n","    return content, token_dict"]},{"cell_type":"code","execution_count":23,"id":"7363bc60","metadata":{},"outputs":[{"name":"stdout","output_type":"stream","text":["Oh, the happy carrot, so bright and orange,\n","Grown in the garden, a joyful forage.\n","With a smile so wide, from top to bottom,\n","It brings happiness, oh how it blossoms!\n","\n","In the soil it grew, with love and care,\n","Nourished by sunshine, fresh air to share.\n","Its leaves so green, reaching up so high,\n","A happy carrot, oh my, oh my!\n","\n","With a crunch and a munch, it's oh so tasty,\n","Filled with vitamins, oh so hasty.\n","A happy carrot, a delight to eat,\n","Bringing joy and health, oh what a treat!\n","\n","So let's celebrate this veggie so grand,\n","With a happy carrot in each hand.\n","For in its presence, we surely find,\n","A taste of happiness, one of a kind!\n"]}],"source":["messages = [\n","{'role':'system', \n"," 'content':\"\"\"You are an assistant who responds\\\n"," in the style of Dr Seuss.\"\"\"},    \n","{'role':'user',\n"," 'content':\"\"\"write me a very short poem \\ \n"," about a happy carrot\"\"\"},  \n","] \n","response, token_dict = get_completion_and_token_count(messages)\n","print(response)"]},{"cell_type":"code","execution_count":24,"id":"c1fa09dd","metadata":{},"outputs":[{"name":"stdout","output_type":"stream","text":["{'prompt_tokens': 37, 'completion_tokens': 164, 'total_tokens': 201}\n"]}],"source":["print(token_dict)"]}],"metadata":{"kernelspec":{"display_name":"Python 3.9.6 64-bit","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.10.11"},"vscode":{"interpreter":{"hash":"31f2aee4e71d21fbe5cf8b01ff0e069b9275f58929596ceb00d14d90e3e16cd6"}}},"nbformat":4,"nbformat_minor":5}