diff --git a/content/LangChain for LLM Application Development/6.评估.ipynb b/content/LangChain for LLM Application Development/6.评估.ipynb index 1125634..4601453 100644 --- a/content/LangChain for LLM Application Development/6.评估.ipynb +++ b/content/LangChain for LLM Application Development/6.评估.ipynb @@ -5,9 +5,7 @@ "id": "52824b89-532a-4e54-87e9-1410813cd39e", "metadata": {}, "source": [ - "# 6. 评估\n", - "\n", - "
\n", + "# 6.评估\n", "\n", "当使用llm构建复杂应用程序时，评估应用程序的表现是一个重要但有时棘手的步骤，它是否满足某些准确性标准？\n", "通常更有用的是从许多不同的数据点中获得更全面的模型表现情况\n", @@ -16,7 +14,7 @@ }, { "cell_type": "code", - "execution_count": 2, + "execution_count": 1, "id": "b7ed03ed-1322-49e3-b2a2-33e94fb592ef", "metadata": { "height": 81, @@ -35,13 +33,14 @@ "id": "28008949", "metadata": {}, "source": [ - "## 6.1 创建LLM应用\n", + "## 6.1 英文原版\n", + "### 6.1.1 创建LLM应用\n", "按照langchain链的方式进行构建" ] }, { "cell_type": "code", - "execution_count": 3, + "execution_count": 2, "id": "974acf8e-8f88-42de-88f8-40a82cb58e8b", "metadata": { "height": 98 @@ -57,7 +56,7 @@ }, { "cell_type": "code", - "execution_count": 13, + "execution_count": 3, "id": "9ec1106d", "metadata": { "height": 64 @@ -72,7 +71,7 @@ }, { "cell_type": "code", - "execution_count": 6, + "execution_count": 4, "id": "06b1ffae", "metadata": {}, "outputs": [ @@ -175,8 +174,8 @@ "" ], "text/plain": [ - " 0 1 \n", - "0 NaN name \\\n", + " 0 1 \\\n", + "0 NaN name \n", "1 0.0 Women's Campside Oxfords \n", "2 1.0 Recycled Waterhog Dog Mat, Chevron Weave \n", "3 2.0 Infant and Toddler Girls' Coastal Chill Swimsu... \n", @@ -204,7 +203,7 @@ "[1001 rows x 3 columns]" ] }, - "execution_count": 6, + "execution_count": 4, "metadata": {}, "output_type": "execute_result" } @@ -218,12 +217,20 @@ }, { "cell_type": "code", - "execution_count": 10, + "execution_count": 6, "id": "b31c218f", "metadata": { "height": 64 }, - "outputs": [], + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Retrying langchain.embeddings.openai.embed_with_retry.._embed_with_retry in 4.0 seconds as it raised RateLimitError: Rate limit reached for default-text-embedding-ada-002 in organization org-2YlJyPMl62f07XPJCAlXfDxj on tokens per min. Limit: 150000 / min. Current: 127135 / min. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n" + ] + } + ], "source": [ "'''\n", "将指定向量存储类,创建完成后，我们将从加载器中调用,通过文档记载器列表加载\n", @@ -235,7 +242,7 @@ }, { "cell_type": "code", - "execution_count": 11, + "execution_count": 7, "id": "a2006054", "metadata": { "height": 183 @@ -260,14 +267,15 @@ "id": "791ebd73", "metadata": {}, "source": [ - "### 6.1.1 创建评估数据点\n", - "们需要做的第一件事是真正弄清楚我们想要评估它的一些数据点，我们将介绍几种不同的方法来完成这个任务\n", + "#### 6.1.1.1 创建评估数据点\n", + "我们需要做的第一件事是真正弄清楚我们想要评估它的一些数据点，我们将介绍几种不同的方法来完成这个任务\n", + "\n", "1、将自己想出好的数据点作为例子，查看一些数据，然后想出例子问题和答案，以便以后用于评估" ] }, { "cell_type": "code", - "execution_count": 14, + "execution_count": 8, "id": "fb04a0f9", "metadata": { "height": 30 @@ -276,10 +284,10 @@ { "data": { "text/plain": [ - "Document(page_content=\": 10\\nname: Cozy Comfort Pullover Set, Stripe\\ndescription: Perfect for lounging, this striped knit set lives up to its name. We used ultrasoft fabric and an easy design that's as comfortable at bedtime as it is when we have to make a quick run out.\\n\\nSize & Fit\\n- Pants are Favorite Fit: Sits lower on the waist.\\n- Relaxed Fit: Our most generous fit sits farthest from the body.\\n\\nFabric & Care\\n- In the softest blend of 63% polyester, 35% rayon and 2% spandex.\\n\\nAdditional Features\\n- Relaxed fit top with raglan sleeves and rounded hem.\\n- Pull-on pants have a wide elastic waistband and drawstring, side pockets and a modern slim leg.\\n\\nImported.\", metadata={'source': 'OutdoorClothingCatalog_1000.csv', 'row': 10})" + "Document(page_content=\": 10\\nname: Cozy Comfort Pullover Set, Stripe\\ndescription: Perfect for lounging, this striped knit set lives up to its name. We used ultrasoft fabric and an easy design that's as comfortable at bedtime as it is when we have to make a quick run out.\\r\\n\\r\\nSize & Fit\\r\\n- Pants are Favorite Fit: Sits lower on the waist.\\r\\n- Relaxed Fit: Our most generous fit sits farthest from the body.\\r\\n\\r\\nFabric & Care\\r\\n- In the softest blend of 63% polyester, 35% rayon and 2% spandex.\\r\\n\\r\\nAdditional Features\\r\\n- Relaxed fit top with raglan sleeves and rounded hem.\\r\\n- Pull-on pants have a wide elastic waistband and drawstring, side pockets and a modern slim leg.\\r\\n\\r\\nImported.\", metadata={'source': 'OutdoorClothingCatalog_1000.csv', 'row': 10})" ] }, - "execution_count": 14, + "execution_count": 8, "metadata": {}, "output_type": "execute_result" } @@ -290,7 +298,7 @@ }, { "cell_type": "code", - "execution_count": 15, + "execution_count": 9, "id": "fe4a88c2", "metadata": { "height": 30 @@ -302,7 +310,7 @@ "Document(page_content=': 11\\nname: Ultra-Lofty 850 Stretch Down Hooded Jacket\\ndescription: This technical stretch down jacket from our DownTek collection is sure to keep you warm and comfortable with its full-stretch construction providing exceptional range of motion. With a slightly fitted style that falls at the hip and best with a midweight layer, this jacket is suitable for light activity up to 20° and moderate activity up to -30°. The soft and durable 100% polyester shell offers complete windproof protection and is insulated with warm, lofty goose down. Other features include welded baffles for a no-stitch construction and excellent stretch, an adjustable hood, an interior media port and mesh stash pocket and a hem drawcord. Machine wash and dry. Imported.', metadata={'source': 'OutdoorClothingCatalog_1000.csv', 'row': 11})" ] }, - "execution_count": 15, + "execution_count": 9, "metadata": {}, "output_type": "execute_result" } @@ -324,12 +332,12 @@ "id": "8d548aef", "metadata": {}, "source": [ - "### 6.1.2 创建测试用例数据\n" + "#### 6.1.1.2 创建测试用例数据\n" ] }, { "cell_type": "code", - "execution_count": 17, + "execution_count": 10, "id": "c2d59bf2", "metadata": { "height": 217 @@ -364,12 +372,12 @@ "id": "c7ce3e4f", "metadata": {}, "source": [ - "### 6.1.3 通过LLM生成测试用例" + "#### 6.1.1.3 通过LLM生成测试用例" ] }, { "cell_type": "code", - "execution_count": 18, + "execution_count": 11, "id": "d44f8376", "metadata": { "height": 47 @@ -381,7 +389,7 @@ }, { "cell_type": "code", - "execution_count": 19, + "execution_count": 12, "id": "34e87816", "metadata": { "height": 30 @@ -393,12 +401,52 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 13, + "id": "b93e7a5d", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[Document(page_content=\": 0\\nname: Women's Campside Oxfords\\ndescription: This ultracomfortable lace-to-toe Oxford boasts a super-soft canvas, thick cushioning, and quality construction for a broken-in feel from the first time you put them on. \\r\\n\\r\\nSize & Fit: Order regular shoe size. For half sizes not offered, order up to next whole size. \\r\\n\\r\\nSpecs: Approx. weight: 1 lb.1 oz. per pair. \\r\\n\\r\\nConstruction: Soft canvas material for a broken-in feel and look. Comfortable EVA innersole with Cleansport NXT® antimicrobial odor control. Vintage hunt, fish and camping motif on innersole. Moderate arch contour of innersole. EVA foam midsole for cushioning and support. Chain-tread-inspired molded rubber outsole with modified chain-tread pattern. Imported. \\r\\n\\r\\nQuestions? Please contact us for any inquiries.\", metadata={'source': 'OutdoorClothingCatalog_1000.csv', 'row': 0}),\n", + " Document(page_content=': 1\\nname: Recycled Waterhog Dog Mat, Chevron Weave\\ndescription: Protect your floors from spills and splashing with our ultradurable recycled Waterhog dog mat made right here in the USA. \\r\\n\\r\\nSpecs\\r\\nSmall - Dimensions: 18\" x 28\". \\r\\nMedium - Dimensions: 22.5\" x 34.5\".\\r\\n\\r\\nWhy We Love It\\r\\nMother nature, wet shoes and muddy paws have met their match with our Recycled Waterhog mats. Ruggedly constructed from recycled plastic materials, these ultratough mats help keep dirt and water off your floors and plastic out of landfills, trails and oceans. Now, that\\'s a win-win for everyone.\\r\\n\\r\\nFabric & Care\\r\\nVacuum or hose clean.\\r\\n\\r\\nConstruction\\r\\n24 oz. polyester fabric made from 94% recycled materials.\\r\\nRubber backing.\\r\\n\\r\\nAdditional Features\\r\\nFeatures an -exclusive design.\\r\\nFeatures thick and thin fibers for scraping dirt and absorbing water.\\r\\nDries quickly and resists fading, rotting, mildew and shedding.\\r\\nUse indoors or out.\\r\\nMade in the USA.\\r\\n\\r\\nHave questions? Reach out to our customer service team with any questions you may have.', metadata={'source': 'OutdoorClothingCatalog_1000.csv', 'row': 1}),\n", + " Document(page_content=\": 2\\nname: Infant and Toddler Girls' Coastal Chill Swimsuit, Two-Piece\\ndescription: She'll love the bright colors, ruffles and exclusive whimsical prints of this toddler's two-piece swimsuit! Our four-way-stretch and chlorine-resistant fabric keeps its shape and resists snags. The UPF 50+ rated fabric provides the highest rated sun protection possible, blocking 98% of the sun's harmful rays. The crossover no-slip straps and fully lined bottom ensure a secure fit and maximum coverage. Machine wash and line dry for best results. Imported.\", metadata={'source': 'OutdoorClothingCatalog_1000.csv', 'row': 2}),\n", + " Document(page_content=\": 3\\nname: Refresh Swimwear, V-Neck Tankini Contrasts\\ndescription: Whether you're going for a swim or heading out on an SUP, this watersport-ready tankini top is designed to move with you and stay comfortable. All while looking great in an eye-catching colorblock style. \\r\\n\\r\\nSize & Fit\\r\\nFitted: Sits close to the body.\\r\\n\\r\\nWhy We Love It\\r\\nNot only does this swimtop feel good to wear, its fabric is good for the earth too. In recycled nylon, with Lycra® spandex for the perfect amount of stretch. \\r\\n\\r\\nFabric & Care\\r\\nThe premium Italian-blend is breathable, quick drying and abrasion resistant. \\r\\nBody in 82% recycled nylon with 18% Lycra® spandex. \\r\\nLined in 90% recycled nylon with 10% Lycra® spandex. \\r\\nUPF 50+ rated – the highest rated sun protection possible. \\r\\nHandwash, line dry.\\r\\n\\r\\nAdditional Features\\r\\nLightweight racerback straps are easy to get on and off, and won't get in your way. \\r\\nFlattering V-neck silhouette. \\r\\nImported.\\r\\n\\r\\nSun Protection That Won't Wear Off\\r\\nOur high-performance fabric provides SPF\", metadata={'source': 'OutdoorClothingCatalog_1000.csv', 'row': 3}),\n", + " Document(page_content=\": 4\\nname: EcoFlex 3L Storm Pants\\ndescription: Our new TEK O2 technology makes our four-season waterproof pants even more breathable. It's guaranteed to keep you dry and comfortable – whatever the activity and whatever the weather. Size & Fit: Slightly Fitted through hip and thigh. \\r\\n\\r\\nWhy We Love It: Our state-of-the-art TEK O2 technology offers the most breathability we've ever tested. Great as ski pants, they're ideal for a variety of outdoor activities year-round. Plus, they're loaded with features outdoor enthusiasts appreciate, including weather-blocking gaiters and handy side zips. Air In. Water Out. See how our air-permeable TEK O2 technology keeps you dry and comfortable. \\r\\n\\r\\nFabric & Care: 100% nylon, exclusive of trim. Machine wash and dry. \\r\\n\\r\\nAdditional Features: Three-layer shell delivers waterproof protection. Brand new TEK O2 technology provides enhanced breathability. Interior gaiters keep out rain and snow. Full side zips for easy on/off over boots. Two zippered hand pockets. Thigh pocket. Imported.\\r\\n\\r\\n – Official Supplier to the U.S. Ski Team\\r\\nTHEIR WILL\", metadata={'source': 'OutdoorClothingCatalog_1000.csv', 'row': 4})]" + ] + }, + "execution_count": 13, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "data[:5]" + ] + }, + { + "cell_type": "code", + "execution_count": 15, "id": "62abae09", "metadata": { "height": 64 }, - "outputs": [], + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 1.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 2.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 4.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 8.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 1.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 2.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 4.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 8.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n" + ] + } + ], "source": [ "new_examples = example_gen_chain.apply_and_parse(\n", " [{\"doc\": t} for t in data[:5]]\n", @@ -407,26 +455,26 @@ }, { "cell_type": "code", - "execution_count": 21, + "execution_count": 16, "id": "31c9f786", "metadata": {}, "outputs": [ { "data": { "text/plain": [ - "[{'query': \"What is the weight of the Women's Campside Oxfords?\",\n", - " 'answer': \"The Women's Campside Oxfords weigh approximately 1 lb.1 oz. per pair.\"},\n", - " {'query': 'What are the dimensions of the medium Recycled Waterhog dog mat?',\n", - " 'answer': 'The dimensions of the medium Recycled Waterhog dog mat are 22.5\" x 34.5\".'},\n", - " {'query': \"What are some features of the Infant and Toddler Girls' Coastal Chill Swimsuit?\",\n", - " 'answer': \"The swimsuit has bright colors, ruffles, and exclusive whimsical prints. It is made of four-way-stretch and chlorine-resistant fabric that keeps its shape and resists snags. The swimsuit is also UPF 50+ rated, providing the highest rated sun protection possible, blocking 98% of the sun's harmful rays. The crossover no-slip straps and fully lined bottom ensure a secure fit and maximum coverage. It is machine washable and should be line dried for best results. The swimsuit is imported.\"},\n", + "[{'query': \"What is the description of the Women's Campside Oxfords?\",\n", + " 'answer': \"The Women's Campside Oxfords are described as an ultracomfortable lace-to-toe Oxford made of soft canvas material, featuring thick cushioning, quality construction, and a broken-in feel from the first time they are worn.\"},\n", + " {'query': 'What are the dimensions of the small and medium sizes of the Recycled Waterhog Dog Mat, Chevron Weave?',\n", + " 'answer': 'The small size of the Recycled Waterhog Dog Mat, Chevron Weave has dimensions of 18\" x 28\", while the medium size has dimensions of 22.5\" x 34.5\".'},\n", + " {'query': \"What are some features of the Infant and Toddler Girls' Coastal Chill Swimsuit, Two-Piece?\",\n", + " 'answer': \"The swimsuit features bright colors, ruffles, and exclusive whimsical prints. It is made of a four-way-stretch and chlorine-resistant fabric that maintains its shape and resists snags. The fabric is also UPF 50+ rated, providing the highest rated sun protection possible by blocking 98% of the sun's harmful rays. The swimsuit has crossover no-slip straps and a fully lined bottom for a secure fit and maximum coverage.\"},\n", " {'query': 'What is the fabric composition of the Refresh Swimwear, V-Neck Tankini Contrasts?',\n", - " 'answer': 'The Refresh Swimwear, V-Neck Tankini Contrasts is made of 82% recycled nylon with 18% Lycra® spandex for the body and 90% recycled nylon with 10% Lycra® spandex for the lining.'},\n", - " {'query': 'What is the name of the waterproof pants and what technology makes them more breathable?',\n", - " 'answer': 'The name of the pants is EcoFlex 3L Storm Pants and the TEK O2 technology makes them more breathable.'}]" + " 'answer': 'The Refresh Swimwear, V-Neck Tankini Contrasts is made of 82% recycled nylon with 18% Lycra® spandex for the body, and 90% recycled nylon with 10% Lycra® spandex for the lining.'},\n", + " {'query': 'What is the technology used in the EcoFlex 3L Storm Pants that makes them more breathable?',\n", + " 'answer': 'The EcoFlex 3L Storm Pants use TEK O2 technology to make them more breathable.'}]" ] }, - "execution_count": 21, + "execution_count": 16, "metadata": {}, "output_type": "execute_result" } @@ -437,7 +485,7 @@ }, { "cell_type": "code", - "execution_count": 22, + "execution_count": 17, "id": "97ab28b5", "metadata": { "height": 30 @@ -446,11 +494,11 @@ { "data": { "text/plain": [ - "{'query': \"What is the weight of the Women's Campside Oxfords?\",\n", - " 'answer': \"The Women's Campside Oxfords weigh approximately 1 lb.1 oz. per pair.\"}" + "{'query': \"What is the description of the Women's Campside Oxfords?\",\n", + " 'answer': \"The Women's Campside Oxfords are described as an ultracomfortable lace-to-toe Oxford made of soft canvas material, featuring thick cushioning, quality construction, and a broken-in feel from the first time they are worn.\"}" ] }, - "execution_count": 22, + "execution_count": 17, "metadata": {}, "output_type": "execute_result" } @@ -461,7 +509,7 @@ }, { "cell_type": "code", - "execution_count": 23, + "execution_count": 18, "id": "0ebe4228", "metadata": { "height": 30 @@ -470,10 +518,10 @@ { "data": { "text/plain": [ - "Document(page_content=\": 0\\nname: Women's Campside Oxfords\\ndescription: This ultracomfortable lace-to-toe Oxford boasts a super-soft canvas, thick cushioning, and quality construction for a broken-in feel from the first time you put them on. \\n\\nSize & Fit: Order regular shoe size. For half sizes not offered, order up to next whole size. \\n\\nSpecs: Approx. weight: 1 lb.1 oz. per pair. \\n\\nConstruction: Soft canvas material for a broken-in feel and look. Comfortable EVA innersole with Cleansport NXT® antimicrobial odor control. Vintage hunt, fish and camping motif on innersole. Moderate arch contour of innersole. EVA foam midsole for cushioning and support. Chain-tread-inspired molded rubber outsole with modified chain-tread pattern. Imported. \\n\\nQuestions? Please contact us for any inquiries.\", metadata={'source': 'OutdoorClothingCatalog_1000.csv', 'row': 0})" + "Document(page_content=\": 0\\nname: Women's Campside Oxfords\\ndescription: This ultracomfortable lace-to-toe Oxford boasts a super-soft canvas, thick cushioning, and quality construction for a broken-in feel from the first time you put them on. \\r\\n\\r\\nSize & Fit: Order regular shoe size. For half sizes not offered, order up to next whole size. \\r\\n\\r\\nSpecs: Approx. weight: 1 lb.1 oz. per pair. \\r\\n\\r\\nConstruction: Soft canvas material for a broken-in feel and look. Comfortable EVA innersole with Cleansport NXT® antimicrobial odor control. Vintage hunt, fish and camping motif on innersole. Moderate arch contour of innersole. EVA foam midsole for cushioning and support. Chain-tread-inspired molded rubber outsole with modified chain-tread pattern. Imported. \\r\\n\\r\\nQuestions? Please contact us for any inquiries.\", metadata={'source': 'OutdoorClothingCatalog_1000.csv', 'row': 0})" ] }, - "execution_count": 23, + "execution_count": 18, "metadata": {}, "output_type": "execute_result" } @@ -487,12 +535,12 @@ "id": "faf25f2f", "metadata": {}, "source": [ - "### 6.1.4 组合用例数据" + "#### 6.1.1.4 组合用例数据" ] }, { "cell_type": "code", - "execution_count": 24, + "execution_count": 19, "id": "ada2a3fc", "metadata": { "height": 30 @@ -504,7 +552,7 @@ }, { "cell_type": "code", - "execution_count": 25, + "execution_count": 20, "id": "9cdf5cf5", "metadata": { "height": 30 @@ -516,7 +564,7 @@ "text": [ "\n", "\n", - "\u001b[1m> Entering new RetrievalQA chain...\u001b[0m\n", + "\u001b[1m> Entering new chain...\u001b[0m\n", "\n", "\u001b[1m> Finished chain.\u001b[0m\n" ] @@ -524,10 +572,10 @@ { "data": { "text/plain": [ - "'The Cozy Comfort Pullover Set, Stripe has side pockets.'" + "'Yes, the Cozy Comfort Pullover Set does have side pockets.'" ] }, - "execution_count": 25, + "execution_count": 20, "metadata": {}, "output_type": "execute_result" } @@ -541,7 +589,7 @@ "id": "63f3cb08", "metadata": {}, "source": [ - "## 6.2 人工评估\n", + "### 6.1.2 人工评估\n", "现在有了这些示例，但是我们如何评估正在发生的事情呢？\n", "通过运行一个示例通过链，并查看它产生的输出\n", "在这里我们传递一个查询，然后我们得到一个答案。实际上正在发生的事情，进入语言模型的实际提示是什么？ \n", @@ -552,7 +600,7 @@ }, { "cell_type": "code", - "execution_count": 26, + "execution_count": 21, "id": "fcaf622e", "metadata": { "height": 47 @@ -568,7 +616,7 @@ }, { "cell_type": "code", - "execution_count": 27, + "execution_count": 22, "id": "1e1deab0", "metadata": {}, "outputs": [ @@ -585,23 +633,23 @@ "\u001b[32;1m\u001b[1;3m[chain/start]\u001b[0m \u001b[1m[1:chain:RetrievalQA > 2:chain:StuffDocumentsChain > 3:chain:LLMChain] Entering Chain run with input:\n", "\u001b[0m{\n", " \"question\": \"Do the Cozy Comfort Pullover Set have side pockets?\",\n", - " \"context\": \": 10\\nname: Cozy Comfort Pullover Set, Stripe\\ndescription: Perfect for lounging, this striped knit set lives up to its name. We used ultrasoft fabric and an easy design that's as comfortable at bedtime as it is when we have to make a quick run out.\\n\\nSize & Fit\\n- Pants are Favorite Fit: Sits lower on the waist.\\n- Relaxed Fit: Our most generous fit sits farthest from the body.\\n\\nFabric & Care\\n- In the softest blend of 63% polyester, 35% rayon and 2% spandex.\\n\\nAdditional Features\\n- Relaxed fit top with raglan sleeves and rounded hem.\\n- Pull-on pants have a wide elastic waistband and drawstring, side pockets and a modern slim leg.\\n\\nImported.<<<<>>>>>: 73\\nname: Cozy Cuddles Knit Pullover Set\\ndescription: Perfect for lounging, this knit set lives up to its name. We used ultrasoft fabric and an easy design that's as comfortable at bedtime as it is when we have to make a quick run out. \\n\\nSize & Fit \\nPants are Favorite Fit: Sits lower on the waist. \\nRelaxed Fit: Our most generous fit sits farthest from the body. \\n\\nFabric & Care \\nIn the softest blend of 63% polyester, 35% rayon and 2% spandex.\\n\\nAdditional Features \\nRelaxed fit top with raglan sleeves and rounded hem. \\nPull-on pants have a wide elastic waistband and drawstring, side pockets and a modern slim leg. \\nImported.<<<<>>>>>: 632\\nname: Cozy Comfort Fleece Pullover\\ndescription: The ultimate sweater fleece \\u2013 made from superior fabric and offered at an unbeatable price. \\n\\nSize & Fit\\nSlightly Fitted: Softly shapes the body. Falls at hip. \\n\\nWhy We Love It\\nOur customers (and employees) love the rugged construction and heritage-inspired styling of our popular Sweater Fleece Pullover and wear it for absolutely everything. From high-intensity activities to everyday tasks, you'll find yourself reaching for it every time.\\n\\nFabric & Care\\nRugged sweater-knit exterior and soft brushed interior for exceptional warmth and comfort. Made from soft, 100% polyester. Machine wash and dry.\\n\\nAdditional Features\\nFeatures our classic Mount Katahdin logo. Snap placket. Front princess seams create a feminine shape. Kangaroo handwarmer pockets. Cuffs and hem reinforced with jersey binding. Imported.\\n\\n \\u2013 Official Supplier to the U.S. Ski Team\\nTHEIR WILL TO WIN, WOVEN RIGHT IN. LEARN MORE<<<<>>>>>: 151\\nname: Cozy Quilted Sweatshirt\\ndescription: Our sweatshirt is an instant classic with its great quilted texture and versatile weight that easily transitions between seasons. With a traditional fit that is relaxed through the chest, sleeve, and waist, this pullover is lightweight enough to be worn most months of the year. The cotton blend fabric is super soft and comfortable, making it the perfect casual layer. To make dressing easy, this sweatshirt also features a snap placket and a heritage-inspired Mt. Katahdin logo patch. For care, machine wash and dry. Imported.\"\n", + " \"context\": \": 10\\nname: Cozy Comfort Pullover Set, Stripe\\ndescription: Perfect for lounging, this striped knit set lives up to its name. We used ultrasoft fabric and an easy design that's as comfortable at bedtime as it is when we have to make a quick run out.\\r\\n\\r\\nSize & Fit\\r\\n- Pants are Favorite Fit: Sits lower on the waist.\\r\\n- Relaxed Fit: Our most generous fit sits farthest from the body.\\r\\n\\r\\nFabric & Care\\r\\n- In the softest blend of 63% polyester, 35% rayon and 2% spandex.\\r\\n\\r\\nAdditional Features\\r\\n- Relaxed fit top with raglan sleeves and rounded hem.\\r\\n- Pull-on pants have a wide elastic waistband and drawstring, side pockets and a modern slim leg.\\r\\n\\r\\nImported.<<<<>>>>>: 73\\nname: Cozy Cuddles Knit Pullover Set\\ndescription: Perfect for lounging, this knit set lives up to its name. We used ultrasoft fabric and an easy design that's as comfortable at bedtime as it is when we have to make a quick run out. \\r\\n\\r\\nSize & Fit \\r\\nPants are Favorite Fit: Sits lower on the waist. \\r\\nRelaxed Fit: Our most generous fit sits farthest from the body. \\r\\n\\r\\nFabric & Care \\r\\nIn the softest blend of 63% polyester, 35% rayon and 2% spandex.\\r\\n\\r\\nAdditional Features \\r\\nRelaxed fit top with raglan sleeves and rounded hem. \\r\\nPull-on pants have a wide elastic waistband and drawstring, side pockets and a modern slim leg. \\r\\nImported.<<<<>>>>>: 632\\nname: Cozy Comfort Fleece Pullover\\ndescription: The ultimate sweater fleece – made from superior fabric and offered at an unbeatable price. \\r\\n\\r\\nSize & Fit\\r\\nSlightly Fitted: Softly shapes the body. Falls at hip. \\r\\n\\r\\nWhy We Love It\\r\\nOur customers (and employees) love the rugged construction and heritage-inspired styling of our popular Sweater Fleece Pullover and wear it for absolutely everything. From high-intensity activities to everyday tasks, you'll find yourself reaching for it every time.\\r\\n\\r\\nFabric & Care\\r\\nRugged sweater-knit exterior and soft brushed interior for exceptional warmth and comfort. Made from soft, 100% polyester. Machine wash and dry.\\r\\n\\r\\nAdditional Features\\r\\nFeatures our classic Mount Katahdin logo. Snap placket. Front princess seams create a feminine shape. Kangaroo handwarmer pockets. Cuffs and hem reinforced with jersey binding. Imported.\\r\\n\\r\\n – Official Supplier to the U.S. Ski Team\\r\\nTHEIR WILL TO WIN, WOVEN RIGHT IN. LEARN MORE<<<<>>>>>: 265\\nname: Cozy Workout Vest\\ndescription: For serious warmth that won't weigh you down, reach for this fleece-lined vest, which provides you with layering options whether you're inside or outdoors.\\r\\nSize & Fit\\r\\nRelaxed Fit. Falls at hip.\\r\\nFabric & Care\\r\\nSoft, textured fleece lining. Nylon shell. Machine wash and dry. \\r\\nAdditional Features \\r\\nTwo handwarmer pockets. Knit side panels stretch for a more flattering fit. Shell fabric is treated to resist water and stains. Imported.\"\n", "}\n", "\u001b[32;1m\u001b[1;3m[llm/start]\u001b[0m \u001b[1m[1:chain:RetrievalQA > 2:chain:StuffDocumentsChain > 3:chain:LLMChain > 4:llm:ChatOpenAI] Entering LLM run with input:\n", "\u001b[0m{\n", " \"prompts\": [\n", - " \"System: Use the following pieces of context to answer the users question. \\nIf you don't know the answer, just say that you don't know, don't try to make up an answer.\\n----------------\\n: 10\\nname: Cozy Comfort Pullover Set, Stripe\\ndescription: Perfect for lounging, this striped knit set lives up to its name. We used ultrasoft fabric and an easy design that's as comfortable at bedtime as it is when we have to make a quick run out.\\n\\nSize & Fit\\n- Pants are Favorite Fit: Sits lower on the waist.\\n- Relaxed Fit: Our most generous fit sits farthest from the body.\\n\\nFabric & Care\\n- In the softest blend of 63% polyester, 35% rayon and 2% spandex.\\n\\nAdditional Features\\n- Relaxed fit top with raglan sleeves and rounded hem.\\n- Pull-on pants have a wide elastic waistband and drawstring, side pockets and a modern slim leg.\\n\\nImported.<<<<>>>>>: 73\\nname: Cozy Cuddles Knit Pullover Set\\ndescription: Perfect for lounging, this knit set lives up to its name. We used ultrasoft fabric and an easy design that's as comfortable at bedtime as it is when we have to make a quick run out. \\n\\nSize & Fit \\nPants are Favorite Fit: Sits lower on the waist. \\nRelaxed Fit: Our most generous fit sits farthest from the body. \\n\\nFabric & Care \\nIn the softest blend of 63% polyester, 35% rayon and 2% spandex.\\n\\nAdditional Features \\nRelaxed fit top with raglan sleeves and rounded hem. \\nPull-on pants have a wide elastic waistband and drawstring, side pockets and a modern slim leg. \\nImported.<<<<>>>>>: 632\\nname: Cozy Comfort Fleece Pullover\\ndescription: The ultimate sweater fleece \\u2013 made from superior fabric and offered at an unbeatable price. \\n\\nSize & Fit\\nSlightly Fitted: Softly shapes the body. Falls at hip. \\n\\nWhy We Love It\\nOur customers (and employees) love the rugged construction and heritage-inspired styling of our popular Sweater Fleece Pullover and wear it for absolutely everything. From high-intensity activities to everyday tasks, you'll find yourself reaching for it every time.\\n\\nFabric & Care\\nRugged sweater-knit exterior and soft brushed interior for exceptional warmth and comfort. Made from soft, 100% polyester. Machine wash and dry.\\n\\nAdditional Features\\nFeatures our classic Mount Katahdin logo. Snap placket. Front princess seams create a feminine shape. Kangaroo handwarmer pockets. Cuffs and hem reinforced with jersey binding. Imported.\\n\\n \\u2013 Official Supplier to the U.S. Ski Team\\nTHEIR WILL TO WIN, WOVEN RIGHT IN. LEARN MORE<<<<>>>>>: 151\\nname: Cozy Quilted Sweatshirt\\ndescription: Our sweatshirt is an instant classic with its great quilted texture and versatile weight that easily transitions between seasons. With a traditional fit that is relaxed through the chest, sleeve, and waist, this pullover is lightweight enough to be worn most months of the year. The cotton blend fabric is super soft and comfortable, making it the perfect casual layer. To make dressing easy, this sweatshirt also features a snap placket and a heritage-inspired Mt. Katahdin logo patch. For care, machine wash and dry. Imported.\\nHuman: Do the Cozy Comfort Pullover Set have side pockets?\"\n", + " \"System: Use the following pieces of context to answer the users question. \\nIf you don't know the answer, just say that you don't know, don't try to make up an answer.\\n----------------\\n: 10\\nname: Cozy Comfort Pullover Set, Stripe\\ndescription: Perfect for lounging, this striped knit set lives up to its name. We used ultrasoft fabric and an easy design that's as comfortable at bedtime as it is when we have to make a quick run out.\\r\\n\\r\\nSize & Fit\\r\\n- Pants are Favorite Fit: Sits lower on the waist.\\r\\n- Relaxed Fit: Our most generous fit sits farthest from the body.\\r\\n\\r\\nFabric & Care\\r\\n- In the softest blend of 63% polyester, 35% rayon and 2% spandex.\\r\\n\\r\\nAdditional Features\\r\\n- Relaxed fit top with raglan sleeves and rounded hem.\\r\\n- Pull-on pants have a wide elastic waistband and drawstring, side pockets and a modern slim leg.\\r\\n\\r\\nImported.<<<<>>>>>: 73\\nname: Cozy Cuddles Knit Pullover Set\\ndescription: Perfect for lounging, this knit set lives up to its name. We used ultrasoft fabric and an easy design that's as comfortable at bedtime as it is when we have to make a quick run out. \\r\\n\\r\\nSize & Fit \\r\\nPants are Favorite Fit: Sits lower on the waist. \\r\\nRelaxed Fit: Our most generous fit sits farthest from the body. \\r\\n\\r\\nFabric & Care \\r\\nIn the softest blend of 63% polyester, 35% rayon and 2% spandex.\\r\\n\\r\\nAdditional Features \\r\\nRelaxed fit top with raglan sleeves and rounded hem. \\r\\nPull-on pants have a wide elastic waistband and drawstring, side pockets and a modern slim leg. \\r\\nImported.<<<<>>>>>: 632\\nname: Cozy Comfort Fleece Pullover\\ndescription: The ultimate sweater fleece – made from superior fabric and offered at an unbeatable price. \\r\\n\\r\\nSize & Fit\\r\\nSlightly Fitted: Softly shapes the body. Falls at hip. \\r\\n\\r\\nWhy We Love It\\r\\nOur customers (and employees) love the rugged construction and heritage-inspired styling of our popular Sweater Fleece Pullover and wear it for absolutely everything. From high-intensity activities to everyday tasks, you'll find yourself reaching for it every time.\\r\\n\\r\\nFabric & Care\\r\\nRugged sweater-knit exterior and soft brushed interior for exceptional warmth and comfort. Made from soft, 100% polyester. Machine wash and dry.\\r\\n\\r\\nAdditional Features\\r\\nFeatures our classic Mount Katahdin logo. Snap placket. Front princess seams create a feminine shape. Kangaroo handwarmer pockets. Cuffs and hem reinforced with jersey binding. Imported.\\r\\n\\r\\n – Official Supplier to the U.S. Ski Team\\r\\nTHEIR WILL TO WIN, WOVEN RIGHT IN. LEARN MORE<<<<>>>>>: 265\\nname: Cozy Workout Vest\\ndescription: For serious warmth that won't weigh you down, reach for this fleece-lined vest, which provides you with layering options whether you're inside or outdoors.\\r\\nSize & Fit\\r\\nRelaxed Fit. Falls at hip.\\r\\nFabric & Care\\r\\nSoft, textured fleece lining. Nylon shell. Machine wash and dry. \\r\\nAdditional Features \\r\\nTwo handwarmer pockets. Knit side panels stretch for a more flattering fit. Shell fabric is treated to resist water and stains. Imported.\\nHuman: Do the Cozy Comfort Pullover Set have side pockets?\"\n", " ]\n", "}\n", - "\u001b[36;1m\u001b[1;3m[llm/end]\u001b[0m \u001b[1m[1:chain:RetrievalQA > 2:chain:StuffDocumentsChain > 3:chain:LLMChain > 4:llm:ChatOpenAI] [1.68s] Exiting LLM run with output:\n", + "\u001b[36;1m\u001b[1;3m[llm/end]\u001b[0m \u001b[1m[1:chain:RetrievalQA > 2:chain:StuffDocumentsChain > 3:chain:LLMChain > 4:llm:ChatOpenAI] [2.29s] Exiting LLM run with output:\n", "\u001b[0m{\n", " \"generations\": [\n", " [\n", " {\n", - " \"text\": \"The Cozy Comfort Pullover Set, Stripe has side pockets on the pull-on pants.\",\n", + " \"text\": \"Yes, the Cozy Comfort Pullover Set does have side pockets.\",\n", " \"generation_info\": null,\n", " \"message\": {\n", - " \"content\": \"The Cozy Comfort Pullover Set, Stripe has side pockets on the pull-on pants.\",\n", + " \"content\": \"Yes, the Cozy Comfort Pullover Set does have side pockets.\",\n", " \"additional_kwargs\": {},\n", " \"example\": false\n", " }\n", @@ -610,34 +658,35 @@ " ],\n", " \"llm_output\": {\n", " \"token_usage\": {\n", - " \"prompt_tokens\": 734,\n", - " \"completion_tokens\": 18,\n", - " \"total_tokens\": 752\n", + " \"prompt_tokens\": 717,\n", + " \"completion_tokens\": 14,\n", + " \"total_tokens\": 731\n", " },\n", " \"model_name\": \"gpt-3.5-turbo\"\n", - " }\n", + " },\n", + " \"run\": null\n", "}\n", - "\u001b[36;1m\u001b[1;3m[chain/end]\u001b[0m \u001b[1m[1:chain:RetrievalQA > 2:chain:StuffDocumentsChain > 3:chain:LLMChain] [1.68s] Exiting Chain run with output:\n", + "\u001b[36;1m\u001b[1;3m[chain/end]\u001b[0m \u001b[1m[1:chain:RetrievalQA > 2:chain:StuffDocumentsChain > 3:chain:LLMChain] [2.29s] Exiting Chain run with output:\n", "\u001b[0m{\n", - " \"text\": \"The Cozy Comfort Pullover Set, Stripe has side pockets on the pull-on pants.\"\n", + " \"text\": \"Yes, the Cozy Comfort Pullover Set does have side pockets.\"\n", "}\n", - "\u001b[36;1m\u001b[1;3m[chain/end]\u001b[0m \u001b[1m[1:chain:RetrievalQA > 2:chain:StuffDocumentsChain] [1.71s] Exiting Chain run with output:\n", + "\u001b[36;1m\u001b[1;3m[chain/end]\u001b[0m \u001b[1m[1:chain:RetrievalQA > 2:chain:StuffDocumentsChain] [2.29s] Exiting Chain run with output:\n", "\u001b[0m{\n", - " \"output_text\": \"The Cozy Comfort Pullover Set, Stripe has side pockets on the pull-on pants.\"\n", + " \"output_text\": \"Yes, the Cozy Comfort Pullover Set does have side pockets.\"\n", "}\n", - "\u001b[36;1m\u001b[1;3m[chain/end]\u001b[0m \u001b[1m[1:chain:RetrievalQA] [2.86s] Exiting Chain run with output:\n", + "\u001b[36;1m\u001b[1;3m[chain/end]\u001b[0m \u001b[1m[1:chain:RetrievalQA] [2.93s] Exiting Chain run with output:\n", "\u001b[0m{\n", - " \"result\": \"The Cozy Comfort Pullover Set, Stripe has side pockets on the pull-on pants.\"\n", + " \"result\": \"Yes, the Cozy Comfort Pullover Set does have side pockets.\"\n", "}\n" ] }, { "data": { "text/plain": [ - "'The Cozy Comfort Pullover Set, Stripe has side pockets on the pull-on pants.'" + "'Yes, the Cozy Comfort Pullover Set does have side pockets.'" ] }, - "execution_count": 27, + "execution_count": 22, "metadata": {}, "output_type": "execute_result" } @@ -666,12 +715,13 @@ "id": "7b37c7bc", "metadata": {}, "source": [ - "**如何评估新创建的实例**: 与创建它们类似，可以运行链条来处理所有示例，然后查看输出并尝试弄清楚，发生了什么，它是否正确" + "#### 6.1.2.1 如何评估新创建的实例\n", + "与创建它们类似，可以运行链条来处理所有示例，然后查看输出并尝试弄清楚，发生了什么，它是否正确" ] }, { "cell_type": "code", - "execution_count": 28, + "execution_count": 23, "id": "b3d6bef0", "metadata": { "height": 47 @@ -687,12 +737,12 @@ "id": "d5bdbdce", "metadata": {}, "source": [ - "## 6.3 通过LLM进行评估实例" + "### 6.1.3 通过LLM进行评估实例" ] }, { "cell_type": "code", - "execution_count": 32, + "execution_count": 24, "id": "a4dca05a", "metadata": { "height": 30 @@ -704,19 +754,21 @@ "text": [ "\n", "\n", - "\u001b[1m> Entering new RetrievalQA chain...\u001b[0m\n", + "\u001b[1m> Entering new chain...\u001b[0m\n", "\n", "\u001b[1m> Finished chain.\u001b[0m\n", "\n", "\n", - "\u001b[1m> Entering new RetrievalQA chain...\u001b[0m\n" + "\u001b[1m> Entering new chain...\u001b[0m\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ - "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 1.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-Nul9WsPYdsnjttqS3f0hDSWd on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n" + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 1.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 2.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 4.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n" ] }, { @@ -727,18 +779,17 @@ "\u001b[1m> Finished chain.\u001b[0m\n", "\n", "\n", - "\u001b[1m> Entering new RetrievalQA chain...\u001b[0m\n" + "\u001b[1m> Entering new chain...\u001b[0m\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ - "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 1.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-Nul9WsPYdsnjttqS3f0hDSWd on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", - "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 2.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-Nul9WsPYdsnjttqS3f0hDSWd on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", - "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 4.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-Nul9WsPYdsnjttqS3f0hDSWd on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", - "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 8.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-Nul9WsPYdsnjttqS3f0hDSWd on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", - "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 16.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-Nul9WsPYdsnjttqS3f0hDSWd on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n" + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 1.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 2.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 4.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 8.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n" ] }, { @@ -749,14 +800,17 @@ "\u001b[1m> Finished chain.\u001b[0m\n", "\n", "\n", - "\u001b[1m> Entering new RetrievalQA chain...\u001b[0m\n" + "\u001b[1m> Entering new chain...\u001b[0m\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ - "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 1.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-Nul9WsPYdsnjttqS3f0hDSWd on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n" + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 1.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 2.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 4.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 8.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n" ] }, { @@ -767,18 +821,17 @@ "\u001b[1m> Finished chain.\u001b[0m\n", "\n", "\n", - "\u001b[1m> Entering new RetrievalQA chain...\u001b[0m\n" + "\u001b[1m> Entering new chain...\u001b[0m\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ - "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 1.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-Nul9WsPYdsnjttqS3f0hDSWd on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", - "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 2.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-Nul9WsPYdsnjttqS3f0hDSWd on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", - "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 4.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-Nul9WsPYdsnjttqS3f0hDSWd on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", - "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 8.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-Nul9WsPYdsnjttqS3f0hDSWd on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", - "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 16.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-Nul9WsPYdsnjttqS3f0hDSWd on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n" + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 1.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 2.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 4.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 8.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n" ] }, { @@ -789,22 +842,38 @@ "\u001b[1m> Finished chain.\u001b[0m\n", "\n", "\n", - "\u001b[1m> Entering new RetrievalQA chain...\u001b[0m\n", - "\n", - "\u001b[1m> Finished chain.\u001b[0m\n", - "\n", - "\n", - "\u001b[1m> Entering new RetrievalQA chain...\u001b[0m\n" + "\u001b[1m> Entering new chain...\u001b[0m\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ - "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 1.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-Nul9WsPYdsnjttqS3f0hDSWd on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", - "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 2.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-Nul9WsPYdsnjttqS3f0hDSWd on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", - "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 4.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-Nul9WsPYdsnjttqS3f0hDSWd on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", - "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 8.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-Nul9WsPYdsnjttqS3f0hDSWd on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n" + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 1.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 2.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 4.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 8.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "\u001b[1m> Finished chain.\u001b[0m\n", + "\n", + "\n", + "\u001b[1m> Entering new chain...\u001b[0m\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 1.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 2.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 4.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 8.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n" ] }, { @@ -822,7 +891,7 @@ }, { "cell_type": "code", - "execution_count": 35, + "execution_count": 25, "id": "6012a3e0", "metadata": { "height": 30 @@ -837,7 +906,7 @@ }, { "cell_type": "code", - "execution_count": 36, + "execution_count": 26, "id": "724b1c0b", "metadata": { "height": 47 @@ -851,12 +920,46 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 27, "id": "8b46ae55", "metadata": { "height": 30 }, - "outputs": [], + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 1.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 2.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 4.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 8.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 1.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 2.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 4.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 8.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 1.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 2.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 4.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 8.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 1.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 2.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 4.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 8.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 16.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 1.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 2.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 1.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 2.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 4.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 8.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 16.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 1.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 2.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 4.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n" + ] + } + ], "source": [ "graded_outputs = eval_chain.evaluate(examples, predictions)#在此链上调用evaluate，进行评估" ] @@ -866,13 +969,13 @@ "id": "9ad64f72", "metadata": {}, "source": [ - "### 6.3.1 评估思路\n", + "#### 6.1.3.1 评估思路\n", "当它面前有整个文档时，它可以生成一个真实的答案，我们将打印出预测的答，当它进行QA链时，使用embedding和向量数据库进行检索时，将其传递到语言模型中，然后尝试猜测预测的答案，我们还将打印出成绩，这也是语言模型生成的。当它要求评估链评估正在发生的事情时，以及它是否正确或不正确。因此，当我们循环遍历所有这些示例并将它们打印出来时，可以详细了解每个示例" ] }, { "cell_type": "code", - "execution_count": 24, + "execution_count": 28, "id": "3437cfbe", "metadata": { "height": 132 @@ -883,45 +986,53 @@ "output_type": "stream", "text": [ "Example 0:\n", - "Question: Do the Cozy Comfort Pullover Set have side pockets?\n", + "Question: Do the Cozy Comfort Pullover Set have side pockets?\n", "Real Answer: Yes\n", - "Predicted Answer: The Cozy Comfort Pullover Set, Stripe does have side pockets.\n", + "Predicted Answer: Yes, the Cozy Comfort Pullover Set does have side pockets.\n", "Predicted Grade: CORRECT\n", "\n", "Example 1:\n", - "Question: What collection is the Ultra-Lofty 850 Stretch Down Hooded Jacket from?\n", + "Question: What collection is the Ultra-Lofty 850 Stretch Down Hooded Jacket from?\n", "Real Answer: The DownTek collection\n", "Predicted Answer: The Ultra-Lofty 850 Stretch Down Hooded Jacket is from the DownTek collection.\n", "Predicted Grade: CORRECT\n", "\n", "Example 2:\n", - "Question: What is the weight of each pair of Women's Campside Oxfords?\n", - "Real Answer: The approximate weight of each pair of Women's Campside Oxfords is 1 lb. 1 oz.\n", - "Predicted Answer: The weight of each pair of Women's Campside Oxfords is approximately 1 lb. 1 oz.\n", + "Question: What is the description of the Women's Campside Oxfords?\n", + "Real Answer: The Women's Campside Oxfords are described as an ultracomfortable lace-to-toe Oxford made of soft canvas material, featuring thick cushioning, quality construction, and a broken-in feel from the first time they are worn.\n", + "Predicted Answer: The description of the Women's Campside Oxfords is: \"This ultracomfortable lace-to-toe Oxford boasts a super-soft canvas, thick cushioning, and quality construction for a broken-in feel from the first time you put them on.\"\n", "Predicted Grade: CORRECT\n", "\n", "Example 3:\n", - "Question: What are the dimensions of the small and medium Recycled Waterhog Dog Mat?\n", - "Real Answer: The dimensions of the small Recycled Waterhog Dog Mat are 18\" x 28\" and the dimensions of the medium Recycled Waterhog Dog Mat are 22.5\" x 34.5\".\n", - "Predicted Answer: The small Recycled Waterhog Dog Mat has dimensions of 18\" x 28\" and the medium size has dimensions of 22.5\" x 34.5\".\n", + "Question: What are the dimensions of the small and medium sizes of the Recycled Waterhog Dog Mat, Chevron Weave?\n", + "Real Answer: The small size of the Recycled Waterhog Dog Mat, Chevron Weave has dimensions of 18\" x 28\", while the medium size has dimensions of 22.5\" x 34.5\".\n", + "Predicted Answer: The dimensions of the small size of the Recycled Waterhog Dog Mat, Chevron Weave are 18\" x 28\". The dimensions of the medium size are 22.5\" x 34.5\".\n", "Predicted Grade: CORRECT\n", "\n", "Example 4:\n", - "Question: What are some features of the Infant and Toddler Girls' Coastal Chill Swimsuit?\n", - "Real Answer: The swimsuit features bright colors, ruffles, and exclusive whimsical prints. It is made of four-way-stretch and chlorine-resistant fabric, ensuring that it keeps its shape and resists snags. The swimsuit is also UPF 50+ rated, providing the highest rated sun protection possible by blocking 98% of the sun's harmful rays. The crossover no-slip straps and fully lined bottom ensure a secure fit and maximum coverage. Finally, it can be machine washed and line dried for best results.\n", - "Predicted Answer: The Infant and Toddler Girls' Coastal Chill Swimsuit is a two-piece swimsuit with bright colors, ruffles, and exclusive whimsical prints. It is made of four-way-stretch and chlorine-resistant fabric that keeps its shape and resists snags. The swimsuit has UPF 50+ rated fabric that provides the highest rated sun protection possible, blocking 98% of the sun's harmful rays. The crossover no-slip straps and fully lined bottom ensure a secure fit and maximum coverage. It is machine washable and should be line dried for best results.\n", + "Question: What are some features of the Infant and Toddler Girls' Coastal Chill Swimsuit, Two-Piece?\n", + "Real Answer: The swimsuit features bright colors, ruffles, and exclusive whimsical prints. It is made of a four-way-stretch and chlorine-resistant fabric that maintains its shape and resists snags. The fabric is also UPF 50+ rated, providing the highest rated sun protection possible by blocking 98% of the sun's harmful rays. The swimsuit has crossover no-slip straps and a fully lined bottom for a secure fit and maximum coverage.\n", + "Predicted Answer: Some features of the Infant and Toddler Girls' Coastal Chill Swimsuit, Two-Piece are:\n", + "- Bright colors and ruffles\n", + "- Exclusive whimsical prints\n", + "- Four-way-stretch and chlorine-resistant fabric\n", + "- UPF 50+ rated fabric for sun protection\n", + "- Crossover no-slip straps\n", + "- Fully lined bottom for secure fit and maximum coverage\n", + "- Machine washable and line dry for best results\n", + "- Imported\n", "Predicted Grade: CORRECT\n", "\n", "Example 5:\n", - "Question: What is the fabric composition of the Refresh Swimwear V-Neck Tankini Contrasts?\n", - "Real Answer: The body of the Refresh Swimwear V-Neck Tankini Contrasts is made of 82% recycled nylon and 18% Lycra® spandex, while the lining is made of 90% recycled nylon and 10% Lycra® spandex.\n", - "Predicted Answer: The Refresh Swimwear V-Neck Tankini Contrasts is made of 82% recycled nylon with 18% Lycra® spandex for the body and 90% recycled nylon with 10% Lycra® spandex for the lining.\n", + "Question: What is the fabric composition of the Refresh Swimwear, V-Neck Tankini Contrasts?\n", + "Real Answer: The Refresh Swimwear, V-Neck Tankini Contrasts is made of 82% recycled nylon with 18% Lycra® spandex for the body, and 90% recycled nylon with 10% Lycra® spandex for the lining.\n", + "Predicted Answer: The fabric composition of the Refresh Swimwear, V-Neck Tankini Contrasts is 82% recycled nylon with 18% Lycra® spandex for the body, and 90% recycled nylon with 10% Lycra® spandex for the lining.\n", "Predicted Grade: CORRECT\n", "\n", "Example 6:\n", - "Question: What is the fabric composition of the EcoFlex 3L Storm Pants?\n", - "Real Answer: The EcoFlex 3L Storm Pants are made of 100% nylon, exclusive of trim.\n", - "Predicted Answer: The fabric composition of the EcoFlex 3L Storm Pants is 100% nylon, exclusive of trim.\n", + "Question: What is the technology used in the EcoFlex 3L Storm Pants that makes them more breathable?\n", + "Real Answer: The EcoFlex 3L Storm Pants use TEK O2 technology to make them more breathable.\n", + "Predicted Answer: The technology used in the EcoFlex 3L Storm Pants that makes them more breathable is called TEK O2 technology.\n", "Predicted Grade: CORRECT\n", "\n" ] @@ -943,10 +1054,10 @@ "id": "87ecb476", "metadata": {}, "source": [ - "### 6.3.2 结果分析\n", + "#### 6.1.3.2 结果分析\n", "对于每个示例，它看起来都是正确的，让我们看看第一个例子。\n", "这里的问题是，舒适的套头衫套装，有侧口袋吗？真正的答案，我们创建了这个，是肯定的。模型预测的答案是舒适的套头衫套装条纹，确实有侧口袋。因此，我们可以理解这是一个正确的答案。它将其评为正确。 \n", - "### 6.3.3 使用模型评估的优势\n", + "#### 6.1.3.3 使用模型评估的优势\n", "\n", "你有这些答案，它们是任意的字符串。没有单一的真实字符串是最好的可能答案，有许多不同的变体，只要它们具有相同的语义，它们应该被评为相似。如果使用正则进行精准匹配就会丢失语义信息，到目前为止存在的许多评估指标都不够好。目前最有趣和最受欢迎的之一就是使用语言模型进行评估。" ] @@ -954,7 +1065,1314 @@ { "cell_type": "code", "execution_count": null, - "id": "e4014482", + "id": "fe07d2ef", + "metadata": {}, + "outputs": [], + "source": [] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "146806a9", + "metadata": {}, + "outputs": [], + "source": [] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "44a464d5", + "metadata": {}, + "outputs": [], + "source": [] + }, + { + "attachments": {}, + "cell_type": "markdown", + "id": "0b6680d2", + "metadata": {}, + "source": [ + "## 6.2 中文版\n", + "### 6.2.1 创建LLM应用\n", + "按照langchain链的方式进行构建" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "c045628e", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain.chains import RetrievalQA #检索QA链，在文档上进行检索\n", + "from langchain.chat_models import ChatOpenAI #openai模型\n", + "from langchain.document_loaders import CSVLoader #文档加载器，采用csv格式存储\n", + "from langchain.indexes import VectorstoreIndexCreator #导入向量存储索引创建器\n", + "from langchain.vectorstores import DocArrayInMemorySearch #向量存储\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "bfd04e93", + "metadata": {}, + "outputs": [], + "source": [ + "#加载中文数据\n", + "file = 'product_data.csv'\n", + "loader = CSVLoader(file_path=file)\n", + "data = loader.load()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "da8eb973", + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
01
0product_namedescription
1全自动咖啡机规格:\\n大型 - 尺寸：13.8'' x 17.3''。\\n中型 - 尺寸：11.5'' ...
2电动牙刷规格:\\n一般大小 - 高度：9.5''，宽度：1''。\\n\\n为什么我们热爱它:\\n我们的...
3橙味维生素C泡腾片规格:\\n每盒含有20片。\\n\\n为什么我们热爱它:\\n我们的橙味维生素C泡腾片是快速补充维...
4无线蓝牙耳机规格:\\n单个耳机尺寸：1.5'' x 1.3''。\\n\\n为什么我们热爱它:\\n这款无线蓝...
5瑜伽垫规格:\\n尺寸：24'' x 68''。\\n\\n为什么我们热爱它:\\n我们的瑜伽垫拥有出色的...
6防水运动手表规格:\\n表盘直径：40mm。\\n\\n为什么我们热爱它:\\n这款防水运动手表配备了心率监测和...
7书籍:《机器学习基础》规格:\\n页数：580页。\\n\\n为什么我们热爱它:\\n《机器学习基础》以易懂的语言讲解了机...
8空气净化器规格:\\n尺寸：15'' x 15'' x 20''。\\n\\n为什么我们热爱它:\\n我们的空...
9陶瓷保温杯规格:\\n容量：350ml。\\n\\n为什么我们热爱它:\\n我们的陶瓷保温杯设计优雅，保温效果...
10宠物自动喂食器规格:\\n尺寸：14'' x 9'' x 15''。\\n\\n为什么我们热爱它:\\n我们的宠物...
11高清电视机规格:\\n尺寸：50''。\\n\\n为什么我们热爱它:\\n我们的高清电视机拥有出色的画质和强大...
12旅行背包规格:\\n尺寸：18'' x 12'' x 6''。\\n\\n为什么我们热爱它:\\n我们的旅行...
13太阳能庭院灯规格:\\n高度：18''。\\n\\n为什么我们热爱它:\\n我们的太阳能庭院灯无需电源，只需将其...
14厨房刀具套装规格:\\n一套包括8把刀。\\n\\n为什么我们热爱它:\\n我们的厨房刀具套装由专业级不锈钢制成...
15迷你无线蓝牙音箱规格:\\n直径：3''，高度：2''。\\n\\n为什么我们热爱它:\\n我们的迷你无线蓝牙音箱体...
16抗菌洗手液规格:\\n容量：500ml。\\n\\n为什么我们热爱它:\\n我们的抗菌洗手液含有天然植物精华，...
17纯棉T恤规格:\\n尺码：S, M, L, XL, XXL。\\n\\n为什么我们热爱它:\\n我们的纯棉T...
18自动咖啡机规格:\\n尺寸：12'' x 8'' x 14''。\\n\\n为什么我们热爱它:\\n我们的自动...
19摄像头保护套规格:\\n适用于各种品牌和型号的摄像头。\\n\\n为什么我们热爱它:\\n我们的摄像头保护套可以...
20玻璃保护膜规格:\\n适用于各种尺寸的手机屏幕。\\n\\n为什么我们热爱它:\\n我们的玻璃保护膜可以有效防...
21儿童益智玩具规格:\\n适合3岁以上的儿童。\\n\\n为什么我们热爱它:\\n我们的儿童益智玩具设计独特，色彩...
22迷你书架规格:\\n尺寸：20'' x 8'' x 24''。\\n\\n为什么我们热爱它:\\n我们的迷你...
23防滑瑜伽垫规格:\\n尺寸：72'' x 24''。\\n\\n为什么我们热爱它:\\n我们的防滑瑜伽垫采用高...
24LED台灯规格:\\n尺寸：6'' x 6'' x 18''。\\n\\n为什么我们热爱它:\\n我们的LED...
25水晶酒杯规格:\\n容量：250ml。\\n\\n为什么我们热爱它:\\n我们的水晶酒杯采用高品质水晶玻璃制...
\n", + "
" + ], + "text/plain": [ + " 0 1\n", + "0 product_name description\n", + "1 全自动咖啡机 规格:\\n大型 - 尺寸：13.8'' x 17.3''。\\n中型 - 尺寸：11.5'' ...\n", + "2 电动牙刷 规格:\\n一般大小 - 高度：9.5''，宽度：1''。\\n\\n为什么我们热爱它:\\n我们的...\n", + "3 橙味维生素C泡腾片 规格:\\n每盒含有20片。\\n\\n为什么我们热爱它:\\n我们的橙味维生素C泡腾片是快速补充维...\n", + "4 无线蓝牙耳机 规格:\\n单个耳机尺寸：1.5'' x 1.3''。\\n\\n为什么我们热爱它:\\n这款无线蓝...\n", + "5 瑜伽垫 规格:\\n尺寸：24'' x 68''。\\n\\n为什么我们热爱它:\\n我们的瑜伽垫拥有出色的...\n", + "6 防水运动手表 规格:\\n表盘直径：40mm。\\n\\n为什么我们热爱它:\\n这款防水运动手表配备了心率监测和...\n", + "7 书籍:《机器学习基础》 规格:\\n页数：580页。\\n\\n为什么我们热爱它:\\n《机器学习基础》以易懂的语言讲解了机...\n", + "8 空气净化器 规格:\\n尺寸：15'' x 15'' x 20''。\\n\\n为什么我们热爱它:\\n我们的空...\n", + "9 陶瓷保温杯 规格:\\n容量：350ml。\\n\\n为什么我们热爱它:\\n我们的陶瓷保温杯设计优雅，保温效果...\n", + "10 宠物自动喂食器 规格:\\n尺寸：14'' x 9'' x 15''。\\n\\n为什么我们热爱它:\\n我们的宠物...\n", + "11 高清电视机 规格:\\n尺寸：50''。\\n\\n为什么我们热爱它:\\n我们的高清电视机拥有出色的画质和强大...\n", + "12 旅行背包 规格:\\n尺寸：18'' x 12'' x 6''。\\n\\n为什么我们热爱它:\\n我们的旅行...\n", + "13 太阳能庭院灯 规格:\\n高度：18''。\\n\\n为什么我们热爱它:\\n我们的太阳能庭院灯无需电源，只需将其...\n", + "14 厨房刀具套装 规格:\\n一套包括8把刀。\\n\\n为什么我们热爱它:\\n我们的厨房刀具套装由专业级不锈钢制成...\n", + "15 迷你无线蓝牙音箱 规格:\\n直径：3''，高度：2''。\\n\\n为什么我们热爱它:\\n我们的迷你无线蓝牙音箱体...\n", + "16 抗菌洗手液 规格:\\n容量：500ml。\\n\\n为什么我们热爱它:\\n我们的抗菌洗手液含有天然植物精华，...\n", + "17 纯棉T恤 规格:\\n尺码：S, M, L, XL, XXL。\\n\\n为什么我们热爱它:\\n我们的纯棉T...\n", + "18 自动咖啡机 规格:\\n尺寸：12'' x 8'' x 14''。\\n\\n为什么我们热爱它:\\n我们的自动...\n", + "19 摄像头保护套 规格:\\n适用于各种品牌和型号的摄像头。\\n\\n为什么我们热爱它:\\n我们的摄像头保护套可以...\n", + "20 玻璃保护膜 规格:\\n适用于各种尺寸的手机屏幕。\\n\\n为什么我们热爱它:\\n我们的玻璃保护膜可以有效防...\n", + "21 儿童益智玩具 规格:\\n适合3岁以上的儿童。\\n\\n为什么我们热爱它:\\n我们的儿童益智玩具设计独特，色彩...\n", + "22 迷你书架 规格:\\n尺寸：20'' x 8'' x 24''。\\n\\n为什么我们热爱它:\\n我们的迷你...\n", + "23 防滑瑜伽垫 规格:\\n尺寸：72'' x 24''。\\n\\n为什么我们热爱它:\\n我们的防滑瑜伽垫采用高...\n", + "24 LED台灯 规格:\\n尺寸：6'' x 6'' x 18''。\\n\\n为什么我们热爱它:\\n我们的LED...\n", + "25 水晶酒杯 规格:\\n容量：250ml。\\n\\n为什么我们热爱它:\\n我们的水晶酒杯采用高品质水晶玻璃制..." + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "#查看数据\n", + "import pandas as pd\n", + "test_data = pd.read_csv(file,header=None)\n", + "test_data" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "95890a3b", + "metadata": {}, + "outputs": [], + "source": [ + "'''\n", + "将指定向量存储类,创建完成后，我们将从加载器中调用,通过文档记载器列表加载\n", + "'''\n", + "index = VectorstoreIndexCreator(\n", + " vectorstore_cls=DocArrayInMemorySearch\n", + ").from_loaders([loader])" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f5230594", + "metadata": {}, + "outputs": [], + "source": [ + "#通过指定语言模型、链类型、检索器和我们要打印的详细程度来创建检索QA链\n", + "llm = ChatOpenAI(temperature = 0.0)\n", + "qa = RetrievalQA.from_chain_type(\n", + " llm=llm, \n", + " chain_type=\"stuff\", \n", + " retriever=index.vectorstore.as_retriever(), \n", + " verbose=True,\n", + " chain_type_kwargs = {\n", + " \"document_separator\": \"<<<<>>>>>\"\n", + " }\n", + ")" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "id": "952fc528", + "metadata": {}, + "source": [ + "#### 6.2.1.1 创建评估数据点\n", + "我们需要做的第一件事是真正弄清楚我们想要评估它的一些数据点，我们将介绍几种不同的方法来完成这个任务\n", + "\n", + "1、将自己想出好的数据点作为例子，查看一些数据，然后想出例子问题和答案，以便以后用于评估" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "eae9bd65", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "Document(page_content=\"product_name: 高清电视机\\ndescription: 规格:\\n尺寸：50''。\\n\\n为什么我们热爱它:\\n我们的高清电视机拥有出色的画质和强大的音效，带来沉浸式的观看体验。\\n\\n材质与护理:\\n使用干布清洁。\\n\\n构造:\\n由塑料、金属和电子元件制成。\\n\\n其他特性:\\n支持网络连接，可以在线观看视频。\\n配备遥控器。\\n在韩国制造。\\n\\n有问题？请随时联系我们的客户服务团队，他们会解答您的所有问题。\", metadata={'source': 'product_data.csv', 'row': 10})" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "data[10]#查看这里的一些文档，我们可以对其中发生的事情有所了解" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "5ef28a34", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "Document(page_content=\"product_name: 旅行背包\\ndescription: 规格:\\n尺寸：18'' x 12'' x 6''。\\n\\n为什么我们热爱它:\\n我们的旅行背包拥有多个实用的内外袋，轻松装下您的必需品，是短途旅行的理想选择。\\n\\n材质与护理:\\n可以手洗，自然晾干。\\n\\n构造:\\n由防水尼龙制成。\\n\\n其他特性:\\n附带可调节背带和安全锁。\\n在中国制造。\\n\\n有问题？请随时联系我们的客户服务团队，他们会解答您的所有问题。\", metadata={'source': 'product_data.csv', 'row': 11})" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "data[11]" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "id": "b5b994b0", + "metadata": {}, + "source": [ + "看上面的第一个文档中有高清电视机，第二个文档中有旅行背包，从这些细节中，我们可以创建一些例子查询和答案" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "id": "0214fe01", + "metadata": {}, + "source": [ + "#### 6.2.1.2 创建测试用例数据\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "8936f72d", + "metadata": {}, + "outputs": [], + "source": [ + "examples = [\n", + " {\n", + " \"query\": \"高清电视机怎么进行护理？\",\n", + " \"answer\": \"使用干布清洁。\"\n", + " },\n", + " {\n", + " \"query\": \"旅行背包有内外袋吗？\",\n", + " \"answer\": \"有。\"\n", + " }\n", + "]" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "id": "27f3bc68", + "metadata": {}, + "source": [ + "#### 6.2.1.3 通过LLM生成测试用例" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "d9c7342c", + "metadata": {}, + "outputs": [], + "source": [ + "from langchain.evaluation.qa import QAGenerateChain #导入QA生成链，它将接收文档，并从每个文档中创建一个问题答案对\n" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "id": "d2117da8", + "metadata": {}, + "source": [ + "由于`QAGenerateChain`类中使用的`PROMPT`是英文，故我们继承`QAGenerateChain`类，将`PROMPT`加上“请使用中文输出”。" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "id": "9e76a2ae", + "metadata": {}, + "source": [ + "下面是`generate_chain.py`文件中的`QAGenerateChain`类的源码" + ] + }, + { + "cell_type": "code", + "execution_count": 35, + "id": "0401f602", + "metadata": {}, + "outputs": [], + "source": [ + "\"\"\"LLM Chain specifically for generating examples for question answering.\"\"\"\n", + "from __future__ import annotations\n", + "\n", + "from typing import Any\n", + "\n", + "from langchain.base_language import BaseLanguageModel\n", + "from langchain.chains.llm import LLMChain\n", + "from langchain.evaluation.qa.generate_prompt import PROMPT\n", + "\n", + "class QAGenerateChain(LLMChain):\n", + " \"\"\"LLM Chain specifically for generating examples for question answering.\"\"\"\n", + "\n", + " @classmethod\n", + " def from_llm(cls, llm: BaseLanguageModel, **kwargs: Any) -> QAGenerateChain:\n", + " \"\"\"Load QA Generate Chain from LLM.\"\"\"\n", + " return cls(llm=llm, prompt=PROMPT, **kwargs)" + ] + }, + { + "cell_type": "code", + "execution_count": 36, + "id": "9fb1e63e", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "PromptTemplate(input_variables=['doc'], output_parser=RegexParser(regex='QUESTION: (.*?)\\\\nANSWER: (.*)', output_keys=['query', 'answer'], default_output_key=None), partial_variables={}, template='You are a teacher coming up with questions to ask on a quiz. \\nGiven the following document, please generate a question and answer based on that document.\\n\\nExample Format:\\n\\n...\\n\\nQUESTION: question here\\nANSWER: answer here\\n\\nThese questions should be detailed and be based explicitly on information in the document. Begin!\\n\\n\\n{doc}\\n\\n请用中文。', template_format='f-string', validate_template=True)" + ] + }, + "execution_count": 36, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "PROMPT" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "id": "13e72db4", + "metadata": {}, + "source": [ + "我们可以看到`PROMPT`为英文，下面我们将`PROMPT`添加上“请使用中文输出”" + ] + }, + { + "cell_type": "code", + "execution_count": 43, + "id": "a33dc3af", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "PromptTemplate(input_variables=['doc'], output_parser=RegexParser(regex='QUESTION: (.*?)\\\\nANSWER: (.*)', output_keys=['query', 'answer'], default_output_key=None), partial_variables={}, template='You are a teacher coming up with questions to ask on a quiz. \\nGiven the following document, please generate a question and answer based on that document.\\n\\nExample Format:\\n\\n...\\n\\nQUESTION: question here\\nANSWER: answer here\\n\\nThese questions should be detailed and be based explicitly on information in the document. Begin!\\n\\n\\n{doc}\\n\\n请使用中文输出。\\n', template_format='f-string', validate_template=True)" + ] + }, + "execution_count": 43, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "# 下面是langchain.evaluation.qa.generate_prompt中的源码，我们在template的最后加上“请使用中文输出”\n", + "# flake8: noqa\n", + "from langchain.output_parsers.regex import RegexParser\n", + "from langchain.prompts import PromptTemplate\n", + "\n", + "template = \"\"\"You are a teacher coming up with questions to ask on a quiz. \n", + "Given the following document, please generate a question and answer based on that document.\n", + "\n", + "Example Format:\n", + "\n", + "...\n", + "\n", + "QUESTION: question here\n", + "ANSWER: answer here\n", + "\n", + "These questions should be detailed and be based explicitly on information in the document. Begin!\n", + "\n", + "\n", + "{doc}\n", + "\n", + "请使用中文输出。\n", + "\"\"\"\n", + "output_parser = RegexParser(\n", + " regex=r\"QUESTION: (.*?)\\nANSWER: (.*)\", output_keys=[\"query\", \"answer\"]\n", + ")\n", + "PROMPT = PromptTemplate(\n", + " input_variables=[\"doc\"], template=template, output_parser=output_parser\n", + ")\n", + "\n", + "PROMPT\n" + ] + }, + { + "cell_type": "code", + "execution_count": 39, + "metadata": {}, + "outputs": [], + "source": [ + "# 继承QAGenerateChain\n", + "class MyQAGenerateChain(QAGenerateChain):\n", + " \"\"\"LLM Chain specifically for generating examples for question answering.\"\"\"\n", + "\n", + " @classmethod\n", + " def from_llm(cls, llm: BaseLanguageModel, **kwargs: Any) -> QAGenerateChain:\n", + " \"\"\"Load QA Generate Chain from LLM.\"\"\"\n", + " return cls(llm=llm, prompt=PROMPT, **kwargs)" + ] + }, + { + "cell_type": "code", + "execution_count": 40, + "id": "8dfc4372", + "metadata": {}, + "outputs": [], + "source": [ + "example_gen_chain = MyQAGenerateChain.from_llm(ChatOpenAI())#通过传递chat open AI语言模型来创建这个链" + ] + }, + { + "cell_type": "code", + "execution_count": 41, + "id": "d25c6a0e", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[Document(page_content=\"product_name: 全自动咖啡机\\ndescription: 规格:\\n大型 - 尺寸：13.8'' x 17.3''。\\n中型 - 尺寸：11.5'' x 15.2''。\\n\\n为什么我们热爱它:\\n这款全自动咖啡机是爱好者的理想选择。 一键操作，即可研磨豆子并沏制出您喜爱的咖啡。它的耐用性和一致性使它成为家庭和办公室的理想选择。\\n\\n材质与护理:\\n清洁时只需轻擦。\\n\\n构造:\\n由高品质不锈钢制成。\\n\\n其他特性:\\n内置研磨器和滤网。\\n预设多种咖啡模式。\\n在中国制造。\\n\\n有问题？ 请随时联系我们的客户服务团队，他们会解答您的所有问题。\", metadata={'source': 'product_data.csv', 'row': 0}),\n", + " Document(page_content=\"product_name: 电动牙刷\\ndescription: 规格:\\n一般大小 - 高度：9.5''，宽度：1''。\\n\\n为什么我们热爱它:\\n我们的电动牙刷采用先进的刷头设计和强大的电机，为您提供超凡的清洁力和舒适的刷牙体验。\\n\\n材质与护理:\\n不可水洗，只需用湿布清洁。\\n\\n构造:\\n由食品级塑料和尼龙刷毛制成。\\n\\n其他特性:\\n具有多种清洁模式和定时功能。\\nUSB充电。\\n在日本制造。\\n\\n有问题？请随时联系我们的客户服务团队，他们会解答您的所有问题。\", metadata={'source': 'product_data.csv', 'row': 1}),\n", + " Document(page_content='product_name: 橙味维生素C泡腾片\\ndescription: 规格:\\n每盒含有20片。\\n\\n为什么我们热爱它:\\n我们的橙味维生素C泡腾片是快速补充维生素C的理想方式。每片含有500mg的维生素C，可以帮助提升免疫力，保护您的健康。\\n\\n材质与护理:\\n请存放在阴凉干燥的地方，避免阳光直射。\\n\\n构造:\\n主要成分为维生素C和柠檬酸钠。\\n\\n其他特性:\\n含有天然橙味。\\n易于携带。\\n在美国制造。\\n\\n有问题？请随时联系我们的客户服务团队，他们会解答您的所有问题。', metadata={'source': 'product_data.csv', 'row': 2}),\n", + " Document(page_content=\"product_name: 无线蓝牙耳机\\ndescription: 规格:\\n单个耳机尺寸：1.5'' x 1.3''。\\n\\n为什么我们热爱它:\\n这款无线蓝牙耳机配备了降噪技术和长达8小时的电池续航力，让您无论在哪里都可以享受无障碍的音乐体验。\\n\\n材质与护理:\\n只需用湿布清洁。\\n\\n构造:\\n由耐用的塑料和金属构成，配备有软质耳塞。\\n\\n其他特性:\\n快速充电功能。\\n内置麦克风，支持接听电话。\\n在韩国制造。\\n\\n有问题？请随时联系我们的客户服务团队，他们会解答您的所有问题。\", metadata={'source': 'product_data.csv', 'row': 3}),\n", + " Document(page_content=\"product_name: 瑜伽垫\\ndescription: 规格:\\n尺寸：24'' x 68''。\\n\\n为什么我们热爱它:\\n我们的瑜伽垫拥有出色的抓地力和舒适度，无论是做瑜伽还是健身，都是理想的选择。\\n\\n材质与护理:\\n可用清水清洁，自然晾干。\\n\\n构造:\\n由环保PVC材料制成。\\n\\n其他特性:\\n附带便携包和绑带。\\n在印度制造。\\n\\n有问题？请随时联系我们的客户服务团队，他们会解答您的所有问题。\", metadata={'source': 'product_data.csv', 'row': 4})]" + ] + }, + "execution_count": 41, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "data[:5]" + ] + }, + { + "cell_type": "code", + "execution_count": 42, + "id": "ef0f5cad", + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "d:\\anaconda3\\envs\\gpt_flask\\lib\\site-packages\\langchain\\chains\\llm.py:303: UserWarning: The apply_and_parse method is deprecated, instead pass an output parser directly to LLMChain.\n", + " warnings.warn(\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 1.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 2.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 4.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 8.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n" + ] + } + ], + "source": [ + "new_examples = example_gen_chain.apply_and_parse(\n", + " [{\"doc\": t} for t in data[:5]]\n", + ") #我们可以创建许多例子" + ] + }, + { + "cell_type": "code", + "execution_count": 44, + "id": "7bc64a85", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[{'query': '这款全自动咖啡机的规格是什么？',\n", + " 'answer': \"大型尺寸为13.8'' x 17.3''，中型尺寸为11.5'' x 15.2''。\"},\n", + " {'query': '这款电动牙刷的高度和宽度分别是多少？', 'answer': '这款电动牙刷的高度是9.5英寸，宽度是1英寸。'},\n", + " {'query': '这款橙味维生素C泡腾片的规格是什么？', 'answer': '每盒含有20片。'},\n", + " {'query': '这款无线蓝牙耳机的尺寸是多少？', 'answer': \"这款无线蓝牙耳机的尺寸是1.5'' x 1.3''。\"},\n", + " {'query': '这款瑜伽垫的尺寸是多少？', 'answer': \"这款瑜伽垫的尺寸是24'' x 68''。\"}]" + ] + }, + "execution_count": 44, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "new_examples #查看用例数据" + ] + }, + { + "cell_type": "code", + "execution_count": 46, + "id": "1e6b2fe4", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "{'query': '这款全自动咖啡机的规格是什么？',\n", + " 'answer': \"大型尺寸为13.8'' x 17.3''，中型尺寸为11.5'' x 15.2''。\"}" + ] + }, + "execution_count": 46, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "new_examples[0]" + ] + }, + { + "cell_type": "code", + "execution_count": 47, + "id": "7ec72577", + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "Document(page_content=\"product_name: 全自动咖啡机\\ndescription: 规格:\\n大型 - 尺寸：13.8'' x 17.3''。\\n中型 - 尺寸：11.5'' x 15.2''。\\n\\n为什么我们热爱它:\\n这款全自动咖啡机是爱好者的理想选择。 一键操作，即可研磨豆子并沏制出您喜爱的咖啡。它的耐用性和一致性使它成为家庭和办公室的理想选择。\\n\\n材质与护理:\\n清洁时只需轻擦。\\n\\n构造:\\n由高品质不锈钢制成。\\n\\n其他特性:\\n内置研磨器和滤网。\\n预设多种咖啡模式。\\n在中国制造。\\n\\n有问题？ 请随时联系我们的客户服务团队，他们会解答您的所有问题。\", metadata={'source': 'product_data.csv', 'row': 0})" + ] + }, + "execution_count": 47, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "data[0]" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "id": "0f076f4c", + "metadata": {}, + "source": [ + "#### 6.2.1.4 组合用例数据" + ] + }, + { + "cell_type": "code", + "execution_count": 48, + "id": "c0636823", + "metadata": {}, + "outputs": [], + "source": [ + "examples += new_examples" + ] + }, + { + "cell_type": "code", + "execution_count": 49, + "id": "8d33b5de", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "\n", + "\u001b[1m> Entering new chain...\u001b[0m\n", + "\n", + "\u001b[1m> Finished chain.\u001b[0m\n" + ] + }, + { + "data": { + "text/plain": [ + "'高清电视机的护理非常简单。您只需要使用干布清洁即可。避免使用湿布或化学清洁剂，以免损坏电视机的表面。'" + ] + }, + "execution_count": 49, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "qa.run(examples[0][\"query\"])" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "id": "9737c0cc", + "metadata": {}, + "source": [ + "### 6.2.2 人工评估\n", + "现在有了这些示例，但是我们如何评估正在发生的事情呢？\n", + "通过运行一个示例通过链，并查看它产生的输出\n", + "在这里我们传递一个查询，然后我们得到一个答案。实际上正在发生的事情，进入语言模型的实际提示是什么？ \n", + "它检索的文档是什么？ \n", + "中间结果是什么？ \n", + "仅仅查看最终答案通常不足以了解链中出现了什么问题或可能出现了什么问题" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "45f60c6e", + "metadata": {}, + "outputs": [], + "source": [ + "''' \n", + "LingChainDebug工具可以了解运行一个实例通过链中间所经历的步骤\n", + "'''\n", + "import langchain\n", + "langchain.debug = True" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "3f216f9a", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\u001b[32;1m\u001b[1;3m[chain/start]\u001b[0m \u001b[1m[1:chain:RetrievalQA] Entering Chain run with input:\n", + "\u001b[0m{\n", + " \"query\": \"高清电视机怎么进行护理？\"\n", + "}\n", + "\u001b[32;1m\u001b[1;3m[chain/start]\u001b[0m \u001b[1m[1:chain:RetrievalQA > 2:chain:StuffDocumentsChain] Entering Chain run with input:\n", + "\u001b[0m[inputs]\n", + "\u001b[32;1m\u001b[1;3m[chain/start]\u001b[0m \u001b[1m[1:chain:RetrievalQA > 2:chain:StuffDocumentsChain > 3:chain:LLMChain] Entering Chain run with input:\n", + "\u001b[0m{\n", + " \"question\": \"高清电视机怎么进行护理？\",\n", + " \"context\": \"product_name: 高清电视机\\ndescription: 规格:\\n尺寸：50''。\\n\\n为什么我们热爱它:\\n我们的高清电视机拥有出色的画质和强大的音效，带来沉浸式的观看体验。\\n\\n材质与护理:\\n使用干布清洁。\\n\\n构造:\\n由塑料、金属和电子元件制成。\\n\\n其他特性:\\n支持网络连接，可以在线观看视频。\\n配备遥控器。\\n在韩国制造。\\n\\n有问题？请随时联系我们的客户服务团队，他们会解答您的所有问题。<<<<>>>>>product_name: 空气净化器\\ndescription: 规格:\\n尺寸：15'' x 15'' x 20''。\\n\\n为什么我们热爱它:\\n我们的空气净化器采用了先进的HEPA过滤技术，能有效去除空气中的微粒和异味，为您提供清新的室内环境。\\n\\n材质与护理:\\n清洁时使用干布擦拭。\\n\\n构造:\\n由塑料和电子元件制成。\\n\\n其他特性:\\n三档风速，附带定时功能。\\n在德国制造。\\n\\n有问题？请随时联系我们的客户服务团队，他们会解答您的所有问题。<<<<>>>>>product_name: 宠物自动喂食器\\ndescription: 规格:\\n尺寸：14'' x 9'' x 15''。\\n\\n为什么我们热爱它:\\n我们的宠物自动喂食器可以定时定量投放食物，让您无论在家或外出都能确保宠物的饮食。\\n\\n材质与护理:\\n可用湿布清洁。\\n\\n构造:\\n由塑料和电子元件制成。\\n\\n其他特性:\\n配备LCD屏幕，操作简单。\\n可以设置多次投食。\\n在美国制造。\\n\\n有问题？请随时联系我们的客户服务团队，他们会解答您的所有问题。<<<<>>>>>product_name: 玻璃保护膜\\ndescription: 规格:\\n适用于各种尺寸的手机屏幕。\\n\\n为什么我们热爱它:\\n我们的玻璃保护膜可以有效防止手机屏幕刮伤和破裂，而且不影响触控的灵敏度。\\n\\n材质与护理:\\n使用干布擦拭。\\n\\n构造:\\n由高强度的玻璃材料制成。\\n\\n其他特性:\\n安装简单，适合自行安装。\\n在日本制造。\\n\\n有问题？请随时联系我们的客户服务团队，他们会解答您的所有问题。\"\n", + "}\n", + "\u001b[32;1m\u001b[1;3m[llm/start]\u001b[0m \u001b[1m[1:chain:RetrievalQA > 2:chain:StuffDocumentsChain > 3:chain:LLMChain > 4:llm:ChatOpenAI] Entering LLM run with input:\n", + "\u001b[0m{\n", + " \"prompts\": [\n", + " \"System: Use the following pieces of context to answer the users question. \\nIf you don't know the answer, just say that you don't know, don't try to make up an answer.\\n----------------\\nproduct_name: 高清电视机\\ndescription: 规格:\\n尺寸：50''。\\n\\n为什么我们热爱它:\\n我们的高清电视机拥有出色的画质和强大的音效，带来沉浸式的观看体验。\\n\\n材质与护理:\\n使用干布清洁。\\n\\n构造:\\n由塑料、金属和电子元件制成。\\n\\n其他特性:\\n支持网络连接，可以在线观看视频。\\n配备遥控器。\\n在韩国制造。\\n\\n有问题？请随时联系我们的客户服务团队，他们会解答您的所有问题。<<<<>>>>>product_name: 空气净化器\\ndescription: 规格:\\n尺寸：15'' x 15'' x 20''。\\n\\n为什么我们热爱它:\\n我们的空气净化器采用了先进的HEPA过滤技术，能有效去除空气中的微粒和异味，为您提供清新的室内环境。\\n\\n材质与护理:\\n清洁时使用干布擦拭。\\n\\n构造:\\n由塑料和电子元件制成。\\n\\n其他特性:\\n三档风速，附带定时功能。\\n在德国制造。\\n\\n有问题？请随时联系我们的客户服务团队，他们会解答您的所有问题。<<<<>>>>>product_name: 宠物自动喂食器\\ndescription: 规格:\\n尺寸：14'' x 9'' x 15''。\\n\\n为什么我们热爱它:\\n我们的宠物自动喂食器可以定时定量投放食物，让您无论在家或外出都能确保宠物的饮食。\\n\\n材质与护理:\\n可用湿布清洁。\\n\\n构造:\\n由塑料和电子元件制成。\\n\\n其他特性:\\n配备LCD屏幕，操作简单。\\n可以设置多次投食。\\n在美国制造。\\n\\n有问题？请随时联系我们的客户服务团队，他们会解答您的所有问题。<<<<>>>>>product_name: 玻璃保护膜\\ndescription: 规格:\\n适用于各种尺寸的手机屏幕。\\n\\n为什么我们热爱它:\\n我们的玻璃保护膜可以有效防止手机屏幕刮伤和破裂，而且不影响触控的灵敏度。\\n\\n材质与护理:\\n使用干布擦拭。\\n\\n构造:\\n由高强度的玻璃材料制成。\\n\\n其他特性:\\n安装简单，适合自行安装。\\n在日本制造。\\n\\n有问题？请随时联系我们的客户服务团队，他们会解答您的所有问题。\\nHuman: 高清电视机怎么进行护理？\"\n", + " ]\n", + "}\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 1.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 2.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 4.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 8.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\u001b[36;1m\u001b[1;3m[llm/end]\u001b[0m \u001b[1m[1:chain:RetrievalQA > 2:chain:StuffDocumentsChain > 3:chain:LLMChain > 4:llm:ChatOpenAI] [21.02s] Exiting LLM run with output:\n", + "\u001b[0m{\n", + " \"generations\": [\n", + " [\n", + " {\n", + " \"text\": \"高清电视机的护理非常简单。您只需要使用干布清洁即可。避免使用湿布或化学清洁剂，以免损坏电视机的表面。\",\n", + " \"generation_info\": null,\n", + " \"message\": {\n", + " \"content\": \"高清电视机的护理非常简单。您只需要使用干布清洁即可。避免使用湿布或化学清洁剂，以免损坏电视机的表面。\",\n", + " \"additional_kwargs\": {},\n", + " \"example\": false\n", + " }\n", + " }\n", + " ]\n", + " ],\n", + " \"llm_output\": {\n", + " \"token_usage\": {\n", + " \"prompt_tokens\": 823,\n", + " \"completion_tokens\": 58,\n", + " \"total_tokens\": 881\n", + " },\n", + " \"model_name\": \"gpt-3.5-turbo\"\n", + " },\n", + " \"run\": null\n", + "}\n", + "\u001b[36;1m\u001b[1;3m[chain/end]\u001b[0m \u001b[1m[1:chain:RetrievalQA > 2:chain:StuffDocumentsChain > 3:chain:LLMChain] [21.02s] Exiting Chain run with output:\n", + "\u001b[0m{\n", + " \"text\": \"高清电视机的护理非常简单。您只需要使用干布清洁即可。避免使用湿布或化学清洁剂，以免损坏电视机的表面。\"\n", + "}\n", + "\u001b[36;1m\u001b[1;3m[chain/end]\u001b[0m \u001b[1m[1:chain:RetrievalQA > 2:chain:StuffDocumentsChain] [21.02s] Exiting Chain run with output:\n", + "\u001b[0m{\n", + " \"output_text\": \"高清电视机的护理非常简单。您只需要使用干布清洁即可。避免使用湿布或化学清洁剂，以免损坏电视机的表面。\"\n", + "}\n", + "\u001b[36;1m\u001b[1;3m[chain/end]\u001b[0m \u001b[1m[1:chain:RetrievalQA] [21.81s] Exiting Chain run with output:\n", + "\u001b[0m{\n", + " \"result\": \"高清电视机的护理非常简单。您只需要使用干布清洁即可。避免使用湿布或化学清洁剂，以免损坏电视机的表面。\"\n", + "}\n" + ] + }, + { + "data": { + "text/plain": [ + "'高清电视机的护理非常简单。您只需要使用干布清洁即可。避免使用湿布或化学清洁剂，以免损坏电视机的表面。'" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "qa.run(examples[0][\"query\"])#重新运行与上面相同的示例，可以看到它开始打印出更多的信息" + ] + }, + { + "cell_type": "markdown", + "id": "24536d9b", + "metadata": {}, + "source": [ + "我们可以看到它首先深入到检索QA链中，然后它进入了一些文档链。如上所述，我们正在使用stuff方法，现在我们正在传递这个上下文，可以看到，这个上下文是由我们检索到的不同文档创建的。因此，在进行问答时，当返回错误结果时，通常不是语言模型本身出错了，实际上是检索步骤出错了，仔细查看问题的确切内容和上下文可以帮助调试出错的原因。 \n", + "然后，我们可以再向下一级，看看进入语言模型的确切内容，以及 OpenAI 自身，在这里，我们可以看到传递的完整提示，我们有一个系统消息，有所使用的提示的描述，这是问题回答链使用的提示，我们可以看到提示打印出来，使用以下上下文片段回答用户的问题。\n", + "如果您不知道答案，只需说您不知道即可，不要试图编造答案。然后我们看到一堆之前插入的上下文，我们还可以看到有关实际返回类型的更多信息。我们不仅仅返回一个答案，还有token的使用情况，可以了解到token数的使用情况\n", + "\n", + "\n", + "由于这是一个相对简单的链，我们现在可以看到最终的响应，舒适的毛衣套装，条纹款，有侧袋，正在起泡，通过链返回给用户，我们刚刚讲解了如何查看和调试单个输入到该链的情况。\n", + "\n", + "\n" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "id": "9f3ebc93", + "metadata": {}, + "source": [ + "#### 6.2.2.1 如何评估新创建的实例\n", + "与创建它们类似，可以运行链条来处理所有示例，然后查看输出并尝试弄清楚，发生了什么，它是否正确" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a32fbb61", + "metadata": {}, + "outputs": [], + "source": [ + "# 我们需要为所有示例创建预测，关闭调试模式，以便不将所有内容打印到屏幕上\n", + "langchain.debug = False" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "id": "dd57860a", + "metadata": {}, + "source": [ + "### 6.2.3 通过LLM进行评估实例" + ] + }, + { + "cell_type": "code", + "execution_count": 50, + "id": "40d908c9", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "\n", + "\u001b[1m> Entering new chain...\u001b[0m\n", + "\n", + "\u001b[1m> Finished chain.\u001b[0m\n", + "\n", + "\n", + "\u001b[1m> Entering new chain...\u001b[0m\n", + "\n", + "\u001b[1m> Finished chain.\u001b[0m\n", + "\n", + "\n", + "\u001b[1m> Entering new chain...\u001b[0m\n", + "\n", + "\u001b[1m> Finished chain.\u001b[0m\n", + "\n", + "\n", + "\u001b[1m> Entering new chain...\u001b[0m\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 1.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "\u001b[1m> Finished chain.\u001b[0m\n", + "\n", + "\n", + "\u001b[1m> Entering new chain...\u001b[0m\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 1.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 2.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 4.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 8.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "\u001b[1m> Finished chain.\u001b[0m\n", + "\n", + "\n", + "\u001b[1m> Entering new chain...\u001b[0m\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 1.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 2.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 4.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 8.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "\u001b[1m> Finished chain.\u001b[0m\n", + "\n", + "\n", + "\u001b[1m> Entering new chain...\u001b[0m\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 1.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 2.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 4.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 8.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "\u001b[1m> Finished chain.\u001b[0m\n", + "\n", + "\n", + "\u001b[1m> Entering new chain...\u001b[0m\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 1.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 2.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 4.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 8.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 16.0 seconds as it raised APIError: Bad gateway. {\"error\":{\"code\":502,\"message\":\"Bad gateway.\",\"param\":null,\"type\":\"cf_bad_gateway\"}} 502 {'error': {'code': 502, 'message': 'Bad gateway.', 'param': None, 'type': 'cf_bad_gateway'}} {'Date': 'Thu, 29 Jun 2023 02:20:57 GMT', 'Content-Type': 'application/json', 'Content-Length': '84', 'Connection': 'keep-alive', 'X-Frame-Options': 'SAMEORIGIN', 'Referrer-Policy': 'same-origin', 'Cache-Control': 'private, max-age=0, no-store, no-cache, must-revalidate, post-check=0, pre-check=0', 'Expires': 'Thu, 01 Jan 1970 00:00:01 GMT', 'Server': 'cloudflare', 'CF-RAY': '7deaa9d4cd2640da-SIN', 'alt-svc': 'h3=\":443\"; ma=86400'}.\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "\u001b[1m> Finished chain.\u001b[0m\n", + "\n", + "\n", + "\u001b[1m> Entering new chain...\u001b[0m\n", + "\n", + "\u001b[1m> Finished chain.\u001b[0m\n", + "\n", + "\n", + "\u001b[1m> Entering new chain...\u001b[0m\n", + "\n", + "\u001b[1m> Finished chain.\u001b[0m\n", + "\n", + "\n", + "\u001b[1m> Entering new chain...\u001b[0m\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 1.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 2.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 4.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 8.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "\u001b[1m> Finished chain.\u001b[0m\n", + "\n", + "\n", + "\u001b[1m> Entering new chain...\u001b[0m\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 1.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 2.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 4.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 8.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "\u001b[1m> Finished chain.\u001b[0m\n" + ] + } + ], + "source": [ + "predictions = qa.apply(examples) #为所有不同的示例创建预测" + ] + }, + { + "cell_type": "code", + "execution_count": 51, + "id": "c71d3d2f", + "metadata": {}, + "outputs": [], + "source": [ + "''' \n", + "对预测的结果进行评估，导入QA问题回答，评估链，通过语言模型创建此链\n", + "'''\n", + "from langchain.evaluation.qa import QAEvalChain #导入QA问题回答，评估链" + ] + }, + { + "cell_type": "code", + "execution_count": 52, + "id": "ac1c848f", + "metadata": {}, + "outputs": [], + "source": [ + "#通过调用chatGPT进行评估\n", + "llm = ChatOpenAI(temperature=0)\n", + "eval_chain = QAEvalChain.from_llm(llm)" + ] + }, + { + "cell_type": "code", + "execution_count": 53, + "id": "50c1250a", + "metadata": {}, + "outputs": [ + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 1.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 2.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 4.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 8.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 16.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 1.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 2.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 1.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 2.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 4.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 8.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 16.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 1.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 2.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 1.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 2.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 4.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 8.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 16.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 1.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 2.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 4.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 1.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 2.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 4.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 8.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 1.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 2.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 4.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 8.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 16.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 1.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 2.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 4.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n" + ] + } + ], + "source": [ + "graded_outputs = eval_chain.evaluate(examples, predictions)#在此链上调用evaluate，进行评估" + ] + }, + { + "cell_type": "markdown", + "id": "72dc8595", + "metadata": {}, + "source": [ + "##### 6.2.3.1 评估思路\n", + "当它面前有整个文档时，它可以生成一个真实的答案，我们将打印出预测的答，当它进行QA链时，使用embedding和向量数据库进行检索时，将其传递到语言模型中，然后尝试猜测预测的答案，我们还将打印出成绩，这也是语言模型生成的。当它要求评估链评估正在发生的事情时，以及它是否正确或不正确。因此，当我们循环遍历所有这些示例并将它们打印出来时，可以详细了解每个示例" + ] + }, + { + "cell_type": "code", + "execution_count": 54, + "id": "bf21e40a", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Example 0:\n", + "Question: 高清电视机怎么进行护理？\n", + "Real Answer: 使用干布清洁。\n", + "Predicted Answer: 高清电视机的护理非常简单。您只需要使用干布清洁即可。避免使用湿布或化学清洁剂，以免损坏电视机的表面。\n", + "Predicted Grade: CORRECT\n", + "\n", + "Example 1:\n", + "Question: 旅行背包有内外袋吗？\n", + "Real Answer: 有。\n", + "Predicted Answer: 是的，旅行背包有多个实用的内外袋，可以轻松装下您的必需品。\n", + "Predicted Grade: CORRECT\n", + "\n", + "Example 2:\n", + "Question: 这款全自动咖啡机有什么特点和优势？\n", + "Real Answer: 这款全自动咖啡机的特点和优势包括一键操作、内置研磨器和滤网、预设多种咖啡模式、耐用性和一致性，以及由高品质不锈钢制成的构造。\n", + "Predicted Answer: 这款全自动咖啡机有以下特点和优势：\n", + "1. 一键操作：只需按下按钮，即可研磨咖啡豆并沏制出您喜爱的咖啡，非常方便。\n", + "2. 耐用性和一致性：这款咖啡机具有耐用性和一致性，使其成为家庭和办公室的理想选择。\n", + "3. 内置研磨器和滤网：咖啡机内置研磨器和滤网，可以确保咖啡的新鲜和口感。\n", + "4. 多种咖啡模式：咖啡机预设了多种咖啡模式，可以根据个人口味选择不同的咖啡。\n", + "5. 高品质材料：咖啡机由高品质不锈钢制成，具有优良的耐用性和质感。\n", + "6. 中国制造：这款咖啡机是在中国制造的，具有可靠的品质保证。\n", + "Predicted Grade: CORRECT\n", + "\n", + "Example 3:\n", + "Question: 这款电动牙刷的规格是什么？\n", + "Real Answer: 这款电动牙刷的规格是一般大小，高度为9.5英寸，宽度为1英寸。\n", + "Predicted Answer: 这款电动牙刷的规格是：高度为9.5英寸，宽度为1英寸。\n", + "Predicted Grade: CORRECT\n", + "\n", + "Example 4:\n", + "Question: 这款橙味维生素C泡腾片的规格是什么？\n", + "Real Answer: 这款橙味维生素C泡腾片每盒含有20片。\n", + "Predicted Answer: 这款橙味维生素C泡腾片的规格是每盒含有20片。\n", + "Predicted Grade: CORRECT\n", + "\n", + "Example 5:\n", + "Question: 这款无线蓝牙耳机的规格是什么？\n", + "Real Answer: 单个耳机尺寸为1.5'' x 1.3''。\n", + "Predicted Answer: 这款无线蓝牙耳机的规格是单个耳机尺寸为1.5'' x 1.3''。\n", + "Predicted Grade: CORRECT\n", + "\n", + "Example 6:\n", + "Question: 这个产品的名称是什么？\n", + "Real Answer: 瑜伽垫\n", + "Predicted Answer: 这个产品的名称是儿童益智玩具。\n", + "Predicted Grade: INCORRECT\n", + "\n", + "Example 7:\n", + "Question: 这款全自动咖啡机的规格是什么？\n", + "Real Answer: 大型尺寸为13.8'' x 17.3''，中型尺寸为11.5'' x 15.2''。\n", + "Predicted Answer: 这款全自动咖啡机有两种规格：\n", + "- 大型尺寸为13.8'' x 17.3''。\n", + "- 中型尺寸为11.5'' x 15.2''。\n", + "Predicted Grade: CORRECT\n", + "\n", + "Example 8:\n", + "Question: 这款电动牙刷的高度和宽度分别是多少？\n", + "Real Answer: 这款电动牙刷的高度是9.5英寸，宽度是1英寸。\n", + "Predicted Answer: 这款电动牙刷的高度是9.5英寸，宽度是1英寸。\n", + "Predicted Grade: CORRECT\n", + "\n", + "Example 9:\n", + "Question: 这款橙味维生素C泡腾片的规格是什么？\n", + "Real Answer: 每盒含有20片。\n", + "Predicted Answer: 这款橙味维生素C泡腾片的规格是每盒含有20片。\n", + "Predicted Grade: CORRECT\n", + "\n", + "Example 10:\n", + "Question: 这款无线蓝牙耳机的尺寸是多少？\n", + "Real Answer: 这款无线蓝牙耳机的尺寸是1.5'' x 1.3''。\n", + "Predicted Answer: 这款无线蓝牙耳机的尺寸是1.5'' x 1.3''。\n", + "Predicted Grade: CORRECT\n", + "\n", + "Example 11:\n", + "Question: 这款瑜伽垫的尺寸是多少？\n", + "Real Answer: 这款瑜伽垫的尺寸是24'' x 68''。\n", + "Predicted Answer: 这款瑜伽垫的尺寸是24'' x 68''。\n", + "Predicted Grade: CORRECT\n", + "\n" + ] + } + ], + "source": [ + "#我们将传入示例和预测，得到一堆分级输出，循环遍历它们打印答案\n", + "for i, eg in enumerate(examples):\n", + " print(f\"Example {i}:\")\n", + " print(\"Question: \" + predictions[i]['query'])\n", + " print(\"Real Answer: \" + predictions[i]['answer'])\n", + " print(\"Predicted Answer: \" + predictions[i]['result'])\n", + " print(\"Predicted Grade: \" + graded_outputs[i]['text'])\n", + " print()" + ] + }, + { + "attachments": {}, + "cell_type": "markdown", + "id": "ece7c7b4", + "metadata": {}, + "source": [ + "#### 6.2.3.2 结果分析\n", + "对于每个示例，它看起来都是正确的，让我们看看第一个例子。\n", + "这里的问题是，旅行背包有内外袋吗？真正的答案，我们创建了这个，是肯定的。模型预测的答案是是的，旅行背包有多个实用的内外袋，可以轻松装下您的必需品。因此，我们可以理解这是一个正确的答案。它将其评为正确。 \n", + "#### 6.2.3.3 使用模型评估的优势\n", + "\n", + "你有这些答案，它们是任意的字符串。没有单一的真实字符串是最好的可能答案，有许多不同的变体，只要它们具有相同的语义，它们应该被评为相似。如果使用正则进行精准匹配就会丢失语义信息，到目前为止存在的许多评估指标都不够好。目前最有趣和最受欢迎的之一就是使用语言模型进行评估。" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "314095a6", + "metadata": {}, + "outputs": [], + "source": [] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "cafb368e", + "metadata": {}, + "outputs": [], + "source": [] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a9e546e4", "metadata": {}, "outputs": [], "source": [] diff --git a/content/LangChain for LLM Application Development/7.代理.ipynb b/content/LangChain for LLM Application Development/7.代理.ipynb index c54fa3a..d746699 100644 --- a/content/LangChain for LLM Application Development/7.代理.ipynb +++ b/content/LangChain for LLM Application Development/7.代理.ipynb @@ -1,6 +1,7 @@ { "cells": [ { + "attachments": {}, "cell_type": "markdown", "id": "2caa79ba-45e3-437c-9cb6-e433f443f0bf", "metadata": {}, @@ -47,6 +48,7 @@ ] }, { + "attachments": {}, "cell_type": "markdown", "id": "631c764b-68fa-483a-80a5-9a322cd1117c", "metadata": {}, @@ -56,7 +58,7 @@ }, { "cell_type": "code", - "execution_count": 3, + "execution_count": 2, "id": "8ab03352-1554-474c-afda-ec8974250e31", "metadata": {}, "outputs": [], @@ -69,7 +71,7 @@ }, { "cell_type": "code", - "execution_count": 5, + "execution_count": 3, "id": "bc80a8a7-a66e-4fbb-8d57-05004a3e77c7", "metadata": {}, "outputs": [], @@ -81,6 +83,7 @@ ] }, { + "attachments": {}, "cell_type": "markdown", "id": "d5a3655c-4d5e-4a86-8bf4-a11bd1525059", "metadata": {}, @@ -89,6 +92,7 @@ ] }, { + "attachments": {}, "cell_type": "markdown", "id": "e484d285-2110-4a06-a360-55313d6d9ffc", "metadata": {}, @@ -102,7 +106,7 @@ }, { "cell_type": "code", - "execution_count": 24, + "execution_count": 4, "id": "71e5894f-fc50-478b-a3d3-58891cc46ffc", "metadata": {}, "outputs": [], @@ -112,6 +116,7 @@ ] }, { + "attachments": {}, "cell_type": "markdown", "id": "d7f75023-1825-4d74-bc8e-2c362f551fd1", "metadata": { @@ -125,7 +130,7 @@ }, { "cell_type": "code", - "execution_count": 25, + "execution_count": 5, "id": "3541b866-8a35-4b6f-bfe6-66edb1d666cd", "metadata": {}, "outputs": [], @@ -137,6 +142,7 @@ ] }, { + "attachments": {}, "cell_type": "markdown", "id": "e5b4fcc8-8817-4a94-b154-3f328480e441", "metadata": {}, @@ -150,7 +156,7 @@ }, { "cell_type": "code", - "execution_count": 26, + "execution_count": 6, "id": "e7cdc782-2b8c-4649-ae1e-cabacee1a757", "metadata": {}, "outputs": [], @@ -165,6 +171,7 @@ ] }, { + "attachments": {}, "cell_type": "markdown", "id": "b980137a-a1d2-4c19-80c8-ec380f9c1efe", "metadata": { @@ -176,7 +183,7 @@ }, { "cell_type": "code", - "execution_count": 9, + "execution_count": 7, "id": "1e2bcd3b-ae72-4fa1-b38e-d0ce650e1f31", "metadata": {}, "outputs": [ @@ -186,22 +193,26 @@ "text": [ "\n", "\n", - "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n", - "\u001b[32;1m\u001b[1;3mThought: We need to calculate 25% of 300, which involves multiplication and division.\n", - "\n", + "\u001b[1m> Entering new chain...\u001b[0m\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\u001b[32;1m\u001b[1;3mQuestion: 计算300的25%\n", + "Thought: 我可以使用计算器来计算这个百分比\n", "Action:\n", "```\n", "{\n", " \"action\": \"Calculator\",\n", - " \"action_input\": \"300*0.25\"\n", + " \"action_input\": \"300 * 25 / 100\"\n", "}\n", "```\n", - "\n", "\u001b[0m\n", "Observation: \u001b[36;1m\u001b[1;3mAnswer: 75.0\u001b[0m\n", - "Thought:\u001b[32;1m\u001b[1;3mWe have the answer to the question.\n", - "\n", - "Final Answer: 75.0\u001b[0m\n", + "Thought:\u001b[32;1m\u001b[1;3m我现在知道最终答案了\n", + "Final Answer: 300的25%是75.0\u001b[0m\n", "\n", "\u001b[1m> Finished chain.\u001b[0m\n" ] @@ -209,19 +220,20 @@ { "data": { "text/plain": [ - "{'input': 'What is the 25% of 300?', 'output': '75.0'}" + "{'input': '计算300的25%，请用中文', 'output': '300的25%是75.0'}" ] }, - "execution_count": 9, + "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ - "agent(\"What is the 25% of 300?\")" + "agent(\"计算300的25%，思考过程请使用中文。\") " ] }, { + "attachments": {}, "cell_type": "markdown", "id": "05bb0811-d71e-4016-868b-efbb651d8e59", "metadata": {}, @@ -246,6 +258,7 @@ ] }, { + "attachments": {}, "cell_type": "markdown", "id": "bdd13d06-0f0e-4918-ac28-339f524f8c76", "metadata": {}, @@ -255,7 +268,7 @@ }, { "cell_type": "code", - "execution_count": 29, + "execution_count": 18, "id": "f43b5cbf-1011-491b-90e8-09cf903fb866", "metadata": {}, "outputs": [ @@ -265,28 +278,272 @@ "text": [ "\n", "\n", - "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n", - "\u001b[32;1m\u001b[1;3mThought: I should use Wikipedia to find the answer to this question.\n", - "\n", - "Action:\n", - "```\n", - "{\n", - " \"action\": \"Wikipedia\",\n", - " \"action_input\": \"Tom M. Mitchell\"\n", - "}\n", - "```\n", - "\n", - "\u001b[0m\n", - "Observation: \u001b[33;1m\u001b[1;3mPage: Tom M. Mitchell\n", - "Summary: Tom Michael Mitchell (born August 9, 1951) is an American computer scientist and the Founders University Professor at Carnegie Mellon University (CMU). He is a founder and former Chair of the Machine Learning Department at CMU. Mitchell is known for his contributions to the advancement of machine learning, artificial intelligence, and cognitive neuroscience and is the author of the textbook Machine Learning. He is a member of the United States National Academy of Engineering since 2010. He is also a Fellow of the American Academy of Arts and Sciences, the American Association for the Advancement of Science and a Fellow and past President of the Association for the Advancement of Artificial Intelligence. In October 2018, Mitchell was appointed as the Interim Dean of the School of Computer Science at Carnegie Mellon.\n", - "\n", - "Page: Tom Mitchell (Australian footballer)\n", - "Summary: Thomas Mitchell (born 31 May 1993) is a professional Australian rules footballer playing for the Collingwood Football Club in the Australian Football League (AFL). He previously played for the Sydney Swans from 2012 to 2016, and the Hawthorn Football Club between 2017 and 2022. Mitchell won the Brownlow Medal as the league's best and fairest player in 2018 and set the record for the most disposals in a VFL/AFL match, accruing 54 in a game against Collingwood during that season.\n", - "\n", - "\u001b[0m\n", - "Thought:\u001b[32;1m\u001b[1;3mThe book that Tom M. Mitchell wrote is \"Machine Learning\".\n", - "\n", - "Final Answer: Machine Learning.\u001b[0m\n", + "\u001b[1m> Entering new chain...\u001b[0m\n", + "\u001b[32;1m\u001b[1;3mI need to find out what book Tom M. Mitchell wrote.\n", + "Action: Search for Tom M. Mitchell's books.\n", + "Action Input: \"Tom M. Mitchell books\"\u001b[0m\n", + "Observation: Search for Tom M. Mitchell's books. is not a valid tool, try another one.\n", + "Thought:\u001b[32;1m\u001b[1;3mI can try searching for Tom M. Mitchell's books using a search engine.\n", + "Action: Search for \"Tom M. Mitchell books\"\n", + "Action Input: \"Tom M. Mitchell books\"\u001b[0m\n", + "Observation: Search for \"Tom M. Mitchell books\" is not a valid tool, try another one.\n", + "Thought:\u001b[32;1m\u001b[1;3mI can try searching for Tom M. Mitchell's books using a search engine.\n", + "Action: Search for \"Tom M. Mitchell books\"\n", + "Action Input: \"Tom M. Mitchell books\"\u001b[0m\n", + "Observation: Search for \"Tom M. Mitchell books\" is not a valid tool, try another one.\n", + "Thought:" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 1.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 2.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 4.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 8.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\u001b[32;1m\u001b[1;3mI can try searching for Tom M. Mitchell's books using a search engine.\n", + "Action: Search for \"Tom M. Mitchell books\"\n", + "Action Input: \"Tom M. Mitchell books\"\u001b[0m\n", + "Observation: Search for \"Tom M. Mitchell books\" is not a valid tool, try another one.\n", + "Thought:" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 1.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 2.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 4.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 8.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\u001b[32;1m\u001b[1;3mI can try searching for Tom M. Mitchell's books using a search engine.\n", + "Action: Search for \"Tom M. Mitchell books\"\n", + "Action Input: \"Tom M. Mitchell books\"\u001b[0m\n", + "Observation: Search for \"Tom M. Mitchell books\" is not a valid tool, try another one.\n", + "Thought:" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 1.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 2.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 4.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 8.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\u001b[32;1m\u001b[1;3mI can try searching for Tom M. Mitchell's books using a search engine.\n", + "Action: Search for \"Tom M. Mitchell books\"\n", + "Action Input: \"Tom M. Mitchell books\"\u001b[0m\n", + "Observation: Search for \"Tom M. Mitchell books\" is not a valid tool, try another one.\n", + "Thought:" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 1.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 2.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 4.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 8.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\u001b[32;1m\u001b[1;3mI can try searching for Tom M. Mitchell's books using a search engine.\n", + "Action: Search for \"Tom M. Mitchell books\"\n", + "Action Input: \"Tom M. Mitchell books\"\u001b[0m\n", + "Observation: Search for \"Tom M. Mitchell books\" is not a valid tool, try another one.\n", + "Thought:" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 1.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 2.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 4.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 8.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\u001b[32;1m\u001b[1;3mI can try searching for Tom M. Mitchell's books using a search engine.\n", + "Action: Search for \"Tom M. Mitchell books\"\n", + "Action Input: \"Tom M. Mitchell books\"\u001b[0m\n", + "Observation: Search for \"Tom M. Mitchell books\" is not a valid tool, try another one.\n", + "Thought:" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 1.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 2.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 4.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 8.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\u001b[32;1m\u001b[1;3mI can try searching for Tom M. Mitchell's books using a search engine.\n", + "Action: Search for \"Tom M. Mitchell books\"\n", + "Action Input: \"Tom M. Mitchell books\"\u001b[0m\n", + "Observation: Search for \"Tom M. Mitchell books\" is not a valid tool, try another one.\n", + "Thought:" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 1.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 2.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 4.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 8.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\u001b[32;1m\u001b[1;3mI can try searching for Tom M. Mitchell's books using a search engine.\n", + "Action: Search for \"Tom M. Mitchell books\"\n", + "Action Input: \"Tom M. Mitchell books\"\u001b[0m\n", + "Observation: Search for \"Tom M. Mitchell books\" is not a valid tool, try another one.\n", + "Thought:" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 1.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 2.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 4.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 8.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\u001b[32;1m\u001b[1;3mI can try searching for Tom M. Mitchell's books using a search engine.\n", + "Action: Search for \"Tom M. Mitchell books\"\n", + "Action Input: \"Tom M. Mitchell books\"\u001b[0m\n", + "Observation: Search for \"Tom M. Mitchell books\" is not a valid tool, try another one.\n", + "Thought:" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 1.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 2.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 4.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 8.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\u001b[32;1m\u001b[1;3mI can try searching for Tom M. Mitchell's books using a search engine.\n", + "Action: Search for \"Tom M. Mitchell books\"\n", + "Action Input: \"Tom M. Mitchell books\"\u001b[0m\n", + "Observation: Search for \"Tom M. Mitchell books\" is not a valid tool, try another one.\n", + "Thought:" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 1.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 2.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 4.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 8.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\u001b[32;1m\u001b[1;3mI can try searching for Tom M. Mitchell's books using a search engine.\n", + "Action: Search for \"Tom M. Mitchell books\"\n", + "Action Input: \"Tom M. Mitchell books\"\u001b[0m\n", + "Observation: Search for \"Tom M. Mitchell books\" is not a valid tool, try another one.\n", + "Thought:" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 1.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 2.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 4.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 8.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\u001b[32;1m\u001b[1;3mI can try searching for Tom M. Mitchell's books using a search engine.\n", + "Action: Search for \"Tom M. Mitchell books\"\n", + "Action Input: \"Tom M. Mitchell books\"\u001b[0m\n", + "Observation: Search for \"Tom M. Mitchell books\" is not a valid tool, try another one.\n", + "Thought:" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 1.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 2.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 4.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 8.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\u001b[32;1m\u001b[1;3mI can try searching for Tom M. Mitchell's books using a search engine.\n", + "Action: Search for \"Tom M. Mitchell books\"\n", + "Action Input: \"Tom M. Mitchell books\"\u001b[0m\n", + "Observation: Search for \"Tom M. Mitchell books\" is not a valid tool, try another one.\n", + "Thought:\u001b[32;1m\u001b[1;3m\u001b[0m\n", "\n", "\u001b[1m> Finished chain.\u001b[0m\n" ] @@ -295,10 +552,10 @@ "data": { "text/plain": [ "{'input': 'Tom M. Mitchell is an American computer scientist and the Founders University Professor at Carnegie Mellon University (CMU)what book did he write?',\n", - " 'output': 'Machine Learning.'}" + " 'output': 'Agent stopped due to iteration limit or time limit.'}" ] }, - "execution_count": 29, + "execution_count": 18, "metadata": {}, "output_type": "execute_result" } @@ -311,6 +568,7 @@ ] }, { + "attachments": {}, "cell_type": "markdown", "id": "39e2f6fb-f229-4235-ad58-98a5f505800c", "metadata": {}, @@ -337,6 +595,7 @@ ] }, { + "attachments": {}, "cell_type": "markdown", "id": "5901cab6-a7c9-4590-b35d-d41c29e39a39", "metadata": {}, @@ -345,6 +604,7 @@ ] }, { + "attachments": {}, "cell_type": "markdown", "id": "12b8d837-524c-46b8-9189-504808cf1f93", "metadata": {}, @@ -354,7 +614,7 @@ }, { "cell_type": "code", - "execution_count": 31, + "execution_count": 8, "id": "6c60d933-168d-4834-b29c-b2b43ade80cc", "metadata": {}, "outputs": [], @@ -370,6 +630,7 @@ ] }, { + "attachments": {}, "cell_type": "markdown", "id": "d32ed40c-cbcd-4efd-b044-57e611f5fab5", "metadata": { @@ -381,24 +642,27 @@ }, { "cell_type": "code", - "execution_count": 32, + "execution_count": 14, "id": "3d416f28-8fdb-4761-a4f9-d46c6d37fb3d", "metadata": {}, "outputs": [], "source": [ - "customer_list = [[\"Harrison\", \"Chase\"], \n", - " [\"Lang\", \"Chain\"],\n", - " [\"Dolly\", \"Too\"],\n", - " [\"Elle\", \"Elem\"], \n", - " [\"Geoff\",\"Fusion\"], \n", - " [\"Trance\",\"Former\"],\n", - " [\"Jen\",\"Ayai\"]\n", - " ]" + "customer_list = [\"小明\",\"小黄\",\"小红\",\"小蓝\",\"小橘\",\"小绿\",]" ] }, { "cell_type": "code", - "execution_count": 33, + "execution_count": 22, + "id": "d0fbfc4c", + "metadata": {}, + "outputs": [], + "source": [ + "!pip install -q pinyin" + ] + }, + { + "cell_type": "code", + "execution_count": 23, "id": "ba7f9a50-acf6-4fba-a1b9-503baa80266b", "metadata": {}, "outputs": [ @@ -408,26 +672,44 @@ "text": [ "\n", "\n", - "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n", - "\u001b[32;1m\u001b[1;3mI can use the sorted() function to sort the list of customers by last name and then first name. I will need to provide a key function to sorted() that returns a tuple of the last name and first name in that order.\n", - "Action: Python REPL\n", - "Action Input:\n", - "```\n", - "customers = [['Harrison', 'Chase'], ['Lang', 'Chain'], ['Dolly', 'Too'], ['Elle', 'Elem'], ['Geoff', 'Fusion'], ['Trance', 'Former'], ['Jen', 'Ayai']]\n", - "sorted_customers = sorted(customers, key=lambda x: (x[1], x[0]))\n", - "for customer in sorted_customers:\n", - " print(customer)\n", - "```\u001b[0m\n", - "Observation: \u001b[36;1m\u001b[1;3m['Jen', 'Ayai']\n", - "['Lang', 'Chain']\n", - "['Harrison', 'Chase']\n", - "['Elle', 'Elem']\n", - "['Trance', 'Former']\n", - "['Geoff', 'Fusion']\n", - "['Dolly', 'Too']\n", - "\u001b[0m\n", - "Thought:\u001b[32;1m\u001b[1;3mThe customers are now sorted by last name and then first name. \n", - "Final Answer: [['Jen', 'Ayai'], ['Lang', 'Chain'], ['Harrison', 'Chase'], ['Elle', 'Elem'], ['Trance', 'Former'], ['Geoff', 'Fusion'], ['Dolly', 'Too']]\u001b[0m\n", + "\u001b[1m> Entering new chain...\u001b[0m\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\u001b[32;1m\u001b[1;3m我需要使用拼音库来将客户名字转换为拼音。我可以使用Python的pinyin库来实现这个功能。\n", + "Action: Python_REPL\n", + "Action Input: import pinyin\u001b[0m\n", + "Observation: \u001b[36;1m\u001b[1;3m\u001b[0m\n", + "Thought:\u001b[32;1m\u001b[1;3m我现在可以使用pinyin库来将客户名字转换为拼音了。\n", + "Action: Python_REPL\n", + "Action Input: names = ['小明', '小黄', '小红', '小蓝', '小橘', '小绿']\u001b[0m\n", + "Observation: \u001b[36;1m\u001b[1;3m\u001b[0m\n", + "Thought:\u001b[32;1m\u001b[1;3m我现在可以使用pinyin库将names列表中的客户名字转换为拼音了。\n", + "Action: Python_REPL\n", + "Action Input: pinyin_names = [pinyin.get(i, format='strip', delimiter='') for i in names]\u001b[0m\n", + "Observation: \u001b[36;1m\u001b[1;3m\u001b[0m\n", + "Thought:" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 1.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 2.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 4.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 8.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\u001b[32;1m\u001b[1;3m我已经成功将客户名字转换为拼音并存储在pinyin_names列表中了。\n", + "Final Answer: pinyin_names = ['xiaoming', 'xiaohuang', 'xiaohong', 'xiaolan', 'xiaojv', 'xiaolv']\u001b[0m\n", "\n", "\u001b[1m> Finished chain.\u001b[0m\n" ] @@ -435,66 +717,22 @@ { "data": { "text/plain": [ - "\"[['Jen', 'Ayai'], ['Lang', 'Chain'], ['Harrison', 'Chase'], ['Elle', 'Elem'], ['Trance', 'Former'], ['Geoff', 'Fusion'], ['Dolly', 'Too']]\"" + "\"pinyin_names = ['xiaoming', 'xiaohuang', 'xiaohong', 'xiaolan', 'xiaojv', 'xiaolv']\"" ] }, - "execution_count": 33, + "execution_count": 23, "metadata": {}, "output_type": "execute_result" } ], "source": [ - "agent.run(f\"\"\"Sort these customers by \\\n", - "last name and then first name \\\n", - "and print the output: {customer_list}\"\"\") " - ] - }, - { - "cell_type": "markdown", - "id": "c6c6fd4b-1e9f-4f37-9a0f-3f69cc121ab0", - "metadata": { - "tags": [] - }, - "source": [ - "✅ **总结**\n", - "\n", - "1. 模型对于接下来需要做什么，给出思考（Thought） \n", - "

思考：我可以使用`sorted()`函数按姓氏和名字的顺序对客户列表进行排序。我需要为`sorted()`提供一个键函数，该函数返回一个元组，元组中按顺序包含姓氏和名字。

\n", - "\n", - "2. 模型基于思考采取行动（Action), 因为使用的工具不同，Action的输出也和之前有所不同，这里输出的为python代码\n", - "

行动: 使用Python编程，输入以下代码

\n", - " \n", - " \n", - " ```python\n", - " customers = [['Harrison', 'Chase'], ['Lang', 'Chain'], ['Dolly', 'Too'], ['Elle', 'Elem'], ['Geoff', 'Fusion'], ['Trance', 'Former'], ['Jen', 'Ayai']]\n", - " sorted_customers = sorted(customers, key=lambda x: (x[1], x[0]))\n", - " for customer in sorted_customers:\n", - " print(customer)\n", - " ```\n", - "
\n", - "3. 模型得到观察（Observation）\n", - "

观测:

\n", - " \n", - " ```\n", - " ['Jen', 'Ayai']\n", - " ['Lang', 'Chain']\n", - " ['Harrison', 'Chase']\n", - " ['Elle', 'Elem']\n", - " ['Trance', 'Former']\n", - " ['Geoff', 'Fusion']\n", - " ['Dolly', 'Too']\n", - " ```\n", - "
\n", - " \n", - "4. 基于观察，模型对于接下来需要做什么，给出思考（Thought）\n", - "

思考: 现在客户按照姓氏和名字的顺序进行了排序。

\n", - "\n", - "5. 给出最终答案（Final Answer）\n", - "

最终答案: [['Jen', 'Ayai'], ['Lang', 'Chain'], ['Harrison', 'Chase'], ['Elle', 'Elem'], ['Trance', 'Former'], ['Geoff', 'Fusion'], ['Dolly', 'Too']]

\n", - "6. 返回最终答案。" + "agent.run(f\"\"\"在这些客户名字转换为拼音\\\n", + "并打印输出列表： {customer_list}\\\n", + "思考过程请使用中文。\"\"\") " ] }, { + "attachments": {}, "cell_type": "markdown", "id": "16f530a5-8c50-4648-951f-4e65d15ca93f", "metadata": { @@ -506,6 +744,7 @@ ] }, { + "attachments": {}, "cell_type": "markdown", "id": "951048dc-0414-410d-a0e9-6ef32bbdda89", "metadata": { @@ -536,7 +775,7 @@ }, { "cell_type": "code", - "execution_count": 36, + "execution_count": 25, "id": "92b8961b-b990-4540-a7f9-e4701dfaa616", "metadata": {}, "outputs": [ @@ -546,11 +785,11 @@ "text": [ "\u001b[32;1m\u001b[1;3m[chain/start]\u001b[0m \u001b[1m[1:chain:AgentExecutor] Entering Chain run with input:\n", "\u001b[0m{\n", - " \"input\": \"Sort these customers by last name and then first name and print the output: [['Harrison', 'Chase'], ['Lang', 'Chain'], ['Dolly', 'Too'], ['Elle', 'Elem'], ['Geoff', 'Fusion'], ['Trance', 'Former'], ['Jen', 'Ayai']]\"\n", + " \"input\": \"在这些客户名字转换为拼音并打印输出列表： ['小明', '小黄', '小红', '小蓝', '小橘', '小绿']思考过程请使用中文。\"\n", "}\n", "\u001b[32;1m\u001b[1;3m[chain/start]\u001b[0m \u001b[1m[1:chain:AgentExecutor > 2:chain:LLMChain] Entering Chain run with input:\n", "\u001b[0m{\n", - " \"input\": \"Sort these customers by last name and then first name and print the output: [['Harrison', 'Chase'], ['Lang', 'Chain'], ['Dolly', 'Too'], ['Elle', 'Elem'], ['Geoff', 'Fusion'], ['Trance', 'Former'], ['Jen', 'Ayai']]\",\n", + " \"input\": \"在这些客户名字转换为拼音并打印输出列表： ['小明', '小黄', '小红', '小蓝', '小橘', '小绿']思考过程请使用中文。\",\n", " \"agent_scratchpad\": \"\",\n", " \"stop\": [\n", " \"\\nObservation:\",\n", @@ -560,18 +799,18 @@ "\u001b[32;1m\u001b[1;3m[llm/start]\u001b[0m \u001b[1m[1:chain:AgentExecutor > 2:chain:LLMChain > 3:llm:ChatOpenAI] Entering LLM run with input:\n", "\u001b[0m{\n", " \"prompts\": [\n", - " \"Human: You are an agent designed to write and execute python code to answer questions.\\nYou have access to a python REPL, which you can use to execute python code.\\nIf you get an error, debug your code and try again.\\nOnly use the output of your code to answer the question. \\nYou might know the answer without running any code, but you should still run the code to get the answer.\\nIf it does not seem like you can write code to answer the question, just return \\\"I don't know\\\" as the answer.\\n\\n\\nPython REPL: A Python shell. Use this to execute python commands. Input should be a valid python command. If you want to see the output of a value, you should print it out with `print(...)`.\\n\\nUse the following format:\\n\\nQuestion: the input question you must answer\\nThought: you should always think about what to do\\nAction: the action to take, should be one of [Python REPL]\\nAction Input: the input to the action\\nObservation: the result of the action\\n... (this Thought/Action/Action Input/Observation can repeat N times)\\nThought: I now know the final answer\\nFinal Answer: the final answer to the original input question\\n\\nBegin!\\n\\nQuestion: Sort these customers by last name and then first name and print the output: [['Harrison', 'Chase'], ['Lang', 'Chain'], ['Dolly', 'Too'], ['Elle', 'Elem'], ['Geoff', 'Fusion'], ['Trance', 'Former'], ['Jen', 'Ayai']]\\nThought:\"\n", + " \"Human: You are an agent designed to write and execute python code to answer questions.\\nYou have access to a python REPL, which you can use to execute python code.\\nIf you get an error, debug your code and try again.\\nOnly use the output of your code to answer the question. \\nYou might know the answer without running any code, but you should still run the code to get the answer.\\nIf it does not seem like you can write code to answer the question, just return \\\"I don't know\\\" as the answer.\\n\\n\\nPython_REPL: A Python shell. Use this to execute python commands. Input should be a valid python command. If you want to see the output of a value, you should print it out with `print(...)`.\\n\\nUse the following format:\\n\\nQuestion: the input question you must answer\\nThought: you should always think about what to do\\nAction: the action to take, should be one of [Python_REPL]\\nAction Input: the input to the action\\nObservation: the result of the action\\n... (this Thought/Action/Action Input/Observation can repeat N times)\\nThought: I now know the final answer\\nFinal Answer: the final answer to the original input question\\n\\nBegin!\\n\\nQuestion: 在这些客户名字转换为拼音并打印输出列表： ['小明', '小黄', '小红', '小蓝', '小橘', '小绿']思考过程请使用中文。\\nThought:\"\n", " ]\n", "}\n", - "\u001b[36;1m\u001b[1;3m[llm/end]\u001b[0m \u001b[1m[1:chain:AgentExecutor > 2:chain:LLMChain > 3:llm:ChatOpenAI] [12.25s] Exiting LLM run with output:\n", + "\u001b[36;1m\u001b[1;3m[llm/end]\u001b[0m \u001b[1m[1:chain:AgentExecutor > 2:chain:LLMChain > 3:llm:ChatOpenAI] [2.91s] Exiting LLM run with output:\n", "\u001b[0m{\n", " \"generations\": [\n", " [\n", " {\n", - " \"text\": \"I can use the sorted() function to sort the list of customers by last name and then first name. I will need to provide a key function to sorted() that returns a tuple of the last name and first name in that order.\\nAction: Python REPL\\nAction Input:\\n```\\ncustomers = [['Harrison', 'Chase'], ['Lang', 'Chain'], ['Dolly', 'Too'], ['Elle', 'Elem'], ['Geoff', 'Fusion'], ['Trance', 'Former'], ['Jen', 'Ayai']]\\nsorted_customers = sorted(customers, key=lambda x: (x[1], x[0]))\\nprint(sorted_customers)\\n```\",\n", + " \"text\": \"我需要使用拼音转换库来将这些客户名字转换为拼音。我可以使用Python的pinyin库来完成这个任务。\\nAction: Python_REPL\\nAction Input: import pinyin\",\n", " \"generation_info\": null,\n", " \"message\": {\n", - " \"content\": \"I can use the sorted() function to sort the list of customers by last name and then first name. I will need to provide a key function to sorted() that returns a tuple of the last name and first name in that order.\\nAction: Python REPL\\nAction Input:\\n```\\ncustomers = [['Harrison', 'Chase'], ['Lang', 'Chain'], ['Dolly', 'Too'], ['Elle', 'Elem'], ['Geoff', 'Fusion'], ['Trance', 'Former'], ['Jen', 'Ayai']]\\nsorted_customers = sorted(customers, key=lambda x: (x[1], x[0]))\\nprint(sorted_customers)\\n```\",\n", + " \"content\": \"我需要使用拼音转换库来将这些客户名字转换为拼音。我可以使用Python的pinyin库来完成这个任务。\\nAction: Python_REPL\\nAction Input: import pinyin\",\n", " \"additional_kwargs\": {},\n", " \"example\": false\n", " }\n", @@ -580,29 +819,26 @@ " ],\n", " \"llm_output\": {\n", " \"token_usage\": {\n", - " \"prompt_tokens\": 327,\n", - " \"completion_tokens\": 138,\n", - " \"total_tokens\": 465\n", + " \"prompt_tokens\": 322,\n", + " \"completion_tokens\": 50,\n", + " \"total_tokens\": 372\n", " },\n", " \"model_name\": \"gpt-3.5-turbo\"\n", - " }\n", + " },\n", + " \"run\": null\n", "}\n", - "\u001b[36;1m\u001b[1;3m[chain/end]\u001b[0m \u001b[1m[1:chain:AgentExecutor > 2:chain:LLMChain] [12.25s] Exiting Chain run with output:\n", + "\u001b[36;1m\u001b[1;3m[chain/end]\u001b[0m \u001b[1m[1:chain:AgentExecutor > 2:chain:LLMChain] [2.91s] Exiting Chain run with output:\n", "\u001b[0m{\n", - " \"text\": \"I can use the sorted() function to sort the list of customers by last name and then first name. I will need to provide a key function to sorted() that returns a tuple of the last name and first name in that order.\\nAction: Python REPL\\nAction Input:\\n```\\ncustomers = [['Harrison', 'Chase'], ['Lang', 'Chain'], ['Dolly', 'Too'], ['Elle', 'Elem'], ['Geoff', 'Fusion'], ['Trance', 'Former'], ['Jen', 'Ayai']]\\nsorted_customers = sorted(customers, key=lambda x: (x[1], x[0]))\\nprint(sorted_customers)\\n```\"\n", + " \"text\": \"我需要使用拼音转换库来将这些客户名字转换为拼音。我可以使用Python的pinyin库来完成这个任务。\\nAction: Python_REPL\\nAction Input: import pinyin\"\n", "}\n", - "\u001b[32;1m\u001b[1;3m[tool/start]\u001b[0m \u001b[1m[1:chain:AgentExecutor > 4:tool:Python REPL] Entering Tool run with input:\n", - "\u001b[0m\"```\n", - "customers = [['Harrison', 'Chase'], ['Lang', 'Chain'], ['Dolly', 'Too'], ['Elle', 'Elem'], ['Geoff', 'Fusion'], ['Trance', 'Former'], ['Jen', 'Ayai']]\n", - "sorted_customers = sorted(customers, key=lambda x: (x[1], x[0]))\n", - "print(sorted_customers)\n", - "```\"\n", - "\u001b[36;1m\u001b[1;3m[tool/end]\u001b[0m \u001b[1m[1:chain:AgentExecutor > 4:tool:Python REPL] [2.2239999999999998ms] Exiting Tool run with output:\n", - "\u001b[0m\"[['Jen', 'Ayai'], ['Lang', 'Chain'], ['Harrison', 'Chase'], ['Elle', 'Elem'], ['Trance', 'Former'], ['Geoff', 'Fusion'], ['Dolly', 'Too']]\"\n", + "\u001b[32;1m\u001b[1;3m[tool/start]\u001b[0m \u001b[1m[1:chain:AgentExecutor > 4:tool:Python_REPL] Entering Tool run with input:\n", + "\u001b[0m\"import pinyin\"\n", + "\u001b[36;1m\u001b[1;3m[tool/end]\u001b[0m \u001b[1m[1:chain:AgentExecutor > 4:tool:Python_REPL] [0.0ms] Exiting Tool run with output:\n", + "\u001b[0m\"\"\n", "\u001b[32;1m\u001b[1;3m[chain/start]\u001b[0m \u001b[1m[1:chain:AgentExecutor > 5:chain:LLMChain] Entering Chain run with input:\n", "\u001b[0m{\n", - " \"input\": \"Sort these customers by last name and then first name and print the output: [['Harrison', 'Chase'], ['Lang', 'Chain'], ['Dolly', 'Too'], ['Elle', 'Elem'], ['Geoff', 'Fusion'], ['Trance', 'Former'], ['Jen', 'Ayai']]\",\n", - " \"agent_scratchpad\": \"I can use the sorted() function to sort the list of customers by last name and then first name. I will need to provide a key function to sorted() that returns a tuple of the last name and first name in that order.\\nAction: Python REPL\\nAction Input:\\n```\\ncustomers = [['Harrison', 'Chase'], ['Lang', 'Chain'], ['Dolly', 'Too'], ['Elle', 'Elem'], ['Geoff', 'Fusion'], ['Trance', 'Former'], ['Jen', 'Ayai']]\\nsorted_customers = sorted(customers, key=lambda x: (x[1], x[0]))\\nprint(sorted_customers)\\n```\\nObservation: [['Jen', 'Ayai'], ['Lang', 'Chain'], ['Harrison', 'Chase'], ['Elle', 'Elem'], ['Trance', 'Former'], ['Geoff', 'Fusion'], ['Dolly', 'Too']]\\n\\nThought:\",\n", + " \"input\": \"在这些客户名字转换为拼音并打印输出列表： ['小明', '小黄', '小红', '小蓝', '小橘', '小绿']思考过程请使用中文。\",\n", + " \"agent_scratchpad\": \"我需要使用拼音转换库来将这些客户名字转换为拼音。我可以使用Python的pinyin库来完成这个任务。\\nAction: Python_REPL\\nAction Input: import pinyin\\nObservation: \\nThought:\",\n", " \"stop\": [\n", " \"\\nObservation:\",\n", " \"\\n\\tObservation:\"\n", @@ -611,18 +847,33 @@ "\u001b[32;1m\u001b[1;3m[llm/start]\u001b[0m \u001b[1m[1:chain:AgentExecutor > 5:chain:LLMChain > 6:llm:ChatOpenAI] Entering LLM run with input:\n", "\u001b[0m{\n", " \"prompts\": [\n", - " \"Human: You are an agent designed to write and execute python code to answer questions.\\nYou have access to a python REPL, which you can use to execute python code.\\nIf you get an error, debug your code and try again.\\nOnly use the output of your code to answer the question. \\nYou might know the answer without running any code, but you should still run the code to get the answer.\\nIf it does not seem like you can write code to answer the question, just return \\\"I don't know\\\" as the answer.\\n\\n\\nPython REPL: A Python shell. Use this to execute python commands. Input should be a valid python command. If you want to see the output of a value, you should print it out with `print(...)`.\\n\\nUse the following format:\\n\\nQuestion: the input question you must answer\\nThought: you should always think about what to do\\nAction: the action to take, should be one of [Python REPL]\\nAction Input: the input to the action\\nObservation: the result of the action\\n... (this Thought/Action/Action Input/Observation can repeat N times)\\nThought: I now know the final answer\\nFinal Answer: the final answer to the original input question\\n\\nBegin!\\n\\nQuestion: Sort these customers by last name and then first name and print the output: [['Harrison', 'Chase'], ['Lang', 'Chain'], ['Dolly', 'Too'], ['Elle', 'Elem'], ['Geoff', 'Fusion'], ['Trance', 'Former'], ['Jen', 'Ayai']]\\nThought:I can use the sorted() function to sort the list of customers by last name and then first name. I will need to provide a key function to sorted() that returns a tuple of the last name and first name in that order.\\nAction: Python REPL\\nAction Input:\\n```\\ncustomers = [['Harrison', 'Chase'], ['Lang', 'Chain'], ['Dolly', 'Too'], ['Elle', 'Elem'], ['Geoff', 'Fusion'], ['Trance', 'Former'], ['Jen', 'Ayai']]\\nsorted_customers = sorted(customers, key=lambda x: (x[1], x[0]))\\nprint(sorted_customers)\\n```\\nObservation: [['Jen', 'Ayai'], ['Lang', 'Chain'], ['Harrison', 'Chase'], ['Elle', 'Elem'], ['Trance', 'Former'], ['Geoff', 'Fusion'], ['Dolly', 'Too']]\\n\\nThought:\"\n", + " \"Human: You are an agent designed to write and execute python code to answer questions.\\nYou have access to a python REPL, which you can use to execute python code.\\nIf you get an error, debug your code and try again.\\nOnly use the output of your code to answer the question. \\nYou might know the answer without running any code, but you should still run the code to get the answer.\\nIf it does not seem like you can write code to answer the question, just return \\\"I don't know\\\" as the answer.\\n\\n\\nPython_REPL: A Python shell. Use this to execute python commands. Input should be a valid python command. If you want to see the output of a value, you should print it out with `print(...)`.\\n\\nUse the following format:\\n\\nQuestion: the input question you must answer\\nThought: you should always think about what to do\\nAction: the action to take, should be one of [Python_REPL]\\nAction Input: the input to the action\\nObservation: the result of the action\\n... (this Thought/Action/Action Input/Observation can repeat N times)\\nThought: I now know the final answer\\nFinal Answer: the final answer to the original input question\\n\\nBegin!\\n\\nQuestion: 在这些客户名字转换为拼音并打印输出列表： ['小明', '小黄', '小红', '小蓝', '小橘', '小绿']思考过程请使用中文。\\nThought:我需要使用拼音转换库来将这些客户名字转换为拼音。我可以使用Python的pinyin库来完成这个任务。\\nAction: Python_REPL\\nAction Input: import pinyin\\nObservation: \\nThought:\"\n", " ]\n", - "}\n", - "\u001b[36;1m\u001b[1;3m[llm/end]\u001b[0m \u001b[1m[1:chain:AgentExecutor > 5:chain:LLMChain > 6:llm:ChatOpenAI] [6.94s] Exiting LLM run with output:\n", + "}\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 1.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 2.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 4.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\u001b[36;1m\u001b[1;3m[llm/end]\u001b[0m \u001b[1m[1:chain:AgentExecutor > 5:chain:LLMChain > 6:llm:ChatOpenAI] [12.76s] Exiting LLM run with output:\n", "\u001b[0m{\n", " \"generations\": [\n", " [\n", " {\n", - " \"text\": \"The customers are now sorted by last name and then first name.\\nFinal Answer: [['Jen', 'Ayai'], ['Lang', 'Chain'], ['Harrison', 'Chase'], ['Elle', 'Elem'], ['Trance', 'Former'], ['Geoff', 'Fusion'], ['Dolly', 'Too']]\",\n", + " \"text\": \"我现在可以使用pinyin库来将客户名字转换为拼音。\\nAction: Python_REPL\\nAction Input: names = ['小明', '小黄', '小红', '小蓝', '小橘', '小绿']\\npinyin_names = [pinyin.get(i, format='strip', delimiter='') for i in names]\\nprint(pinyin_names)\",\n", " \"generation_info\": null,\n", " \"message\": {\n", - " \"content\": \"The customers are now sorted by last name and then first name.\\nFinal Answer: [['Jen', 'Ayai'], ['Lang', 'Chain'], ['Harrison', 'Chase'], ['Elle', 'Elem'], ['Trance', 'Former'], ['Geoff', 'Fusion'], ['Dolly', 'Too']]\",\n", + " \"content\": \"我现在可以使用pinyin库来将客户名字转换为拼音。\\nAction: Python_REPL\\nAction Input: names = ['小明', '小黄', '小红', '小蓝', '小橘', '小绿']\\npinyin_names = [pinyin.get(i, format='strip', delimiter='') for i in names]\\nprint(pinyin_names)\",\n", " \"additional_kwargs\": {},\n", " \"example\": false\n", " }\n", @@ -631,20 +882,87 @@ " ],\n", " \"llm_output\": {\n", " \"token_usage\": {\n", - " \"prompt_tokens\": 521,\n", - " \"completion_tokens\": 67,\n", - " \"total_tokens\": 588\n", + " \"prompt_tokens\": 379,\n", + " \"completion_tokens\": 89,\n", + " \"total_tokens\": 468\n", " },\n", " \"model_name\": \"gpt-3.5-turbo\"\n", - " }\n", + " },\n", + " \"run\": null\n", "}\n", - "\u001b[36;1m\u001b[1;3m[chain/end]\u001b[0m \u001b[1m[1:chain:AgentExecutor > 5:chain:LLMChain] [6.94s] Exiting Chain run with output:\n", + "\u001b[36;1m\u001b[1;3m[chain/end]\u001b[0m \u001b[1m[1:chain:AgentExecutor > 5:chain:LLMChain] [12.77s] Exiting Chain run with output:\n", "\u001b[0m{\n", - " \"text\": \"The customers are now sorted by last name and then first name.\\nFinal Answer: [['Jen', 'Ayai'], ['Lang', 'Chain'], ['Harrison', 'Chase'], ['Elle', 'Elem'], ['Trance', 'Former'], ['Geoff', 'Fusion'], ['Dolly', 'Too']]\"\n", + " \"text\": \"我现在可以使用pinyin库来将客户名字转换为拼音。\\nAction: Python_REPL\\nAction Input: names = ['小明', '小黄', '小红', '小蓝', '小橘', '小绿']\\npinyin_names = [pinyin.get(i, format='strip', delimiter='') for i in names]\\nprint(pinyin_names)\"\n", "}\n", - "\u001b[36;1m\u001b[1;3m[chain/end]\u001b[0m \u001b[1m[1:chain:AgentExecutor] [19.20s] Exiting Chain run with output:\n", + "\u001b[32;1m\u001b[1;3m[tool/start]\u001b[0m \u001b[1m[1:chain:AgentExecutor > 7:tool:Python_REPL] Entering Tool run with input:\n", + "\u001b[0m\"names = ['小明', '小黄', '小红', '小蓝', '小橘', '小绿']\n", + "pinyin_names = [pinyin.get(i, format='strip', delimiter='') for i in names]\n", + "print(pinyin_names)\"\n", + "\u001b[36;1m\u001b[1;3m[tool/end]\u001b[0m \u001b[1m[1:chain:AgentExecutor > 7:tool:Python_REPL] [0.0ms] Exiting Tool run with output:\n", + "\u001b[0m\"['xiaoming', 'xiaohuang', 'xiaohong', 'xiaolan', 'xiaoju', 'xiaolv']\"\n", + "\u001b[32;1m\u001b[1;3m[chain/start]\u001b[0m \u001b[1m[1:chain:AgentExecutor > 8:chain:LLMChain] Entering Chain run with input:\n", "\u001b[0m{\n", - " \"output\": \"[['Jen', 'Ayai'], ['Lang', 'Chain'], ['Harrison', 'Chase'], ['Elle', 'Elem'], ['Trance', 'Former'], ['Geoff', 'Fusion'], ['Dolly', 'Too']]\"\n", + " \"input\": \"在这些客户名字转换为拼音并打印输出列表： ['小明', '小黄', '小红', '小蓝', '小橘', '小绿']思考过程请使用中文。\",\n", + " \"agent_scratchpad\": \"我需要使用拼音转换库来将这些客户名字转换为拼音。我可以使用Python的pinyin库来完成这个任务。\\nAction: Python_REPL\\nAction Input: import pinyin\\nObservation: \\nThought:我现在可以使用pinyin库来将客户名字转换为拼音。\\nAction: Python_REPL\\nAction Input: names = ['小明', '小黄', '小红', '小蓝', '小橘', '小绿']\\npinyin_names = [pinyin.get(i, format='strip', delimiter='') for i in names]\\nprint(pinyin_names)\\nObservation: ['xiaoming', 'xiaohuang', 'xiaohong', 'xiaolan', 'xiaoju', 'xiaolv']\\n\\nThought:\",\n", + " \"stop\": [\n", + " \"\\nObservation:\",\n", + " \"\\n\\tObservation:\"\n", + " ]\n", + "}\n", + "\u001b[32;1m\u001b[1;3m[llm/start]\u001b[0m \u001b[1m[1:chain:AgentExecutor > 8:chain:LLMChain > 9:llm:ChatOpenAI] Entering LLM run with input:\n", + "\u001b[0m{\n", + " \"prompts\": [\n", + " \"Human: You are an agent designed to write and execute python code to answer questions.\\nYou have access to a python REPL, which you can use to execute python code.\\nIf you get an error, debug your code and try again.\\nOnly use the output of your code to answer the question. \\nYou might know the answer without running any code, but you should still run the code to get the answer.\\nIf it does not seem like you can write code to answer the question, just return \\\"I don't know\\\" as the answer.\\n\\n\\nPython_REPL: A Python shell. Use this to execute python commands. Input should be a valid python command. If you want to see the output of a value, you should print it out with `print(...)`.\\n\\nUse the following format:\\n\\nQuestion: the input question you must answer\\nThought: you should always think about what to do\\nAction: the action to take, should be one of [Python_REPL]\\nAction Input: the input to the action\\nObservation: the result of the action\\n... (this Thought/Action/Action Input/Observation can repeat N times)\\nThought: I now know the final answer\\nFinal Answer: the final answer to the original input question\\n\\nBegin!\\n\\nQuestion: 在这些客户名字转换为拼音并打印输出列表： ['小明', '小黄', '小红', '小蓝', '小橘', '小绿']思考过程请使用中文。\\nThought:我需要使用拼音转换库来将这些客户名字转换为拼音。我可以使用Python的pinyin库来完成这个任务。\\nAction: Python_REPL\\nAction Input: import pinyin\\nObservation: \\nThought:我现在可以使用pinyin库来将客户名字转换为拼音。\\nAction: Python_REPL\\nAction Input: names = ['小明', '小黄', '小红', '小蓝', '小橘', '小绿']\\npinyin_names = [pinyin.get(i, format='strip', delimiter='') for i in names]\\nprint(pinyin_names)\\nObservation: ['xiaoming', 'xiaohuang', 'xiaohong', 'xiaolan', 'xiaoju', 'xiaolv']\\n\\nThought:\"\n", + " ]\n", + "}\n" + ] + }, + { + "name": "stderr", + "output_type": "stream", + "text": [ + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 1.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 2.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 4.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n", + "Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.._completion_with_retry in 8.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-2YlJyPMl62f07XPJCAlXfDxj on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method..\n" + ] + }, + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\u001b[36;1m\u001b[1;3m[llm/end]\u001b[0m \u001b[1m[1:chain:AgentExecutor > 8:chain:LLMChain > 9:llm:ChatOpenAI] [20.25s] Exiting LLM run with output:\n", + "\u001b[0m{\n", + " \"generations\": [\n", + " [\n", + " {\n", + " \"text\": \"我现在知道了最终答案。\\nFinal Answer: ['xiaoming', 'xiaohuang', 'xiaohong', 'xiaolan', 'xiaoju', 'xiaolv']\",\n", + " \"generation_info\": null,\n", + " \"message\": {\n", + " \"content\": \"我现在知道了最终答案。\\nFinal Answer: ['xiaoming', 'xiaohuang', 'xiaohong', 'xiaolan', 'xiaoju', 'xiaolv']\",\n", + " \"additional_kwargs\": {},\n", + " \"example\": false\n", + " }\n", + " }\n", + " ]\n", + " ],\n", + " \"llm_output\": {\n", + " \"token_usage\": {\n", + " \"prompt_tokens\": 500,\n", + " \"completion_tokens\": 43,\n", + " \"total_tokens\": 543\n", + " },\n", + " \"model_name\": \"gpt-3.5-turbo\"\n", + " },\n", + " \"run\": null\n", + "}\n", + "\u001b[36;1m\u001b[1;3m[chain/end]\u001b[0m \u001b[1m[1:chain:AgentExecutor > 8:chain:LLMChain] [20.25s] Exiting Chain run with output:\n", + "\u001b[0m{\n", + " \"text\": \"我现在知道了最终答案。\\nFinal Answer: ['xiaoming', 'xiaohuang', 'xiaohong', 'xiaolan', 'xiaoju', 'xiaolv']\"\n", + "}\n", + "\u001b[36;1m\u001b[1;3m[chain/end]\u001b[0m \u001b[1m[1:chain:AgentExecutor] [35.93s] Exiting Chain run with output:\n", + "\u001b[0m{\n", + " \"output\": \"['xiaoming', 'xiaohuang', 'xiaohong', 'xiaolan', 'xiaoju', 'xiaolv']\"\n", "}\n" ] } @@ -652,13 +970,14 @@ "source": [ "import langchain\n", "langchain.debug=True\n", - "agent.run(f\"\"\"Sort these customers by \\\n", - "last name and then first name \\\n", - "and print the output: {customer_list}\"\"\") \n", + "agent.run(f\"\"\"在这些客户名字转换为拼音\\\n", + "并打印输出列表： {customer_list}\\\n", + "思考过程请使用中文。\"\"\") \n", "langchain.debug=False" ] }, { + "attachments": {}, "cell_type": "markdown", "id": "6b2ba50d-53c4-4eab-9467-98f562a072cc", "metadata": { @@ -670,7 +989,7 @@ }, { "cell_type": "code", - "execution_count": 11, + "execution_count": 26, "id": "d8e88bdf-01d2-4adc-b2af-086ece6ba97d", "metadata": {}, "outputs": [], @@ -680,6 +999,7 @@ ] }, { + "attachments": {}, "cell_type": "markdown", "id": "fb92f6e4-21ab-494d-9c22-d0678050fd37", "metadata": {}, @@ -689,7 +1009,7 @@ }, { "cell_type": "code", - "execution_count": 12, + "execution_count": 27, "id": "4b7e80e4-48d5-48f0-abeb-3dc33678ed9e", "metadata": {}, "outputs": [], @@ -700,6 +1020,7 @@ ] }, { + "attachments": {}, "cell_type": "markdown", "id": "8990b3c6-fe05-45b6-9567-06baec267a99", "metadata": {}, @@ -723,23 +1044,23 @@ }, { "cell_type": "code", - "execution_count": 13, + "execution_count": 28, "id": "e22a823f-3749-4bda-bac3-1b892da77929", "metadata": {}, "outputs": [], "source": [ "@tool\n", "def time(text: str) -> str:\n", - " \"\"\"Returns todays date, use this for any \\\n", - " questions related to knowing todays date. \\\n", - " The input should always be an empty string, \\\n", - " and this function will always return todays \\\n", - " date - any date mathmatics should occur \\\n", - " outside this function.\"\"\"\n", + " \"\"\"\n", + " 返回今天的日期，用于任何需要知道今天日期的问题。\\\n", + " 输入应该总是一个空字符串，\\\n", + " 这个函数将总是返回今天的日期，任何日期计算应该在这个函数之外进行。\n", + " \"\"\"\n", " return str(date.today())" ] }, { + "attachments": {}, "cell_type": "markdown", "id": "440cb348-0ec6-4999-a6cd-81cb48b8c546", "metadata": {}, @@ -749,14 +1070,14 @@ }, { "cell_type": "code", - "execution_count": 37, + "execution_count": 36, "id": "09c9f59e-c14c-41b9-b6bd-0e97e8be72ab", "metadata": {}, "outputs": [], "source": [ "agent= initialize_agent(\n", - " tools + [time], #将刚刚创建的时间工具加入到已有的工具中\n", - " llm, #初始化的模型\n", + " tools=[time], #将刚刚创建的时间工具加入代理\n", + " llm=llm, #初始化的模型\n", " agent=AgentType.CHAT_ZERO_SHOT_REACT_DESCRIPTION, #代理类型\n", " handle_parsing_errors=True, #处理解析错误\n", " verbose = True #输出中间步骤\n", @@ -764,6 +1085,7 @@ ] }, { + "attachments": {}, "cell_type": "markdown", "id": "f1a10657-23f2-4f78-a247-73f272119eb4", "metadata": {}, @@ -774,7 +1096,7 @@ }, { "cell_type": "code", - "execution_count": 39, + "execution_count": 37, "id": "f868e30b-ec67-4d76-9dc9-81ecd3332115", "metadata": {}, "outputs": [ @@ -784,8 +1106,11 @@ "text": [ "\n", "\n", - "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n", - "\u001b[32;1m\u001b[1;3mThought: I need to use the `time` tool to get today's date.\n", + "\u001b[1m> Entering new chain...\u001b[0m\n", + "\u001b[32;1m\u001b[1;3m根据提供的工具，我们可以使用`time`函数来获取今天的日期。\n", + "\n", + "Thought: 使用`time`函数来获取今天的日期。\n", + "\n", "Action:\n", "```\n", "{\n", @@ -793,10 +1118,11 @@ " \"action_input\": \"\"\n", "}\n", "```\n", + "\n", "\u001b[0m\n", - "Observation: \u001b[38;5;200m\u001b[1;3m2023-06-03\u001b[0m\n", - "Thought:\u001b[32;1m\u001b[1;3mI have successfully retrieved today's date using the `time` tool.\n", - "Final Answer: Today's date is 2023-06-03.\u001b[0m\n", + "Observation: \u001b[36;1m\u001b[1;3m2023-07-04\u001b[0m\n", + "Thought:\u001b[32;1m\u001b[1;3m我现在知道了最终答案。\n", + "Final Answer: 今天的日期是2023-07-04。\u001b[0m\n", "\n", "\u001b[1m> Finished chain.\u001b[0m\n" ] @@ -804,19 +1130,20 @@ { "data": { "text/plain": [ - "{'input': 'whats the date today?', 'output': \"Today's date is 2023-06-03.\"}" + "{'input': '今天的日期是？', 'output': '今天的日期是2023-07-04。'}" ] }, - "execution_count": 39, + "execution_count": 37, "metadata": {}, "output_type": "execute_result" } ], "source": [ - "agent(\"whats the date today?\") " + "agent(\"今天的日期是？\") " ] }, { + "attachments": {}, "cell_type": "markdown", "id": "923a199f-08e8-4e70-afb5-70cd3e123acf", "metadata": {}, @@ -831,7 +1158,7 @@ " \n", " \n", "3. 模型得到观察（Observation）\n", - "

观测: 2023-06-03

\n", + "

观测: 2023-07-04

\n", " \n", " \n", " \n", @@ -839,7 +1166,7 @@ "

思考: 我已成功使用 time 工具检索到了今天的日期

\n", "\n", "5. 给出最终答案（Final Answer）\n", - "

最终答案: 今天的日期是2023-06-03.

\n", + "

最终答案: 今天的日期是2023-07-04.

\n", "6. 返回最终答案。" ] } @@ -860,7 +1187,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.9.12" + "version": "3.9.16" }, "toc": { "base_numbering": 1, diff --git a/content/LangChain for LLM Application Development/product_data.csv b/content/LangChain for LLM Application Development/product_data.csv new file mode 100644 index 0000000..a3f8362 --- /dev/null +++ b/content/LangChain for LLM Application Development/product_data.csv @@ -0,0 +1,434 @@ +product_name,description +全自动咖啡机,"规格: +大型 - 尺寸：13.8'' x 17.3''。 +中型 - 尺寸：11.5'' x 15.2''。 + +为什么我们热爱它: +这款全自动咖啡机是爱好者的理想选择。 一键操作，即可研磨豆子并沏制出您喜爱的咖啡。它的耐用性和一致性使它成为家庭和办公室的理想选择。 + +材质与护理: +清洁时只需轻擦。 + +构造: +由高品质不锈钢制成。 + +其他特性: +内置研磨器和滤网。 +预设多种咖啡模式。 +在中国制造。 + +有问题？ 请随时联系我们的客户服务团队，他们会解答您的所有问题。" +电动牙刷,"规格: +一般大小 - 高度：9.5''，宽度：1''。 + +为什么我们热爱它: +我们的电动牙刷采用先进的刷头设计和强大的电机，为您提供超凡的清洁力和舒适的刷牙体验。 + +材质与护理: +不可水洗，只需用湿布清洁。 + +构造: +由食品级塑料和尼龙刷毛制成。 + +其他特性: +具有多种清洁模式和定时功能。 +USB充电。 +在日本制造。 + +有问题？请随时联系我们的客户服务团队，他们会解答您的所有问题。" +橙味维生素C泡腾片,"规格: +每盒含有20片。 + +为什么我们热爱它: +我们的橙味维生素C泡腾片是快速补充维生素C的理想方式。每片含有500mg的维生素C，可以帮助提升免疫力，保护您的健康。 + +材质与护理: +请存放在阴凉干燥的地方，避免阳光直射。 + +构造: +主要成分为维生素C和柠檬酸钠。 + +其他特性: +含有天然橙味。 +易于携带。 +在美国制造。 + +有问题？请随时联系我们的客户服务团队，他们会解答您的所有问题。" +无线蓝牙耳机,"规格: +单个耳机尺寸：1.5'' x 1.3''。 + +为什么我们热爱它: +这款无线蓝牙耳机配备了降噪技术和长达8小时的电池续航力，让您无论在哪里都可以享受无障碍的音乐体验。 + +材质与护理: +只需用湿布清洁。 + +构造: +由耐用的塑料和金属构成，配备有软质耳塞。 + +其他特性: +快速充电功能。 +内置麦克风，支持接听电话。 +在韩国制造。 + +有问题？请随时联系我们的客户服务团队，他们会解答您的所有问题。" +瑜伽垫,"规格: +尺寸：24'' x 68''。 + +为什么我们热爱它: +我们的瑜伽垫拥有出色的抓地力和舒适度，无论是做瑜伽还是健身，都是理想的选择。 + +材质与护理: +可用清水清洁，自然晾干。 + +构造: +由环保PVC材料制成。 + +其他特性: +附带便携包和绑带。 +在印度制造。 + +有问题？请随时联系我们的客户服务团队，他们会解答您的所有问题。" +防水运动手表,"规格: +表盘直径：40mm。 + +为什么我们热爱它: +这款防水运动手表配备了心率监测和计步功能，不仅适合日常佩戴，也是户外运动的理想伙伴。 + +材质与护理: +只需用湿布清洁。 + +构造: +由不锈钢和硅胶制成。 + +其他特性: +内置可充电电池。 +支持蓝牙连接手机。 +在瑞士制造。 + +有问题？请随时联系我们的客户服务团队，他们会解答您的所有问题。" +书籍:《机器学习基础》,"规格: +页数：580页。 + +为什么我们热爱它: +《机器学习基础》以易懂的语言讲解了机器学习的基本概念和方法，适合初学者阅读。 + +材质与护理: +纸质封面，存放在阴凉干燥处。 + +构造: +由环保纸张和油墨印刷。 + +其他特性: +附带实例和习题。 +在美国印刷。 + +有问题？请随时联系我们的客户服务团队，他们会解答您的所有问题。" +空气净化器,"规格: +尺寸：15'' x 15'' x 20''。 + +为什么我们热爱它: +我们的空气净化器采用了先进的HEPA过滤技术，能有效去除空气中的微粒和异味，为您提供清新的室内环境。 + +材质与护理: +清洁时使用干布擦拭。 + +构造: +由塑料和电子元件制成。 + +其他特性: +三档风速，附带定时功能。 +在德国制造。 + +有问题？请随时联系我们的客户服务团队，他们会解答您的所有问题。" +陶瓷保温杯,"规格: +容量：350ml。 + +为什么我们热爱它: +我们的陶瓷保温杯设计优雅，保温效果好，是办公室和户外活动的理想伴侣。 + +材质与护理: +可以手洗或洗碗机清洗。 + +构造: +由高品质陶瓷和不锈钢制成。 + +其他特性: +有多种颜色可选。 +在中国制造。 + +有问题？请随时联系我们的客户服务团队，他们会解答您的所有问题。" +宠物自动喂食器,"规格: +尺寸：14'' x 9'' x 15''。 + +为什么我们热爱它: +我们的宠物自动喂食器可以定时定量投放食物，让您无论在家或外出都能确保宠物的饮食。 + +材质与护理: +可用湿布清洁。 + +构造: +由塑料和电子元件制成。 + +其他特性: +配备LCD屏幕，操作简单。 +可以设置多次投食。 +在美国制造。 + +有问题？请随时联系我们的客户服务团队，他们会解答您的所有问题。" +高清电视机,"规格: +尺寸：50''。 + +为什么我们热爱它: +我们的高清电视机拥有出色的画质和强大的音效，带来沉浸式的观看体验。 + +材质与护理: +使用干布清洁。 + +构造: +由塑料、金属和电子元件制成。 + +其他特性: +支持网络连接，可以在线观看视频。 +配备遥控器。 +在韩国制造。 + +有问题？请随时联系我们的客户服务团队，他们会解答您的所有问题。" +旅行背包,"规格: +尺寸：18'' x 12'' x 6''。 + +为什么我们热爱它: +我们的旅行背包拥有多个实用的内外袋，轻松装下您的必需品，是短途旅行的理想选择。 + +材质与护理: +可以手洗，自然晾干。 + +构造: +由防水尼龙制成。 + +其他特性: +附带可调节背带和安全锁。 +在中国制造。 + +有问题？请随时联系我们的客户服务团队，他们会解答您的所有问题。" +太阳能庭院灯,"规格: +高度：18''。 + +为什么我们热爱它: +我们的太阳能庭院灯无需电源，只需将其放在阳光下即可充电。到了晚上，它会自动点亮，为您的庭院增添温馨的氛围。 + +材质与护理: +可以用湿布清洁。 + +构造: +由塑料和太阳能板制成。 + +其他特性: +自动感应，日落后自动点亮。 +在中国制造。 + +有问题？请随时联系我们的客户服务团队，他们会解答您的所有问题。" +厨房刀具套装,"规格: +一套包括8把刀。 + +为什么我们热爱它: +我们的厨房刀具套装由专业级不锈钢制成，刀身坚固，刀刃锋利，切割效果出色，是厨房必备工具。 + +材质与护理: +建议手洗并立即擦干，以保持最佳状况。 + +构造: +由高品质不锈钢制成。 + +其他特性: +包括多种类型的刀，适合不同的用途。 +在德国制造。 + +有问题？请随时联系我们的客户服务团队，他们会解答您的所有问题。" +迷你无线蓝牙音箱,"规格: +直径：3''，高度：2''。 + +为什么我们热爱它: +我们的迷你无线蓝牙音箱体积小巧，音质出色，可以轻松带给您音乐的享受。 + +材质与护理: +使用湿布擦拭清洁。 + +构造: +由塑料和金属构成。 + +其他特性: +内置可充电电池，充满后可播放5小时。 +在中国制造。 + +有问题？请随时联系我们的客户服务团队，他们会解答您的所有问题。" +抗菌洗手液,"规格: +容量：500ml。 + +为什么我们热爱它: +我们的抗菌洗手液含有天然植物精华，能有效杀灭细菌，同时滋润双手。 + +材质与护理: +直接涂抹在手上，然后用水冲洗。 + +构造: +由天然植物精华和抗菌成分制成。 + +其他特性: +具有清新的柠檬香味。 +在法国制造。 + +有问题？请随时联系我们的客户服务团队，他们会解答您的所有问题。" +纯棉T恤,"规格: +尺码：S, M, L, XL, XXL。 + +为什么我们热爱它: +我们的纯棉T恤采用高品质棉质，柔软舒适，透气性好，是日常穿着的理想选择。 + +材质与护理: +可以机洗，建议反面洗涤，以保持颜色鲜艳。 + +构造: +由100%纯棉制成。 + +其他特性: +有多种颜色可选。 +在中国制造。 + +有问题？请随时联系我们的客户服务团队，他们会解答您的所有问题。" +自动咖啡机,"规格: +尺寸：12'' x 8'' x 14''。 + +为什么我们热爱它: +我们的自动咖啡机可以轻松制作美味的咖啡，只需按一下钮，就能享受到新鲜的咖啡。 + +材质与护理: +可以用湿布清洁外部，内部配件可以拆卸清洗。 + +构造: +由不锈钢和塑料制成。 + +其他特性: +可调节咖啡浓度。 +在意大利制造。 + +有问题？请随时联系我们的客户服务团队，他们会解答您的所有问题。" +摄像头保护套,"规格: +适用于各种品牌和型号的摄像头。 + +为什么我们热爱它: +我们的摄像头保护套可以有效保护您的摄像头不受灰尘和刮擦的影响。 + +材质与护理: +使用湿布擦拭。 + +构造: +由耐磨的合成纤维制成。 + +其他特性: +设计简洁，易于安装。 +在中国制造。 + +有问题？请随时联系我们的客户服务团队，他们会解答您的所有问题。" +玻璃保护膜,"规格: +适用于各种尺寸的手机屏幕。 + +为什么我们热爱它: +我们的玻璃保护膜可以有效防止手机屏幕刮伤和破裂，而且不影响触控的灵敏度。 + +材质与护理: +使用干布擦拭。 + +构造: +由高强度的玻璃材料制成。 + +其他特性: +安装简单，适合自行安装。 +在日本制造。 + +有问题？请随时联系我们的客户服务团队，他们会解答您的所有问题。" +儿童益智玩具,"规格: +适合3岁以上的儿童。 + +为什么我们热爱它: +我们的儿童益智玩具设计独特，色彩鲜艳，能够引发孩子的兴趣，同时训练他们的手眼协调能力和逻辑思维。 + +材质与护理: +使用湿布擦拭。 + +构造: +由环保无毒的塑料制成。 + +其他特性: +具有教育和娱乐的双重功能。 +在中国制造。 + +有问题？请随时联系我们的客户服务团队，他们会解答您的所有问题。" +迷你书架,"规格: +尺寸：20'' x 8'' x 24''。 + +为什么我们热爱它: +我们的迷你书架占地面积小，设计简约，能够有效存放和展示您的书籍。 + +材质与护理: +使用干布擦拭。 + +构造: +由实木制成。 + +其他特性: +拥有3层收纳空间。 +在中国制造。 + +有问题？请随时联系我们的客户服务团队，他们会解答您的所有问题。" +防滑瑜伽垫,"规格: +尺寸：72'' x 24''。 + +为什么我们热爱它: +我们的防滑瑜伽垫采用高品质环保材料，触感舒适，防滑效果好，是瑜伽爱好者的理想选择。 + +材质与护理: +可以用湿布清洁。 + +构造: +由环保PVC材料制成。 + +其他特性: +有多种颜色可选。 +在中国制造。 + +有问题？请随时联系我们的客户服务团队，他们会解答您的所有问题。" +LED台灯,"规格: +尺寸：6'' x 6'' x 18''。 + +为什么我们热爱它: +我们的LED台灯亮度可调，采用节能LED灯珠，不仅可以提供明亮的光照，还可以节省能源。 + +材质与护理: +使用湿布擦拭。 + +构造: +由金属和塑料制成。 + +其他特性: +具有触控开关和调光功能。 +在中国制造。 + +有问题？请随时联系我们的客户服务团队，他们会解答您的所有问题。" +水晶酒杯,"规格: +容量：250ml。 + +为什么我们热爱它: +我们的水晶酒杯采用高品质水晶玻璃制成，造型优雅，是享受美酒的理想选择。 + +材质与护理: +建议手洗。 + +构造: +由水晶玻璃制成。 + +其他特性: +适合各种场合使用。 +在法国制造。 + +有问题？请随时联系我们的客户服务团队，他们会解答您的所有问题。"