kennethreitz/langchain
1
0
Fork 0
mirror of https://github.com/kennethreitz/langchain.git synced 2026-06-05 23:00:18 +00:00
Code Issues Packages Projects Releases Wiki Activity

mv module integrations docs (#8101)

Browse Source
This commit is contained in:
Bagatur
2023-07-23 23:23:16 -07:00
committed by GitHub
parent 8ea840432f
commit c8c8635dc9
619 changed files with 2322 additions and 449 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/extras/modules/agents/toolkits/amadeus.ipynb
-242
View File
@@ -1,242 +0,0 @@
{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"# Amadeus Toolkit\n",
"\n",
"This notebook walks you through connecting LangChain to the Amadeus travel information API\n",
"\n",
"To use this toolkit, you will need to set up your credentials explained in the [Amadeus for developers getting started overview](https://developers.amadeus.com/get-started/get-started-with-self-service-apis-335). Once you've received a AMADEUS_CLIENT_ID and AMADEUS_CLIENT_SECRET, you can input them as environmental variables below."
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"!pip install --upgrade amadeus > /dev/null"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Assign Environmental Variables\n",
"\n",
"The toolkit will read the AMADEUS_CLIENT_ID and AMADEUS_CLIENT_SECRET environmental variables to authenticate the user so you need to set them here. You will also need to set your OPENAI_API_KEY to use the agent later."
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"# Set environmental variables here\n",
"import os\n",
"\n",
"os.environ[\"AMADEUS_CLIENT_ID\"] = \"CLIENT_ID\"\n",
"os.environ[\"AMADEUS_CLIENT_SECRET\"] = \"CLIENT_SECRET\"\n",
"os.environ[\"OPENAI_API_KEY\"] = \"API_KEY\""
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Create the Amadeus Toolkit and Get Tools\n",
"\n",
"To start, you need to create the toolkit, so you can access its tools later."
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"from langchain.agents.agent_toolkits.amadeus.toolkit import AmadeusToolkit\n",
"\n",
"toolkit = AmadeusToolkit()\n",
"tools = toolkit.get_tools()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Use Amadeus Toolkit within an Agent"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"from langchain import OpenAI\n",
"from langchain.agents import initialize_agent, AgentType"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"llm = OpenAI(temperature=0)\n",
"agent = initialize_agent(\n",
" tools=tools,\n",
" llm=llm,\n",
" verbose=False,\n",
" agent=AgentType.STRUCTURED_CHAT_ZERO_SHOT_REACT_DESCRIPTION,\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"tags": []
},
"outputs": [
{
"data": {
"text/plain": [
"'The closest airport to Cali, Colombia is Alfonso Bonilla Aragón International Airport (CLO).'"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"agent.run(\"What is the name of the airport in Cali, Colombia?\")"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"tags": []
},
"outputs": [
{
"data": {
"text/plain": [
"'The cheapest flight on August 23, 2023 leaving Dallas, Texas before noon to Lincoln, Nebraska has a departure time of 16:42 and a total price of 276.08 EURO.'"
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"agent.run(\n",
" \"What is the departure time of the cheapest flight on August 23, 2023 leaving Dallas, Texas before noon to Lincoln, Nebraska?\"\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'The earliest flight on August 23, 2023 leaving Dallas, Texas to Lincoln, Nebraska lands in Lincoln, Nebraska at 16:07.'"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"agent.run(\n",
" \"At what time does earliest flight on August 23, 2023 leaving Dallas, Texas to Lincoln, Nebraska land in Nebraska?\"\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'The cheapest flight between Portland, Oregon to Dallas, TX on October 3, 2023 is a Spirit Airlines flight with a total price of 84.02 EURO and a total travel time of 8 hours and 43 minutes.'"
]
},
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"agent.run(\n",
" \"What is the full travel time for the cheapest flight between Portland, Oregon to Dallas, TX on October 3, 2023?\"\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'Dear Paul,\\n\\nI am writing to request that you book the earliest flight from DFW to DCA on Aug 28, 2023. The flight details are as follows:\\n\\nFlight 1: DFW to ATL, departing at 7:15 AM, arriving at 10:25 AM, flight number 983, carrier Delta Air Lines\\nFlight 2: ATL to DCA, departing at 12:15 PM, arriving at 2:02 PM, flight number 759, carrier Delta Air Lines\\n\\nThank you for your help.\\n\\nSincerely,\\nSantiago'"
]
},
"execution_count": 10,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"agent.run(\n",
" \"Please draft a concise email from Santiago to Paul, Santiago's travel agent, asking him to book the earliest flight from DFW to DCA on Aug 28, 2023. Include all flight details in the email.\"\n",
")"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.4"
}
},
"nbformat": 4,
"nbformat_minor": 4
}
docs/extras/modules/agents/toolkits/azure_cognitive_services.ipynb
-272
View File
@@ -1,272 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Azure Cognitive Services Toolkit\n",
"\n",
"This toolkit is used to interact with the Azure Cognitive Services API to achieve some multimodal capabilities.\n",
"\n",
"Currently There are four tools bundled in this toolkit:\n",
"- AzureCogsImageAnalysisTool: used to extract caption, objects, tags, and text from images. (Note: this tool is not available on Mac OS yet, due to the dependency on `azure-ai-vision` package, which is only supported on Windows and Linux currently.)\n",
"- AzureCogsFormRecognizerTool: used to extract text, tables, and key-value pairs from documents.\n",
"- AzureCogsSpeech2TextTool: used to transcribe speech to text.\n",
"- AzureCogsText2SpeechTool: used to synthesize text to speech."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"First, you need to set up an Azure account and create a Cognitive Services resource. You can follow the instructions [here](https://docs.microsoft.com/en-us/azure/cognitive-services/cognitive-services-apis-create-account?tabs=multiservice%2Cwindows) to create a resource. \n",
"\n",
"Then, you need to get the endpoint, key and region of your resource, and set them as environment variables. You can find them in the \"Keys and Endpoint\" page of your resource."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# !pip install --upgrade azure-ai-formrecognizer > /dev/null\n",
"# !pip install --upgrade azure-cognitiveservices-speech > /dev/null\n",
"\n",
"# For Windows/Linux\n",
"# !pip install --upgrade azure-ai-vision > /dev/null"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"\n",
"os.environ[\"OPENAI_API_KEY\"] = \"sk-\"\n",
"os.environ[\"AZURE_COGS_KEY\"] = \"\"\n",
"os.environ[\"AZURE_COGS_ENDPOINT\"] = \"\"\n",
"os.environ[\"AZURE_COGS_REGION\"] = \"\""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Create the Toolkit"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from langchain.agents.agent_toolkits import AzureCognitiveServicesToolkit\n",
"\n",
"toolkit = AzureCognitiveServicesToolkit()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"['Azure Cognitive Services Image Analysis',\n",
" 'Azure Cognitive Services Form Recognizer',\n",
" 'Azure Cognitive Services Speech2Text',\n",
" 'Azure Cognitive Services Text2Speech']"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"[tool.name for tool in toolkit.get_tools()]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Use within an Agent"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from langchain import OpenAI\n",
"from langchain.agents import initialize_agent, AgentType"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"llm = OpenAI(temperature=0)\n",
"agent = initialize_agent(\n",
" tools=toolkit.get_tools(),\n",
" llm=llm,\n",
" agent=AgentType.STRUCTURED_CHAT_ZERO_SHOT_REACT_DESCRIPTION,\n",
" verbose=True,\n",
")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3m\n",
"Action:\n",
"```\n",
"{\n",
" \"action\": \"Azure Cognitive Services Image Analysis\",\n",
" \"action_input\": \"https://images.openai.com/blob/9ad5a2ab-041f-475f-ad6a-b51899c50182/ingredients.png\"\n",
"}\n",
"```\n",
"\n",
"\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3mCaption: a group of eggs and flour in bowls\n",
"Objects: Egg, Egg, Food\n",
"Tags: dairy, ingredient, indoor, thickening agent, food, mixing bowl, powder, flour, egg, bowl\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I can use the objects and tags to suggest recipes\n",
"Action:\n",
"```\n",
"{\n",
" \"action\": \"Final Answer\",\n",
" \"action_input\": \"You can make pancakes, omelettes, or quiches with these ingredients!\"\n",
"}\n",
"```\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"'You can make pancakes, omelettes, or quiches with these ingredients!'"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"agent.run(\n",
" \"What can I make with these ingredients?\"\n",
" \"https://images.openai.com/blob/9ad5a2ab-041f-475f-ad6a-b51899c50182/ingredients.png\"\n",
")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3mAction:\n",
"```\n",
"{\n",
" \"action\": \"Azure Cognitive Services Text2Speech\",\n",
" \"action_input\": \"Why did the chicken cross the playground? To get to the other slide!\"\n",
"}\n",
"```\n",
"\n",
"\u001b[0m\n",
"Observation: \u001b[31;1m\u001b[1;3m/tmp/tmpa3uu_j6b.wav\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I have the audio file of the joke\n",
"Action:\n",
"```\n",
"{\n",
" \"action\": \"Final Answer\",\n",
" \"action_input\": \"/tmp/tmpa3uu_j6b.wav\"\n",
"}\n",
"```\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"'/tmp/tmpa3uu_j6b.wav'"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"audio_file = agent.run(\"Tell me a joke and read it out for me.\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from IPython import display\n",
"\n",
"audio = display.Audio(audio_file)\n",
"display.display(audio)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.3"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
docs/extras/modules/agents/toolkits/csv.ipynb
-313
View File
@@ -1,313 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "7094e328",
"metadata": {},
"source": [
"# CSV Agent\n",
"\n",
"This notebook shows how to use agents to interact with a csv. It is mostly optimized for question answering.\n",
"\n",
"**NOTE: this agent calls the Pandas DataFrame agent under the hood, which in turn calls the Python agent, which executes LLM generated Python code - this can be bad if the LLM generated Python code is harmful. Use cautiously.**\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "827982c7",
"metadata": {},
"outputs": [],
"source": [
"from langchain.agents import create_csv_agent"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "caae0bec",
"metadata": {},
"outputs": [],
"source": [
"from langchain.llms import OpenAI\n",
"from langchain.chat_models import ChatOpenAI\n",
"from langchain.agents.agent_types import AgentType"
]
},
{
"cell_type": "markdown",
"id": "bd806175",
"metadata": {},
"source": [
"## Using ZERO_SHOT_REACT_DESCRIPTION\n",
"\n",
"This shows how to initialize the agent using the ZERO_SHOT_REACT_DESCRIPTION agent type. Note that this is an alternative to the above."
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "a1717204",
"metadata": {},
"outputs": [],
"source": [
"agent = create_csv_agent(\n",
" OpenAI(temperature=0),\n",
" \"titanic.csv\",\n",
" verbose=True,\n",
" agent_type=AgentType.ZERO_SHOT_REACT_DESCRIPTION,\n",
")"
]
},
{
"cell_type": "markdown",
"id": "c31bb8a6",
"metadata": {},
"source": [
"## Using OpenAI Functions\n",
"\n",
"This shows how to initialize the agent using the OPENAI_FUNCTIONS agent type. Note that this is an alternative to the above."
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "16c4dc59",
"metadata": {},
"outputs": [],
"source": [
"agent = create_csv_agent(\n",
" ChatOpenAI(temperature=0, model=\"gpt-3.5-turbo-0613\"),\n",
" \"titanic.csv\",\n",
" verbose=True,\n",
" agent_type=AgentType.OPENAI_FUNCTIONS,\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "46b9489d",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Error in on_chain_start callback: 'name'\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\u001b[32;1m\u001b[1;3m\n",
"Invoking: `python_repl_ast` with `df.shape[0]`\n",
"\n",
"\n",
"\u001b[0m\u001b[36;1m\u001b[1;3m891\u001b[0m\u001b[32;1m\u001b[1;3mThere are 891 rows in the dataframe.\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"'There are 891 rows in the dataframe.'"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"agent.run(\"how many rows are there?\")"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "a96309be",
"metadata": {
"scrolled": false
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Error in on_chain_start callback: 'name'\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\u001b[32;1m\u001b[1;3m\n",
"Invoking: `python_repl_ast` with `df[df['SibSp'] > 3]['PassengerId'].count()`\n",
"\n",
"\n",
"\u001b[0m\u001b[36;1m\u001b[1;3m30\u001b[0m\u001b[32;1m\u001b[1;3mThere are 30 people in the dataframe who have more than 3 siblings.\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"'There are 30 people in the dataframe who have more than 3 siblings.'"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"agent.run(\"how many people have more than 3 siblings\")"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "964a09f7",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Error in on_chain_start callback: 'name'\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\u001b[32;1m\u001b[1;3m\n",
"Invoking: `python_repl_ast` with `import pandas as pd\n",
"import math\n",
"\n",
"# Create a dataframe\n",
"data = {'Age': [22, 38, 26, 35, 35]}\n",
"df = pd.DataFrame(data)\n",
"\n",
"# Calculate the average age\n",
"average_age = df['Age'].mean()\n",
"\n",
"# Calculate the square root of the average age\n",
"square_root = math.sqrt(average_age)\n",
"\n",
"square_root`\n",
"\n",
"\n",
"\u001b[0m\u001b[36;1m\u001b[1;3m5.585696017507576\u001b[0m\u001b[32;1m\u001b[1;3mThe square root of the average age is approximately 5.59.\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"'The square root of the average age is approximately 5.59.'"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"agent.run(\"whats the square root of the average age?\")"
]
},
{
"cell_type": "markdown",
"id": "09539c18",
"metadata": {},
"source": [
"### Multi CSV Example\n",
"\n",
"This next part shows how the agent can interact with multiple csv files passed in as a list."
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "15f11fbd",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Error in on_chain_start callback: 'name'\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\u001b[32;1m\u001b[1;3m\n",
"Invoking: `python_repl_ast` with `df1['Age'].nunique() - df2['Age'].nunique()`\n",
"\n",
"\n",
"\u001b[0m\u001b[36;1m\u001b[1;3m-1\u001b[0m\u001b[32;1m\u001b[1;3mThere is 1 row in the age column that is different between the two dataframes.\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"'There is 1 row in the age column that is different between the two dataframes.'"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"agent = create_csv_agent(\n",
" ChatOpenAI(temperature=0, model=\"gpt-3.5-turbo-0613\"),\n",
" [\"titanic.csv\", \"titanic_age_fillna.csv\"],\n",
" verbose=True,\n",
" agent_type=AgentType.OPENAI_FUNCTIONS,\n",
")\n",
"agent.run(\"how many rows in the age column are different between the two dfs?\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f2909808",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.1"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
docs/extras/modules/agents/toolkits/document_comparison_toolkit.ipynb
-435
View File
File diff suppressed because one or more lines are too long
docs/extras/modules/agents/toolkits/github.ipynb
-167
View File
@@ -1,167 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# GitHub\n",
"\n",
"This notebook goes over how to use the GitHub tool.\n",
"The GitHub tool allows agents to interact with a given GitHub repository. It implements CRUD operations for modifying files and can read/comment on Issues. The tool wraps the [PyGitHub](https://github.com/PyGithub/PyGithub) library.\n",
"\n",
"In order to interact with the GitHub API you must create a [GitHub app](https://docs.github.com/en/apps/creating-github-apps/about-creating-github-apps/about-creating-github-apps). Next, you must set the following environment variables:\n",
"```\n",
"GITHUB_APP_ID\n",
"GITHUB_APP_PRIVATE_KEY\n",
"GITHUB_REPOSITORY\n",
"GITHUB_BRANCH\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"vscode": {
"languageId": "shellscript"
}
},
"outputs": [],
"source": [
"%pip install pygithub"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"from langchain.agents import AgentType\n",
"from langchain.agents import initialize_agent\n",
"from langchain.agents.agent_toolkits.github.toolkit import GitHubToolkit\n",
"from langchain.llms import OpenAI\n",
"from langchain.utilities.github import GitHubAPIWrapper"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [],
"source": [
"os.environ[\"GITHUB_APP_ID\"] = \"your-github-app-id\"\n",
"os.environ[\"GITHUB_APP_PRIVATE_KEY\"] = \"/path/to/your/private/key\"\n",
"os.environ[\"GITHUB_REPOSITORY\"] = \"user/repo\"\n",
"os.environ[\"GITHUB_BRANCH\"] = \"branch-name\"\n",
"os.environ[\"OPENAI_API_KEY\"] = \"your-openai-api-key\""
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [],
"source": [
"llm = OpenAI(temperature=0)\n",
"github = GitHubAPIWrapper()\n",
"toolkit = GitHubToolkit.from_github_api_wrapper(github)\n",
"agent = initialize_agent(\n",
" toolkit.get_tools(), llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=True\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3m I need to figure out what issues need to be completed and how to complete them.\n",
"Action: Get Issues\n",
"Action Input: N/A\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3mFound 1 issues:\n",
"[{'title': 'Change the main script to print Hello AI!', 'number': 1}]\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I need to get more information about this issue.\n",
"Action: Get Issue\n",
"Action Input: 1\u001b[0m\n",
"Observation: \u001b[33;1m\u001b[1;3m{'title': 'Change the main script to print Hello AI!', 'body': None, 'comments': '[]'}\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I need to update the main script to print Hello AI!\n",
"Action: Update File\n",
"Action Input: main.py\n",
"OLD <<<<\n",
"print(\"Hello World!\")\n",
">>>> OLD\n",
"NEW <<<<\n",
"print(\"Hello AI!\")\n",
">>>> NEW\u001b[0m\n",
"Observation: \u001b[38;5;200m\u001b[1;3mFile content was not updated because the old content was not found. It may be helpful to use the read_file action to get the current file contents.\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I need to read the current file contents.\n",
"Action: Read File\n",
"Action Input: main.py\u001b[0m\n",
"Observation: \u001b[33;1m\u001b[1;3mprint(\"Hello world!\")\n",
"\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I need to update the main script to print Hello AI!\n",
"Action: Update File\n",
"Action Input: main.py\n",
"OLD <<<<\n",
"print(\"Hello world!\")\n",
">>>> OLD\n",
"NEW <<<<\n",
"print(\"Hello AI!\")\n",
">>>> NEW\u001b[0m\n",
"Observation: \u001b[38;5;200m\u001b[1;3mUpdated file main.py\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I now know the final answer\n",
"Final Answer: The main script has been updated to print \"Hello AI!\"\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"'The main script has been updated to print \"Hello AI!\"'"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"agent.run(\n",
" \"You have the software engineering capabilities of a Google Principle engineer. You are tasked with completing issues on a github repository. Please look at the existing issues and complete them.\"\n",
")"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.4"
},
"orig_nbformat": 4
},
"nbformat": 4,
"nbformat_minor": 2
}
docs/extras/modules/agents/toolkits/gmail.ipynb
-234
View File
@@ -1,234 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Gmail Toolkit\n",
"\n",
"This notebook walks through connecting a LangChain email to the Gmail API.\n",
"\n",
"To use this toolkit, you will need to set up your credentials explained in the [Gmail API docs](https://developers.google.com/gmail/api/quickstart/python#authorize_credentials_for_a_desktop_application). Once you've downloaded the `credentials.json` file, you can start using the Gmail API. Once this is done, we'll install the required libraries."
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"!pip install --upgrade google-api-python-client > /dev/null\n",
"!pip install --upgrade google-auth-oauthlib > /dev/null\n",
"!pip install --upgrade google-auth-httplib2 > /dev/null\n",
"!pip install beautifulsoup4 > /dev/null # This is optional but is useful for parsing HTML messages"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Create the Toolkit\n",
"\n",
"By default the toolkit reads the local `credentials.json` file. You can also manually provide a `Credentials` object."
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"from langchain.agents.agent_toolkits import GmailToolkit\n",
"\n",
"toolkit = GmailToolkit()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Customizing Authentication\n",
"\n",
"Behind the scenes, a `googleapi` resource is created using the following methods. \n",
"you can manually build a `googleapi` resource for more auth control. "
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"from langchain.tools.gmail.utils import build_resource_service, get_gmail_credentials\n",
"\n",
"# Can review scopes here https://developers.google.com/gmail/api/auth/scopes\n",
"# For instance, readonly scope is 'https://www.googleapis.com/auth/gmail.readonly'\n",
"credentials = get_gmail_credentials(\n",
" token_file=\"token.json\",\n",
" scopes=[\"https://mail.google.com/\"],\n",
" client_secrets_file=\"credentials.json\",\n",
")\n",
"api_resource = build_resource_service(credentials=credentials)\n",
"toolkit = GmailToolkit(api_resource=api_resource)"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"tags": []
},
"outputs": [
{
"data": {
"text/plain": [
"[GmailCreateDraft(name='create_gmail_draft', description='Use this tool to create a draft email with the provided message fields.', args_schema=<class 'langchain.tools.gmail.create_draft.CreateDraftSchema'>, return_direct=False, verbose=False, callbacks=None, callback_manager=None, api_resource=<googleapiclient.discovery.Resource object at 0x10e5c6d10>),\n",
" GmailSendMessage(name='send_gmail_message', description='Use this tool to send email messages. The input is the message, recipents', args_schema=None, return_direct=False, verbose=False, callbacks=None, callback_manager=None, api_resource=<googleapiclient.discovery.Resource object at 0x10e5c6d10>),\n",
" GmailSearch(name='search_gmail', description=('Use this tool to search for email messages or threads. The input must be a valid Gmail query. The output is a JSON list of the requested resource.',), args_schema=<class 'langchain.tools.gmail.search.SearchArgsSchema'>, return_direct=False, verbose=False, callbacks=None, callback_manager=None, api_resource=<googleapiclient.discovery.Resource object at 0x10e5c6d10>),\n",
" GmailGetMessage(name='get_gmail_message', description='Use this tool to fetch an email by message ID. Returns the thread ID, snipet, body, subject, and sender.', args_schema=<class 'langchain.tools.gmail.get_message.SearchArgsSchema'>, return_direct=False, verbose=False, callbacks=None, callback_manager=None, api_resource=<googleapiclient.discovery.Resource object at 0x10e5c6d10>),\n",
" GmailGetThread(name='get_gmail_thread', description=('Use this tool to search for email messages. The input must be a valid Gmail query. The output is a JSON list of messages.',), args_schema=<class 'langchain.tools.gmail.get_thread.GetThreadSchema'>, return_direct=False, verbose=False, callbacks=None, callback_manager=None, api_resource=<googleapiclient.discovery.Resource object at 0x10e5c6d10>)]"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"tools = toolkit.get_tools()\n",
"tools"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Use within an Agent"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"from langchain import OpenAI\n",
"from langchain.agents import initialize_agent, AgentType"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"llm = OpenAI(temperature=0)\n",
"agent = initialize_agent(\n",
" tools=toolkit.get_tools(),\n",
" llm=llm,\n",
" agent=AgentType.STRUCTURED_CHAT_ZERO_SHOT_REACT_DESCRIPTION,\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {
"tags": []
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"WARNING:root:Failed to load default session, using empty session: 0\n",
"WARNING:root:Failed to persist run: {\"detail\":\"Not Found\"}\n"
]
},
{
"data": {
"text/plain": [
"'I have created a draft email for you to edit. The draft Id is r5681294731961864018.'"
]
},
"execution_count": 19,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"agent.run(\n",
" \"Create a gmail draft for me to edit of a letter from the perspective of a sentient parrot\"\n",
" \" who is looking to collaborate on some research with her\"\n",
" \" estranged friend, a cat. Under no circumstances may you send the message, however.\"\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 24,
"metadata": {
"tags": []
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"WARNING:root:Failed to load default session, using empty session: 0\n",
"WARNING:root:Failed to persist run: {\"detail\":\"Not Found\"}\n"
]
},
{
"data": {
"text/plain": [
"\"The latest email in your drafts is from hopefulparrot@gmail.com with the subject 'Collaboration Opportunity'. The body of the email reads: 'Dear [Friend], I hope this letter finds you well. I am writing to you in the hopes of rekindling our friendship and to discuss the possibility of collaborating on some research together. I know that we have had our differences in the past, but I believe that we can put them aside and work together for the greater good. I look forward to hearing from you. Sincerely, [Parrot]'\""
]
},
"execution_count": 24,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"agent.run(\"Could you search in my drafts for the latest email?\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.2"
}
},
"nbformat": 4,
"nbformat_minor": 4
}
docs/extras/modules/agents/toolkits/jira.ipynb
-166
View File
@@ -1,166 +0,0 @@
{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"id": "245a954a",
"metadata": {},
"source": [
"# Jira\n",
"\n",
"This notebook goes over how to use the Jira tool.\n",
"The Jira tool allows agents to interact with a given Jira instance, performing actions such as searching for issues and creating issues, the tool wraps the atlassian-python-api library, for more see: https://atlassian-python-api.readthedocs.io/jira.html\n",
"\n",
"To use this tool, you must first set as environment variables:\n",
" JIRA_API_TOKEN\n",
" JIRA_USERNAME\n",
" JIRA_INSTANCE_URL"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "961b3689",
"metadata": {
"vscode": {
"languageId": "shellscript"
},
"ExecuteTime": {
"start_time": "2023-04-17T10:21:18.698672Z",
"end_time": "2023-04-17T10:21:20.168639Z"
}
},
"outputs": [],
"source": [
"%pip install atlassian-python-api"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "34bb5968",
"metadata": {
"ExecuteTime": {
"start_time": "2023-04-17T10:21:22.911233Z",
"end_time": "2023-04-17T10:21:23.730922Z"
}
},
"outputs": [],
"source": [
"import os\n",
"from langchain.agents import AgentType\n",
"from langchain.agents import initialize_agent\n",
"from langchain.agents.agent_toolkits.jira.toolkit import JiraToolkit\n",
"from langchain.llms import OpenAI\n",
"from langchain.utilities.jira import JiraAPIWrapper"
]
},
{
"cell_type": "code",
"execution_count": 4,
"outputs": [],
"source": [
"os.environ[\"JIRA_API_TOKEN\"] = \"abc\"\n",
"os.environ[\"JIRA_USERNAME\"] = \"123\"\n",
"os.environ[\"JIRA_INSTANCE_URL\"] = \"https://jira.atlassian.com\"\n",
"os.environ[\"OPENAI_API_KEY\"] = \"xyz\""
],
"metadata": {
"collapsed": false,
"ExecuteTime": {
"start_time": "2023-04-17T10:22:42.499447Z",
"end_time": "2023-04-17T10:22:42.505412Z"
}
},
"id": "b3050b55"
},
{
"cell_type": "code",
"execution_count": 5,
"id": "ac4910f8",
"metadata": {
"ExecuteTime": {
"start_time": "2023-04-17T10:22:44.664481Z",
"end_time": "2023-04-17T10:22:44.720538Z"
}
},
"outputs": [],
"source": [
"llm = OpenAI(temperature=0)\n",
"jira = JiraAPIWrapper()\n",
"toolkit = JiraToolkit.from_jira_api_wrapper(jira)\n",
"agent = initialize_agent(\n",
" toolkit.get_tools(), llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=True\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 9,
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3m I need to create an issue in project PW\n",
"Action: Create Issue\n",
"Action Input: {\"summary\": \"Make more fried rice\", \"description\": \"Reminder to make more fried rice\", \"issuetype\": {\"name\": \"Task\"}, \"priority\": {\"name\": \"Low\"}, \"project\": {\"key\": \"PW\"}}\u001b[0m\n",
"Observation: \u001b[38;5;200m\u001b[1;3mNone\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I now know the final answer\n",
"Final Answer: A new issue has been created in project PW with the summary \"Make more fried rice\" and description \"Reminder to make more fried rice\".\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": "'A new issue has been created in project PW with the summary \"Make more fried rice\" and description \"Reminder to make more fried rice\".'"
},
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"agent.run(\"make a new issue in project PW to remind me to make more fried rice\")"
],
"metadata": {
"collapsed": false,
"ExecuteTime": {
"start_time": "2023-04-17T10:23:33.662454Z",
"end_time": "2023-04-17T10:23:38.121883Z"
}
},
"id": "d5461370"
}
],
"metadata": {
"kernelspec": {
"display_name": ".venv",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.7"
},
"vscode": {
"interpreter": {
"hash": "53f3bc57609c7a84333bb558594977aa5b4026b1d6070b93987956689e367341"
}
}
},
"nbformat": 4,
"nbformat_minor": 5
}
docs/extras/modules/agents/toolkits/json.ipynb
-187
View File
@@ -1,187 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "85fb2c03-ab88-4c8c-97e3-a7f2954555ab",
"metadata": {},
"source": [
"# JSON Agent\n",
"\n",
"This notebook showcases an agent designed to interact with large JSON/dict objects. This is useful when you want to answer questions about a JSON blob that's too large to fit in the context window of an LLM. The agent is able to iteratively explore the blob to find what it needs to answer the user's question.\n",
"\n",
"In the below example, we are using the OpenAPI spec for the OpenAI API, which you can find [here](https://github.com/openai/openai-openapi/blob/master/openapi.yaml).\n",
"\n",
"We will use the JSON agent to answer some questions about the API spec."
]
},
{
"cell_type": "markdown",
"id": "893f90fd-f8f6-470a-a76d-1f200ba02e2f",
"metadata": {},
"source": [
"## Initialization"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "ff988466-c389-4ec6-b6ac-14364a537fd5",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"import os\n",
"import yaml\n",
"\n",
"from langchain.agents import create_json_agent, AgentExecutor\n",
"from langchain.agents.agent_toolkits import JsonToolkit\n",
"from langchain.chains import LLMChain\n",
"from langchain.llms.openai import OpenAI\n",
"from langchain.requests import TextRequestsWrapper\n",
"from langchain.tools.json.tool import JsonSpec"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "9ecd1ba0-3937-4359-a41e-68605f0596a1",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"with open(\"openai_openapi.yml\") as f:\n",
" data = yaml.load(f, Loader=yaml.FullLoader)\n",
"json_spec = JsonSpec(dict_=data, max_value_length=4000)\n",
"json_toolkit = JsonToolkit(spec=json_spec)\n",
"\n",
"json_agent_executor = create_json_agent(\n",
" llm=OpenAI(temperature=0), toolkit=json_toolkit, verbose=True\n",
")"
]
},
{
"cell_type": "markdown",
"id": "05cfcb24-4389-4b8f-ad9e-466e3fca8db0",
"metadata": {},
"source": [
"## Example: getting the required POST parameters for a request"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "faf13702-50f0-4d1b-b91f-48c750ccfd98",
"metadata": {
"tags": []
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3mAction: json_spec_list_keys\n",
"Action Input: data\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3m['openapi', 'info', 'servers', 'tags', 'paths', 'components', 'x-oaiMeta']\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I should look at the paths key to see what endpoints exist\n",
"Action: json_spec_list_keys\n",
"Action Input: data[\"paths\"]\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3m['/engines', '/engines/{engine_id}', '/completions', '/edits', '/images/generations', '/images/edits', '/images/variations', '/embeddings', '/engines/{engine_id}/search', '/files', '/files/{file_id}', '/files/{file_id}/content', '/answers', '/classifications', '/fine-tunes', '/fine-tunes/{fine_tune_id}', '/fine-tunes/{fine_tune_id}/cancel', '/fine-tunes/{fine_tune_id}/events', '/models', '/models/{model}', '/moderations']\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I should look at the /completions endpoint to see what parameters are required\n",
"Action: json_spec_list_keys\n",
"Action Input: data[\"paths\"][\"/completions\"]\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3m['post']\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I should look at the post key to see what parameters are required\n",
"Action: json_spec_list_keys\n",
"Action Input: data[\"paths\"][\"/completions\"][\"post\"]\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3m['operationId', 'tags', 'summary', 'requestBody', 'responses', 'x-oaiMeta']\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I should look at the requestBody key to see what parameters are required\n",
"Action: json_spec_list_keys\n",
"Action Input: data[\"paths\"][\"/completions\"][\"post\"][\"requestBody\"]\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3m['required', 'content']\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I should look at the required key to see what parameters are required\n",
"Action: json_spec_get_value\n",
"Action Input: data[\"paths\"][\"/completions\"][\"post\"][\"requestBody\"][\"required\"]\u001b[0m\n",
"Observation: \u001b[33;1m\u001b[1;3mTrue\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I should look at the content key to see what parameters are required\n",
"Action: json_spec_list_keys\n",
"Action Input: data[\"paths\"][\"/completions\"][\"post\"][\"requestBody\"][\"content\"]\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3m['application/json']\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I should look at the application/json key to see what parameters are required\n",
"Action: json_spec_list_keys\n",
"Action Input: data[\"paths\"][\"/completions\"][\"post\"][\"requestBody\"][\"content\"][\"application/json\"]\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3m['schema']\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I should look at the schema key to see what parameters are required\n",
"Action: json_spec_list_keys\n",
"Action Input: data[\"paths\"][\"/completions\"][\"post\"][\"requestBody\"][\"content\"][\"application/json\"][\"schema\"]\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3m['$ref']\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I should look at the $ref key to see what parameters are required\n",
"Action: json_spec_get_value\n",
"Action Input: data[\"paths\"][\"/completions\"][\"post\"][\"requestBody\"][\"content\"][\"application/json\"][\"schema\"][\"$ref\"]\u001b[0m\n",
"Observation: \u001b[33;1m\u001b[1;3m#/components/schemas/CreateCompletionRequest\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I should look at the CreateCompletionRequest schema to see what parameters are required\n",
"Action: json_spec_list_keys\n",
"Action Input: data[\"components\"][\"schemas\"][\"CreateCompletionRequest\"]\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3m['type', 'properties', 'required']\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I should look at the required key to see what parameters are required\n",
"Action: json_spec_get_value\n",
"Action Input: data[\"components\"][\"schemas\"][\"CreateCompletionRequest\"][\"required\"]\u001b[0m\n",
"Observation: \u001b[33;1m\u001b[1;3m['model']\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I now know the final answer\n",
"Final Answer: The required parameters in the request body to the /completions endpoint are 'model'.\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"\"The required parameters in the request body to the /completions endpoint are 'model'.\""
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"json_agent_executor.run(\n",
" \"What are the required parameters in the request body to the /completions endpoint?\"\n",
")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ba9c9d30",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.9"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
docs/extras/modules/agents/toolkits/multion.ipynb
-129
View File
@@ -1,129 +0,0 @@
{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"# Multion Toolkit\n",
"\n",
"This notebook walks you through connecting LangChain to the MultiOn Client in your browser\n",
"\n",
"To use this toolkit, you will need to add MultiOn Extension to your browser as explained in the [MultiOn for Chrome](https://multion.notion.site/Download-MultiOn-ddddcfe719f94ab182107ca2612c07a5)."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"!pip install --upgrade multion > /dev/null"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## MultiOn Setup\n",
"\n",
"Login to establish connection with your extension."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Authorize connection to your Browser extention\n",
"import multion \n",
"multion.login()\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Use Multion Toolkit within an Agent"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"from langchain.agents.agent_toolkits import create_multion_agent\n",
"from langchain.tools.multion.tool import MultionClientTool\n",
"from langchain.agents.agent_types import AgentType\n",
"from langchain.chat_models import ChatOpenAI"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"\n",
"agent_executor = create_multion_agent(\n",
" llm=ChatOpenAI(temperature=0),\n",
" tool=MultionClientTool(),\n",
" agent_type=AgentType.OPENAI_FUNCTIONS,\n",
" verbose=True\n",
")\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"agent.run(\"show me the weather today\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"agent.run(\n",
" \"Tweet about Elon Musk\"\n",
")"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.4"
}
},
"nbformat": 4,
"nbformat_minor": 4
}
docs/extras/modules/agents/toolkits/office365.ipynb
-246
View File
@@ -1,246 +0,0 @@
{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"# Office365 Toolkit\n",
"\n",
"This notebook walks through connecting LangChain to Office365 email and calendar.\n",
"\n",
"To use this toolkit, you will need to set up your credentials explained in the [Microsoft Graph authentication and authorization overview](https://learn.microsoft.com/en-us/graph/auth/). Once you've received a CLIENT_ID and CLIENT_SECRET, you can input them as environmental variables below."
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"!pip install --upgrade O365 > /dev/null\n",
"!pip install beautifulsoup4 > /dev/null # This is optional but is useful for parsing HTML messages"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Assign Environmental Variables\n",
"\n",
"The toolkit will read the CLIENT_ID and CLIENT_SECRET environmental variables to authenticate the user so you need to set them here. You will also need to set your OPENAI_API_KEY to use the agent later."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Set environmental variables here"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Create the Toolkit and Get Tools\n",
"\n",
"To start, you need to create the toolkit, so you can access its tools later."
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"tags": []
},
"outputs": [
{
"data": {
"text/plain": [
"[O365SearchEvents(name='events_search', description=\" Use this tool to search for the user's calendar events. The input must be the start and end datetimes for the search query. The output is a JSON list of all the events in the user's calendar between the start and end times. You can assume that the user can not schedule any meeting over existing meetings, and that the user is busy during meetings. Any times without events are free for the user. \", args_schema=<class 'langchain.tools.office365.events_search.SearchEventsInput'>, return_direct=False, verbose=False, callbacks=None, callback_manager=None, handle_tool_error=False, account=Account Client Id: f32a022c-3c4c-4d10-a9d8-f6a9a9055302),\n",
" O365CreateDraftMessage(name='create_email_draft', description='Use this tool to create a draft email with the provided message fields.', args_schema=<class 'langchain.tools.office365.create_draft_message.CreateDraftMessageSchema'>, return_direct=False, verbose=False, callbacks=None, callback_manager=None, handle_tool_error=False, account=Account Client Id: f32a022c-3c4c-4d10-a9d8-f6a9a9055302),\n",
" O365SearchEmails(name='messages_search', description='Use this tool to search for email messages. The input must be a valid Microsoft Graph v1.0 $search query. The output is a JSON list of the requested resource.', args_schema=<class 'langchain.tools.office365.messages_search.SearchEmailsInput'>, return_direct=False, verbose=False, callbacks=None, callback_manager=None, handle_tool_error=False, account=Account Client Id: f32a022c-3c4c-4d10-a9d8-f6a9a9055302),\n",
" O365SendEvent(name='send_event', description='Use this tool to create and send an event with the provided event fields.', args_schema=<class 'langchain.tools.office365.send_event.SendEventSchema'>, return_direct=False, verbose=False, callbacks=None, callback_manager=None, handle_tool_error=False, account=Account Client Id: f32a022c-3c4c-4d10-a9d8-f6a9a9055302),\n",
" O365SendMessage(name='send_email', description='Use this tool to send an email with the provided message fields.', args_schema=<class 'langchain.tools.office365.send_message.SendMessageSchema'>, return_direct=False, verbose=False, callbacks=None, callback_manager=None, handle_tool_error=False, account=Account Client Id: f32a022c-3c4c-4d10-a9d8-f6a9a9055302)]"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from langchain.agents.agent_toolkits import O365Toolkit\n",
"\n",
"toolkit = O365Toolkit()\n",
"tools = toolkit.get_tools()\n",
"tools"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Use within an Agent"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"from langchain import OpenAI\n",
"from langchain.agents import initialize_agent, AgentType"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"llm = OpenAI(temperature=0)\n",
"agent = initialize_agent(\n",
" tools=toolkit.get_tools(),\n",
" llm=llm,\n",
" verbose=False,\n",
" agent=AgentType.STRUCTURED_CHAT_ZERO_SHOT_REACT_DESCRIPTION,\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"tags": []
},
"outputs": [
{
"data": {
"text/plain": [
"'The draft email was created correctly.'"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"agent.run(\n",
" \"Create an email draft for me to edit of a letter from the perspective of a sentient parrot\"\n",
" \" who is looking to collaborate on some research with her\"\n",
" \" estranged friend, a cat. Under no circumstances may you send the message, however.\"\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {
"tags": []
},
"outputs": [
{
"data": {
"text/plain": [
"\"I found one draft in your drafts folder about collaboration. It was sent on 2023-06-16T18:22:17+0000 and the subject was 'Collaboration Request'.\""
]
},
"execution_count": 12,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"agent.run(\n",
" \"Could you search in my drafts folder and let me know if any of them are about collaboration?\"\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"/home/vscode/langchain-py-env/lib/python3.11/site-packages/O365/utils/windows_tz.py:639: PytzUsageWarning: The zone attribute is specific to pytz's interface; please migrate to a new time zone provider. For more details on how to do so, see https://pytz-deprecation-shim.readthedocs.io/en/latest/migration.html\n",
" iana_tz.zone if isinstance(iana_tz, tzinfo) else iana_tz)\n",
"/home/vscode/langchain-py-env/lib/python3.11/site-packages/O365/utils/utils.py:463: PytzUsageWarning: The zone attribute is specific to pytz's interface; please migrate to a new time zone provider. For more details on how to do so, see https://pytz-deprecation-shim.readthedocs.io/en/latest/migration.html\n",
" timezone = date_time.tzinfo.zone if date_time.tzinfo is not None else None\n"
]
},
{
"data": {
"text/plain": [
"'I have scheduled a meeting with a sentient parrot to discuss research collaborations on October 3, 2023 at 2 pm Easter Time. Please let me know if you need to make any changes.'"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"agent.run(\n",
" \"Can you schedule a 30 minute meeting with a sentient parrot to discuss research collaborations on October 3, 2023 at 2 pm Easter Time?\"\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"\"Yes, you have an event on October 3, 2023 with a sentient parrot. The event is titled 'Meeting with sentient parrot' and is scheduled from 6:00 PM to 6:30 PM.\""
]
},
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"agent.run(\n",
" \"Can you tell me if I have any events on October 3, 2023 in Eastern Time, and if so, tell me if any of them are with a sentient parrot?\"\n",
")"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.3"
}
},
"nbformat": 4,
"nbformat_minor": 4
}
docs/extras/modules/agents/toolkits/openapi.ipynb
-781
View File
@@ -1,781 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "85fb2c03-ab88-4c8c-97e3-a7f2954555ab",
"metadata": {},
"source": [
"# OpenAPI agents\n",
"\n",
"We can construct agents to consume arbitrary APIs, here APIs conformant to the OpenAPI/Swagger specification."
]
},
{
"cell_type": "markdown",
"id": "a389367b",
"metadata": {},
"source": [
"## 1st example: hierarchical planning agent\n",
"\n",
"In this example, we'll consider an approach called hierarchical planning, common in robotics and appearing in recent works for LLMs X robotics. We'll see it's a viable approach to start working with a massive API spec AND to assist with user queries that require multiple steps against the API.\n",
"\n",
"The idea is simple: to get coherent agent behavior over long sequences behavior & to save on tokens, we'll separate concerns: a \"planner\" will be responsible for what endpoints to call and a \"controller\" will be responsible for how to call them.\n",
"\n",
"In the initial implementation, the planner is an LLM chain that has the name and a short description for each endpoint in context. The controller is an LLM agent that is instantiated with documentation for only the endpoints for a particular plan. There's a lot left to get this working very robustly :)\n",
"\n",
"---"
]
},
{
"cell_type": "markdown",
"id": "4b6ecf6e",
"metadata": {},
"source": [
"### To start, let's collect some OpenAPI specs."
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "0adf3537",
"metadata": {},
"outputs": [],
"source": [
"import os, yaml"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "eb15cea0",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"--2023-03-31 15:45:56-- https://raw.githubusercontent.com/openai/openai-openapi/master/openapi.yaml\n",
"Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.110.133, 185.199.109.133, 185.199.111.133, ...\n",
"Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.110.133|:443... connected.\n",
"HTTP request sent, awaiting response... 200 OK\n",
"Length: 122995 (120K) [text/plain]\n",
"Saving to: ‘openapi.yaml’\n",
"\n",
"openapi.yaml 100%[===================>] 120.11K --.-KB/s in 0.01s \n",
"\n",
"2023-03-31 15:45:56 (10.4 MB/s) - ‘openapi.yaml’ saved [122995/122995]\n",
"\n",
"--2023-03-31 15:45:57-- https://www.klarna.com/us/shopping/public/openai/v0/api-docs\n",
"Resolving www.klarna.com (www.klarna.com)... 52.84.150.34, 52.84.150.46, 52.84.150.61, ...\n",
"Connecting to www.klarna.com (www.klarna.com)|52.84.150.34|:443... connected.\n",
"HTTP request sent, awaiting response... 200 OK\n",
"Length: unspecified [application/json]\n",
"Saving to: ‘api-docs’\n",
"\n",
"api-docs [ <=> ] 1.87K --.-KB/s in 0s \n",
"\n",
"2023-03-31 15:45:57 (261 MB/s) - ‘api-docs’ saved [1916]\n",
"\n",
"--2023-03-31 15:45:57-- https://raw.githubusercontent.com/APIs-guru/openapi-directory/main/APIs/spotify.com/1.0.0/openapi.yaml\n",
"Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.110.133, 185.199.109.133, 185.199.111.133, ...\n",
"Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.110.133|:443... connected.\n",
"HTTP request sent, awaiting response... 200 OK\n",
"Length: 286747 (280K) [text/plain]\n",
"Saving to: ‘openapi.yaml’\n",
"\n",
"openapi.yaml 100%[===================>] 280.03K --.-KB/s in 0.02s \n",
"\n",
"2023-03-31 15:45:58 (13.3 MB/s) - ‘openapi.yaml’ saved [286747/286747]\n",
"\n"
]
}
],
"source": [
"!wget https://raw.githubusercontent.com/openai/openai-openapi/master/openapi.yaml\n",
"!mv openapi.yaml openai_openapi.yaml\n",
"!wget https://www.klarna.com/us/shopping/public/openai/v0/api-docs\n",
"!mv api-docs klarna_openapi.yaml\n",
"!wget https://raw.githubusercontent.com/APIs-guru/openapi-directory/main/APIs/spotify.com/1.0.0/openapi.yaml\n",
"!mv openapi.yaml spotify_openapi.yaml"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "690a35bf",
"metadata": {},
"outputs": [],
"source": [
"from langchain.agents.agent_toolkits.openapi.spec import reduce_openapi_spec"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "69a8e1b9",
"metadata": {},
"outputs": [],
"source": [
"with open(\"openai_openapi.yaml\") as f:\n",
" raw_openai_api_spec = yaml.load(f, Loader=yaml.Loader)\n",
"openai_api_spec = reduce_openapi_spec(raw_openai_api_spec)\n",
"\n",
"with open(\"klarna_openapi.yaml\") as f:\n",
" raw_klarna_api_spec = yaml.load(f, Loader=yaml.Loader)\n",
"klarna_api_spec = reduce_openapi_spec(raw_klarna_api_spec)\n",
"\n",
"with open(\"spotify_openapi.yaml\") as f:\n",
" raw_spotify_api_spec = yaml.load(f, Loader=yaml.Loader)\n",
"spotify_api_spec = reduce_openapi_spec(raw_spotify_api_spec)"
]
},
{
"cell_type": "markdown",
"id": "ba833d49",
"metadata": {},
"source": [
"---\n",
"\n",
"We'll work with the Spotify API as one of the examples of a somewhat complex API. There's a bit of auth-related setup to do if you want to replicate this.\n",
"\n",
"- You'll have to set up an application in the Spotify developer console, documented [here](https://developer.spotify.com/documentation/general/guides/authorization/), to get credentials: `CLIENT_ID`, `CLIENT_SECRET`, and `REDIRECT_URI`.\n",
"- To get an access tokens (and keep them fresh), you can implement the oauth flows, or you can use `spotipy`. If you've set your Spotify creedentials as environment variables `SPOTIPY_CLIENT_ID`, `SPOTIPY_CLIENT_SECRET`, and `SPOTIPY_REDIRECT_URI`, you can use the helper functions below:"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "a82c2cfa",
"metadata": {},
"outputs": [],
"source": [
"import spotipy.util as util\n",
"from langchain.requests import RequestsWrapper\n",
"\n",
"\n",
"def construct_spotify_auth_headers(raw_spec: dict):\n",
" scopes = list(\n",
" raw_spec[\"components\"][\"securitySchemes\"][\"oauth_2_0\"][\"flows\"][\n",
" \"authorizationCode\"\n",
" ][\"scopes\"].keys()\n",
" )\n",
" access_token = util.prompt_for_user_token(scope=\",\".join(scopes))\n",
" return {\"Authorization\": f\"Bearer {access_token}\"}\n",
"\n",
"\n",
"# Get API credentials.\n",
"headers = construct_spotify_auth_headers(raw_spotify_api_spec)\n",
"requests_wrapper = RequestsWrapper(headers=headers)"
]
},
{
"cell_type": "markdown",
"id": "76349780",
"metadata": {},
"source": [
"### How big is this spec?"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "2a93271e",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"63"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"endpoints = [\n",
" (route, operation)\n",
" for route, operations in raw_spotify_api_spec[\"paths\"].items()\n",
" for operation in operations\n",
" if operation in [\"get\", \"post\"]\n",
"]\n",
"len(endpoints)"
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "eb829190",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"80326"
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"import tiktoken\n",
"\n",
"enc = tiktoken.encoding_for_model(\"text-davinci-003\")\n",
"\n",
"\n",
"def count_tokens(s):\n",
" return len(enc.encode(s))\n",
"\n",
"\n",
"count_tokens(yaml.dump(raw_spotify_api_spec))"
]
},
{
"cell_type": "markdown",
"id": "cbc4964e",
"metadata": {},
"source": [
"### Let's see some examples!\n",
"\n",
"Starting with GPT-4. (Some robustness iterations under way for GPT-3 family.)"
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "7f42ee84",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"/Users/jeremywelborn/src/langchain/langchain/llms/openai.py:169: UserWarning: You are trying to use a chat model. This way of initializing it is no longer supported. Instead, please use: `from langchain.chat_models import ChatOpenAI`\n",
" warnings.warn(\n",
"/Users/jeremywelborn/src/langchain/langchain/llms/openai.py:608: UserWarning: You are trying to use a chat model. This way of initializing it is no longer supported. Instead, please use: `from langchain.chat_models import ChatOpenAI`\n",
" warnings.warn(\n"
]
}
],
"source": [
"from langchain.llms.openai import OpenAI\n",
"from langchain.agents.agent_toolkits.openapi import planner\n",
"\n",
"llm = OpenAI(model_name=\"gpt-4\", temperature=0.0)"
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "38762cc0",
"metadata": {
"scrolled": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3mAction: api_planner\n",
"Action Input: I need to find the right API calls to create a playlist with the first song from Kind of Blue and name it Machine Blues\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3m1. GET /search to search for the album \"Kind of Blue\"\n",
"2. GET /albums/{id}/tracks to get the tracks from the \"Kind of Blue\" album\n",
"3. GET /me to get the current user's information\n",
"4. POST /users/{user_id}/playlists to create a new playlist named \"Machine Blues\" for the current user\n",
"5. POST /playlists/{playlist_id}/tracks to add the first song from \"Kind of Blue\" to the \"Machine Blues\" playlist\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3mI have the plan, now I need to execute the API calls.\n",
"Action: api_controller\n",
"Action Input: 1. GET /search to search for the album \"Kind of Blue\"\n",
"2. GET /albums/{id}/tracks to get the tracks from the \"Kind of Blue\" album\n",
"3. GET /me to get the current user's information\n",
"4. POST /users/{user_id}/playlists to create a new playlist named \"Machine Blues\" for the current user\n",
"5. POST /playlists/{playlist_id}/tracks to add the first song from \"Kind of Blue\" to the \"Machine Blues\" playlist\u001b[0m\n",
"\n",
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3mAction: requests_get\n",
"Action Input: {\"url\": \"https://api.spotify.com/v1/search?q=Kind%20of%20Blue&type=album\", \"output_instructions\": \"Extract the id of the first album in the search results\"}\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3m1weenld61qoidwYuZ1GESA\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3mAction: requests_get\n",
"Action Input: {\"url\": \"https://api.spotify.com/v1/albums/1weenld61qoidwYuZ1GESA/tracks\", \"output_instructions\": \"Extract the id of the first track in the album\"}\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3m7q3kkfAVpmcZ8g6JUThi3o\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3mAction: requests_get\n",
"Action Input: {\"url\": \"https://api.spotify.com/v1/me\", \"output_instructions\": \"Extract the id of the current user\"}\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3m22rhrz4m4kvpxlsb5hezokzwi\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3mAction: requests_post\n",
"Action Input: {\"url\": \"https://api.spotify.com/v1/users/22rhrz4m4kvpxlsb5hezokzwi/playlists\", \"data\": {\"name\": \"Machine Blues\"}, \"output_instructions\": \"Extract the id of the created playlist\"}\u001b[0m\n",
"Observation: \u001b[33;1m\u001b[1;3m7lzoEi44WOISnFYlrAIqyX\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3mAction: requests_post\n",
"Action Input: {\"url\": \"https://api.spotify.com/v1/playlists/7lzoEi44WOISnFYlrAIqyX/tracks\", \"data\": {\"uris\": [\"spotify:track:7q3kkfAVpmcZ8g6JUThi3o\"]}, \"output_instructions\": \"Confirm that the track was added to the playlist\"}\n",
"\u001b[0m\n",
"Observation: \u001b[33;1m\u001b[1;3mThe track was added to the playlist, confirmed by the snapshot_id: MiwxODMxNTMxZTFlNzg3ZWFlZmMxYTlmYWQyMDFiYzUwNDEwMTAwZmE1.\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3mI am finished executing the plan.\n",
"Final Answer: The first song from the \"Kind of Blue\" album has been added to the \"Machine Blues\" playlist.\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n",
"\n",
"Observation: \u001b[33;1m\u001b[1;3mThe first song from the \"Kind of Blue\" album has been added to the \"Machine Blues\" playlist.\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3mI am finished executing the plan and have created the playlist with the first song from Kind of Blue.\n",
"Final Answer: I have created a playlist called \"Machine Blues\" with the first song from the \"Kind of Blue\" album.\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"'I have created a playlist called \"Machine Blues\" with the first song from the \"Kind of Blue\" album.'"
]
},
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"spotify_agent = planner.create_openapi_agent(spotify_api_spec, requests_wrapper, llm)\n",
"user_query = (\n",
" \"make me a playlist with the first song from kind of blue. call it machine blues.\"\n",
")\n",
"spotify_agent.run(user_query)"
]
},
{
"cell_type": "code",
"execution_count": 12,
"id": "96184181",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3mAction: api_planner\n",
"Action Input: I need to find the right API calls to get a blues song recommendation for the user\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3m1. GET /me to get the current user's information\n",
"2. GET /recommendations/available-genre-seeds to retrieve a list of available genres\n",
"3. GET /recommendations with the seed_genre parameter set to \"blues\" to get a blues song recommendation for the user\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3mI have the plan, now I need to execute the API calls.\n",
"Action: api_controller\n",
"Action Input: 1. GET /me to get the current user's information\n",
"2. GET /recommendations/available-genre-seeds to retrieve a list of available genres\n",
"3. GET /recommendations with the seed_genre parameter set to \"blues\" to get a blues song recommendation for the user\u001b[0m\n",
"\n",
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3mAction: requests_get\n",
"Action Input: {\"url\": \"https://api.spotify.com/v1/me\", \"output_instructions\": \"Extract the user's id and username\"}\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3mID: 22rhrz4m4kvpxlsb5hezokzwi, Username: Jeremy Welborn\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3mAction: requests_get\n",
"Action Input: {\"url\": \"https://api.spotify.com/v1/recommendations/available-genre-seeds\", \"output_instructions\": \"Extract the list of available genres\"}\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3macoustic, afrobeat, alt-rock, alternative, ambient, anime, black-metal, bluegrass, blues, bossanova, brazil, breakbeat, british, cantopop, chicago-house, children, chill, classical, club, comedy, country, dance, dancehall, death-metal, deep-house, detroit-techno, disco, disney, drum-and-bass, dub, dubstep, edm, electro, electronic, emo, folk, forro, french, funk, garage, german, gospel, goth, grindcore, groove, grunge, guitar, happy, hard-rock, hardcore, hardstyle, heavy-metal, hip-hop, holidays, honky-tonk, house, idm, indian, indie, indie-pop, industrial, iranian, j-dance, j-idol, j-pop, j-rock, jazz, k-pop, kids, latin, latino, malay, mandopop, metal, metal-misc, metalcore, minimal-techno, movies, mpb, new-age, new-release, opera, pagode, party, philippines-\u001b[0m\n",
"Thought:"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"Retrying langchain.llms.openai.completion_with_retry.<locals>._completion_with_retry in 4.0 seconds as it raised RateLimitError: That model is currently overloaded with other requests. You can retry your request, or contact us through our help center at help.openai.com if the error persists. (Please include the request ID 2167437a0072228238f3c0c5b3882764 in your message.).\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\u001b[32;1m\u001b[1;3mAction: requests_get\n",
"Action Input: {\"url\": \"https://api.spotify.com/v1/recommendations?seed_genres=blues\", \"output_instructions\": \"Extract the list of recommended tracks with their ids and names\"}\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3m[\n",
" {\n",
" id: '03lXHmokj9qsXspNsPoirR',\n",
" name: 'Get Away Jordan'\n",
" }\n",
"]\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3mI am finished executing the plan.\n",
"Final Answer: The recommended blues song for user Jeremy Welborn (ID: 22rhrz4m4kvpxlsb5hezokzwi) is \"Get Away Jordan\" with the track ID: 03lXHmokj9qsXspNsPoirR.\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n",
"\n",
"Observation: \u001b[33;1m\u001b[1;3mThe recommended blues song for user Jeremy Welborn (ID: 22rhrz4m4kvpxlsb5hezokzwi) is \"Get Away Jordan\" with the track ID: 03lXHmokj9qsXspNsPoirR.\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3mI am finished executing the plan and have the information the user asked for.\n",
"Final Answer: The recommended blues song for you is \"Get Away Jordan\" with the track ID: 03lXHmokj9qsXspNsPoirR.\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"'The recommended blues song for you is \"Get Away Jordan\" with the track ID: 03lXHmokj9qsXspNsPoirR.'"
]
},
"execution_count": 12,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"user_query = \"give me a song I'd like, make it blues-ey\"\n",
"spotify_agent.run(user_query)"
]
},
{
"cell_type": "markdown",
"id": "d5317926",
"metadata": {},
"source": [
"#### Try another API.\n"
]
},
{
"cell_type": "code",
"execution_count": 23,
"id": "06c3d6a8",
"metadata": {},
"outputs": [],
"source": [
"headers = {\"Authorization\": f\"Bearer {os.getenv('OPENAI_API_KEY')}\"}\n",
"openai_requests_wrapper = RequestsWrapper(headers=headers)"
]
},
{
"cell_type": "code",
"execution_count": 28,
"id": "3a9cc939",
"metadata": {
"scrolled": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3mAction: api_planner\n",
"Action Input: I need to find the right API calls to generate a short piece of advice\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3m1. GET /engines to retrieve the list of available engines\n",
"2. POST /completions with the selected engine and a prompt for generating a short piece of advice\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3mI have the plan, now I need to execute the API calls.\n",
"Action: api_controller\n",
"Action Input: 1. GET /engines to retrieve the list of available engines\n",
"2. POST /completions with the selected engine and a prompt for generating a short piece of advice\u001b[0m\n",
"\n",
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3mAction: requests_get\n",
"Action Input: {\"url\": \"https://api.openai.com/v1/engines\", \"output_instructions\": \"Extract the ids of the engines\"}\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3mbabbage, davinci, text-davinci-edit-001, babbage-code-search-code, text-similarity-babbage-001, code-davinci-edit-001, text-davinci-001, ada, babbage-code-search-text, babbage-similarity, whisper-1, code-search-babbage-text-001, text-curie-001, code-search-babbage-code-001, text-ada-001, text-embedding-ada-002, text-similarity-ada-001, curie-instruct-beta, ada-code-search-code, ada-similarity, text-davinci-003, code-search-ada-text-001, text-search-ada-query-001, davinci-search-document, ada-code-search-text, text-search-ada-doc-001, davinci-instruct-beta, text-similarity-curie-001, code-search-ada-code-001\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3mI will use the \"davinci\" engine to generate a short piece of advice.\n",
"Action: requests_post\n",
"Action Input: {\"url\": \"https://api.openai.com/v1/completions\", \"data\": {\"engine\": \"davinci\", \"prompt\": \"Give me a short piece of advice on how to be more productive.\"}, \"output_instructions\": \"Extract the text from the first choice\"}\u001b[0m\n",
"Observation: \u001b[33;1m\u001b[1;3m\"you must provide a model parameter\"\u001b[0m\n",
"Thought:!! Could not _extract_tool_and_input from \"I cannot finish executing the plan without knowing how to provide the model parameter correctly.\" in _get_next_action\n",
"\u001b[32;1m\u001b[1;3mI cannot finish executing the plan without knowing how to provide the model parameter correctly.\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n",
"\n",
"Observation: \u001b[33;1m\u001b[1;3mI need more information on how to provide the model parameter correctly in the POST request to generate a short piece of advice.\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3mI need to adjust my plan to include the model parameter in the POST request.\n",
"Action: api_planner\n",
"Action Input: I need to find the right API calls to generate a short piece of advice, including the model parameter in the POST request\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3m1. GET /models to retrieve the list of available models\n",
"2. Choose a suitable model from the list\n",
"3. POST /completions with the chosen model as a parameter to generate a short piece of advice\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3mI have an updated plan, now I need to execute the API calls.\n",
"Action: api_controller\n",
"Action Input: 1. GET /models to retrieve the list of available models\n",
"2. Choose a suitable model from the list\n",
"3. POST /completions with the chosen model as a parameter to generate a short piece of advice\u001b[0m\n",
"\n",
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3mAction: requests_get\n",
"Action Input: {\"url\": \"https://api.openai.com/v1/models\", \"output_instructions\": \"Extract the ids of the available models\"}\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3mbabbage, davinci, text-davinci-edit-001, babbage-code-search-code, text-similarity-babbage-001, code-davinci-edit-001, text-davinci-edit-001, ada\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3mAction: requests_post\n",
"Action Input: {\"url\": \"https://api.openai.com/v1/completions\", \"data\": {\"model\": \"davinci\", \"prompt\": \"Give me a short piece of advice on how to improve communication skills.\"}, \"output_instructions\": \"Extract the text from the first choice\"}\u001b[0m\n",
"Observation: \u001b[33;1m\u001b[1;3m\"I'd like to broaden my horizon.\\n\\nI was trying to\"\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3mI cannot finish executing the plan without knowing some other information.\n",
"\n",
"Final Answer: The generated text is not a piece of advice on improving communication skills. I would need to retry the API call with a different prompt or model to get a more relevant response.\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n",
"\n",
"Observation: \u001b[33;1m\u001b[1;3mThe generated text is not a piece of advice on improving communication skills. I would need to retry the API call with a different prompt or model to get a more relevant response.\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3mI need to adjust my plan to include a more specific prompt for generating a short piece of advice on improving communication skills.\n",
"Action: api_planner\n",
"Action Input: I need to find the right API calls to generate a short piece of advice on improving communication skills, including the model parameter in the POST request\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3m1. GET /models to retrieve the list of available models\n",
"2. Choose a suitable model for generating text (e.g., text-davinci-002)\n",
"3. POST /completions with the chosen model and a prompt related to improving communication skills to generate a short piece of advice\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3mI have an updated plan, now I need to execute the API calls.\n",
"Action: api_controller\n",
"Action Input: 1. GET /models to retrieve the list of available models\n",
"2. Choose a suitable model for generating text (e.g., text-davinci-002)\n",
"3. POST /completions with the chosen model and a prompt related to improving communication skills to generate a short piece of advice\u001b[0m\n",
"\n",
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3mAction: requests_get\n",
"Action Input: {\"url\": \"https://api.openai.com/v1/models\", \"output_instructions\": \"Extract the names of the models\"}\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3mbabbage, davinci, text-davinci-edit-001, babbage-code-search-code, text-similarity-babbage-001, code-davinci-edit-001, text-davinci-edit-001, ada\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3mAction: requests_post\n",
"Action Input: {\"url\": \"https://api.openai.com/v1/completions\", \"data\": {\"model\": \"text-davinci-002\", \"prompt\": \"Give a short piece of advice on how to improve communication skills\"}, \"output_instructions\": \"Extract the text from the first choice\"}\u001b[0m\n",
"Observation: \u001b[33;1m\u001b[1;3m\"Some basic advice for improving communication skills would be to make sure to listen\"\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3mI am finished executing the plan.\n",
"\n",
"Final Answer: Some basic advice for improving communication skills would be to make sure to listen.\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n",
"\n",
"Observation: \u001b[33;1m\u001b[1;3mSome basic advice for improving communication skills would be to make sure to listen.\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3mI am finished executing the plan and have the information the user asked for.\n",
"Final Answer: A short piece of advice for improving communication skills is to make sure to listen.\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"'A short piece of advice for improving communication skills is to make sure to listen.'"
]
},
"execution_count": 28,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Meta!\n",
"llm = OpenAI(model_name=\"gpt-4\", temperature=0.25)\n",
"openai_agent = planner.create_openapi_agent(\n",
" openai_api_spec, openai_requests_wrapper, llm\n",
")\n",
"user_query = \"generate a short piece of advice\"\n",
"openai_agent.run(user_query)"
]
},
{
"cell_type": "markdown",
"id": "f32bc6ec",
"metadata": {},
"source": [
"Takes awhile to get there!"
]
},
{
"cell_type": "markdown",
"id": "461229e4",
"metadata": {},
"source": [
"## 2nd example: \"json explorer\" agent\n",
"\n",
"Here's an agent that's not particularly practical, but neat! The agent has access to 2 toolkits. One comprises tools to interact with json: one tool to list the keys of a json object and another tool to get the value for a given key. The other toolkit comprises `requests` wrappers to send GET and POST requests. This agent consumes a lot calls to the language model, but does a surprisingly decent job.\n"
]
},
{
"cell_type": "code",
"execution_count": 29,
"id": "f8dfa1d3",
"metadata": {},
"outputs": [],
"source": [
"from langchain.agents import create_openapi_agent\n",
"from langchain.agents.agent_toolkits import OpenAPIToolkit\n",
"from langchain.llms.openai import OpenAI\n",
"from langchain.requests import TextRequestsWrapper\n",
"from langchain.tools.json.tool import JsonSpec"
]
},
{
"cell_type": "code",
"execution_count": 32,
"id": "9ecd1ba0-3937-4359-a41e-68605f0596a1",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"with open(\"openai_openapi.yaml\") as f:\n",
" data = yaml.load(f, Loader=yaml.FullLoader)\n",
"json_spec = JsonSpec(dict_=data, max_value_length=4000)\n",
"\n",
"\n",
"openapi_toolkit = OpenAPIToolkit.from_llm(\n",
" OpenAI(temperature=0), json_spec, openai_requests_wrapper, verbose=True\n",
")\n",
"openapi_agent_executor = create_openapi_agent(\n",
" llm=OpenAI(temperature=0), toolkit=openapi_toolkit, verbose=True\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 33,
"id": "548db7f7-337b-4ba8-905c-e7fd58c01799",
"metadata": {
"tags": []
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3mAction: json_explorer\n",
"Action Input: What is the base url for the API?\u001b[0m\n",
"\n",
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3mAction: json_spec_list_keys\n",
"Action Input: data\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3m['openapi', 'info', 'servers', 'tags', 'paths', 'components', 'x-oaiMeta']\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I should look at the servers key to see what the base url is\n",
"Action: json_spec_list_keys\n",
"Action Input: data[\"servers\"][0]\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3mValueError('Value at path `data[\"servers\"][0]` is not a dict, get the value directly.')\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I should get the value of the servers key\n",
"Action: json_spec_get_value\n",
"Action Input: data[\"servers\"][0]\u001b[0m\n",
"Observation: \u001b[33;1m\u001b[1;3m{'url': 'https://api.openai.com/v1'}\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I now know the base url for the API\n",
"Final Answer: The base url for the API is https://api.openai.com/v1\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n",
"\n",
"Observation: \u001b[33;1m\u001b[1;3mThe base url for the API is https://api.openai.com/v1\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I should find the path for the /completions endpoint.\n",
"Action: json_explorer\n",
"Action Input: What is the path for the /completions endpoint?\u001b[0m\n",
"\n",
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3mAction: json_spec_list_keys\n",
"Action Input: data\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3m['openapi', 'info', 'servers', 'tags', 'paths', 'components', 'x-oaiMeta']\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I should look at the paths key to see what endpoints exist\n",
"Action: json_spec_list_keys\n",
"Action Input: data[\"paths\"]\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3m['/engines', '/engines/{engine_id}', '/completions', '/chat/completions', '/edits', '/images/generations', '/images/edits', '/images/variations', '/embeddings', '/audio/transcriptions', '/audio/translations', '/engines/{engine_id}/search', '/files', '/files/{file_id}', '/files/{file_id}/content', '/answers', '/classifications', '/fine-tunes', '/fine-tunes/{fine_tune_id}', '/fine-tunes/{fine_tune_id}/cancel', '/fine-tunes/{fine_tune_id}/events', '/models', '/models/{model}', '/moderations']\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I now know the path for the /completions endpoint\n",
"Final Answer: The path for the /completions endpoint is data[\"paths\"][2]\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n",
"\n",
"Observation: \u001b[33;1m\u001b[1;3mThe path for the /completions endpoint is data[\"paths\"][2]\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I should find the required parameters for the POST request.\n",
"Action: json_explorer\n",
"Action Input: What are the required parameters for a POST request to the /completions endpoint?\u001b[0m\n",
"\n",
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3mAction: json_spec_list_keys\n",
"Action Input: data\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3m['openapi', 'info', 'servers', 'tags', 'paths', 'components', 'x-oaiMeta']\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I should look at the paths key to see what endpoints exist\n",
"Action: json_spec_list_keys\n",
"Action Input: data[\"paths\"]\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3m['/engines', '/engines/{engine_id}', '/completions', '/chat/completions', '/edits', '/images/generations', '/images/edits', '/images/variations', '/embeddings', '/audio/transcriptions', '/audio/translations', '/engines/{engine_id}/search', '/files', '/files/{file_id}', '/files/{file_id}/content', '/answers', '/classifications', '/fine-tunes', '/fine-tunes/{fine_tune_id}', '/fine-tunes/{fine_tune_id}/cancel', '/fine-tunes/{fine_tune_id}/events', '/models', '/models/{model}', '/moderations']\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I should look at the /completions endpoint to see what parameters are required\n",
"Action: json_spec_list_keys\n",
"Action Input: data[\"paths\"][\"/completions\"]\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3m['post']\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I should look at the post key to see what parameters are required\n",
"Action: json_spec_list_keys\n",
"Action Input: data[\"paths\"][\"/completions\"][\"post\"]\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3m['operationId', 'tags', 'summary', 'requestBody', 'responses', 'x-oaiMeta']\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I should look at the requestBody key to see what parameters are required\n",
"Action: json_spec_list_keys\n",
"Action Input: data[\"paths\"][\"/completions\"][\"post\"][\"requestBody\"]\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3m['required', 'content']\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I should look at the content key to see what parameters are required\n",
"Action: json_spec_list_keys\n",
"Action Input: data[\"paths\"][\"/completions\"][\"post\"][\"requestBody\"][\"content\"]\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3m['application/json']\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I should look at the application/json key to see what parameters are required\n",
"Action: json_spec_list_keys\n",
"Action Input: data[\"paths\"][\"/completions\"][\"post\"][\"requestBody\"][\"content\"][\"application/json\"]\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3m['schema']\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I should look at the schema key to see what parameters are required\n",
"Action: json_spec_list_keys\n",
"Action Input: data[\"paths\"][\"/completions\"][\"post\"][\"requestBody\"][\"content\"][\"application/json\"][\"schema\"]\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3m['$ref']\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I should look at the $ref key to see what parameters are required\n",
"Action: json_spec_list_keys\n",
"Action Input: data[\"paths\"][\"/completions\"][\"post\"][\"requestBody\"][\"content\"][\"application/json\"][\"schema\"][\"$ref\"]\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3mValueError('Value at path `data[\"paths\"][\"/completions\"][\"post\"][\"requestBody\"][\"content\"][\"application/json\"][\"schema\"][\"$ref\"]` is not a dict, get the value directly.')\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I should look at the $ref key to get the value directly\n",
"Action: json_spec_get_value\n",
"Action Input: data[\"paths\"][\"/completions\"][\"post\"][\"requestBody\"][\"content\"][\"application/json\"][\"schema\"][\"$ref\"]\u001b[0m\n",
"Observation: \u001b[33;1m\u001b[1;3m#/components/schemas/CreateCompletionRequest\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I should look at the CreateCompletionRequest schema to see what parameters are required\n",
"Action: json_spec_list_keys\n",
"Action Input: data[\"components\"][\"schemas\"][\"CreateCompletionRequest\"]\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3m['type', 'properties', 'required']\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I should look at the required key to see what parameters are required\n",
"Action: json_spec_get_value\n",
"Action Input: data[\"components\"][\"schemas\"][\"CreateCompletionRequest\"][\"required\"]\u001b[0m\n",
"Observation: \u001b[33;1m\u001b[1;3m['model']\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I now know the final answer\n",
"Final Answer: The required parameters for a POST request to the /completions endpoint are 'model'.\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n",
"\n",
"Observation: \u001b[33;1m\u001b[1;3mThe required parameters for a POST request to the /completions endpoint are 'model'.\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I now know the parameters needed to make the request.\n",
"Action: requests_post\n",
"Action Input: { \"url\": \"https://api.openai.com/v1/completions\", \"data\": { \"model\": \"davinci\", \"prompt\": \"tell me a joke\" } }\u001b[0m\n",
"Observation: \u001b[33;1m\u001b[1;3m{\"id\":\"cmpl-70Ivzip3dazrIXU8DSVJGzFJj2rdv\",\"object\":\"text_completion\",\"created\":1680307139,\"model\":\"davinci\",\"choices\":[{\"text\":\" with mummy not there”\\n\\nYou dig deep and come up with,\",\"index\":0,\"logprobs\":null,\"finish_reason\":\"length\"}],\"usage\":{\"prompt_tokens\":4,\"completion_tokens\":16,\"total_tokens\":20}}\n",
"\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I now know the final answer.\n",
"Final Answer: The response of the POST request is {\"id\":\"cmpl-70Ivzip3dazrIXU8DSVJGzFJj2rdv\",\"object\":\"text_completion\",\"created\":1680307139,\"model\":\"davinci\",\"choices\":[{\"text\":\" with mummy not there”\\n\\nYou dig deep and come up with,\",\"index\":0,\"logprobs\":null,\"finish_reason\":\"length\"}],\"usage\":{\"prompt_tokens\":4,\"completion_tokens\":16,\"total_tokens\":20}}\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"'The response of the POST request is {\"id\":\"cmpl-70Ivzip3dazrIXU8DSVJGzFJj2rdv\",\"object\":\"text_completion\",\"created\":1680307139,\"model\":\"davinci\",\"choices\":[{\"text\":\" with mummy not there”\\\\n\\\\nYou dig deep and come up with,\",\"index\":0,\"logprobs\":null,\"finish_reason\":\"length\"}],\"usage\":{\"prompt_tokens\":4,\"completion_tokens\":16,\"total_tokens\":20}}'"
]
},
"execution_count": 33,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"openapi_agent_executor.run(\n",
" \"Make a post request to openai /completions. The prompt should be 'tell me a joke.'\"\n",
")"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.1"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
docs/extras/modules/agents/toolkits/openapi_nla.ipynb
-428
View File
@@ -1,428 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "c7ad998d",
"metadata": {},
"source": [
"# Natural Language APIs\n",
"\n",
"Natural Language API Toolkits (NLAToolkits) permit LangChain Agents to efficiently plan and combine calls across endpoints. This notebook demonstrates a sample composition of the Speak, Klarna, and Spoonacluar APIs.\n",
"\n",
"For a detailed walkthrough of the OpenAPI chains wrapped within the NLAToolkit, see the [OpenAPI Operation Chain](/docs/modules/chains/additional/openapi.html) notebook.\n",
"\n",
"### First, import dependencies and load the LLM"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "6593f793",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"from typing import List, Optional\n",
"from langchain.chains import LLMChain\n",
"from langchain.llms import OpenAI\n",
"from langchain.prompts import PromptTemplate\n",
"from langchain.requests import Requests\n",
"from langchain.tools import APIOperation, OpenAPISpec\n",
"from langchain.agents import AgentType, Tool, initialize_agent\n",
"from langchain.agents.agent_toolkits import NLAToolkit"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "dd720860",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"# Select the LLM to use. Here, we use text-davinci-003\n",
"llm = OpenAI(\n",
" temperature=0, max_tokens=700\n",
") # You can swap between different core LLM's here."
]
},
{
"cell_type": "markdown",
"id": "4cadac9d",
"metadata": {
"tags": []
},
"source": [
"### Next, load the Natural Language API Toolkits"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "6b208ab0",
"metadata": {
"scrolled": true,
"tags": []
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Attempting to load an OpenAPI 3.0.1 spec. This may result in degraded performance. Convert your OpenAPI spec to 3.1.* spec for better support.\n",
"Attempting to load an OpenAPI 3.0.1 spec. This may result in degraded performance. Convert your OpenAPI spec to 3.1.* spec for better support.\n",
"Attempting to load an OpenAPI 3.0.1 spec. This may result in degraded performance. Convert your OpenAPI spec to 3.1.* spec for better support.\n"
]
}
],
"source": [
"speak_toolkit = NLAToolkit.from_llm_and_url(llm, \"https://api.speak.com/openapi.yaml\")\n",
"klarna_toolkit = NLAToolkit.from_llm_and_url(\n",
" llm, \"https://www.klarna.com/us/shopping/public/openai/v0/api-docs/\"\n",
")"
]
},
{
"cell_type": "markdown",
"id": "16c7336f",
"metadata": {},
"source": [
"### Create the Agent"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "730a0dc2-b4d0-46d5-a1e9-583803220973",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"# Slightly tweak the instructions from the default agent\n",
"openapi_format_instructions = \"\"\"Use the following format:\n",
"\n",
"Question: the input question you must answer\n",
"Thought: you should always think about what to do\n",
"Action: the action to take, should be one of [{tool_names}]\n",
"Action Input: what to instruct the AI Action representative.\n",
"Observation: The Agent's response\n",
"... (this Thought/Action/Action Input/Observation can repeat N times)\n",
"Thought: I now know the final answer. User can't see any of my observations, API responses, links, or tools.\n",
"Final Answer: the final answer to the original input question with the right amount of detail\n",
"\n",
"When responding with your Final Answer, remember that the person you are responding to CANNOT see any of your Thought/Action/Action Input/Observations, so if there is any relevant information there you need to include it explicitly in your response.\"\"\""
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "40a979c3",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"natural_language_tools = speak_toolkit.get_tools() + klarna_toolkit.get_tools()\n",
"mrkl = initialize_agent(\n",
" natural_language_tools,\n",
" llm,\n",
" agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION,\n",
" verbose=True,\n",
" agent_kwargs={\"format_instructions\": openapi_format_instructions},\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "794380ba",
"metadata": {
"tags": []
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3m I need to find out what kind of Italian clothes are available\n",
"Action: Open_AI_Klarna_product_Api.productsUsingGET\n",
"Action Input: Italian clothes\u001b[0m\n",
"Observation: \u001b[31;1m\u001b[1;3mThe API response contains two products from the Alé brand in Italian Blue. The first is the Alé Colour Block Short Sleeve Jersey Men - Italian Blue, which costs $86.49, and the second is the Alé Dolid Flash Jersey Men - Italian Blue, which costs $40.00.\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I now know what kind of Italian clothes are available and how much they cost.\n",
"Final Answer: You can buy two products from the Alé brand in Italian Blue for your end of year party. The Alé Colour Block Short Sleeve Jersey Men - Italian Blue costs $86.49, and the Alé Dolid Flash Jersey Men - Italian Blue costs $40.00.\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"'You can buy two products from the Alé brand in Italian Blue for your end of year party. The Alé Colour Block Short Sleeve Jersey Men - Italian Blue costs $86.49, and the Alé Dolid Flash Jersey Men - Italian Blue costs $40.00.'"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"mrkl.run(\n",
" \"I have an end of year party for my Italian class and have to buy some Italian clothes for it\"\n",
")"
]
},
{
"cell_type": "markdown",
"id": "c61d92a8",
"metadata": {},
"source": [
"### Using Auth + Adding more Endpoints\n",
"\n",
"Some endpoints may require user authentication via things like access tokens. Here we show how to pass in the authentication information via the `Requests` wrapper object.\n",
"\n",
"Since each NLATool exposes a concisee natural language interface to its wrapped API, the top level conversational agent has an easier job incorporating each endpoint to satisfy a user's request."
]
},
{
"cell_type": "markdown",
"id": "f0d132cc",
"metadata": {},
"source": [
"**Adding the Spoonacular endpoints.**\n",
"\n",
"1. Go to the [Spoonacular API Console](https://spoonacular.com/food-api/console#Profile) and make a free account.\n",
"2. Click on `Profile` and copy your API key below."
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "c2368b9c",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"spoonacular_api_key = \"\" # Copy from the API Console"
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "fbd97c28-fef6-41b5-9600-a9611a32bfb3",
"metadata": {
"tags": []
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Attempting to load an OpenAPI 3.0.0 spec. This may result in degraded performance. Convert your OpenAPI spec to 3.1.* spec for better support.\n",
"Unsupported APIPropertyLocation \"header\" for parameter Content-Type. Valid values are ['path', 'query'] Ignoring optional parameter\n",
"Unsupported APIPropertyLocation \"header\" for parameter Accept. Valid values are ['path', 'query'] Ignoring optional parameter\n",
"Unsupported APIPropertyLocation \"header\" for parameter Content-Type. Valid values are ['path', 'query'] Ignoring optional parameter\n",
"Unsupported APIPropertyLocation \"header\" for parameter Accept. Valid values are ['path', 'query'] Ignoring optional parameter\n",
"Unsupported APIPropertyLocation \"header\" for parameter Content-Type. Valid values are ['path', 'query'] Ignoring optional parameter\n",
"Unsupported APIPropertyLocation \"header\" for parameter Accept. Valid values are ['path', 'query'] Ignoring optional parameter\n",
"Unsupported APIPropertyLocation \"header\" for parameter Content-Type. Valid values are ['path', 'query'] Ignoring optional parameter\n",
"Unsupported APIPropertyLocation \"header\" for parameter Accept. Valid values are ['path', 'query'] Ignoring optional parameter\n",
"Unsupported APIPropertyLocation \"header\" for parameter Content-Type. Valid values are ['path', 'query'] Ignoring optional parameter\n",
"Unsupported APIPropertyLocation \"header\" for parameter Content-Type. Valid values are ['path', 'query'] Ignoring optional parameter\n",
"Unsupported APIPropertyLocation \"header\" for parameter Content-Type. Valid values are ['path', 'query'] Ignoring optional parameter\n",
"Unsupported APIPropertyLocation \"header\" for parameter Content-Type. Valid values are ['path', 'query'] Ignoring optional parameter\n",
"Unsupported APIPropertyLocation \"header\" for parameter Accept. Valid values are ['path', 'query'] Ignoring optional parameter\n",
"Unsupported APIPropertyLocation \"header\" for parameter Content-Type. Valid values are ['path', 'query'] Ignoring optional parameter\n",
"Unsupported APIPropertyLocation \"header\" for parameter Accept. Valid values are ['path', 'query'] Ignoring optional parameter\n",
"Unsupported APIPropertyLocation \"header\" for parameter Accept. Valid values are ['path', 'query'] Ignoring optional parameter\n",
"Unsupported APIPropertyLocation \"header\" for parameter Accept. Valid values are ['path', 'query'] Ignoring optional parameter\n",
"Unsupported APIPropertyLocation \"header\" for parameter Content-Type. Valid values are ['path', 'query'] Ignoring optional parameter\n"
]
}
],
"source": [
"requests = Requests(headers={\"x-api-key\": spoonacular_api_key})\n",
"spoonacular_toolkit = NLAToolkit.from_llm_and_url(\n",
" llm,\n",
" \"https://spoonacular.com/application/frontend/downloads/spoonacular-openapi-3.json\",\n",
" requests=requests,\n",
" max_text_length=1800, # If you want to truncate the response text\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "81a6edac",
"metadata": {
"scrolled": true,
"tags": []
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"34 tools loaded.\n"
]
}
],
"source": [
"natural_language_api_tools = (\n",
" speak_toolkit.get_tools()\n",
" + klarna_toolkit.get_tools()\n",
" + spoonacular_toolkit.get_tools()[:30]\n",
")\n",
"print(f\"{len(natural_language_api_tools)} tools loaded.\")"
]
},
{
"cell_type": "code",
"execution_count": 10,
"id": "831f772d-5cd1-4467-b494-a3172af2ff48",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"# Create an agent with the new tools\n",
"mrkl = initialize_agent(\n",
" natural_language_api_tools,\n",
" llm,\n",
" agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION,\n",
" verbose=True,\n",
" agent_kwargs={\"format_instructions\": openapi_format_instructions},\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 11,
"id": "0385e04b",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"# Make the query more complex!\n",
"user_input = (\n",
" \"I'm learning Italian, and my language class is having an end of year party... \"\n",
" \" Could you help me find an Italian outfit to wear and\"\n",
" \" an appropriate recipe to prepare so I can present for the class in Italian?\"\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 12,
"id": "6ebd3f55",
"metadata": {
"tags": []
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3m I need to find a recipe and an outfit that is Italian-themed.\n",
"Action: spoonacular_API.searchRecipes\n",
"Action Input: Italian\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3mThe API response contains 10 Italian recipes, including Turkey Tomato Cheese Pizza, Broccolini Quinoa Pilaf, Bruschetta Style Pork & Pasta, Salmon Quinoa Risotto, Italian Tuna Pasta, Roasted Brussels Sprouts With Garlic, Asparagus Lemon Risotto, Italian Steamed Artichokes, Crispy Italian Cauliflower Poppers Appetizer, and Pappa Al Pomodoro.\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I need to find an Italian-themed outfit.\n",
"Action: Open_AI_Klarna_product_Api.productsUsingGET\n",
"Action Input: Italian\u001b[0m\n",
"Observation: \u001b[31;1m\u001b[1;3mI found 10 products related to 'Italian' in the API response. These products include Italian Gold Sparkle Perfectina Necklace - Gold, Italian Design Miami Cuban Link Chain Necklace - Gold, Italian Gold Miami Cuban Link Chain Necklace - Gold, Italian Gold Herringbone Necklace - Gold, Italian Gold Claddagh Ring - Gold, Italian Gold Herringbone Chain Necklace - Gold, Garmin QuickFit 22mm Italian Vacchetta Leather Band, Macy's Italian Horn Charm - Gold, Dolce & Gabbana Light Blue Italian Love Pour Homme EdT 1.7 fl oz.\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I now know the final answer.\n",
"Final Answer: To present for your Italian language class, you could wear an Italian Gold Sparkle Perfectina Necklace - Gold, an Italian Design Miami Cuban Link Chain Necklace - Gold, or an Italian Gold Miami Cuban Link Chain Necklace - Gold. For a recipe, you could make Turkey Tomato Cheese Pizza, Broccolini Quinoa Pilaf, Bruschetta Style Pork & Pasta, Salmon Quinoa Risotto, Italian Tuna Pasta, Roasted Brussels Sprouts With Garlic, Asparagus Lemon Risotto, Italian Steamed Artichokes, Crispy Italian Cauliflower Poppers Appetizer, or Pappa Al Pomodoro.\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"'To present for your Italian language class, you could wear an Italian Gold Sparkle Perfectina Necklace - Gold, an Italian Design Miami Cuban Link Chain Necklace - Gold, or an Italian Gold Miami Cuban Link Chain Necklace - Gold. For a recipe, you could make Turkey Tomato Cheese Pizza, Broccolini Quinoa Pilaf, Bruschetta Style Pork & Pasta, Salmon Quinoa Risotto, Italian Tuna Pasta, Roasted Brussels Sprouts With Garlic, Asparagus Lemon Risotto, Italian Steamed Artichokes, Crispy Italian Cauliflower Poppers Appetizer, or Pappa Al Pomodoro.'"
]
},
"execution_count": 12,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"mrkl.run(user_input)"
]
},
{
"cell_type": "markdown",
"id": "a2959462",
"metadata": {},
"source": [
"## Thank you!"
]
},
{
"cell_type": "code",
"execution_count": 13,
"id": "6fcda5f0",
"metadata": {
"scrolled": true
},
"outputs": [
{
"data": {
"text/plain": [
"\"In Italian, you can say 'Buon appetito' to someone to wish them to enjoy their meal. This phrase is commonly used in Italy when someone is about to eat, often at the beginning of a meal. It's similar to saying 'Bon appétit' in French or 'Guten Appetit' in German.\""
]
},
"execution_count": 13,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"natural_language_api_tools[1].run(\n",
" \"Tell the LangChain audience to 'enjoy the meal' in Italian, please!\"\n",
")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ab366dc0",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.3"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
docs/extras/modules/agents/toolkits/pandas.ipynb
-300
View File
@@ -1,300 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "c81da886",
"metadata": {},
"source": [
"# Pandas Dataframe Agent\n",
"\n",
"This notebook shows how to use agents to interact with a pandas dataframe. It is mostly optimized for question answering.\n",
"\n",
"**NOTE: this agent calls the Python agent under the hood, which executes LLM generated Python code - this can be bad if the LLM generated Python code is harmful. Use cautiously.**"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "0cdd9bf5",
"metadata": {},
"outputs": [],
"source": [
"from langchain.agents import create_pandas_dataframe_agent\n",
"from langchain.chat_models import ChatOpenAI\n",
"from langchain.agents.agent_types import AgentType"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "051ebe84",
"metadata": {},
"outputs": [],
"source": [
"from langchain.llms import OpenAI\n",
"import pandas as pd\n",
"\n",
"df = pd.read_csv(\"titanic.csv\")"
]
},
{
"cell_type": "markdown",
"id": "a62858e2",
"metadata": {},
"source": [
"## Using ZERO_SHOT_REACT_DESCRIPTION\n",
"\n",
"This shows how to initialize the agent using the ZERO_SHOT_REACT_DESCRIPTION agent type. Note that this is an alternative to the above."
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "4185ff46",
"metadata": {},
"outputs": [],
"source": [
"agent = create_pandas_dataframe_agent(OpenAI(temperature=0), df, verbose=True)"
]
},
{
"cell_type": "markdown",
"id": "7233ab56",
"metadata": {},
"source": [
"## Using OpenAI Functions\n",
"\n",
"This shows how to initialize the agent using the OPENAI_FUNCTIONS agent type. Note that this is an alternative to the above."
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "a8ea710e",
"metadata": {},
"outputs": [],
"source": [
"agent = create_pandas_dataframe_agent(\n",
" ChatOpenAI(temperature=0, model=\"gpt-3.5-turbo-0613\"),\n",
" df,\n",
" verbose=True,\n",
" agent_type=AgentType.OPENAI_FUNCTIONS,\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "a9207a2e",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3m\n",
"Invoking: `python_repl_ast` with `df.shape[0]`\n",
"\n",
"\n",
"\u001b[0m\u001b[36;1m\u001b[1;3m891\u001b[0m\u001b[32;1m\u001b[1;3mThere are 891 rows in the dataframe.\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"'There are 891 rows in the dataframe.'"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"agent.run(\"how many rows are there?\")"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "bd43617c",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3mThought: I need to count the number of people with more than 3 siblings\n",
"Action: python_repl_ast\n",
"Action Input: df[df['SibSp'] > 3].shape[0]\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3m30\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I now know the final answer\n",
"Final Answer: 30 people have more than 3 siblings.\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"'30 people have more than 3 siblings.'"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"agent.run(\"how many people have more than 3 siblings\")"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "94e64b58",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3mThought: I need to calculate the average age first\n",
"Action: python_repl_ast\n",
"Action Input: df['Age'].mean()\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3m29.69911764705882\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I now need to calculate the square root of the average age\n",
"Action: python_repl_ast\n",
"Action Input: math.sqrt(df['Age'].mean())\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3mNameError(\"name 'math' is not defined\")\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I need to import the math library\n",
"Action: python_repl_ast\n",
"Action Input: import math\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3m\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I now need to calculate the square root of the average age\n",
"Action: python_repl_ast\n",
"Action Input: math.sqrt(df['Age'].mean())\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3m5.449689683556195\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I now know the final answer\n",
"Final Answer: The square root of the average age is 5.449689683556195.\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"'The square root of the average age is 5.449689683556195.'"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"agent.run(\"whats the square root of the average age?\")"
]
},
{
"cell_type": "markdown",
"id": "c4bc0584",
"metadata": {},
"source": [
"### Multi DataFrame Example\n",
"\n",
"This next part shows how the agent can interact with multiple dataframes passed in as a list."
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "42a15bd9",
"metadata": {},
"outputs": [],
"source": [
"df1 = df.copy()\n",
"df1[\"Age\"] = df1[\"Age\"].fillna(df1[\"Age\"].mean())"
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "eba13b4d",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3mThought: I need to compare the age columns in both dataframes\n",
"Action: python_repl_ast\n",
"Action Input: len(df1[df1['Age'] != df2['Age']])\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3m177\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I now know the final answer\n",
"Final Answer: 177 rows in the age column are different.\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"'177 rows in the age column are different.'"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"agent = create_pandas_dataframe_agent(OpenAI(temperature=0), [df, df1], verbose=True)\n",
"agent.run(\"how many rows in the age column are different?\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "60d08a56",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.1"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
docs/extras/modules/agents/toolkits/playwright.ipynb
-335
View File
File diff suppressed because one or more lines are too long
docs/extras/modules/agents/toolkits/powerbi.ipynb
-231
View File
@@ -1,231 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"source": [
"# PowerBI Dataset Agent\n",
"\n",
"This notebook showcases an agent designed to interact with a Power BI Dataset. The agent is designed to answer more general questions about a dataset, as well as recover from errors.\n",
"\n",
"Note that, as this agent is in active development, all answers might not be correct. It runs against the [executequery endpoint](https://learn.microsoft.com/en-us/rest/api/power-bi/datasets/execute-queries), which does not allow deletes.\n",
"\n",
"### Some notes\n",
"- It relies on authentication with the azure.identity package, which can be installed with `pip install azure-identity`. Alternatively you can create the powerbi dataset with a token as a string without supplying the credentials.\n",
"- You can also supply a username to impersonate for use with datasets that have RLS enabled. \n",
"- The toolkit uses a LLM to create the query from the question, the agent uses the LLM for the overall execution.\n",
"- Testing was done mostly with a `text-davinci-003` model, codex models did not seem to perform ver well."
],
"metadata": {},
"attachments": {},
"id": "9363398d"
},
{
"cell_type": "markdown",
"source": [
"## Initialization"
],
"metadata": {
"tags": []
},
"id": "0725445e"
},
{
"cell_type": "code",
"execution_count": null,
"source": [
"from langchain.agents.agent_toolkits import create_pbi_agent\n",
"from langchain.agents.agent_toolkits import PowerBIToolkit\n",
"from langchain.utilities.powerbi import PowerBIDataset\n",
"from langchain.chat_models import ChatOpenAI\n",
"from langchain.agents import AgentExecutor\n",
"from azure.identity import DefaultAzureCredential"
],
"outputs": [],
"metadata": {
"tags": []
},
"id": "c82f33e9"
},
{
"cell_type": "code",
"execution_count": null,
"source": [
"fast_llm = ChatOpenAI(\n",
" temperature=0.5, max_tokens=1000, model_name=\"gpt-3.5-turbo\", verbose=True\n",
")\n",
"smart_llm = ChatOpenAI(temperature=0, max_tokens=100, model_name=\"gpt-4\", verbose=True)\n",
"\n",
"toolkit = PowerBIToolkit(\n",
" powerbi=PowerBIDataset(\n",
" dataset_id=\"<dataset_id>\",\n",
" table_names=[\"table1\", \"table2\"],\n",
" credential=DefaultAzureCredential(),\n",
" ),\n",
" llm=smart_llm,\n",
")\n",
"\n",
"agent_executor = create_pbi_agent(\n",
" llm=fast_llm,\n",
" toolkit=toolkit,\n",
" verbose=True,\n",
")"
],
"outputs": [],
"metadata": {
"tags": []
},
"id": "0b2c5853"
},
{
"cell_type": "markdown",
"source": [
"## Example: describing a table"
],
"metadata": {},
"id": "80c92be3"
},
{
"cell_type": "code",
"execution_count": null,
"source": [
"agent_executor.run(\"Describe table1\")"
],
"outputs": [],
"metadata": {
"tags": []
},
"id": "90f236cb"
},
{
"cell_type": "markdown",
"source": [
"## Example: simple query on a table\n",
"In this example, the agent actually figures out the correct query to get a row count of the table."
],
"metadata": {},
"attachments": {},
"id": "b464930f"
},
{
"cell_type": "code",
"execution_count": null,
"source": [
"agent_executor.run(\"How many records are in table1?\")"
],
"outputs": [],
"metadata": {
"tags": []
},
"id": "b668c907"
},
{
"cell_type": "markdown",
"source": [
"## Example: running queries"
],
"metadata": {},
"id": "f2229a2f"
},
{
"cell_type": "code",
"execution_count": null,
"source": [
"agent_executor.run(\"How many records are there by dimension1 in table2?\")"
],
"outputs": [],
"metadata": {
"tags": []
},
"id": "865a420f"
},
{
"cell_type": "code",
"execution_count": null,
"source": [
"agent_executor.run(\"What unique values are there for dimensions2 in table2\")"
],
"outputs": [],
"metadata": {
"tags": []
},
"id": "120cd49a"
},
{
"cell_type": "markdown",
"source": [
"## Example: add your own few-shot prompts"
],
"metadata": {},
"attachments": {},
"id": "ac584fb2"
},
{
"cell_type": "code",
"execution_count": null,
"source": [
"# fictional example\n",
"few_shots = \"\"\"\n",
"Question: How many rows are in the table revenue?\n",
"DAX: EVALUATE ROW(\"Number of rows\", COUNTROWS(revenue_details))\n",
"----\n",
"Question: How many rows are in the table revenue where year is not empty?\n",
"DAX: EVALUATE ROW(\"Number of rows\", COUNTROWS(FILTER(revenue_details, revenue_details[year] <> \"\")))\n",
"----\n",
"Question: What was the average of value in revenue in dollars?\n",
"DAX: EVALUATE ROW(\"Average\", AVERAGE(revenue_details[dollar_value]))\n",
"----\n",
"\"\"\"\n",
"toolkit = PowerBIToolkit(\n",
" powerbi=PowerBIDataset(\n",
" dataset_id=\"<dataset_id>\",\n",
" table_names=[\"table1\", \"table2\"],\n",
" credential=DefaultAzureCredential(),\n",
" ),\n",
" llm=smart_llm,\n",
" examples=few_shots,\n",
")\n",
"agent_executor = create_pbi_agent(\n",
" llm=fast_llm,\n",
" toolkit=toolkit,\n",
" verbose=True,\n",
")"
],
"outputs": [],
"metadata": {},
"id": "ffa66827"
},
{
"cell_type": "code",
"execution_count": null,
"source": [
"agent_executor.run(\"What was the maximum of value in revenue in dollars in 2022?\")"
],
"outputs": [],
"metadata": {},
"id": "3be44685"
}
],
"metadata": {
"kernelspec": {
"name": "python3",
"display_name": "Python 3.9.16 64-bit"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.16"
},
"interpreter": {
"hash": "397704579725e15f5c7cb49fe5f0341eb7531c82d19f2c29d197e8b64ab5776b"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
docs/extras/modules/agents/toolkits/python.ipynb
-279
View File
@@ -1,279 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "82a4c2cc-20ea-4b20-a565-63e905dee8ff",
"metadata": {},
"source": [
"# Python Agent\n",
"\n",
"This notebook showcases an agent designed to write and execute python code to answer a question."
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "f98e9c90-5c37-4fb9-af3e-d09693af8543",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"from langchain.agents.agent_toolkits import create_python_agent\n",
"from langchain.tools.python.tool import PythonREPLTool\n",
"from langchain.python import PythonREPL\n",
"from langchain.llms.openai import OpenAI\n",
"from langchain.agents.agent_types import AgentType\n",
"from langchain.chat_models import ChatOpenAI"
]
},
{
"cell_type": "markdown",
"id": "ca30d64c",
"metadata": {},
"source": [
"## Using ZERO_SHOT_REACT_DESCRIPTION\n",
"\n",
"This shows how to initialize the agent using the ZERO_SHOT_REACT_DESCRIPTION agent type. Note that this is an alternative to the above."
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "cc422f53-c51c-4694-a834-72ecd1e68363",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"agent_executor = create_python_agent(\n",
" llm=OpenAI(temperature=0, max_tokens=1000),\n",
" tool=PythonREPLTool(),\n",
" verbose=True,\n",
" agent_type=AgentType.ZERO_SHOT_REACT_DESCRIPTION,\n",
")"
]
},
{
"cell_type": "markdown",
"id": "bb487e8e",
"metadata": {},
"source": [
"## Using OpenAI Functions\n",
"\n",
"This shows how to initialize the agent using the OPENAI_FUNCTIONS agent type. Note that this is an alternative to the above."
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "6e651822",
"metadata": {},
"outputs": [],
"source": [
"agent_executor = create_python_agent(\n",
" llm=ChatOpenAI(temperature=0, model=\"gpt-3.5-turbo-0613\"),\n",
" tool=PythonREPLTool(),\n",
" verbose=True,\n",
" agent_type=AgentType.OPENAI_FUNCTIONS,\n",
" agent_executor_kwargs={\"handle_parsing_errors\": True},\n",
")"
]
},
{
"cell_type": "markdown",
"id": "c16161de",
"metadata": {},
"source": [
"## Fibonacci Example\n",
"This example was created by [John Wiseman](https://twitter.com/lemonodor/status/1628270074074398720?s=20)."
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "25cd4f92-ea9b-4fe6-9838-a4f85f81eebe",
"metadata": {
"tags": []
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3m\n",
"Invoking: `Python_REPL` with `def fibonacci(n):\n",
" if n <= 0:\n",
" return 0\n",
" elif n == 1:\n",
" return 1\n",
" else:\n",
" return fibonacci(n-1) + fibonacci(n-2)\n",
"\n",
"fibonacci(10)`\n",
"\n",
"\n",
"\u001b[0m\u001b[36;1m\u001b[1;3m\u001b[0m\u001b[32;1m\u001b[1;3mThe 10th Fibonacci number is 55.\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"'The 10th Fibonacci number is 55.'"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"agent_executor.run(\"What is the 10th fibonacci number?\")"
]
},
{
"cell_type": "markdown",
"id": "7caa30de",
"metadata": {},
"source": [
"## Training neural net\n",
"This example was created by [Samee Ur Rehman](https://twitter.com/sameeurehman/status/1630130518133207046?s=20)."
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "4b9f60e7-eb6a-4f14-8604-498d863d4482",
"metadata": {
"scrolled": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3mCould not parse tool input: {'name': 'python', 'arguments': 'import torch\\nimport torch.nn as nn\\nimport torch.optim as optim\\n\\n# Define the neural network\\nclass SingleNeuron(nn.Module):\\n def __init__(self):\\n super(SingleNeuron, self).__init__()\\n self.linear = nn.Linear(1, 1)\\n \\n def forward(self, x):\\n return self.linear(x)\\n\\n# Create the synthetic data\\nx_train = torch.tensor([[1.0], [2.0], [3.0], [4.0]], dtype=torch.float32)\\ny_train = torch.tensor([[2.0], [4.0], [6.0], [8.0]], dtype=torch.float32)\\n\\n# Create the neural network\\nmodel = SingleNeuron()\\n\\n# Define the loss function and optimizer\\ncriterion = nn.MSELoss()\\noptimizer = optim.SGD(model.parameters(), lr=0.01)\\n\\n# Train the neural network\\nfor epoch in range(1, 1001):\\n # Forward pass\\n y_pred = model(x_train)\\n \\n # Compute loss\\n loss = criterion(y_pred, y_train)\\n \\n # Backward pass and optimization\\n optimizer.zero_grad()\\n loss.backward()\\n optimizer.step()\\n \\n # Print the loss every 100 epochs\\n if epoch % 100 == 0:\\n print(f\"Epoch {epoch}: Loss = {loss.item()}\")\\n\\n# Make a prediction for x = 5\\nx_test = torch.tensor([[5.0]], dtype=torch.float32)\\ny_pred = model(x_test)\\ny_pred.item()'} because the `arguments` is not valid JSON.\u001b[0mInvalid or incomplete response\u001b[32;1m\u001b[1;3m\n",
"Invoking: `Python_REPL` with `import torch\n",
"import torch.nn as nn\n",
"import torch.optim as optim\n",
"\n",
"# Define the neural network\n",
"class SingleNeuron(nn.Module):\n",
" def __init__(self):\n",
" super(SingleNeuron, self).__init__()\n",
" self.linear = nn.Linear(1, 1)\n",
" \n",
" def forward(self, x):\n",
" return self.linear(x)\n",
"\n",
"# Create the synthetic data\n",
"x_train = torch.tensor([[1.0], [2.0], [3.0], [4.0]], dtype=torch.float32)\n",
"y_train = torch.tensor([[2.0], [4.0], [6.0], [8.0]], dtype=torch.float32)\n",
"\n",
"# Create the neural network\n",
"model = SingleNeuron()\n",
"\n",
"# Define the loss function and optimizer\n",
"criterion = nn.MSELoss()\n",
"optimizer = optim.SGD(model.parameters(), lr=0.01)\n",
"\n",
"# Train the neural network\n",
"for epoch in range(1, 1001):\n",
" # Forward pass\n",
" y_pred = model(x_train)\n",
" \n",
" # Compute loss\n",
" loss = criterion(y_pred, y_train)\n",
" \n",
" # Backward pass and optimization\n",
" optimizer.zero_grad()\n",
" loss.backward()\n",
" optimizer.step()\n",
" \n",
" # Print the loss every 100 epochs\n",
" if epoch % 100 == 0:\n",
" print(f\"Epoch {epoch}: Loss = {loss.item()}\")\n",
"\n",
"# Make a prediction for x = 5\n",
"x_test = torch.tensor([[5.0]], dtype=torch.float32)\n",
"y_pred = model(x_test)\n",
"y_pred.item()`\n",
"\n",
"\n",
"\u001b[0m\u001b[36;1m\u001b[1;3mEpoch 100: Loss = 0.03825576975941658\n",
"Epoch 200: Loss = 0.02100197970867157\n",
"Epoch 300: Loss = 0.01152981910854578\n",
"Epoch 400: Loss = 0.006329738534986973\n",
"Epoch 500: Loss = 0.0034749575424939394\n",
"Epoch 600: Loss = 0.0019077073084190488\n",
"Epoch 700: Loss = 0.001047312980517745\n",
"Epoch 800: Loss = 0.0005749554838985205\n",
"Epoch 900: Loss = 0.0003156439634039998\n",
"Epoch 1000: Loss = 0.00017328384274151176\n",
"\u001b[0m\u001b[32;1m\u001b[1;3m\n",
"Invoking: `Python_REPL` with `x_test.item()`\n",
"\n",
"\n",
"\u001b[0m\u001b[36;1m\u001b[1;3m\u001b[0m\u001b[32;1m\u001b[1;3mThe prediction for x = 5 is 10.000173568725586.\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"'The prediction for x = 5 is 10.000173568725586.'"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"agent_executor.run(\n",
" \"\"\"Understand, write a single neuron neural network in PyTorch.\n",
"Take synthetic data for y=2x. Train for 1000 epochs and print every 100 epochs.\n",
"Return prediction for x = 5\"\"\"\n",
")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "eb654671",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.1"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
docs/extras/modules/agents/toolkits/spark.ipynb
-413
View File
@@ -1,413 +0,0 @@
{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"# Spark Dataframe Agent\n",
"\n",
"This notebook shows how to use agents to interact with a Spark dataframe and Spark Connect. It is mostly optimized for question answering.\n",
"\n",
"**NOTE: this agent calls the Python agent under the hood, which executes LLM generated Python code - this can be bad if the LLM generated Python code is harmful. Use cautiously.**"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"\n",
"os.environ[\"OPENAI_API_KEY\"] = \"...input your openai api key here...\""
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"23/05/15 20:33:10 WARN Utils: Your hostname, Mikes-Mac-mini.local resolves to a loopback address: 127.0.0.1; using 192.168.68.115 instead (on interface en1)\n",
"23/05/15 20:33:10 WARN Utils: Set SPARK_LOCAL_IP if you need to bind to another address\n",
"Setting default log level to \"WARN\".\n",
"To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).\n",
"23/05/15 20:33:10 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"+-----------+--------+------+--------------------+------+----+-----+-----+----------------+-------+-----+--------+\n",
"|PassengerId|Survived|Pclass| Name| Sex| Age|SibSp|Parch| Ticket| Fare|Cabin|Embarked|\n",
"+-----------+--------+------+--------------------+------+----+-----+-----+----------------+-------+-----+--------+\n",
"| 1| 0| 3|Braund, Mr. Owen ...| male|22.0| 1| 0| A/5 21171| 7.25| null| S|\n",
"| 2| 1| 1|Cumings, Mrs. Joh...|female|38.0| 1| 0| PC 17599|71.2833| C85| C|\n",
"| 3| 1| 3|Heikkinen, Miss. ...|female|26.0| 0| 0|STON/O2. 3101282| 7.925| null| S|\n",
"| 4| 1| 1|Futrelle, Mrs. Ja...|female|35.0| 1| 0| 113803| 53.1| C123| S|\n",
"| 5| 0| 3|Allen, Mr. Willia...| male|35.0| 0| 0| 373450| 8.05| null| S|\n",
"| 6| 0| 3| Moran, Mr. James| male|null| 0| 0| 330877| 8.4583| null| Q|\n",
"| 7| 0| 1|McCarthy, Mr. Tim...| male|54.0| 0| 0| 17463|51.8625| E46| S|\n",
"| 8| 0| 3|Palsson, Master. ...| male| 2.0| 3| 1| 349909| 21.075| null| S|\n",
"| 9| 1| 3|Johnson, Mrs. Osc...|female|27.0| 0| 2| 347742|11.1333| null| S|\n",
"| 10| 1| 2|Nasser, Mrs. Nich...|female|14.0| 1| 0| 237736|30.0708| null| C|\n",
"| 11| 1| 3|Sandstrom, Miss. ...|female| 4.0| 1| 1| PP 9549| 16.7| G6| S|\n",
"| 12| 1| 1|Bonnell, Miss. El...|female|58.0| 0| 0| 113783| 26.55| C103| S|\n",
"| 13| 0| 3|Saundercock, Mr. ...| male|20.0| 0| 0| A/5. 2151| 8.05| null| S|\n",
"| 14| 0| 3|Andersson, Mr. An...| male|39.0| 1| 5| 347082| 31.275| null| S|\n",
"| 15| 0| 3|Vestrom, Miss. Hu...|female|14.0| 0| 0| 350406| 7.8542| null| S|\n",
"| 16| 1| 2|Hewlett, Mrs. (Ma...|female|55.0| 0| 0| 248706| 16.0| null| S|\n",
"| 17| 0| 3|Rice, Master. Eugene| male| 2.0| 4| 1| 382652| 29.125| null| Q|\n",
"| 18| 1| 2|Williams, Mr. Cha...| male|null| 0| 0| 244373| 13.0| null| S|\n",
"| 19| 0| 3|Vander Planke, Mr...|female|31.0| 1| 0| 345763| 18.0| null| S|\n",
"| 20| 1| 3|Masselmani, Mrs. ...|female|null| 0| 0| 2649| 7.225| null| C|\n",
"+-----------+--------+------+--------------------+------+----+-----+-----+----------------+-------+-----+--------+\n",
"only showing top 20 rows\n",
"\n"
]
}
],
"source": [
"from langchain.llms import OpenAI\n",
"from pyspark.sql import SparkSession\n",
"from langchain.agents import create_spark_dataframe_agent\n",
"\n",
"spark = SparkSession.builder.getOrCreate()\n",
"csv_file_path = \"titanic.csv\"\n",
"df = spark.read.csv(csv_file_path, header=True, inferSchema=True)\n",
"df.show()"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [],
"source": [
"agent = create_spark_dataframe_agent(llm=OpenAI(temperature=0), df=df, verbose=True)"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3mThought: I need to find out how many rows are in the dataframe\n",
"Action: python_repl_ast\n",
"Action Input: df.count()\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3m891\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I now know the final answer\n",
"Final Answer: There are 891 rows in the dataframe.\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"'There are 891 rows in the dataframe.'"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"agent.run(\"how many rows are there?\")"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3mThought: I need to find out how many people have more than 3 siblings\n",
"Action: python_repl_ast\n",
"Action Input: df.filter(df.SibSp > 3).count()\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3m30\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I now know the final answer\n",
"Final Answer: 30 people have more than 3 siblings.\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"'30 people have more than 3 siblings.'"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"agent.run(\"how many people have more than 3 siblings\")"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3mThought: I need to get the average age first\n",
"Action: python_repl_ast\n",
"Action Input: df.agg({\"Age\": \"mean\"}).collect()[0][0]\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3m29.69911764705882\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I now have the average age, I need to get the square root\n",
"Action: python_repl_ast\n",
"Action Input: math.sqrt(29.69911764705882)\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3mname 'math' is not defined\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I need to import math first\n",
"Action: python_repl_ast\n",
"Action Input: import math\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3m\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I now have the math library imported, I can get the square root\n",
"Action: python_repl_ast\n",
"Action Input: math.sqrt(29.69911764705882)\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3m5.449689683556195\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I now know the final answer\n",
"Final Answer: 5.449689683556195\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"'5.449689683556195'"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"agent.run(\"whats the square root of the average age?\")"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [],
"source": [
"spark.stop()"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Spark Connect Example"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# in apache-spark root directory. (tested here with \"spark-3.4.0-bin-hadoop3 and later\")\n",
"# To launch Spark with support for Spark Connect sessions, run the start-connect-server.sh script.\n",
"!./sbin/start-connect-server.sh --packages org.apache.spark:spark-connect_2.12:3.4.0"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"23/05/08 10:06:09 WARN Utils: Service 'SparkUI' could not bind on port 4040. Attempting port 4041.\n"
]
}
],
"source": [
"from pyspark.sql import SparkSession\n",
"\n",
"# Now that the Spark server is running, we can connect to it remotely using Spark Connect. We do this by\n",
"# creating a remote Spark session on the client where our application runs. Before we can do that, we need\n",
"# to make sure to stop the existing regular Spark session because it cannot coexist with the remote\n",
"# Spark Connect session we are about to create.\n",
"SparkSession.builder.master(\"local[*]\").getOrCreate().stop()"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [],
"source": [
"# The command we used above to launch the server configured Spark to run as localhost:15002.\n",
"# So now we can create a remote Spark session on the client using the following command.\n",
"spark = SparkSession.builder.remote(\"sc://localhost:15002\").getOrCreate()"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"+-----------+--------+------+--------------------+------+----+-----+-----+----------------+-------+-----+--------+\n",
"|PassengerId|Survived|Pclass| Name| Sex| Age|SibSp|Parch| Ticket| Fare|Cabin|Embarked|\n",
"+-----------+--------+------+--------------------+------+----+-----+-----+----------------+-------+-----+--------+\n",
"| 1| 0| 3|Braund, Mr. Owen ...| male|22.0| 1| 0| A/5 21171| 7.25| null| S|\n",
"| 2| 1| 1|Cumings, Mrs. Joh...|female|38.0| 1| 0| PC 17599|71.2833| C85| C|\n",
"| 3| 1| 3|Heikkinen, Miss. ...|female|26.0| 0| 0|STON/O2. 3101282| 7.925| null| S|\n",
"| 4| 1| 1|Futrelle, Mrs. Ja...|female|35.0| 1| 0| 113803| 53.1| C123| S|\n",
"| 5| 0| 3|Allen, Mr. Willia...| male|35.0| 0| 0| 373450| 8.05| null| S|\n",
"| 6| 0| 3| Moran, Mr. James| male|null| 0| 0| 330877| 8.4583| null| Q|\n",
"| 7| 0| 1|McCarthy, Mr. Tim...| male|54.0| 0| 0| 17463|51.8625| E46| S|\n",
"| 8| 0| 3|Palsson, Master. ...| male| 2.0| 3| 1| 349909| 21.075| null| S|\n",
"| 9| 1| 3|Johnson, Mrs. Osc...|female|27.0| 0| 2| 347742|11.1333| null| S|\n",
"| 10| 1| 2|Nasser, Mrs. Nich...|female|14.0| 1| 0| 237736|30.0708| null| C|\n",
"| 11| 1| 3|Sandstrom, Miss. ...|female| 4.0| 1| 1| PP 9549| 16.7| G6| S|\n",
"| 12| 1| 1|Bonnell, Miss. El...|female|58.0| 0| 0| 113783| 26.55| C103| S|\n",
"| 13| 0| 3|Saundercock, Mr. ...| male|20.0| 0| 0| A/5. 2151| 8.05| null| S|\n",
"| 14| 0| 3|Andersson, Mr. An...| male|39.0| 1| 5| 347082| 31.275| null| S|\n",
"| 15| 0| 3|Vestrom, Miss. Hu...|female|14.0| 0| 0| 350406| 7.8542| null| S|\n",
"| 16| 1| 2|Hewlett, Mrs. (Ma...|female|55.0| 0| 0| 248706| 16.0| null| S|\n",
"| 17| 0| 3|Rice, Master. Eugene| male| 2.0| 4| 1| 382652| 29.125| null| Q|\n",
"| 18| 1| 2|Williams, Mr. Cha...| male|null| 0| 0| 244373| 13.0| null| S|\n",
"| 19| 0| 3|Vander Planke, Mr...|female|31.0| 1| 0| 345763| 18.0| null| S|\n",
"| 20| 1| 3|Masselmani, Mrs. ...|female|null| 0| 0| 2649| 7.225| null| C|\n",
"+-----------+--------+------+--------------------+------+----+-----+-----+----------------+-------+-----+--------+\n",
"only showing top 20 rows\n",
"\n"
]
}
],
"source": [
"csv_file_path = \"titanic.csv\"\n",
"df = spark.read.csv(csv_file_path, header=True, inferSchema=True)\n",
"df.show()"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {},
"outputs": [],
"source": [
"from langchain.agents import create_spark_dataframe_agent\n",
"from langchain.llms import OpenAI\n",
"import os\n",
"\n",
"os.environ[\"OPENAI_API_KEY\"] = \"...input your openai api key here...\"\n",
"\n",
"agent = create_spark_dataframe_agent(llm=OpenAI(temperature=0), df=df, verbose=True)"
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3m\n",
"Thought: I need to find the row with the highest fare\n",
"Action: python_repl_ast\n",
"Action Input: df.sort(df.Fare.desc()).first()\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3mRow(PassengerId=259, Survived=1, Pclass=1, Name='Ward, Miss. Anna', Sex='female', Age=35.0, SibSp=0, Parch=0, Ticket='PC 17755', Fare=512.3292, Cabin=None, Embarked='C')\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I now know the name of the person who bought the most expensive ticket\n",
"Final Answer: Miss. Anna Ward\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"'Miss. Anna Ward'"
]
},
"execution_count": 16,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"agent.run(\n",
" \"\"\"\n",
"who bought the most expensive ticket?\n",
"You can find all supported function types in https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/dataframe.html\n",
"\"\"\"\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {},
"outputs": [],
"source": [
"spark.stop()"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.1"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
docs/extras/modules/agents/toolkits/spark_sql.ipynb
-344
View File
@@ -1,344 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Spark SQL Agent\n",
"\n",
"This notebook shows how to use agents to interact with a Spark SQL. Similar to [SQL Database Agent](https://python.langchain.com/en/latest/modules/agents/toolkits/examples/sql_database.html), it is designed to address general inquiries about Spark SQL and facilitate error recovery.\n",
"\n",
"**NOTE: Note that, as this agent is in active development, all answers might not be correct. Additionally, it is not guaranteed that the agent won't perform DML statements on your Spark cluster given certain questions. Be careful running it on sensitive data!**"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Initialization"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"from langchain.agents import create_spark_sql_agent\n",
"from langchain.agents.agent_toolkits import SparkSQLToolkit\n",
"from langchain.chat_models import ChatOpenAI\n",
"from langchain.utilities.spark_sql import SparkSQL"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Setting default log level to \"WARN\".\n",
"To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).\n",
"23/05/18 16:03:10 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"+-----------+--------+------+--------------------+------+----+-----+-----+----------------+-------+-----+--------+\n",
"|PassengerId|Survived|Pclass| Name| Sex| Age|SibSp|Parch| Ticket| Fare|Cabin|Embarked|\n",
"+-----------+--------+------+--------------------+------+----+-----+-----+----------------+-------+-----+--------+\n",
"| 1| 0| 3|Braund, Mr. Owen ...| male|22.0| 1| 0| A/5 21171| 7.25| null| S|\n",
"| 2| 1| 1|Cumings, Mrs. Joh...|female|38.0| 1| 0| PC 17599|71.2833| C85| C|\n",
"| 3| 1| 3|Heikkinen, Miss. ...|female|26.0| 0| 0|STON/O2. 3101282| 7.925| null| S|\n",
"| 4| 1| 1|Futrelle, Mrs. Ja...|female|35.0| 1| 0| 113803| 53.1| C123| S|\n",
"| 5| 0| 3|Allen, Mr. Willia...| male|35.0| 0| 0| 373450| 8.05| null| S|\n",
"| 6| 0| 3| Moran, Mr. James| male|null| 0| 0| 330877| 8.4583| null| Q|\n",
"| 7| 0| 1|McCarthy, Mr. Tim...| male|54.0| 0| 0| 17463|51.8625| E46| S|\n",
"| 8| 0| 3|Palsson, Master. ...| male| 2.0| 3| 1| 349909| 21.075| null| S|\n",
"| 9| 1| 3|Johnson, Mrs. Osc...|female|27.0| 0| 2| 347742|11.1333| null| S|\n",
"| 10| 1| 2|Nasser, Mrs. Nich...|female|14.0| 1| 0| 237736|30.0708| null| C|\n",
"| 11| 1| 3|Sandstrom, Miss. ...|female| 4.0| 1| 1| PP 9549| 16.7| G6| S|\n",
"| 12| 1| 1|Bonnell, Miss. El...|female|58.0| 0| 0| 113783| 26.55| C103| S|\n",
"| 13| 0| 3|Saundercock, Mr. ...| male|20.0| 0| 0| A/5. 2151| 8.05| null| S|\n",
"| 14| 0| 3|Andersson, Mr. An...| male|39.0| 1| 5| 347082| 31.275| null| S|\n",
"| 15| 0| 3|Vestrom, Miss. Hu...|female|14.0| 0| 0| 350406| 7.8542| null| S|\n",
"| 16| 1| 2|Hewlett, Mrs. (Ma...|female|55.0| 0| 0| 248706| 16.0| null| S|\n",
"| 17| 0| 3|Rice, Master. Eugene| male| 2.0| 4| 1| 382652| 29.125| null| Q|\n",
"| 18| 1| 2|Williams, Mr. Cha...| male|null| 0| 0| 244373| 13.0| null| S|\n",
"| 19| 0| 3|Vander Planke, Mr...|female|31.0| 1| 0| 345763| 18.0| null| S|\n",
"| 20| 1| 3|Masselmani, Mrs. ...|female|null| 0| 0| 2649| 7.225| null| C|\n",
"+-----------+--------+------+--------------------+------+----+-----+-----+----------------+-------+-----+--------+\n",
"only showing top 20 rows\n",
"\n"
]
}
],
"source": [
"from pyspark.sql import SparkSession\n",
"\n",
"spark = SparkSession.builder.getOrCreate()\n",
"schema = \"langchain_example\"\n",
"spark.sql(f\"CREATE DATABASE IF NOT EXISTS {schema}\")\n",
"spark.sql(f\"USE {schema}\")\n",
"csv_file_path = \"titanic.csv\"\n",
"table = \"titanic\"\n",
"spark.read.csv(csv_file_path, header=True, inferSchema=True).write.saveAsTable(table)\n",
"spark.table(table).show()"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [],
"source": [
"# Note, you can also connect to Spark via Spark connect. For example:\n",
"# db = SparkSQL.from_uri(\"sc://localhost:15002\", schema=schema)\n",
"spark_sql = SparkSQL(schema=schema)\n",
"llm = ChatOpenAI(temperature=0)\n",
"toolkit = SparkSQLToolkit(db=spark_sql, llm=llm)\n",
"agent_executor = create_spark_sql_agent(llm=llm, toolkit=toolkit, verbose=True)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Example: describing a table"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3mAction: list_tables_sql_db\n",
"Action Input: \u001b[0m\n",
"Observation: \u001b[38;5;200m\u001b[1;3mtitanic\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3mI found the titanic table. Now I need to get the schema and sample rows for the titanic table.\n",
"Action: schema_sql_db\n",
"Action Input: titanic\u001b[0m\n",
"Observation: \u001b[33;1m\u001b[1;3mCREATE TABLE langchain_example.titanic (\n",
" PassengerId INT,\n",
" Survived INT,\n",
" Pclass INT,\n",
" Name STRING,\n",
" Sex STRING,\n",
" Age DOUBLE,\n",
" SibSp INT,\n",
" Parch INT,\n",
" Ticket STRING,\n",
" Fare DOUBLE,\n",
" Cabin STRING,\n",
" Embarked STRING)\n",
";\n",
"\n",
"/*\n",
"3 rows from titanic table:\n",
"PassengerId\tSurvived\tPclass\tName\tSex\tAge\tSibSp\tParch\tTicket\tFare\tCabin\tEmbarked\n",
"1\t0\t3\tBraund, Mr. Owen Harris\tmale\t22.0\t1\t0\tA/5 21171\t7.25\tNone\tS\n",
"2\t1\t1\tCumings, Mrs. John Bradley (Florence Briggs Thayer)\tfemale\t38.0\t1\t0\tPC 17599\t71.2833\tC85\tC\n",
"3\t1\t3\tHeikkinen, Miss. Laina\tfemale\t26.0\t0\t0\tSTON/O2. 3101282\t7.925\tNone\tS\n",
"*/\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3mI now know the schema and sample rows for the titanic table.\n",
"Final Answer: The titanic table has the following columns: PassengerId (INT), Survived (INT), Pclass (INT), Name (STRING), Sex (STRING), Age (DOUBLE), SibSp (INT), Parch (INT), Ticket (STRING), Fare (DOUBLE), Cabin (STRING), and Embarked (STRING). Here are some sample rows from the table: \n",
"\n",
"1. PassengerId: 1, Survived: 0, Pclass: 3, Name: Braund, Mr. Owen Harris, Sex: male, Age: 22.0, SibSp: 1, Parch: 0, Ticket: A/5 21171, Fare: 7.25, Cabin: None, Embarked: S\n",
"2. PassengerId: 2, Survived: 1, Pclass: 1, Name: Cumings, Mrs. John Bradley (Florence Briggs Thayer), Sex: female, Age: 38.0, SibSp: 1, Parch: 0, Ticket: PC 17599, Fare: 71.2833, Cabin: C85, Embarked: C\n",
"3. PassengerId: 3, Survived: 1, Pclass: 3, Name: Heikkinen, Miss. Laina, Sex: female, Age: 26.0, SibSp: 0, Parch: 0, Ticket: STON/O2. 3101282, Fare: 7.925, Cabin: None, Embarked: S\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": "'The titanic table has the following columns: PassengerId (INT), Survived (INT), Pclass (INT), Name (STRING), Sex (STRING), Age (DOUBLE), SibSp (INT), Parch (INT), Ticket (STRING), Fare (DOUBLE), Cabin (STRING), and Embarked (STRING). Here are some sample rows from the table: \\n\\n1. PassengerId: 1, Survived: 0, Pclass: 3, Name: Braund, Mr. Owen Harris, Sex: male, Age: 22.0, SibSp: 1, Parch: 0, Ticket: A/5 21171, Fare: 7.25, Cabin: None, Embarked: S\\n2. PassengerId: 2, Survived: 1, Pclass: 1, Name: Cumings, Mrs. John Bradley (Florence Briggs Thayer), Sex: female, Age: 38.0, SibSp: 1, Parch: 0, Ticket: PC 17599, Fare: 71.2833, Cabin: C85, Embarked: C\\n3. PassengerId: 3, Survived: 1, Pclass: 3, Name: Heikkinen, Miss. Laina, Sex: female, Age: 26.0, SibSp: 0, Parch: 0, Ticket: STON/O2. 3101282, Fare: 7.925, Cabin: None, Embarked: S'"
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"agent_executor.run(\"Describe the titanic table\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Example: running queries"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3mAction: list_tables_sql_db\n",
"Action Input: \u001b[0m\n",
"Observation: \u001b[38;5;200m\u001b[1;3mtitanic\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3mI should check the schema of the titanic table to see if there is an age column.\n",
"Action: schema_sql_db\n",
"Action Input: titanic\u001b[0m\n",
"Observation: \u001b[33;1m\u001b[1;3mCREATE TABLE langchain_example.titanic (\n",
" PassengerId INT,\n",
" Survived INT,\n",
" Pclass INT,\n",
" Name STRING,\n",
" Sex STRING,\n",
" Age DOUBLE,\n",
" SibSp INT,\n",
" Parch INT,\n",
" Ticket STRING,\n",
" Fare DOUBLE,\n",
" Cabin STRING,\n",
" Embarked STRING)\n",
";\n",
"\n",
"/*\n",
"3 rows from titanic table:\n",
"PassengerId\tSurvived\tPclass\tName\tSex\tAge\tSibSp\tParch\tTicket\tFare\tCabin\tEmbarked\n",
"1\t0\t3\tBraund, Mr. Owen Harris\tmale\t22.0\t1\t0\tA/5 21171\t7.25\tNone\tS\n",
"2\t1\t1\tCumings, Mrs. John Bradley (Florence Briggs Thayer)\tfemale\t38.0\t1\t0\tPC 17599\t71.2833\tC85\tC\n",
"3\t1\t3\tHeikkinen, Miss. Laina\tfemale\t26.0\t0\t0\tSTON/O2. 3101282\t7.925\tNone\tS\n",
"*/\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3mThere is an Age column in the titanic table. I should write a query to calculate the average age and then find the square root of the result.\n",
"Action: query_checker_sql_db\n",
"Action Input: SELECT SQRT(AVG(Age)) as square_root_of_avg_age FROM titanic\u001b[0m\n",
"Observation: \u001b[31;1m\u001b[1;3mThe original query seems to be correct. Here it is again:\n",
"\n",
"SELECT SQRT(AVG(Age)) as square_root_of_avg_age FROM titanic\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3mThe query is correct, so I can execute it to find the square root of the average age.\n",
"Action: query_sql_db\n",
"Action Input: SELECT SQRT(AVG(Age)) as square_root_of_avg_age FROM titanic\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3m[('5.449689683556195',)]\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3mI now know the final answer\n",
"Final Answer: The square root of the average age is approximately 5.45.\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": "'The square root of the average age is approximately 5.45.'"
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"agent_executor.run(\"whats the square root of the average age?\")"
]
},
{
"cell_type": "code",
"execution_count": 6,
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3mAction: list_tables_sql_db\n",
"Action Input: \u001b[0m\n",
"Observation: \u001b[38;5;200m\u001b[1;3mtitanic\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3mI should check the schema of the titanic table to see what columns are available.\n",
"Action: schema_sql_db\n",
"Action Input: titanic\u001b[0m\n",
"Observation: \u001b[33;1m\u001b[1;3mCREATE TABLE langchain_example.titanic (\n",
" PassengerId INT,\n",
" Survived INT,\n",
" Pclass INT,\n",
" Name STRING,\n",
" Sex STRING,\n",
" Age DOUBLE,\n",
" SibSp INT,\n",
" Parch INT,\n",
" Ticket STRING,\n",
" Fare DOUBLE,\n",
" Cabin STRING,\n",
" Embarked STRING)\n",
";\n",
"\n",
"/*\n",
"3 rows from titanic table:\n",
"PassengerId\tSurvived\tPclass\tName\tSex\tAge\tSibSp\tParch\tTicket\tFare\tCabin\tEmbarked\n",
"1\t0\t3\tBraund, Mr. Owen Harris\tmale\t22.0\t1\t0\tA/5 21171\t7.25\tNone\tS\n",
"2\t1\t1\tCumings, Mrs. John Bradley (Florence Briggs Thayer)\tfemale\t38.0\t1\t0\tPC 17599\t71.2833\tC85\tC\n",
"3\t1\t3\tHeikkinen, Miss. Laina\tfemale\t26.0\t0\t0\tSTON/O2. 3101282\t7.925\tNone\tS\n",
"*/\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3mI can use the titanic table to find the oldest survived passenger. I will query the Name and Age columns, filtering by Survived and ordering by Age in descending order.\n",
"Action: query_checker_sql_db\n",
"Action Input: SELECT Name, Age FROM titanic WHERE Survived = 1 ORDER BY Age DESC LIMIT 1\u001b[0m\n",
"Observation: \u001b[31;1m\u001b[1;3mSELECT Name, Age FROM titanic WHERE Survived = 1 ORDER BY Age DESC LIMIT 1\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3mThe query is correct. Now I will execute it to find the oldest survived passenger.\n",
"Action: query_sql_db\n",
"Action Input: SELECT Name, Age FROM titanic WHERE Survived = 1 ORDER BY Age DESC LIMIT 1\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3m[('Barkworth, Mr. Algernon Henry Wilson', '80.0')]\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3mI now know the final answer.\n",
"Final Answer: The oldest survived passenger is Barkworth, Mr. Algernon Henry Wilson, who was 80 years old.\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": "'The oldest survived passenger is Barkworth, Mr. Algernon Henry Wilson, who was 80 years old.'"
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"agent_executor.run(\"What's the name of the oldest survived passenger?\")"
],
"metadata": {
"collapsed": false
}
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.2"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
docs/extras/modules/agents/toolkits/sql_database.ipynb
-647
View File
@@ -1,647 +0,0 @@
{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"id": "0e499e90-7a6d-4fab-8aab-31a4df417601",
"metadata": {},
"source": [
"# SQL Database Agent\n",
"\n",
"This notebook showcases an agent designed to interact with a sql databases. The agent builds off of [SQLDatabaseChain](https://python.langchain.com/docs/modules/chains/popular/sqlite) and is designed to answer more general questions about a database, as well as recover from errors.\n",
"\n",
"Note that, as this agent is in active development, all answers might not be correct. Additionally, it is not guaranteed that the agent won't perform DML statements on your database given certain questions. Be careful running it on sensitive data!\n",
"\n",
"This uses the example Chinook database. To set it up follow the instructions on https://database.guide/2-sample-databases-sqlite/, placing the .db file in a notebooks folder at the root of this repository."
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "ec927ac6-9b2a-4e8a-9a6e-3e429191875c",
"metadata": {
"tags": []
},
"source": [
"## Initialization"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "53422913-967b-4f2a-8022-00269c1be1b1",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"from langchain.agents import create_sql_agent\n",
"from langchain.agents.agent_toolkits import SQLDatabaseToolkit\n",
"from langchain.sql_database import SQLDatabase\n",
"from langchain.llms.openai import OpenAI\n",
"from langchain.agents import AgentExecutor\n",
"from langchain.agents.agent_types import AgentType\n",
"from langchain.chat_models import ChatOpenAI"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "65ec5bb3",
"metadata": {},
"outputs": [],
"source": [
"db = SQLDatabase.from_uri(\"sqlite:///../../../../../notebooks/Chinook.db\")\n",
"toolkit = SQLDatabaseToolkit(db=db, llm=OpenAI(temperature=0))"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "f74d1792",
"metadata": {},
"source": [
"## Using ZERO_SHOT_REACT_DESCRIPTION\n",
"\n",
"This shows how to initialize the agent using the ZERO_SHOT_REACT_DESCRIPTION agent type. Note that this is an alternative to the above."
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "090f3699-79c6-4ce1-ab96-a94f0121fd64",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"agent_executor = create_sql_agent(\n",
" llm=OpenAI(temperature=0),\n",
" toolkit=toolkit,\n",
" verbose=True,\n",
" agent_type=AgentType.ZERO_SHOT_REACT_DESCRIPTION,\n",
")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "971cc455",
"metadata": {},
"source": [
"## Using OpenAI Functions\n",
"\n",
"This shows how to initialize the agent using the OPENAI_FUNCTIONS agent type. Note that this is an alternative to the above."
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "6426a27d",
"metadata": {},
"outputs": [],
"source": [
"# agent_executor = create_sql_agent(\n",
"# llm=ChatOpenAI(temperature=0, model=\"gpt-3.5-turbo-0613\"),\n",
"# toolkit=toolkit,\n",
"# verbose=True,\n",
"# agent_type=AgentType.OPENAI_FUNCTIONS\n",
"# )"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "54c01168",
"metadata": {},
"source": [
"## Disclamer ⚠️\n",
"\n",
"The query chain may generate insert/update/delete queries. When this is not expected, use a custom prompt or create a SQL users without write permissions.\n",
"\n",
"The final user might overload your SQL database by asking a simple question such as \"run the biggest query possible\". The generated query might look like:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "949772b9",
"metadata": {},
"outputs": [],
"source": [
"SELECT * FROM \"public\".\"users\"\n",
" JOIN \"public\".\"user_permissions\" ON \"public\".\"users\".id = \"public\".\"user_permissions\".user_id\n",
" JOIN \"public\".\"projects\" ON \"public\".\"users\".id = \"public\".\"projects\".user_id\n",
" JOIN \"public\".\"events\" ON \"public\".\"projects\".id = \"public\".\"events\".project_id;"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "5a4a9455",
"metadata": {},
"source": [
"For a transactional SQL database, if one of the table above contains millions of rows, the query might cause trouble to other applications using the same database.\n",
"\n",
"Most datawarehouse oriented databases support user-level quota, for limiting resource usage."
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "36ae48c7-cb08-4fef-977e-c7d4b96a464b",
"metadata": {},
"source": [
"## Example: describing a table"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "ff70e83d-5ad0-4fc7-bb96-27d82ac166d7",
"metadata": {
"tags": []
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3m\n",
"Invoking: `list_tables_sql_db` with `{}`\n",
"\n",
"\n",
"\u001b[0m\u001b[38;5;200m\u001b[1;3mAlbum, Artist, Track, PlaylistTrack, InvoiceLine, sales_table, Playlist, Genre, Employee, Customer, Invoice, MediaType\u001b[0m\u001b[32;1m\u001b[1;3m\n",
"Invoking: `schema_sql_db` with `PlaylistTrack`\n",
"\n",
"\n",
"\u001b[0m\u001b[33;1m\u001b[1;3m\n",
"CREATE TABLE \"PlaylistTrack\" (\n",
"\t\"PlaylistId\" INTEGER NOT NULL, \n",
"\t\"TrackId\" INTEGER NOT NULL, \n",
"\tPRIMARY KEY (\"PlaylistId\", \"TrackId\"), \n",
"\tFOREIGN KEY(\"TrackId\") REFERENCES \"Track\" (\"TrackId\"), \n",
"\tFOREIGN KEY(\"PlaylistId\") REFERENCES \"Playlist\" (\"PlaylistId\")\n",
")\n",
"\n",
"/*\n",
"3 rows from PlaylistTrack table:\n",
"PlaylistId\tTrackId\n",
"1\t3402\n",
"1\t3389\n",
"1\t3390\n",
"*/\u001b[0m\u001b[32;1m\u001b[1;3mThe `PlaylistTrack` table has two columns: `PlaylistId` and `TrackId`. It is a junction table that represents the relationship between playlists and tracks. \n",
"\n",
"Here is the schema of the `PlaylistTrack` table:\n",
"\n",
"```\n",
"CREATE TABLE \"PlaylistTrack\" (\n",
"\t\"PlaylistId\" INTEGER NOT NULL, \n",
"\t\"TrackId\" INTEGER NOT NULL, \n",
"\tPRIMARY KEY (\"PlaylistId\", \"TrackId\"), \n",
"\tFOREIGN KEY(\"TrackId\") REFERENCES \"Track\" (\"TrackId\"), \n",
"\tFOREIGN KEY(\"PlaylistId\") REFERENCES \"Playlist\" (\"PlaylistId\")\n",
")\n",
"```\n",
"\n",
"Here are three sample rows from the `PlaylistTrack` table:\n",
"\n",
"```\n",
"PlaylistId TrackId\n",
"1 3402\n",
"1 3389\n",
"1 3390\n",
"```\n",
"\n",
"Please let me know if there is anything else I can help you with.\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"'The `PlaylistTrack` table has two columns: `PlaylistId` and `TrackId`. It is a junction table that represents the relationship between playlists and tracks. \\n\\nHere is the schema of the `PlaylistTrack` table:\\n\\n```\\nCREATE TABLE \"PlaylistTrack\" (\\n\\t\"PlaylistId\" INTEGER NOT NULL, \\n\\t\"TrackId\" INTEGER NOT NULL, \\n\\tPRIMARY KEY (\"PlaylistId\", \"TrackId\"), \\n\\tFOREIGN KEY(\"TrackId\") REFERENCES \"Track\" (\"TrackId\"), \\n\\tFOREIGN KEY(\"PlaylistId\") REFERENCES \"Playlist\" (\"PlaylistId\")\\n)\\n```\\n\\nHere are three sample rows from the `PlaylistTrack` table:\\n\\n```\\nPlaylistId TrackId\\n1 3402\\n1 3389\\n1 3390\\n```\\n\\nPlease let me know if there is anything else I can help you with.'"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"agent_executor.run(\"Describe the playlisttrack table\")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "9abcfe8e-1868-42a4-8345-ad2d9b44c681",
"metadata": {},
"source": [
"## Example: describing a table, recovering from an error\n",
"\n",
"In this example, the agent tries to search for a table that doesn't exist, but finds the next best result"
]
},
{
"cell_type": "code",
"execution_count": 15,
"id": "bea76658-a65b-47e2-b294-6d52c5556246",
"metadata": {
"tags": []
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3mAction: list_tables_sql_db\n",
"Action Input: \"\"\u001b[0m\n",
"Observation: \u001b[38;5;200m\u001b[1;3mGenre, PlaylistTrack, MediaType, Invoice, InvoiceLine, Track, Playlist, Customer, Album, Employee, Artist\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I should look at the schema of the PlaylistSong table\n",
"Action: schema_sql_db\n",
"Action Input: \"PlaylistSong\"\u001b[0m\n",
"Observation: \u001b[33;1m\u001b[1;3mError: table_names {'PlaylistSong'} not found in database\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I should check the spelling of the table\n",
"Action: list_tables_sql_db\n",
"Action Input: \"\"\u001b[0m\n",
"Observation: \u001b[38;5;200m\u001b[1;3mGenre, PlaylistTrack, MediaType, Invoice, InvoiceLine, Track, Playlist, Customer, Album, Employee, Artist\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m The table is called PlaylistTrack\n",
"Action: schema_sql_db\n",
"Action Input: \"PlaylistTrack\"\u001b[0m\n",
"Observation: \u001b[33;1m\u001b[1;3m\n",
"CREATE TABLE \"PlaylistTrack\" (\n",
"\t\"PlaylistId\" INTEGER NOT NULL, \n",
"\t\"TrackId\" INTEGER NOT NULL, \n",
"\tPRIMARY KEY (\"PlaylistId\", \"TrackId\"), \n",
"\tFOREIGN KEY(\"TrackId\") REFERENCES \"Track\" (\"TrackId\"), \n",
"\tFOREIGN KEY(\"PlaylistId\") REFERENCES \"Playlist\" (\"PlaylistId\")\n",
")\n",
"\n",
"SELECT * FROM 'PlaylistTrack' LIMIT 3;\n",
"PlaylistId TrackId\n",
"1 3402\n",
"1 3389\n",
"1 3390\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I now know the final answer\n",
"Final Answer: The PlaylistTrack table contains two columns, PlaylistId and TrackId, which are both integers and are used to link Playlist and Track tables.\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"'The PlaylistTrack table contains two columns, PlaylistId and TrackId, which are both integers and are used to link Playlist and Track tables.'"
]
},
"execution_count": 15,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"agent_executor.run(\"Describe the playlistsong table\")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "6fbc26af-97e4-4a21-82aa-48bdc992da26",
"metadata": {},
"source": [
"## Example: running queries"
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "17bea710-4a23-4de0-b48e-21d57be48293",
"metadata": {
"tags": []
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3mAction: list_tables_sql_db\n",
"Action Input: \"\"\u001b[0m\n",
"Observation: \u001b[38;5;200m\u001b[1;3mInvoice, MediaType, Artist, InvoiceLine, Genre, Playlist, Employee, Album, PlaylistTrack, Track, Customer\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I should look at the schema of the relevant tables to see what columns I can use.\n",
"Action: schema_sql_db\n",
"Action Input: \"Invoice, Customer\"\u001b[0m\n",
"Observation: \u001b[33;1m\u001b[1;3m\n",
"CREATE TABLE \"Customer\" (\n",
"\t\"CustomerId\" INTEGER NOT NULL, \n",
"\t\"FirstName\" NVARCHAR(40) NOT NULL, \n",
"\t\"LastName\" NVARCHAR(20) NOT NULL, \n",
"\t\"Company\" NVARCHAR(80), \n",
"\t\"Address\" NVARCHAR(70), \n",
"\t\"City\" NVARCHAR(40), \n",
"\t\"State\" NVARCHAR(40), \n",
"\t\"Country\" NVARCHAR(40), \n",
"\t\"PostalCode\" NVARCHAR(10), \n",
"\t\"Phone\" NVARCHAR(24), \n",
"\t\"Fax\" NVARCHAR(24), \n",
"\t\"Email\" NVARCHAR(60) NOT NULL, \n",
"\t\"SupportRepId\" INTEGER, \n",
"\tPRIMARY KEY (\"CustomerId\"), \n",
"\tFOREIGN KEY(\"SupportRepId\") REFERENCES \"Employee\" (\"EmployeeId\")\n",
")\n",
"\n",
"SELECT * FROM 'Customer' LIMIT 3;\n",
"CustomerId FirstName LastName Company Address City State Country PostalCode Phone Fax Email SupportRepId\n",
"1 Luís Gonçalves Embraer - Empresa Brasileira de Aeronáutica S.A. Av. Brigadeiro Faria Lima, 2170 São José dos Campos SP Brazil 12227-000 +55 (12) 3923-5555 +55 (12) 3923-5566 luisg@embraer.com.br 3\n",
"2 Leonie Köhler None Theodor-Heuss-Straße 34 Stuttgart None Germany 70174 +49 0711 2842222 None leonekohler@surfeu.de 5\n",
"3 François Tremblay None 1498 rue Bélanger Montréal QC Canada H2G 1A7 +1 (514) 721-4711 None ftremblay@gmail.com 3\n",
"\n",
"\n",
"CREATE TABLE \"Invoice\" (\n",
"\t\"InvoiceId\" INTEGER NOT NULL, \n",
"\t\"CustomerId\" INTEGER NOT NULL, \n",
"\t\"InvoiceDate\" DATETIME NOT NULL, \n",
"\t\"BillingAddress\" NVARCHAR(70), \n",
"\t\"BillingCity\" NVARCHAR(40), \n",
"\t\"BillingState\" NVARCHAR(40), \n",
"\t\"BillingCountry\" NVARCHAR(40), \n",
"\t\"BillingPostalCode\" NVARCHAR(10), \n",
"\t\"Total\" NUMERIC(10, 2) NOT NULL, \n",
"\tPRIMARY KEY (\"InvoiceId\"), \n",
"\tFOREIGN KEY(\"CustomerId\") REFERENCES \"Customer\" (\"CustomerId\")\n",
")\n",
"\n",
"SELECT * FROM 'Invoice' LIMIT 3;\n",
"InvoiceId CustomerId InvoiceDate BillingAddress BillingCity BillingState BillingCountry BillingPostalCode Total\n",
"1 2 2009-01-01 00:00:00 Theodor-Heuss-Straße 34 Stuttgart None Germany 70174 1.98\n",
"2 4 2009-01-02 00:00:00 Ullevålsveien 14 Oslo None Norway 0171 3.96\n",
"3 8 2009-01-03 00:00:00 Grétrystraat 63 Brussels None Belgium 1000 5.94\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I should query the Invoice and Customer tables to get the total sales per country.\n",
"Action: query_sql_db\n",
"Action Input: SELECT c.Country, SUM(i.Total) AS TotalSales FROM Invoice i INNER JOIN Customer c ON i.CustomerId = c.CustomerId GROUP BY c.Country ORDER BY TotalSales DESC LIMIT 10\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3m[('USA', 523.0600000000003), ('Canada', 303.9599999999999), ('France', 195.09999999999994), ('Brazil', 190.09999999999997), ('Germany', 156.48), ('United Kingdom', 112.85999999999999), ('Czech Republic', 90.24000000000001), ('Portugal', 77.23999999999998), ('India', 75.25999999999999), ('Chile', 46.62)]\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I now know the final answer\n",
"Final Answer: The customers from the USA spent the most, with a total of $523.06.\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"'The customers from the USA spent the most, with a total of $523.06.'"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"agent_executor.run(\n",
" \"List the total sales per country. Which country's customers spent the most?\"\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "474dddda-c067-4eeb-98b1-e763ee78b18c",
"metadata": {
"tags": []
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3mAction: list_tables_sql_db\n",
"Action Input: \"\"\u001b[0m\n",
"Observation: \u001b[38;5;200m\u001b[1;3mInvoice, MediaType, Artist, InvoiceLine, Genre, Playlist, Employee, Album, PlaylistTrack, Track, Customer\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I should look at the schema of the Playlist and PlaylistTrack tables to see what columns I can use.\n",
"Action: schema_sql_db\n",
"Action Input: \"Playlist, PlaylistTrack\"\u001b[0m\n",
"Observation: \u001b[33;1m\u001b[1;3m\n",
"CREATE TABLE \"Playlist\" (\n",
"\t\"PlaylistId\" INTEGER NOT NULL, \n",
"\t\"Name\" NVARCHAR(120), \n",
"\tPRIMARY KEY (\"PlaylistId\")\n",
")\n",
"\n",
"SELECT * FROM 'Playlist' LIMIT 3;\n",
"PlaylistId Name\n",
"1 Music\n",
"2 Movies\n",
"3 TV Shows\n",
"\n",
"\n",
"CREATE TABLE \"PlaylistTrack\" (\n",
"\t\"PlaylistId\" INTEGER NOT NULL, \n",
"\t\"TrackId\" INTEGER NOT NULL, \n",
"\tPRIMARY KEY (\"PlaylistId\", \"TrackId\"), \n",
"\tFOREIGN KEY(\"TrackId\") REFERENCES \"Track\" (\"TrackId\"), \n",
"\tFOREIGN KEY(\"PlaylistId\") REFERENCES \"Playlist\" (\"PlaylistId\")\n",
")\n",
"\n",
"SELECT * FROM 'PlaylistTrack' LIMIT 3;\n",
"PlaylistId TrackId\n",
"1 3402\n",
"1 3389\n",
"1 3390\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I can use a SELECT statement to get the total number of tracks in each playlist.\n",
"Action: query_checker_sql_db\n",
"Action Input: SELECT Playlist.Name, COUNT(PlaylistTrack.TrackId) AS TotalTracks FROM Playlist INNER JOIN PlaylistTrack ON Playlist.PlaylistId = PlaylistTrack.PlaylistId GROUP BY Playlist.Name\u001b[0m\n",
"Observation: \u001b[31;1m\u001b[1;3m\n",
"\n",
"SELECT Playlist.Name, COUNT(PlaylistTrack.TrackId) AS TotalTracks FROM Playlist INNER JOIN PlaylistTrack ON Playlist.PlaylistId = PlaylistTrack.PlaylistId GROUP BY Playlist.Name\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m The query looks correct, I can now execute it.\n",
"Action: query_sql_db\n",
"Action Input: SELECT Playlist.Name, COUNT(PlaylistTrack.TrackId) AS TotalTracks FROM Playlist INNER JOIN PlaylistTrack ON Playlist.PlaylistId = PlaylistTrack.PlaylistId GROUP BY Playlist.Name LIMIT 10\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3m[('90’s Music', 1477), ('Brazilian Music', 39), ('Classical', 75), ('Classical 101 - Deep Cuts', 25), ('Classical 101 - Next Steps', 25), ('Classical 101 - The Basics', 25), ('Grunge', 15), ('Heavy Metal Classic', 26), ('Music', 6580), ('Music Videos', 1)]\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I now know the final answer.\n",
"Final Answer: The total number of tracks in each playlist are: '90’s Music' (1477), 'Brazilian Music' (39), 'Classical' (75), 'Classical 101 - Deep Cuts' (25), 'Classical 101 - Next Steps' (25), 'Classical 101 - The Basics' (25), 'Grunge' (15), 'Heavy Metal Classic' (26), 'Music' (6580), 'Music Videos' (1).\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"\"The total number of tracks in each playlist are: '90’s Music' (1477), 'Brazilian Music' (39), 'Classical' (75), 'Classical 101 - Deep Cuts' (25), 'Classical 101 - Next Steps' (25), 'Classical 101 - The Basics' (25), 'Grunge' (15), 'Heavy Metal Classic' (26), 'Music' (6580), 'Music Videos' (1).\""
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"agent_executor.run(\n",
" \"Show the total number of tracks in each playlist. The Playlist name should be included in the result.\"\n",
")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "7c7503b5-d9d9-4faa-b064-29fcdb5ff213",
"metadata": {},
"source": [
"## Recovering from an error\n",
"\n",
"In this example, the agent is able to recover from an error after initially trying to access an attribute (`Track.ArtistId`) which doesn't exist."
]
},
{
"cell_type": "code",
"execution_count": 16,
"id": "9fe4901e-f9e1-4022-b6bc-80e2b2d6a3a4",
"metadata": {
"tags": []
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3mAction: list_tables_sql_db\n",
"Action Input: \"\"\u001b[0m\n",
"Observation: \u001b[38;5;200m\u001b[1;3mMediaType, Track, Invoice, Album, Playlist, Customer, Employee, InvoiceLine, PlaylistTrack, Genre, Artist\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I should look at the schema of the Artist, InvoiceLine, and Track tables to see what columns I can use.\n",
"Action: schema_sql_db\n",
"Action Input: \"Artist, InvoiceLine, Track\"\u001b[0m\n",
"Observation: \u001b[33;1m\u001b[1;3m\n",
"CREATE TABLE \"Artist\" (\n",
"\t\"ArtistId\" INTEGER NOT NULL, \n",
"\t\"Name\" NVARCHAR(120), \n",
"\tPRIMARY KEY (\"ArtistId\")\n",
")\n",
"\n",
"SELECT * FROM 'Artist' LIMIT 3;\n",
"ArtistId Name\n",
"1 AC/DC\n",
"2 Accept\n",
"3 Aerosmith\n",
"\n",
"\n",
"CREATE TABLE \"Track\" (\n",
"\t\"TrackId\" INTEGER NOT NULL, \n",
"\t\"Name\" NVARCHAR(200) NOT NULL, \n",
"\t\"AlbumId\" INTEGER, \n",
"\t\"MediaTypeId\" INTEGER NOT NULL, \n",
"\t\"GenreId\" INTEGER, \n",
"\t\"Composer\" NVARCHAR(220), \n",
"\t\"Milliseconds\" INTEGER NOT NULL, \n",
"\t\"Bytes\" INTEGER, \n",
"\t\"UnitPrice\" NUMERIC(10, 2) NOT NULL, \n",
"\tPRIMARY KEY (\"TrackId\"), \n",
"\tFOREIGN KEY(\"MediaTypeId\") REFERENCES \"MediaType\" (\"MediaTypeId\"), \n",
"\tFOREIGN KEY(\"GenreId\") REFERENCES \"Genre\" (\"GenreId\"), \n",
"\tFOREIGN KEY(\"AlbumId\") REFERENCES \"Album\" (\"AlbumId\")\n",
")\n",
"\n",
"SELECT * FROM 'Track' LIMIT 3;\n",
"TrackId Name AlbumId MediaTypeId GenreId Composer Milliseconds Bytes UnitPrice\n",
"1 For Those About To Rock (We Salute You) 1 1 1 Angus Young, Malcolm Young, Brian Johnson 343719 11170334 0.99\n",
"2 Balls to the Wall 2 2 1 None 342562 5510424 0.99\n",
"3 Fast As a Shark 3 2 1 F. Baltes, S. Kaufman, U. Dirkscneider & W. Hoffman 230619 3990994 0.99\n",
"\n",
"\n",
"CREATE TABLE \"InvoiceLine\" (\n",
"\t\"InvoiceLineId\" INTEGER NOT NULL, \n",
"\t\"InvoiceId\" INTEGER NOT NULL, \n",
"\t\"TrackId\" INTEGER NOT NULL, \n",
"\t\"UnitPrice\" NUMERIC(10, 2) NOT NULL, \n",
"\t\"Quantity\" INTEGER NOT NULL, \n",
"\tPRIMARY KEY (\"InvoiceLineId\"), \n",
"\tFOREIGN KEY(\"TrackId\") REFERENCES \"Track\" (\"TrackId\"), \n",
"\tFOREIGN KEY(\"InvoiceId\") REFERENCES \"Invoice\" (\"InvoiceId\")\n",
")\n",
"\n",
"SELECT * FROM 'InvoiceLine' LIMIT 3;\n",
"InvoiceLineId InvoiceId TrackId UnitPrice Quantity\n",
"1 1 2 0.99 1\n",
"2 1 4 0.99 1\n",
"3 2 6 0.99 1\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I should query the database to get the top 3 best selling artists.\n",
"Action: query_sql_db\n",
"Action Input: SELECT Artist.Name, SUM(InvoiceLine.Quantity) AS TotalQuantity FROM Artist INNER JOIN Track ON Artist.ArtistId = Track.ArtistId INNER JOIN InvoiceLine ON Track.TrackId = InvoiceLine.TrackId GROUP BY Artist.Name ORDER BY TotalQuantity DESC LIMIT 3\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3mError: (sqlite3.OperationalError) no such column: Track.ArtistId\n",
"[SQL: SELECT Artist.Name, SUM(InvoiceLine.Quantity) AS TotalQuantity FROM Artist INNER JOIN Track ON Artist.ArtistId = Track.ArtistId INNER JOIN InvoiceLine ON Track.TrackId = InvoiceLine.TrackId GROUP BY Artist.Name ORDER BY TotalQuantity DESC LIMIT 3]\n",
"(Background on this error at: https://sqlalche.me/e/14/e3q8)\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I should double check my query before executing it.\n",
"Action: query_checker_sql_db\n",
"Action Input: SELECT Artist.Name, SUM(InvoiceLine.Quantity) AS TotalQuantity FROM Artist INNER JOIN Track ON Artist.ArtistId = Track.ArtistId INNER JOIN InvoiceLine ON Track.TrackId = InvoiceLine.TrackId GROUP BY Artist.Name ORDER BY TotalQuantity DESC LIMIT 3\u001b[0m\n",
"Observation: \u001b[31;1m\u001b[1;3m\n",
"\n",
"SELECT Artist.Name, SUM(InvoiceLine.Quantity) AS TotalQuantity \n",
"FROM Artist \n",
"INNER JOIN Track ON Artist.ArtistId = Track.ArtistId \n",
"INNER JOIN InvoiceLine ON Track.TrackId = InvoiceLine.TrackId \n",
"GROUP BY Artist.Name \n",
"ORDER BY TotalQuantity DESC \n",
"LIMIT 3;\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I now know the final answer.\n",
"Action: query_sql_db\n",
"Action Input: SELECT Artist.Name, SUM(InvoiceLine.Quantity) AS TotalQuantity FROM Artist INNER JOIN Album ON Artist.ArtistId = Album.ArtistId INNER JOIN Track ON Album.AlbumId = Track.AlbumId INNER JOIN InvoiceLine ON Track.TrackId = InvoiceLine.TrackId GROUP BY Artist.Name ORDER BY TotalQuantity DESC LIMIT 3\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3m[('Iron Maiden', 140), ('U2', 107), ('Metallica', 91)]\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I now know the final answer.\n",
"Final Answer: The top 3 best selling artists are Iron Maiden, U2, and Metallica.\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"'The top 3 best selling artists are Iron Maiden, U2, and Metallica.'"
]
},
"execution_count": 16,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"agent_executor.run(\"Who are the top 3 best selling artists?\")"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.1"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
docs/extras/modules/agents/toolkits/vectorstore.ipynb
-430
View File
@@ -1,430 +0,0 @@
{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"id": "18ada398-dce6-4049-9b56-fc0ede63da9c",
"metadata": {},
"source": [
"# Vectorstore Agent\n",
"\n",
"This notebook showcases an agent designed to retrieve information from one or more vectorstores, either with or without sources."
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "eecb683b-3a46-4b9d-81a3-7caefbfec1a1",
"metadata": {},
"source": [
"## Create the Vectorstores"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "9bfd0ed8-a5eb-443e-8e92-90be8cabb0a7",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"from langchain.embeddings.openai import OpenAIEmbeddings\n",
"from langchain.vectorstores import Chroma\n",
"from langchain.text_splitter import CharacterTextSplitter\n",
"from langchain import OpenAI, VectorDBQA\n",
"\n",
"llm = OpenAI(temperature=0)"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "345bb078-4ec1-4e3a-827b-cd238c49054d",
"metadata": {
"tags": []
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Running Chroma using direct local API.\n",
"Using DuckDB in-memory for database. Data will be transient.\n"
]
}
],
"source": [
"from langchain.document_loaders import TextLoader\n",
"\n",
"loader = TextLoader(\"../../../state_of_the_union.txt\")\n",
"documents = loader.load()\n",
"text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)\n",
"texts = text_splitter.split_documents(documents)\n",
"\n",
"embeddings = OpenAIEmbeddings()\n",
"state_of_union_store = Chroma.from_documents(\n",
" texts, embeddings, collection_name=\"state-of-union\"\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "5f50eb82-e1a5-4252-8306-8ec1b478d9b4",
"metadata": {
"tags": []
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Running Chroma using direct local API.\n",
"Using DuckDB in-memory for database. Data will be transient.\n"
]
}
],
"source": [
"from langchain.document_loaders import WebBaseLoader\n",
"\n",
"loader = WebBaseLoader(\"https://beta.ruff.rs/docs/faq/\")\n",
"docs = loader.load()\n",
"ruff_texts = text_splitter.split_documents(docs)\n",
"ruff_store = Chroma.from_documents(ruff_texts, embeddings, collection_name=\"ruff\")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "f4814175-964d-42f1-aa9d-22801ce1e912",
"metadata": {},
"source": [
"## Initialize Toolkit and Agent\n",
"\n",
"First, we'll create an agent with a single vectorstore."
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "5b3b3206",
"metadata": {},
"outputs": [],
"source": [
"from langchain.agents.agent_toolkits import (\n",
" create_vectorstore_agent,\n",
" VectorStoreToolkit,\n",
" VectorStoreInfo,\n",
")\n",
"\n",
"vectorstore_info = VectorStoreInfo(\n",
" name=\"state_of_union_address\",\n",
" description=\"the most recent state of the Union adress\",\n",
" vectorstore=state_of_union_store,\n",
")\n",
"toolkit = VectorStoreToolkit(vectorstore_info=vectorstore_info)\n",
"agent_executor = create_vectorstore_agent(llm=llm, toolkit=toolkit, verbose=True)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "8a38ad10",
"metadata": {},
"source": [
"## Examples"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "3f2f455c",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3m I need to find the answer in the state of the union address\n",
"Action: state_of_union_address\n",
"Action Input: What did biden say about ketanji brown jackson\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3m Biden said that Ketanji Brown Jackson is one of the nation's top legal minds and that she will continue Justice Breyer's legacy of excellence.\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I now know the final answer\n",
"Final Answer: Biden said that Ketanji Brown Jackson is one of the nation's top legal minds and that she will continue Justice Breyer's legacy of excellence.\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"\"Biden said that Ketanji Brown Jackson is one of the nation's top legal minds and that she will continue Justice Breyer's legacy of excellence.\""
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"agent_executor.run(\n",
" \"What did biden say about ketanji brown jackson in the state of the union address?\"\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "d61e1e63",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3m I need to use the state_of_union_address_with_sources tool to answer this question.\n",
"Action: state_of_union_address_with_sources\n",
"Action Input: What did biden say about ketanji brown jackson\u001b[0m\n",
"Observation: \u001b[33;1m\u001b[1;3m{\"answer\": \" Biden said that he nominated Circuit Court of Appeals Judge Ketanji Brown Jackson to the United States Supreme Court, and that she is one of the nation's top legal minds who will continue Justice Breyer's legacy of excellence.\\n\", \"sources\": \"../../state_of_the_union.txt\"}\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I now know the final answer\n",
"Final Answer: Biden said that he nominated Circuit Court of Appeals Judge Ketanji Brown Jackson to the United States Supreme Court, and that she is one of the nation's top legal minds who will continue Justice Breyer's legacy of excellence. Sources: ../../state_of_the_union.txt\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"\"Biden said that he nominated Circuit Court of Appeals Judge Ketanji Brown Jackson to the United States Supreme Court, and that she is one of the nation's top legal minds who will continue Justice Breyer's legacy of excellence. Sources: ../../state_of_the_union.txt\""
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"agent_executor.run(\n",
" \"What did biden say about ketanji brown jackson in the state of the union address? List the source.\"\n",
")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "7ca07707",
"metadata": {},
"source": [
"## Multiple Vectorstores\n",
"We can also easily use this initialize an agent with multiple vectorstores and use the agent to route between them. To do this. This agent is optimized for routing, so it is a different toolkit and initializer."
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "c3209fd3",
"metadata": {},
"outputs": [],
"source": [
"from langchain.agents.agent_toolkits import (\n",
" create_vectorstore_router_agent,\n",
" VectorStoreRouterToolkit,\n",
" VectorStoreInfo,\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "815c4f39-308d-4949-b992-1361036e6e09",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"ruff_vectorstore_info = VectorStoreInfo(\n",
" name=\"ruff\",\n",
" description=\"Information about the Ruff python linting library\",\n",
" vectorstore=ruff_store,\n",
")\n",
"router_toolkit = VectorStoreRouterToolkit(\n",
" vectorstores=[vectorstore_info, ruff_vectorstore_info], llm=llm\n",
")\n",
"agent_executor = create_vectorstore_router_agent(\n",
" llm=llm, toolkit=router_toolkit, verbose=True\n",
")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "71680984-edaf-4a63-90f5-94edbd263550",
"metadata": {},
"source": [
"## Examples"
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "3cd1bf3e-e3df-4e69-bbe1-71c64b1af947",
"metadata": {
"tags": []
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3m I need to use the state_of_union_address tool to answer this question.\n",
"Action: state_of_union_address\n",
"Action Input: What did biden say about ketanji brown jackson\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3m Biden said that Ketanji Brown Jackson is one of the nation's top legal minds and that she will continue Justice Breyer's legacy of excellence.\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I now know the final answer\n",
"Final Answer: Biden said that Ketanji Brown Jackson is one of the nation's top legal minds and that she will continue Justice Breyer's legacy of excellence.\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"\"Biden said that Ketanji Brown Jackson is one of the nation's top legal minds and that she will continue Justice Breyer's legacy of excellence.\""
]
},
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"agent_executor.run(\n",
" \"What did biden say about ketanji brown jackson in the state of the union address?\"\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 10,
"id": "c5998b8d",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3m I need to find out what tool ruff uses to run over Jupyter Notebooks\n",
"Action: ruff\n",
"Action Input: What tool does ruff use to run over Jupyter Notebooks?\u001b[0m\n",
"Observation: \u001b[33;1m\u001b[1;3m Ruff is integrated into nbQA, a tool for running linters and code formatters over Jupyter Notebooks. After installing ruff and nbqa, you can run Ruff over a notebook like so: > nbqa ruff Untitled.html\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I now know the final answer\n",
"Final Answer: Ruff is integrated into nbQA, a tool for running linters and code formatters over Jupyter Notebooks. After installing ruff and nbqa, you can run Ruff over a notebook like so: > nbqa ruff Untitled.html\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"'Ruff is integrated into nbQA, a tool for running linters and code formatters over Jupyter Notebooks. After installing ruff and nbqa, you can run Ruff over a notebook like so: > nbqa ruff Untitled.html'"
]
},
"execution_count": 10,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"agent_executor.run(\"What tool does ruff use to run over Jupyter Notebooks?\")"
]
},
{
"cell_type": "code",
"execution_count": 11,
"id": "744e9b51-fbd9-4778-b594-ea957d0f3467",
"metadata": {
"tags": []
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3m I need to find out what tool ruff uses and if the president mentioned it in the state of the union.\n",
"Action: ruff\n",
"Action Input: What tool does ruff use to run over Jupyter Notebooks?\u001b[0m\n",
"Observation: \u001b[33;1m\u001b[1;3m Ruff is integrated into nbQA, a tool for running linters and code formatters over Jupyter Notebooks. After installing ruff and nbqa, you can run Ruff over a notebook like so: > nbqa ruff Untitled.html\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I need to find out if the president mentioned nbQA in the state of the union.\n",
"Action: state_of_union_address\n",
"Action Input: Did the president mention nbQA in the state of the union?\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3m No, the president did not mention nbQA in the state of the union.\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I now know the final answer.\n",
"Final Answer: No, the president did not mention nbQA in the state of the union.\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"'No, the president did not mention nbQA in the state of the union.'"
]
},
"execution_count": 11,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"agent_executor.run(\n",
" \"What tool does ruff use to run over Jupyter Notebooks? Did the president mention that tool in the state of the union?\"\n",
")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "92203aa9-f63a-4ce1-b562-fadf4474ad9d",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.1"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
docs/extras/modules/agents/toolkits/xorbits.ipynb
-742
View File
@@ -1,742 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Xorbits Agent"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This notebook shows how to use agents to interact with [Xorbits Pandas](https://doc.xorbits.io/en/latest/reference/pandas/index.html) dataframe and [Xorbits Numpy](https://doc.xorbits.io/en/latest/reference/numpy/index.html) ndarray. It is mostly optimized for question answering.\n",
"\n",
"**NOTE: this agent calls the Python agent under the hood, which executes LLM generated Python code - this can be bad if the LLM generated Python code is harmful. Use cautiously.**"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Pandas examples"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"ExecuteTime": {
"end_time": "2023-07-13T08:06:33.955439Z",
"start_time": "2023-07-13T08:06:33.767539500Z"
}
},
"outputs": [
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "05b7c067b1114ce9a8aef4a58a5d5fef",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
" 0%| | 0.00/100 [00:00<?, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"import xorbits.pandas as pd\n",
"\n",
"from langchain.agents import create_xorbits_agent\n",
"from langchain.llms import OpenAI\n",
"\n",
"data = pd.read_csv(\"titanic.csv\")\n",
"agent = create_xorbits_agent(OpenAI(temperature=0), data, verbose=True)"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"ExecuteTime": {
"end_time": "2023-07-13T08:11:06.622471100Z",
"start_time": "2023-07-13T08:11:03.183042Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3mThought: I need to count the number of rows and columns\n",
"Action: python_repl_ast\n",
"Action Input: data.shape\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3m(891, 12)\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I now know the final answer\n",
"Final Answer: There are 891 rows and 12 columns.\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"'There are 891 rows and 12 columns.'"
]
},
"execution_count": 2,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"agent.run(\"How many rows and columns are there?\")"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"ExecuteTime": {
"end_time": "2023-07-13T08:11:23.189275300Z",
"start_time": "2023-07-13T08:11:11.029030900Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new chain...\u001b[0m\n"
]
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "8c63d745a7eb41a484043a5dba357997",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
" 0%| | 0.00/100 [00:00<?, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\u001b[32;1m\u001b[1;3mThought: I need to count the number of people in pclass 1\n",
"Action: python_repl_ast\n",
"Action Input: data[data['Pclass'] == 1].shape[0]\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3m216\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I now know the final answer\n",
"Final Answer: There are 216 people in pclass 1.\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"'There are 216 people in pclass 1.'"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"agent.run(\"How many people are in pclass 1?\")"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3mThought: I need to calculate the mean age\n",
"Action: python_repl_ast\n",
"Action Input: data['Age'].mean()\u001b[0m"
]
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "29af2e29f2d64a3397c212812adf0e9b",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
" 0%| | 0.00/100 [00:00<?, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"Observation: \u001b[36;1m\u001b[1;3m29.69911764705882\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I now know the final answer\n",
"Final Answer: The mean age is 29.69911764705882.\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"'The mean age is 29.69911764705882.'"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"agent.run(\"whats the mean age?\")"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3mThought: I need to group the data by sex and then find the average age for each group\n",
"Action: python_repl_ast\n",
"Action Input: data.groupby('Sex')['Age'].mean()\u001b[0m"
]
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "c3d28625c35946fd91ebc2a47f8d8c5b",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
" 0%| | 0.00/100 [00:00<?, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"Observation: \u001b[36;1m\u001b[1;3mSex\n",
"female 27.915709\n",
"male 30.726645\n",
"Name: Age, dtype: float64\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I now know the average age for each group\n",
"Final Answer: The average age for female passengers is 27.92 and the average age for male passengers is 30.73.\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"'The average age for female passengers is 27.92 and the average age for male passengers is 30.73.'"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"agent.run(\"Group the data by sex and find the average age for each group\")"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new chain...\u001b[0m\n"
]
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "c72aab63b20d47599f4f9806f6887a69",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
" 0%| | 0.00/100 [00:00<?, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\u001b[32;1m\u001b[1;3mThought: I need to filter the dataframe to get the desired result\n",
"Action: python_repl_ast\n",
"Action Input: data[(data['Age'] > 30) & (data['Fare'] > 30) & (data['Fare'] < 50) & ((data['Pclass'] == 1) | (data['Pclass'] == 2))].shape[0]\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3m20\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I now know the final answer\n",
"Final Answer: 20\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"'20'"
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"agent.run(\n",
" \"Show the number of people whose age is greater than 30 and fare is between 30 and 50 , and pclass is either 1 or 2\"\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Numpy examples"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "fa8baf315a0c41c89392edc4a24b76f5",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
" 0%| | 0.00/100 [00:00<?, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"import xorbits.numpy as np\n",
"\n",
"from langchain.agents import create_xorbits_agent\n",
"from langchain.llms import OpenAI\n",
"\n",
"arr = np.array([1, 2, 3, 4, 5, 6])\n",
"agent = create_xorbits_agent(OpenAI(temperature=0), arr, verbose=True)"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3mThought: I need to find out the shape of the array\n",
"Action: python_repl_ast\n",
"Action Input: data.shape\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3m(6,)\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I now know the final answer\n",
"Final Answer: The shape of the array is (6,).\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"'The shape of the array is (6,).'"
]
},
"execution_count": 12,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"agent.run(\"Give the shape of the array \")"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3mThought: I need to access the 2nd element of the array\n",
"Action: python_repl_ast\n",
"Action Input: data[1]\u001b[0m"
]
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "64efcc74f81f404eb0a7d3f0326cd8b3",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
" 0%| | 0.00/100 [00:00<?, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"Observation: \u001b[36;1m\u001b[1;3m2\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I now know the final answer\n",
"Final Answer: 2\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"'2'"
]
},
"execution_count": 14,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"agent.run(\"Give the 2nd element of the array \")"
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3mThought: I need to reshape the array and then transpose it\n",
"Action: python_repl_ast\n",
"Action Input: np.reshape(data, (2,3)).T\u001b[0m"
]
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "fce51acf6fb347c0b400da67c6750534",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
" 0%| | 0.00/100 [00:00<?, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"Observation: \u001b[36;1m\u001b[1;3m[[1 4]\n",
" [2 5]\n",
" [3 6]]\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I now know the final answer\n",
"Final Answer: The reshaped and transposed array is [[1 4], [2 5], [3 6]].\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"'The reshaped and transposed array is [[1 4], [2 5], [3 6]].'"
]
},
"execution_count": 18,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"agent.run(\n",
" \"Reshape the array into a 2-dimensional array with 2 rows and 3 columns, and then transpose it\"\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3mThought: I need to reshape the array and then sum it\n",
"Action: python_repl_ast\n",
"Action Input: np.sum(np.reshape(data, (3,2)), axis=0)\u001b[0m"
]
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "27fd4a0bbf694936bc41a6991064dec2",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
" 0%| | 0.00/100 [00:00<?, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"Observation: \u001b[36;1m\u001b[1;3m[ 9 12]\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I now know the final answer\n",
"Final Answer: The sum of the array along the first axis is [9, 12].\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"'The sum of the array along the first axis is [9, 12].'"
]
},
"execution_count": 20,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"agent.run(\n",
" \"Reshape the array into a 2-dimensional array with 3 rows and 2 columns and sum the array along the first axis\"\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "a591b6d7913f45cba98d2f3b71a5120a",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
" 0%| | 0.00/100 [00:00<?, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"arr = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]])\n",
"agent = create_xorbits_agent(OpenAI(temperature=0), arr, verbose=True)"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3mThought: I need to use the numpy covariance function\n",
"Action: python_repl_ast\n",
"Action Input: np.cov(data)\u001b[0m"
]
},
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "5fe40f83cfae48d0919c147627b5839f",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
" 0%| | 0.00/100 [00:00<?, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"Observation: \u001b[36;1m\u001b[1;3m[[1. 1. 1.]\n",
" [1. 1. 1.]\n",
" [1. 1. 1.]]\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I now know the final answer\n",
"Final Answer: The covariance matrix is [[1. 1. 1.], [1. 1. 1.], [1. 1. 1.]].\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"'The covariance matrix is [[1. 1. 1.], [1. 1. 1.], [1. 1. 1.]].'"
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"agent.run(\"calculate the covariance matrix\")"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3mThought: I need to use the SVD function\n",
"Action: python_repl_ast\n",
"Action Input: U, S, V = np.linalg.svd(data)\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3m\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I now have the U matrix\n",
"Final Answer: U = [[-0.70710678 -0.70710678]\n",
" [-0.70710678 0.70710678]]\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"'U = [[-0.70710678 -0.70710678]\\n [-0.70710678 0.70710678]]'"
]
},
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"agent.run(\"compute the U of Singular Value Decomposition of the matrix\")"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.13"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
docs/extras/modules/agents/tools/how_to/custom_tools.ipynb → docs/extras/modules/agents/tools/custom_tools.ipynb
View File
docs/extras/modules/agents/tools/how_to/human_approval.ipynb → docs/extras/modules/agents/tools/human_approval.ipynb
View File
docs/extras/modules/agents/tools/integrations/_gradio_tools_files/output_7_0.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 875 KiB

docs/extras/modules/agents/tools/integrations/arxiv.ipynb
-258
View File
@@ -1,258 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "245a954a",
"metadata": {},
"source": [
"# ArXiv API Tool\n",
"\n",
"This notebook goes over how to use the `arxiv` component. \n",
"\n",
"First, you need to install `arxiv` python package."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d5a7209e",
"metadata": {
"tags": [],
"vscode": {
"languageId": "shellscript"
}
},
"outputs": [],
"source": [
"!pip install arxiv"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "ce1a4827-ce89-4f31-a041-3246743e513a",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"from langchain.chat_models import ChatOpenAI\n",
"from langchain.agents import load_tools, initialize_agent, AgentType\n",
"\n",
"llm = ChatOpenAI(temperature=0.0)\n",
"tools = load_tools(\n",
" [\"arxiv\"],\n",
")\n",
"\n",
"agent_chain = initialize_agent(\n",
" tools,\n",
" llm,\n",
" agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION,\n",
" verbose=True,\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "ad7dd945-5ae3-49e5-b667-6d86b15050b6",
"metadata": {
"tags": []
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3mI need to use Arxiv to search for the paper.\n",
"Action: Arxiv\n",
"Action Input: \"1605.08386\"\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3mPublished: 2016-05-26\n",
"Title: Heat-bath random walks with Markov bases\n",
"Authors: Caprice Stanley, Tobias Windisch\n",
"Summary: Graphs on lattice points are studied whose edges come from a finite set of\n",
"allowed moves of arbitrary length. We show that the diameter of these graphs on\n",
"fibers of a fixed integer matrix can be bounded from above by a constant. We\n",
"then study the mixing behaviour of heat-bath random walks on these graphs. We\n",
"also state explicit conditions on the set of moves so that the heat-bath random\n",
"walk, a generalization of the Glauber dynamics, is an expander in fixed\n",
"dimension.\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3mThe paper is about heat-bath random walks with Markov bases on graphs of lattice points.\n",
"Final Answer: The paper 1605.08386 is about heat-bath random walks with Markov bases on graphs of lattice points.\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"'The paper 1605.08386 is about heat-bath random walks with Markov bases on graphs of lattice points.'"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"agent_chain.run(\n",
" \"What's the paper 1605.08386 about?\",\n",
")"
]
},
{
"cell_type": "markdown",
"id": "b4183343-d69a-4be0-9b2c-cc98464a6825",
"metadata": {},
"source": [
"## The ArXiv API Wrapper\n",
"\n",
"The tool wraps the API Wrapper. Below, we can explore some of the features it provides."
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "8d32b39a",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"from langchain.utilities import ArxivAPIWrapper"
]
},
{
"cell_type": "markdown",
"id": "c89c110c-96ac-4fe1-ba3e-6056543d1a59",
"metadata": {},
"source": [
"Run a query to get information about some `scientific article`/articles. The query text is limited to 300 characters.\n",
"\n",
"It returns these article fields:\n",
"- Publishing date\n",
"- Title\n",
"- Authors\n",
"- Summary\n",
"\n",
"Next query returns information about one article with arxiv Id equal \"1605.08386\". "
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "34bb5968",
"metadata": {
"tags": []
},
"outputs": [
{
"data": {
"text/plain": [
"'Published: 2016-05-26\\nTitle: Heat-bath random walks with Markov bases\\nAuthors: Caprice Stanley, Tobias Windisch\\nSummary: Graphs on lattice points are studied whose edges come from a finite set of\\nallowed moves of arbitrary length. We show that the diameter of these graphs on\\nfibers of a fixed integer matrix can be bounded from above by a constant. We\\nthen study the mixing behaviour of heat-bath random walks on these graphs. We\\nalso state explicit conditions on the set of moves so that the heat-bath random\\nwalk, a generalization of the Glauber dynamics, is an expander in fixed\\ndimension.'"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"arxiv = ArxivAPIWrapper()\n",
"docs = arxiv.run(\"1605.08386\")\n",
"docs"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "840f70c9-8f80-4680-bb38-46198e931bcf",
"metadata": {},
"source": [
"Now, we want to get information about one author, `Caprice Stanley`.\n",
"\n",
"This query returns information about three articles. By default, the query returns information only about three top articles."
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "b0867fda-e119-4b19-9ec6-e354fa821db3",
"metadata": {
"tags": []
},
"outputs": [
{
"data": {
"text/plain": [
"'Published: 2017-10-10\\nTitle: On Mixing Behavior of a Family of Random Walks Determined by a Linear Recurrence\\nAuthors: Caprice Stanley, Seth Sullivant\\nSummary: We study random walks on the integers mod $G_n$ that are determined by an\\ninteger sequence $\\\\{ G_n \\\\}_{n \\\\geq 1}$ generated by a linear recurrence\\nrelation. Fourier analysis provides explicit formulas to compute the\\neigenvalues of the transition matrices and we use this to bound the mixing time\\nof the random walks.\\n\\nPublished: 2016-05-26\\nTitle: Heat-bath random walks with Markov bases\\nAuthors: Caprice Stanley, Tobias Windisch\\nSummary: Graphs on lattice points are studied whose edges come from a finite set of\\nallowed moves of arbitrary length. We show that the diameter of these graphs on\\nfibers of a fixed integer matrix can be bounded from above by a constant. We\\nthen study the mixing behaviour of heat-bath random walks on these graphs. We\\nalso state explicit conditions on the set of moves so that the heat-bath random\\nwalk, a generalization of the Glauber dynamics, is an expander in fixed\\ndimension.\\n\\nPublished: 2003-03-18\\nTitle: Calculation of fluxes of charged particles and neutrinos from atmospheric showers\\nAuthors: V. Plyaskin\\nSummary: The results on the fluxes of charged particles and neutrinos from a\\n3-dimensional (3D) simulation of atmospheric showers are presented. An\\nagreement of calculated fluxes with data on charged particles from the AMS and\\nCAPRICE detectors is demonstrated. Predictions on neutrino fluxes at different\\nexperimental sites are compared with results from other calculations.'"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"docs = arxiv.run(\"Caprice Stanley\")\n",
"docs"
]
},
{
"cell_type": "markdown",
"id": "2d9b6292-a47d-4f99-9827-8e9f244bf887",
"metadata": {},
"source": [
"Now, we are trying to find information about non-existing article. In this case, the response is \"No good Arxiv Result was found\""
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "3580aeeb-086f-45ba-bcdc-b46f5134b3dd",
"metadata": {
"tags": []
},
"outputs": [
{
"data": {
"text/plain": [
"'No good Arxiv Result was found'"
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"docs = arxiv.run(\"1605.08386WWW\")\n",
"docs"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.4"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
docs/extras/modules/agents/tools/integrations/awslambda.ipynb
-121
View File
@@ -1,121 +0,0 @@
{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## AWS Lambda API"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"This notebook goes over how to use the AWS Lambda Tool component.\n",
"\n",
"AWS Lambda is a serverless computing service provided by Amazon Web Services (AWS), designed to allow developers to build and run applications and services without the need for provisioning or managing servers. This serverless architecture enables you to focus on writing and deploying code, while AWS automatically takes care of scaling, patching, and managing the infrastructure required to run your applications.\n",
"\n",
"By including a `awslambda` in the list of tools provided to an Agent, you can grant your Agent the ability to invoke code running in your AWS Cloud for whatever purposes you need.\n",
"\n",
"When an Agent uses the awslambda tool, it will provide an argument of type string which will in turn be passed into the Lambda function via the event parameter.\n",
"\n",
"First, you need to install `boto3` python package."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"vscode": {
"languageId": "shellscript"
}
},
"outputs": [],
"source": [
"!pip install boto3 > /dev/null"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"In order for an agent to use the tool, you must provide it with the name and description that match the functionality of you lambda function's logic. \n",
"\n",
"You must also provide the name of your function. "
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"Note that because this tool is effectively just a wrapper around the boto3 library, you will need to run `aws configure` in order to make use of the tool. For more detail, see [here](https://docs.aws.amazon.com/cli/index.html)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"vscode": {
"languageId": "shellscript"
}
},
"outputs": [],
"source": [
"from langchain import OpenAI\n",
"from langchain.agents import load_tools, AgentType\n",
"\n",
"llm = OpenAI(temperature=0)\n",
"\n",
"tools = load_tools(\n",
" [\"awslambda\"],\n",
" awslambda_tool_name=\"email-sender\",\n",
" awslambda_tool_description=\"sends an email with the specified content to test@testing123.com\",\n",
" function_name=\"testFunction1\",\n",
")\n",
"\n",
"agent = initialize_agent(\n",
" tools, llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=True\n",
")\n",
"\n",
"agent.run(\"Send an email to test@testing123.com saying hello world.\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"vscode": {
"languageId": "shellscript"
}
},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": ".venv",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.2"
},
"orig_nbformat": 4
},
"nbformat": 4,
"nbformat_minor": 2
}
docs/extras/modules/agents/tools/integrations/bash.ipynb
-192
View File
@@ -1,192 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "8f210ec3",
"metadata": {},
"source": [
"# Shell Tool\n",
"\n",
"Giving agents access to the shell is powerful (though risky outside a sandboxed environment).\n",
"\n",
"The LLM can use it to execute any shell commands. A common use case for this is letting the LLM interact with your local file system."
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "f7b3767b",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"from langchain.tools import ShellTool\n",
"\n",
"shell_tool = ShellTool()"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "c92ac832-556b-4f66-baa4-b78f965dfba0",
"metadata": {
"tags": []
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Hello World!\n",
"\n",
"real\t0m0.000s\n",
"user\t0m0.000s\n",
"sys\t0m0.000s\n",
"\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"/Users/wfh/code/lc/lckg/langchain/tools/shell/tool.py:34: UserWarning: The shell tool has no safeguards by default. Use at your own risk.\n",
" warnings.warn(\n"
]
}
],
"source": [
"print(shell_tool.run({\"commands\": [\"echo 'Hello World!'\", \"time\"]}))"
]
},
{
"cell_type": "markdown",
"id": "2fa952fc",
"metadata": {},
"source": [
"### Use with Agents\n",
"\n",
"As with all tools, these can be given to an agent to accomplish more complex tasks. Let's have the agent fetch some links from a web page."
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "851fee9f",
"metadata": {
"tags": []
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3mQuestion: What is the task?\n",
"Thought: We need to download the langchain.com webpage and extract all the URLs from it. Then we need to sort the URLs and return them.\n",
"Action:\n",
"```\n",
"{\n",
" \"action\": \"shell\",\n",
" \"action_input\": {\n",
" \"commands\": [\n",
" \"curl -s https://langchain.com | grep -o 'http[s]*://[^\\\" ]*' | sort\"\n",
" ]\n",
" }\n",
"}\n",
"```\n",
"\u001b[0m"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"/Users/wfh/code/lc/lckg/langchain/tools/shell/tool.py:34: UserWarning: The shell tool has no safeguards by default. Use at your own risk.\n",
" warnings.warn(\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"Observation: \u001b[36;1m\u001b[1;3mhttps://blog.langchain.dev/\n",
"https://discord.gg/6adMQxSpJS\n",
"https://docs.langchain.com/docs/\n",
"https://github.com/hwchase17/chat-langchain\n",
"https://github.com/hwchase17/langchain\n",
"https://github.com/hwchase17/langchainjs\n",
"https://github.com/sullivan-sean/chat-langchainjs\n",
"https://js.langchain.com/docs/\n",
"https://python.langchain.com/en/latest/\n",
"https://twitter.com/langchainai\n",
"\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3mThe URLs have been successfully extracted and sorted. We can return the list of URLs as the final answer.\n",
"Final Answer: [\"https://blog.langchain.dev/\", \"https://discord.gg/6adMQxSpJS\", \"https://docs.langchain.com/docs/\", \"https://github.com/hwchase17/chat-langchain\", \"https://github.com/hwchase17/langchain\", \"https://github.com/hwchase17/langchainjs\", \"https://github.com/sullivan-sean/chat-langchainjs\", \"https://js.langchain.com/docs/\", \"https://python.langchain.com/en/latest/\", \"https://twitter.com/langchainai\"]\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"'[\"https://blog.langchain.dev/\", \"https://discord.gg/6adMQxSpJS\", \"https://docs.langchain.com/docs/\", \"https://github.com/hwchase17/chat-langchain\", \"https://github.com/hwchase17/langchain\", \"https://github.com/hwchase17/langchainjs\", \"https://github.com/sullivan-sean/chat-langchainjs\", \"https://js.langchain.com/docs/\", \"https://python.langchain.com/en/latest/\", \"https://twitter.com/langchainai\"]'"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from langchain.chat_models import ChatOpenAI\n",
"from langchain.agents import initialize_agent\n",
"from langchain.agents import AgentType\n",
"\n",
"llm = ChatOpenAI(temperature=0)\n",
"\n",
"shell_tool.description = shell_tool.description + f\"args {shell_tool.args}\".replace(\n",
" \"{\", \"{{\"\n",
").replace(\"}\", \"}}\")\n",
"self_ask_with_search = initialize_agent(\n",
" [shell_tool], llm, agent=AgentType.CHAT_ZERO_SHOT_REACT_DESCRIPTION, verbose=True\n",
")\n",
"self_ask_with_search.run(\n",
" \"Download the langchain.com webpage and grep for all urls. Return only a sorted list of them. Be sure to use double quotes.\"\n",
")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "8d0ea3ac-0890-4e39-9cec-74bd80b4b8b8",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.16"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
docs/extras/modules/agents/tools/integrations/bing_search.ipynb
-193
View File
@@ -1,193 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Bing Search"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This notebook goes over how to use the bing search component.\n",
"\n",
"First, you need to set up the proper API keys and environment variables. To set it up, follow the instructions found [here](https://levelup.gitconnected.com/api-tutorial-how-to-use-bing-web-search-api-in-python-4165d5592a7e).\n",
"\n",
"Then we will need to set some environment variables."
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"\n",
"os.environ[\"BING_SUBSCRIPTION_KEY\"] = \"<key>\"\n",
"os.environ[\"BING_SEARCH_URL\"] = \"https://api.bing.microsoft.com/v7.0/search\""
]
},
{
"cell_type": "code",
"execution_count": 21,
"metadata": {},
"outputs": [],
"source": [
"from langchain.utilities import BingSearchAPIWrapper"
]
},
{
"cell_type": "code",
"execution_count": 22,
"metadata": {},
"outputs": [],
"source": [
"search = BingSearchAPIWrapper()"
]
},
{
"cell_type": "code",
"execution_count": 23,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'Thanks to the flexibility of <b>Python</b> and the powerful ecosystem of packages, the Azure CLI supports features such as autocompletion (in shells that support it), persistent credentials, JMESPath result parsing, lazy initialization, network-less unit tests, and more. Building an open-source and cross-platform Azure CLI with <b>Python</b> by Dan Taylor. <b>Python</b> releases by version number: Release version Release date Click for more. <b>Python</b> 3.11.1 Dec. 6, 2022 Download Release Notes. <b>Python</b> 3.10.9 Dec. 6, 2022 Download Release Notes. <b>Python</b> 3.9.16 Dec. 6, 2022 Download Release Notes. <b>Python</b> 3.8.16 Dec. 6, 2022 Download Release Notes. <b>Python</b> 3.7.16 Dec. 6, 2022 Download Release Notes. In this lesson, we will look at the += operator in <b>Python</b> and see how it works with several simple examples.. The operator ‘+=’ is a shorthand for the addition assignment operator.It adds two values and assigns the sum to a variable (left operand). W3Schools offers free online tutorials, references and exercises in all the major languages of the web. Covering popular subjects like HTML, CSS, JavaScript, <b>Python</b>, SQL, Java, and many, many more. This tutorial introduces the reader informally to the basic concepts and features of the <b>Python</b> language and system. It helps to have a <b>Python</b> interpreter handy for hands-on experience, but all examples are self-contained, so the tutorial can be read off-line as well. For a description of standard objects and modules, see The <b>Python</b> Standard ... <b>Python</b> is a general-purpose, versatile, and powerful programming language. It&#39;s a great first language because <b>Python</b> code is concise and easy to read. Whatever you want to do, <b>python</b> can do it. From web development to machine learning to data science, <b>Python</b> is the language for you. To install <b>Python</b> using the Microsoft Store: Go to your Start menu (lower left Windows icon), type &quot;Microsoft Store&quot;, select the link to open the store. Once the store is open, select Search from the upper-right menu and enter &quot;<b>Python</b>&quot;. Select which version of <b>Python</b> you would like to use from the results under Apps. Under the “<b>Python</b> Releases for Mac OS X” heading, click the link for the Latest <b>Python</b> 3 Release - <b>Python</b> 3.x.x. As of this writing, the latest version was <b>Python</b> 3.8.4. Scroll to the bottom and click macOS 64-bit installer to start the download. When the installer is finished downloading, move on to the next step. Step 2: Run the Installer'"
]
},
"execution_count": 23,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"search.run(\"python\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Number of results\n",
"You can use the `k` parameter to set the number of results"
]
},
{
"cell_type": "code",
"execution_count": 24,
"metadata": {},
"outputs": [],
"source": [
"search = BingSearchAPIWrapper(k=1)"
]
},
{
"cell_type": "code",
"execution_count": 25,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'Thanks to the flexibility of <b>Python</b> and the powerful ecosystem of packages, the Azure CLI supports features such as autocompletion (in shells that support it), persistent credentials, JMESPath result parsing, lazy initialization, network-less unit tests, and more. Building an open-source and cross-platform Azure CLI with <b>Python</b> by Dan Taylor.'"
]
},
"execution_count": 25,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"search.run(\"python\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Metadata Results"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Run query through BingSearch and return snippet, title, and link metadata.\n",
"\n",
"- Snippet: The description of the result.\n",
"- Title: The title of the result.\n",
"- Link: The link to the result."
]
},
{
"cell_type": "code",
"execution_count": 26,
"metadata": {},
"outputs": [],
"source": [
"search = BingSearchAPIWrapper()"
]
},
{
"cell_type": "code",
"execution_count": 27,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[{'snippet': 'Lady Alice. Pink Lady <b>apples</b> aren’t the only lady in the apple family. Lady Alice <b>apples</b> were discovered growing, thanks to bees pollinating, in Washington. They are smaller and slightly more stout in appearance than other varieties. Their skin color appears to have red and yellow stripes running from stem to butt.',\n",
" 'title': '25 Types of Apples - Jessica Gavin',\n",
" 'link': 'https://www.jessicagavin.com/types-of-apples/'},\n",
" {'snippet': '<b>Apples</b> can do a lot for you, thanks to plant chemicals called flavonoids. And they have pectin, a fiber that breaks down in your gut. If you take off the apple’s skin before eating it, you won ...',\n",
" 'title': 'Apples: Nutrition &amp; Health Benefits - WebMD',\n",
" 'link': 'https://www.webmd.com/food-recipes/benefits-apples'},\n",
" {'snippet': '<b>Apples</b> boast many vitamins and minerals, though not in high amounts. However, <b>apples</b> are usually a good source of vitamin C. Vitamin C. Also called ascorbic acid, this vitamin is a common ...',\n",
" 'title': 'Apples 101: Nutrition Facts and Health Benefits',\n",
" 'link': 'https://www.healthline.com/nutrition/foods/apples'},\n",
" {'snippet': 'Weight management. The fibers in <b>apples</b> can slow digestion, helping one to feel greater satisfaction after eating. After following three large prospective cohorts of 133,468 men and women for 24 years, researchers found that higher intakes of fiber-rich fruits with a low glycemic load, particularly <b>apples</b> and pears, were associated with the least amount of weight gain over time.',\n",
" 'title': 'Apples | The Nutrition Source | Harvard T.H. Chan School of Public Health',\n",
" 'link': 'https://www.hsph.harvard.edu/nutritionsource/food-features/apples/'}]"
]
},
"execution_count": 27,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"search.results(\"apples\", 5)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.9"
},
"vscode": {
"interpreter": {
"hash": "a0a0263b650d907a3bfe41c0f8d6a63a071b884df3cfdc1579f00cdc1aed6b03"
}
}
},
"nbformat": 4,
"nbformat_minor": 2
}
docs/extras/modules/agents/tools/integrations/brave_search.ipynb
-94
View File
@@ -1,94 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "eda326e4",
"metadata": {},
"source": [
"# Brave Search\n",
"\n",
"This notebook goes over how to use the Brave Search tool."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "a4c896e5",
"metadata": {},
"outputs": [],
"source": [
"from langchain.tools import BraveSearch"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "6784d37c",
"metadata": {},
"outputs": [],
"source": [
"api_key = \"BSAv1neIuQOsxqOyy0sEe_ie2zD_n_V\""
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "5b14008a",
"metadata": {},
"outputs": [],
"source": [
"tool = BraveSearch.from_api_key(api_key=api_key, search_kwargs={\"count\": 3})"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "f11937b2",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'[{\"title\": \"Obama\\'s Middle Name -- My Last Name -- is \\'Hussein.\\' So?\", \"link\": \"https://www.cair.com/cair_in_the_news/obamas-middle-name-my-last-name-is-hussein-so/\", \"snippet\": \"I wasn\\\\u2019t sure whether to laugh or cry a few days back listening to radio talk show host Bill Cunningham repeatedly scream Barack <strong>Obama</strong>\\\\u2019<strong>s</strong> <strong>middle</strong> <strong>name</strong> \\\\u2014 my last <strong>name</strong> \\\\u2014 as if he had anti-Muslim Tourette\\\\u2019s. \\\\u201cHussein,\\\\u201d Cunningham hissed like he was beckoning Satan when shouting the ...\"}, {\"title\": \"What\\'s up with Obama\\'s middle name? - Quora\", \"link\": \"https://www.quora.com/Whats-up-with-Obamas-middle-name\", \"snippet\": \"Answer (1 of 15): A better question would be, \\\\u201cWhat\\\\u2019s up with <strong>Obama</strong>\\\\u2019s first <strong>name</strong>?\\\\u201d President Barack Hussein <strong>Obama</strong>\\\\u2019s father\\\\u2019s <strong>name</strong> was Barack Hussein <strong>Obama</strong>. He was <strong>named</strong> after his father. Hussein, <strong>Obama</strong>\\\\u2019<strong>s</strong> <strong>middle</strong> <strong>name</strong>, is a very common Arabic <strong>name</strong>, meaning &quot;good,&quot; &quot;handsome,&quot; or ...\"}, {\"title\": \"Barack Obama | Biography, Parents, Education, Presidency, Books, ...\", \"link\": \"https://www.britannica.com/biography/Barack-Obama\", \"snippet\": \"Barack <strong>Obama</strong>, in full Barack Hussein <strong>Obama</strong> II, (born August 4, 1961, Honolulu, Hawaii, U.S.), 44th president of the United States (2009\\\\u201317) and the first African American to hold the office. Before winning the presidency, <strong>Obama</strong> represented Illinois in the U.S.\"}]'"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"tool.run(\"obama middle name\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "da9c63d5",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.6"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
docs/extras/modules/agents/tools/integrations/chatgpt_plugins.ipynb
-123
View File
@@ -1,123 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "3f34700b",
"metadata": {},
"source": [
"# ChatGPT Plugins\n",
"\n",
"This example shows how to use ChatGPT Plugins within LangChain abstractions.\n",
"\n",
"Note 1: This currently only works for plugins with no auth.\n",
"\n",
"Note 2: There are almost certainly other ways to do this, this is just a first pass. If you have better ideas, please open a PR!"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "d41405b5",
"metadata": {},
"outputs": [],
"source": [
"from langchain.chat_models import ChatOpenAI\n",
"from langchain.agents import load_tools, initialize_agent\n",
"from langchain.agents import AgentType\n",
"from langchain.tools import AIPluginTool"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "d9e61df5",
"metadata": {},
"outputs": [],
"source": [
"tool = AIPluginTool.from_plugin_url(\"https://www.klarna.com/.well-known/ai-plugin.json\")"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "edc0ea0e",
"metadata": {
"scrolled": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3mI need to check the Klarna Shopping API to see if it has information on available t shirts.\n",
"Action: KlarnaProducts\n",
"Action Input: None\u001b[0m\n",
"Observation: \u001b[33;1m\u001b[1;3mUsage Guide: Use the Klarna plugin to get relevant product suggestions for any shopping or researching purpose. The query to be sent should not include stopwords like articles, prepositions and determinants. The api works best when searching for words that are related to products, like their name, brand, model or category. Links will always be returned and should be shown to the user.\n",
"\n",
"OpenAPI Spec: {'openapi': '3.0.1', 'info': {'version': 'v0', 'title': 'Open AI Klarna product Api'}, 'servers': [{'url': 'https://www.klarna.com/us/shopping'}], 'tags': [{'name': 'open-ai-product-endpoint', 'description': 'Open AI Product Endpoint. Query for products.'}], 'paths': {'/public/openai/v0/products': {'get': {'tags': ['open-ai-product-endpoint'], 'summary': 'API for fetching Klarna product information', 'operationId': 'productsUsingGET', 'parameters': [{'name': 'q', 'in': 'query', 'description': 'query, must be between 2 and 100 characters', 'required': True, 'schema': {'type': 'string'}}, {'name': 'size', 'in': 'query', 'description': 'number of products returned', 'required': False, 'schema': {'type': 'integer'}}, {'name': 'budget', 'in': 'query', 'description': 'maximum price of the matching product in local currency, filters results', 'required': False, 'schema': {'type': 'integer'}}], 'responses': {'200': {'description': 'Products found', 'content': {'application/json': {'schema': {'$ref': '#/components/schemas/ProductResponse'}}}}, '503': {'description': 'one or more services are unavailable'}}, 'deprecated': False}}}, 'components': {'schemas': {'Product': {'type': 'object', 'properties': {'attributes': {'type': 'array', 'items': {'type': 'string'}}, 'name': {'type': 'string'}, 'price': {'type': 'string'}, 'url': {'type': 'string'}}, 'title': 'Product'}, 'ProductResponse': {'type': 'object', 'properties': {'products': {'type': 'array', 'items': {'$ref': '#/components/schemas/Product'}}}, 'title': 'ProductResponse'}}}}\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3mI need to use the Klarna Shopping API to search for t shirts.\n",
"Action: requests_get\n",
"Action Input: https://www.klarna.com/us/shopping/public/openai/v0/products?q=t%20shirts\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3m{\"products\":[{\"name\":\"Lacoste Men's Pack of Plain T-Shirts\",\"url\":\"https://www.klarna.com/us/shopping/pl/cl10001/3202043025/Clothing/Lacoste-Men-s-Pack-of-Plain-T-Shirts/?utm_source=openai\",\"price\":\"$26.60\",\"attributes\":[\"Material:Cotton\",\"Target Group:Man\",\"Color:White,Black\"]},{\"name\":\"Hanes Men's Ultimate 6pk. Crewneck T-Shirts\",\"url\":\"https://www.klarna.com/us/shopping/pl/cl10001/3201808270/Clothing/Hanes-Men-s-Ultimate-6pk.-Crewneck-T-Shirts/?utm_source=openai\",\"price\":\"$13.82\",\"attributes\":[\"Material:Cotton\",\"Target Group:Man\",\"Color:White\"]},{\"name\":\"Nike Boy's Jordan Stretch T-shirts\",\"url\":\"https://www.klarna.com/us/shopping/pl/cl359/3201863202/Children-s-Clothing/Nike-Boy-s-Jordan-Stretch-T-shirts/?utm_source=openai\",\"price\":\"$14.99\",\"attributes\":[\"Material:Cotton\",\"Color:White,Green\",\"Model:Boy\",\"Size (Small-Large):S,XL,L,M\"]},{\"name\":\"Polo Classic Fit Cotton V-Neck T-Shirts 3-Pack\",\"url\":\"https://www.klarna.com/us/shopping/pl/cl10001/3203028500/Clothing/Polo-Classic-Fit-Cotton-V-Neck-T-Shirts-3-Pack/?utm_source=openai\",\"price\":\"$29.95\",\"attributes\":[\"Material:Cotton\",\"Target Group:Man\",\"Color:White,Blue,Black\"]},{\"name\":\"adidas Comfort T-shirts Men's 3-pack\",\"url\":\"https://www.klarna.com/us/shopping/pl/cl10001/3202640533/Clothing/adidas-Comfort-T-shirts-Men-s-3-pack/?utm_source=openai\",\"price\":\"$14.99\",\"attributes\":[\"Material:Cotton\",\"Target Group:Man\",\"Color:White,Black\",\"Neckline:Round\"]}]}\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3mThe available t shirts in Klarna are Lacoste Men's Pack of Plain T-Shirts, Hanes Men's Ultimate 6pk. Crewneck T-Shirts, Nike Boy's Jordan Stretch T-shirts, Polo Classic Fit Cotton V-Neck T-Shirts 3-Pack, and adidas Comfort T-shirts Men's 3-pack.\n",
"Final Answer: The available t shirts in Klarna are Lacoste Men's Pack of Plain T-Shirts, Hanes Men's Ultimate 6pk. Crewneck T-Shirts, Nike Boy's Jordan Stretch T-shirts, Polo Classic Fit Cotton V-Neck T-Shirts 3-Pack, and adidas Comfort T-shirts Men's 3-pack.\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"\"The available t shirts in Klarna are Lacoste Men's Pack of Plain T-Shirts, Hanes Men's Ultimate 6pk. Crewneck T-Shirts, Nike Boy's Jordan Stretch T-shirts, Polo Classic Fit Cotton V-Neck T-Shirts 3-Pack, and adidas Comfort T-shirts Men's 3-pack.\""
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"llm = ChatOpenAI(temperature=0)\n",
"tools = load_tools([\"requests_all\"])\n",
"tools += [tool]\n",
"\n",
"agent_chain = initialize_agent(\n",
" tools, llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=True\n",
")\n",
"agent_chain.run(\"what t shirts are available in klarna?\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e49318a4",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.1"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
docs/extras/modules/agents/tools/integrations/dataforseo.ipynb
-237
View File
@@ -1,237 +0,0 @@
{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"# DataForSeo API Wrapper\n",
"This notebook demonstrates how to use the DataForSeo API wrapper to obtain search engine results. The DataForSeo API allows users to retrieve SERP from most popular search engines like Google, Bing, Yahoo. It also allows to get SERPs from different search engine types like Maps, News, Events, etc.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from langchain.utilities import DataForSeoAPIWrapper"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Setting up the API wrapper with your credentials\n",
"You can obtain your API credentials by registering on the DataForSeo website."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"\n",
"os.environ[\"DATAFORSEO_LOGIN\"] = \"your_api_access_username\"\n",
"os.environ[\"DATAFORSEO_PASSWORD\"] = \"your_api_access_password\"\n",
"\n",
"wrapper = DataForSeoAPIWrapper()"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"The run method will return the first result snippet from one of the following elements: answer_box, knowledge_graph, featured_snippet, shopping, organic."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"wrapper.run(\"Weather in Los Angeles\")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## The Difference Between `run` and `results`\n",
"`run` and `results` are two methods provided by the `DataForSeoAPIWrapper` class.\n",
"\n",
"The `run` method executes the search and returns the first result snippet from the answer box, knowledge graph, featured snippet, shopping, or organic results. These elements are sorted by priority from highest to lowest.\n",
"\n",
"The `results` method returns a JSON response configured according to the parameters set in the wrapper. This allows for more flexibility in terms of what data you want to return from the API."
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Getting Results as JSON\n",
"You can customize the result types and fields you want to return in the JSON response. You can also set a maximum count for the number of top results to return."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"json_wrapper = DataForSeoAPIWrapper(\n",
" json_result_types=[\"organic\", \"knowledge_graph\", \"answer_box\"],\n",
" json_result_fields=[\"type\", \"title\", \"description\", \"text\"],\n",
" top_count=3,\n",
")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"json_wrapper.results(\"Bill Gates\")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Customizing Location and Language\n",
"You can specify the location and language of your search results by passing additional parameters to the API wrapper."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"customized_wrapper = DataForSeoAPIWrapper(\n",
" top_count=10,\n",
" json_result_types=[\"organic\", \"local_pack\"],\n",
" json_result_fields=[\"title\", \"description\", \"type\"],\n",
" params={\"location_name\": \"Germany\", \"language_code\": \"en\"},\n",
")\n",
"customized_wrapper.results(\"coffee near me\")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Customizing the Search Engine\n",
"You can also specify the search engine you want to use."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"customized_wrapper = DataForSeoAPIWrapper(\n",
" top_count=10,\n",
" json_result_types=[\"organic\", \"local_pack\"],\n",
" json_result_fields=[\"title\", \"description\", \"type\"],\n",
" params={\"location_name\": \"Germany\", \"language_code\": \"en\", \"se_name\": \"bing\"},\n",
")\n",
"customized_wrapper.results(\"coffee near me\")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Customizing the Search Type\n",
"The API wrapper also allows you to specify the type of search you want to perform. For example, you can perform a maps search."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"maps_search = DataForSeoAPIWrapper(\n",
" top_count=10,\n",
" json_result_fields=[\"title\", \"value\", \"address\", \"rating\", \"type\"],\n",
" params={\n",
" \"location_coordinate\": \"52.512,13.36,12z\",\n",
" \"language_code\": \"en\",\n",
" \"se_type\": \"maps\",\n",
" },\n",
")\n",
"maps_search.results(\"coffee near me\")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Integration with Langchain Agents\n",
"You can use the `Tool` class from the `langchain.agents` module to integrate the `DataForSeoAPIWrapper` with a langchain agent. The `Tool` class encapsulates a function that the agent can call."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from langchain.agents import Tool\n",
"\n",
"search = DataForSeoAPIWrapper(\n",
" top_count=3,\n",
" json_result_types=[\"organic\"],\n",
" json_result_fields=[\"title\", \"description\", \"type\"],\n",
")\n",
"tool = Tool(\n",
" name=\"google-search-answer\",\n",
" description=\"My new answer tool\",\n",
" func=search.run,\n",
")\n",
"json_tool = Tool(\n",
" name=\"google-search-json\",\n",
" description=\"My new json tool\",\n",
" func=search.results,\n",
")"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.11"
},
"orig_nbformat": 4
},
"nbformat": 4,
"nbformat_minor": 2
}
docs/extras/modules/agents/tools/integrations/ddg.ipynb
-91
View File
@@ -1,91 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "245a954a",
"metadata": {},
"source": [
"# DuckDuckGo Search\n",
"\n",
"This notebook goes over how to use the duck-duck-go search component."
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "21e46d4d",
"metadata": {},
"outputs": [],
"source": [
"# !pip install duckduckgo-search"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "ac4910f8",
"metadata": {},
"outputs": [],
"source": [
"from langchain.tools import DuckDuckGoSearchRun"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "84b8f773",
"metadata": {},
"outputs": [],
"source": [
"search = DuckDuckGoSearchRun()"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "068991a6",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'Barack Obama, in full Barack Hussein Obama II, (born August 4, 1961, Honolulu, Hawaii, U.S.), 44th president of the United States (2009-17) and the first African American to hold the office. Before winning the presidency, Obama represented Illinois in the U.S. Senate (2005-08). Barack Hussein Obama II (/ b ə ˈ r ɑː k h uː ˈ s eɪ n oʊ ˈ b ɑː m ə / bə-RAHK hoo-SAYN oh-BAH-mə; born August 4, 1961) is an American former politician who served as the 44th president of the United States from 2009 to 2017. A member of the Democratic Party, he was the first African-American president of the United States. Obama previously served as a U.S. senator representing ... Barack Obama was the first African American president of the United States (2009-17). He oversaw the recovery of the U.S. economy (from the Great Recession of 2008-09) and the enactment of landmark health care reform (the Patient Protection and Affordable Care Act ). In 2009 he was awarded the Nobel Peace Prize. His birth certificate lists his first name as Barack: That\\'s how Obama has spelled his name throughout his life. His name derives from a Hebrew name which means \"lightning.\". The Hebrew word has been transliterated into English in various spellings, including Barak, Buraq, Burack, and Barack. Most common names of U.S. presidents 1789-2021. Published by. Aaron O\\'Neill , Jun 21, 2022. The most common first name for a U.S. president is James, followed by John and then William. Six U.S ...'"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"search.run(\"Obama's first name?\")"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.1"
},
"vscode": {
"interpreter": {
"hash": "a0a0263b650d907a3bfe41c0f8d6a63a071b884df3cfdc1579f00cdc1aed6b03"
}
}
},
"nbformat": 4,
"nbformat_minor": 5
}
docs/extras/modules/agents/tools/integrations/filesystem.ipynb
-195
View File
@@ -1,195 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# File System Tools\n",
"\n",
"LangChain provides tools for interacting with a local file system out of the box. This notebook walks through some of them.\n",
"\n",
"Note: these tools are not recommended for use outside a sandboxed environment! "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"First, we'll import the tools."
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"from langchain.tools.file_management import (\n",
" ReadFileTool,\n",
" CopyFileTool,\n",
" DeleteFileTool,\n",
" MoveFileTool,\n",
" WriteFileTool,\n",
" ListDirectoryTool,\n",
")\n",
"from langchain.agents.agent_toolkits import FileManagementToolkit\n",
"from tempfile import TemporaryDirectory\n",
"\n",
"# We'll make a temporary directory to avoid clutter\n",
"working_directory = TemporaryDirectory()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## The FileManagementToolkit\n",
"\n",
"If you want to provide all the file tooling to your agent, it's easy to do so with the toolkit. We'll pass the temporary directory in as a root directory as a workspace for the LLM.\n",
"\n",
"It's recommended to always pass in a root directory, since without one, it's easy for the LLM to pollute the working directory, and without one, there isn't any validation against\n",
"straightforward prompt injection."
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"tags": []
},
"outputs": [
{
"data": {
"text/plain": [
"[CopyFileTool(name='copy_file', description='Create a copy of a file in a specified location', args_schema=<class 'langchain.tools.file_management.copy.FileCopyInput'>, return_direct=False, verbose=False, callback_manager=<langchain.callbacks.shared.SharedCallbackManager object at 0x1156f4350>, root_dir='/var/folders/gf/6rnp_mbx5914kx7qmmh7xzmw0000gn/T/tmpxb8c3aug'),\n",
" DeleteFileTool(name='file_delete', description='Delete a file', args_schema=<class 'langchain.tools.file_management.delete.FileDeleteInput'>, return_direct=False, verbose=False, callback_manager=<langchain.callbacks.shared.SharedCallbackManager object at 0x1156f4350>, root_dir='/var/folders/gf/6rnp_mbx5914kx7qmmh7xzmw0000gn/T/tmpxb8c3aug'),\n",
" FileSearchTool(name='file_search', description='Recursively search for files in a subdirectory that match the regex pattern', args_schema=<class 'langchain.tools.file_management.file_search.FileSearchInput'>, return_direct=False, verbose=False, callback_manager=<langchain.callbacks.shared.SharedCallbackManager object at 0x1156f4350>, root_dir='/var/folders/gf/6rnp_mbx5914kx7qmmh7xzmw0000gn/T/tmpxb8c3aug'),\n",
" MoveFileTool(name='move_file', description='Move or rename a file from one location to another', args_schema=<class 'langchain.tools.file_management.move.FileMoveInput'>, return_direct=False, verbose=False, callback_manager=<langchain.callbacks.shared.SharedCallbackManager object at 0x1156f4350>, root_dir='/var/folders/gf/6rnp_mbx5914kx7qmmh7xzmw0000gn/T/tmpxb8c3aug'),\n",
" ReadFileTool(name='read_file', description='Read file from disk', args_schema=<class 'langchain.tools.file_management.read.ReadFileInput'>, return_direct=False, verbose=False, callback_manager=<langchain.callbacks.shared.SharedCallbackManager object at 0x1156f4350>, root_dir='/var/folders/gf/6rnp_mbx5914kx7qmmh7xzmw0000gn/T/tmpxb8c3aug'),\n",
" WriteFileTool(name='write_file', description='Write file to disk', args_schema=<class 'langchain.tools.file_management.write.WriteFileInput'>, return_direct=False, verbose=False, callback_manager=<langchain.callbacks.shared.SharedCallbackManager object at 0x1156f4350>, root_dir='/var/folders/gf/6rnp_mbx5914kx7qmmh7xzmw0000gn/T/tmpxb8c3aug'),\n",
" ListDirectoryTool(name='list_directory', description='List files and directories in a specified folder', args_schema=<class 'langchain.tools.file_management.list_dir.DirectoryListingInput'>, return_direct=False, verbose=False, callback_manager=<langchain.callbacks.shared.SharedCallbackManager object at 0x1156f4350>, root_dir='/var/folders/gf/6rnp_mbx5914kx7qmmh7xzmw0000gn/T/tmpxb8c3aug')]"
]
},
"execution_count": 2,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"toolkit = FileManagementToolkit(\n",
" root_dir=str(working_directory.name)\n",
") # If you don't provide a root_dir, operations will default to the current working directory\n",
"toolkit.get_tools()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Selecting File System Tools\n",
"\n",
"If you only want to select certain tools, you can pass them in as arguments when initializing the toolkit, or you can individually initialize the desired tools."
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"tags": []
},
"outputs": [
{
"data": {
"text/plain": [
"[ReadFileTool(name='read_file', description='Read file from disk', args_schema=<class 'langchain.tools.file_management.read.ReadFileInput'>, return_direct=False, verbose=False, callback_manager=<langchain.callbacks.shared.SharedCallbackManager object at 0x1156f4350>, root_dir='/var/folders/gf/6rnp_mbx5914kx7qmmh7xzmw0000gn/T/tmpxb8c3aug'),\n",
" WriteFileTool(name='write_file', description='Write file to disk', args_schema=<class 'langchain.tools.file_management.write.WriteFileInput'>, return_direct=False, verbose=False, callback_manager=<langchain.callbacks.shared.SharedCallbackManager object at 0x1156f4350>, root_dir='/var/folders/gf/6rnp_mbx5914kx7qmmh7xzmw0000gn/T/tmpxb8c3aug'),\n",
" ListDirectoryTool(name='list_directory', description='List files and directories in a specified folder', args_schema=<class 'langchain.tools.file_management.list_dir.DirectoryListingInput'>, return_direct=False, verbose=False, callback_manager=<langchain.callbacks.shared.SharedCallbackManager object at 0x1156f4350>, root_dir='/var/folders/gf/6rnp_mbx5914kx7qmmh7xzmw0000gn/T/tmpxb8c3aug')]"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"tools = FileManagementToolkit(\n",
" root_dir=str(working_directory.name),\n",
" selected_tools=[\"read_file\", \"write_file\", \"list_directory\"],\n",
").get_tools()\n",
"tools"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'File written successfully to example.txt.'"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"read_tool, write_tool, list_tool = tools\n",
"write_tool.run({\"file_path\": \"example.txt\", \"text\": \"Hello World!\"})"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'example.txt'"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# List files in the working directory\n",
"list_tool.run({})"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.2"
}
},
"nbformat": 4,
"nbformat_minor": 4
}
docs/extras/modules/agents/tools/integrations/golden_query.ipynb
-142
View File
@@ -1,142 +0,0 @@
{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"id": "245a954a",
"metadata": {},
"source": [
"# Golden Query\n",
"\n",
"This notebook goes over how to use the golden-query tool.\n",
"\n",
"- Go to the [Golden API docs](https://docs.golden.com/) to get an overview about the Golden API.\n",
"- Create a Golden account if you don't have one on the [Golden Website](golden.com).\n",
"- Get your API key from the [Golden API Settings](https://golden.com/settings/api) page.\n",
"- Save your API key into GOLDEN_API_KEY env variable"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "34bb5968",
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"\n",
"os.environ[\"GOLDEN_API_KEY\"] = \"\""
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "ac4910f8",
"metadata": {},
"outputs": [],
"source": [
"from langchain.utilities.golden_query import GoldenQueryAPIWrapper"
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "84b8f773",
"metadata": {},
"outputs": [],
"source": [
"golden_query = GoldenQueryAPIWrapper()"
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "068991a6",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"{'results': [{'id': 4673886,\n",
" 'latestVersionId': 60276991,\n",
" 'properties': [{'predicateId': 'name',\n",
" 'instances': [{'value': 'Samsung', 'citations': []}]}]},\n",
" {'id': 7008,\n",
" 'latestVersionId': 61087416,\n",
" 'properties': [{'predicateId': 'name',\n",
" 'instances': [{'value': 'Intel', 'citations': []}]}]},\n",
" {'id': 24193,\n",
" 'latestVersionId': 60274482,\n",
" 'properties': [{'predicateId': 'name',\n",
" 'instances': [{'value': 'Texas Instruments', 'citations': []}]}]},\n",
" {'id': 1142,\n",
" 'latestVersionId': 61406205,\n",
" 'properties': [{'predicateId': 'name',\n",
" 'instances': [{'value': 'Advanced Micro Devices', 'citations': []}]}]},\n",
" {'id': 193948,\n",
" 'latestVersionId': 58326582,\n",
" 'properties': [{'predicateId': 'name',\n",
" 'instances': [{'value': 'Freescale Semiconductor', 'citations': []}]}]},\n",
" {'id': 91316,\n",
" 'latestVersionId': 60387380,\n",
" 'properties': [{'predicateId': 'name',\n",
" 'instances': [{'value': 'Agilent Technologies', 'citations': []}]}]},\n",
" {'id': 90014,\n",
" 'latestVersionId': 60388078,\n",
" 'properties': [{'predicateId': 'name',\n",
" 'instances': [{'value': 'Novartis', 'citations': []}]}]},\n",
" {'id': 237458,\n",
" 'latestVersionId': 61406160,\n",
" 'properties': [{'predicateId': 'name',\n",
" 'instances': [{'value': 'Analog Devices', 'citations': []}]}]},\n",
" {'id': 3941943,\n",
" 'latestVersionId': 60382250,\n",
" 'properties': [{'predicateId': 'name',\n",
" 'instances': [{'value': 'AbbVie Inc.', 'citations': []}]}]},\n",
" {'id': 4178762,\n",
" 'latestVersionId': 60542667,\n",
" 'properties': [{'predicateId': 'name',\n",
" 'instances': [{'value': 'IBM', 'citations': []}]}]}],\n",
" 'next': 'https://golden.com/api/v2/public/queries/59044/results/?cursor=eyJwb3NpdGlvbiI6IFsxNzYxNiwgIklCTS04M1lQM1oiXX0%3D&pageSize=10',\n",
" 'previous': None}"
]
},
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"import json\n",
"\n",
"json.loads(golden_query.run(\"companies in nanotech\"))"
]
}
],
"metadata": {
"kernelspec": {
"display_name": ".venv",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.13"
},
"vscode": {
"interpreter": {
"hash": "53f3bc57609c7a84333bb558594977aa5b4026b1d6070b93987956689e367341"
}
}
},
"nbformat": 4,
"nbformat_minor": 5
}
docs/extras/modules/agents/tools/integrations/google_places.ipynb
-106
View File
@@ -1,106 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "487607cd",
"metadata": {},
"source": [
"# Google Places\n",
"\n",
"This notebook goes through how to use Google Places API"
]
},
{
"cell_type": "code",
"execution_count": 10,
"id": "8690845f",
"metadata": {},
"outputs": [],
"source": [
"#!pip install googlemaps"
]
},
{
"cell_type": "code",
"execution_count": 12,
"id": "fae31ef4",
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"\n",
"os.environ[\"GPLACES_API_KEY\"] = \"\""
]
},
{
"cell_type": "code",
"execution_count": 13,
"id": "abb502b3",
"metadata": {},
"outputs": [],
"source": [
"from langchain.tools import GooglePlacesTool"
]
},
{
"cell_type": "code",
"execution_count": 14,
"id": "a83a02ac",
"metadata": {},
"outputs": [],
"source": [
"places = GooglePlacesTool()"
]
},
{
"cell_type": "code",
"execution_count": 16,
"id": "2b65a285",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"\"1. Delfina Restaurant\\nAddress: 3621 18th St, San Francisco, CA 94110, USA\\nPhone: (415) 552-4055\\nWebsite: https://www.delfinasf.com/\\n\\n\\n2. Piccolo Forno\\nAddress: 725 Columbus Ave, San Francisco, CA 94133, USA\\nPhone: (415) 757-0087\\nWebsite: https://piccolo-forno-sf.com/\\n\\n\\n3. L'Osteria del Forno\\nAddress: 519 Columbus Ave, San Francisco, CA 94133, USA\\nPhone: (415) 982-1124\\nWebsite: Unknown\\n\\n\\n4. Il Fornaio\\nAddress: 1265 Battery St, San Francisco, CA 94111, USA\\nPhone: (415) 986-0100\\nWebsite: https://www.ilfornaio.com/\\n\\n\""
]
},
"execution_count": 16,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"places.run(\"al fornos\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "66d3da8a",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.1"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
docs/extras/modules/agents/tools/integrations/google_search.ipynb
-200
View File
@@ -1,200 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "245a954a",
"metadata": {},
"source": [
"# Google Search\n",
"\n",
"This notebook goes over how to use the google search component.\n",
"\n",
"First, you need to set up the proper API keys and environment variables. To set it up, create the GOOGLE_API_KEY in the Google Cloud credential console (https://console.cloud.google.com/apis/credentials) and a GOOGLE_CSE_ID using the Programmable Search Enginge (https://programmablesearchengine.google.com/controlpanel/create). Next, it is good to follow the instructions found [here](https://stackoverflow.com/questions/37083058/programmatically-searching-google-in-python-using-custom-search).\n",
"\n",
"Then we will need to set some environment variables."
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "34bb5968",
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"\n",
"os.environ[\"GOOGLE_CSE_ID\"] = \"\"\n",
"os.environ[\"GOOGLE_API_KEY\"] = \"\""
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "ac4910f8",
"metadata": {},
"outputs": [],
"source": [
"from langchain.tools import Tool\n",
"from langchain.utilities import GoogleSearchAPIWrapper\n",
"\n",
"search = GoogleSearchAPIWrapper()\n",
"\n",
"tool = Tool(\n",
" name=\"Google Search\",\n",
" description=\"Search Google for recent results.\",\n",
" func=search.run,\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "84b8f773",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"\"STATE OF HAWAII. 1 Child's First Name. (Type or print). 2. Sex. BARACK. 3. This Birth. CERTIFICATE OF LIVE BIRTH. FILE. NUMBER 151 le. lb. Middle Name. Barack Hussein Obama II is an American former politician who served as the 44th president of the United States from 2009 to 2017. A member of the Democratic\\xa0... When Barack Obama was elected president in 2008, he became the first African American to hold ... The Middle East remained a key foreign policy challenge. Jan 19, 2017 ... Jordan Barack Treasure, New York City, born in 2008 ... Jordan Barack Treasure made national news when he was the focus of a New York newspaper\\xa0... Portrait of George Washington, the 1st President of the United States ... Portrait of Barack Obama, the 44th President of the United States\\xa0... His full name is Barack Hussein Obama II. Since the “II” is simply because he was named for his father, his last name is Obama. Mar 22, 2008 ... Barry Obama decided that he didn't like his nickname. A few of his friends at Occidental College had already begun to call him Barack (his\\xa0... Aug 18, 2017 ... It took him several seconds and multiple clues to remember former President Barack Obama's first name. Miller knew that every answer had to\\xa0... Feb 9, 2015 ... Michael Jordan misspelled Barack Obama's first name on 50th-birthday gift ... Knowing Obama is a Chicagoan and huge basketball fan,\\xa0... 4 days ago ... Barack Obama, in full Barack Hussein Obama II, (born August 4, 1961, Honolulu, Hawaii, U.S.), 44th president of the United States (2009–17) and\\xa0...\""
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"tool.run(\"Obama's first name?\")"
]
},
{
"cell_type": "markdown",
"id": "074b7f07",
"metadata": {},
"source": [
"## Number of Results\n",
"You can use the `k` parameter to set the number of results"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "5083fbdd",
"metadata": {},
"outputs": [],
"source": [
"search = GoogleSearchAPIWrapper(k=1)\n",
"\n",
"tool = Tool(\n",
" name=\"I'm Feeling Lucky\",\n",
" description=\"Search Google and return the first result.\",\n",
" func=search.run,\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "77aaa857",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'The official home of the Python Programming Language.'"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"tool.run(\"python\")"
]
},
{
"cell_type": "markdown",
"id": "11c8d94f",
"metadata": {},
"source": [
"'The official home of the Python Programming Language.'"
]
},
{
"cell_type": "markdown",
"id": "73473110",
"metadata": {},
"source": [
"## Metadata Results"
]
},
{
"cell_type": "markdown",
"id": "109fe796",
"metadata": {},
"source": [
"Run query through GoogleSearch and return snippet, title, and link metadata.\n",
"\n",
"- Snippet: The description of the result.\n",
"- Title: The title of the result.\n",
"- Link: The link to the result."
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "028f4cba",
"metadata": {},
"outputs": [],
"source": [
"search = GoogleSearchAPIWrapper()\n",
"\n",
"\n",
"def top5_results(query):\n",
" return search.results(query, 5)\n",
"\n",
"\n",
"tool = Tool(\n",
" name=\"Google Search Snippets\",\n",
" description=\"Search Google for recent results.\",\n",
" func=top5_results,\n",
")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "4d7f92e1",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.1"
},
"vscode": {
"interpreter": {
"hash": "a0a0263b650d907a3bfe41c0f8d6a63a071b884df3cfdc1579f00cdc1aed6b03"
}
}
},
"nbformat": 4,
"nbformat_minor": 5
}
docs/extras/modules/agents/tools/integrations/google_serper.ipynb
-893
View File
@@ -1,893 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "dc23c48e",
"metadata": {},
"source": [
"# Google Serper API\n",
"\n",
"This notebook goes over how to use the Google Serper component to search the web. First you need to sign up for a free account at [serper.dev](https://serper.dev) and get your api key."
]
},
{
"cell_type": "code",
"execution_count": 11,
"outputs": [],
"source": [
"import os\n",
"import pprint\n",
"\n",
"os.environ[\"SERPER_API_KEY\"] = \"\""
],
"metadata": {
"collapsed": false,
"pycharm": {
"is_executing": true
},
"ExecuteTime": {
"end_time": "2023-05-04T00:56:29.336521Z",
"start_time": "2023-05-04T00:56:29.334173Z"
}
},
"id": "a8acfb24"
},
{
"cell_type": "code",
"execution_count": 2,
"id": "54bf5afd",
"metadata": {
"ExecuteTime": {
"end_time": "2023-05-04T00:54:07.676293Z",
"start_time": "2023-05-04T00:54:06.665742Z"
}
},
"outputs": [],
"source": [
"from langchain.utilities import GoogleSerperAPIWrapper"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "31f8f382",
"metadata": {
"ExecuteTime": {
"end_time": "2023-05-04T00:54:08.324245Z",
"start_time": "2023-05-04T00:54:08.321577Z"
}
},
"outputs": [],
"source": [
"search = GoogleSerperAPIWrapper()"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "25ce0225",
"metadata": {
"ExecuteTime": {
"end_time": "2023-05-04T00:54:11.399847Z",
"start_time": "2023-05-04T00:54:09.335597Z"
}
},
"outputs": [
{
"data": {
"text/plain": "'Barack Hussein Obama II'"
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"search.run(\"Obama's first name?\")"
]
},
{
"cell_type": "markdown",
"source": [
"## As part of a Self Ask With Search Chain"
],
"metadata": {
"collapsed": false
},
"id": "1f1c6c22"
},
{
"cell_type": "code",
"execution_count": 5,
"outputs": [],
"source": [
"os.environ[\"OPENAI_API_KEY\"] = \"\""
],
"metadata": {
"collapsed": false,
"ExecuteTime": {
"end_time": "2023-05-04T00:54:14.311773Z",
"start_time": "2023-05-04T00:54:14.304389Z"
}
},
"id": "c1b5edd7"
},
{
"cell_type": "code",
"execution_count": 5,
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3m Yes.\n",
"Follow up: Who is the reigning men's U.S. Open champion?\u001b[0m\n",
"Intermediate answer: \u001b[36;1m\u001b[1;3mCurrent champions Carlos Alcaraz, 2022 men's singles champion.\u001b[0m\n",
"\u001b[32;1m\u001b[1;3mFollow up: Where is Carlos Alcaraz from?\u001b[0m\n",
"Intermediate answer: \u001b[36;1m\u001b[1;3mEl Palmar, Spain\u001b[0m\n",
"\u001b[32;1m\u001b[1;3mSo the final answer is: El Palmar, Spain\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": "'El Palmar, Spain'"
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from langchain.utilities import GoogleSerperAPIWrapper\n",
"from langchain.llms.openai import OpenAI\n",
"from langchain.agents import initialize_agent, Tool\n",
"from langchain.agents import AgentType\n",
"\n",
"llm = OpenAI(temperature=0)\n",
"search = GoogleSerperAPIWrapper()\n",
"tools = [\n",
" Tool(\n",
" name=\"Intermediate Answer\",\n",
" func=search.run,\n",
" description=\"useful for when you need to ask with search\",\n",
" )\n",
"]\n",
"\n",
"self_ask_with_search = initialize_agent(\n",
" tools, llm, agent=AgentType.SELF_ASK_WITH_SEARCH, verbose=True\n",
")\n",
"self_ask_with_search.run(\n",
" \"What is the hometown of the reigning men's U.S. Open champion?\"\n",
")"
],
"metadata": {
"collapsed": false
},
"id": "a8ccea61"
},
{
"cell_type": "markdown",
"source": [
"## Obtaining results with metadata\n",
"If you would also like to obtain the results in a structured way including metadata. For this we will be using the `results` method of the wrapper."
],
"metadata": {
"collapsed": false
},
"id": "3aee3682"
},
{
"cell_type": "code",
"execution_count": 6,
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"{'searchParameters': {'q': 'Apple Inc.',\n",
" 'gl': 'us',\n",
" 'hl': 'en',\n",
" 'num': 10,\n",
" 'type': 'search'},\n",
" 'knowledgeGraph': {'title': 'Apple',\n",
" 'type': 'Technology company',\n",
" 'website': 'http://www.apple.com/',\n",
" 'imageUrl': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcQwGQRv5TjjkycpctY66mOg_e2-npacrmjAb6_jAWhzlzkFE3OTjxyzbA&s=0',\n",
" 'description': 'Apple Inc. is an American multinational '\n",
" 'technology company headquartered in '\n",
" 'Cupertino, California. Apple is the '\n",
" \"world's largest technology company by \"\n",
" 'revenue, with US$394.3 billion in 2022 '\n",
" 'revenue. As of March 2023, Apple is the '\n",
" \"world's biggest...\",\n",
" 'descriptionSource': 'Wikipedia',\n",
" 'descriptionLink': 'https://en.wikipedia.org/wiki/Apple_Inc.',\n",
" 'attributes': {'Customer service': '1 (800) 275-2273',\n",
" 'CEO': 'Tim Cook (Aug 24, 2011–)',\n",
" 'Headquarters': 'Cupertino, CA',\n",
" 'Founded': 'April 1, 1976, Los Altos, CA',\n",
" 'Founders': 'Steve Jobs, Steve Wozniak, '\n",
" 'Ronald Wayne, and more',\n",
" 'Products': 'iPhone, iPad, Apple TV, and '\n",
" 'more'}},\n",
" 'organic': [{'title': 'Apple',\n",
" 'link': 'https://www.apple.com/',\n",
" 'snippet': 'Discover the innovative world of Apple and shop '\n",
" 'everything iPhone, iPad, Apple Watch, Mac, and Apple '\n",
" 'TV, plus explore accessories, entertainment, ...',\n",
" 'sitelinks': [{'title': 'Support',\n",
" 'link': 'https://support.apple.com/'},\n",
" {'title': 'iPhone',\n",
" 'link': 'https://www.apple.com/iphone/'},\n",
" {'title': 'Site Map',\n",
" 'link': 'https://www.apple.com/sitemap/'},\n",
" {'title': 'Business',\n",
" 'link': 'https://www.apple.com/business/'},\n",
" {'title': 'Mac',\n",
" 'link': 'https://www.apple.com/mac/'},\n",
" {'title': 'Watch',\n",
" 'link': 'https://www.apple.com/watch/'}],\n",
" 'position': 1},\n",
" {'title': 'Apple Inc. - Wikipedia',\n",
" 'link': 'https://en.wikipedia.org/wiki/Apple_Inc.',\n",
" 'snippet': 'Apple Inc. is an American multinational technology '\n",
" 'company headquartered in Cupertino, California. '\n",
" \"Apple is the world's largest technology company by \"\n",
" 'revenue, ...',\n",
" 'attributes': {'Products': 'AirPods; Apple Watch; iPad; iPhone; '\n",
" 'Mac; Full list',\n",
" 'Founders': 'Steve Jobs; Steve Wozniak; Ronald '\n",
" 'Wayne; Mike Markkula'},\n",
" 'sitelinks': [{'title': 'History',\n",
" 'link': 'https://en.wikipedia.org/wiki/History_of_Apple_Inc.'},\n",
" {'title': 'Timeline of Apple Inc. products',\n",
" 'link': 'https://en.wikipedia.org/wiki/Timeline_of_Apple_Inc._products'},\n",
" {'title': 'Litigation involving Apple Inc.',\n",
" 'link': 'https://en.wikipedia.org/wiki/Litigation_involving_Apple_Inc.'},\n",
" {'title': 'Apple Store',\n",
" 'link': 'https://en.wikipedia.org/wiki/Apple_Store'}],\n",
" 'imageUrl': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcRvmB5fT1LjqpZx02UM7IJq0Buoqt0DZs_y0dqwxwSWyP4PIN9FaxuTea0&s',\n",
" 'position': 2},\n",
" {'title': 'Apple Inc. | History, Products, Headquarters, & Facts '\n",
" '| Britannica',\n",
" 'link': 'https://www.britannica.com/topic/Apple-Inc',\n",
" 'snippet': 'Apple Inc., formerly Apple Computer, Inc., American '\n",
" 'manufacturer of personal computers, smartphones, '\n",
" 'tablet computers, computer peripherals, and computer '\n",
" '...',\n",
" 'attributes': {'Related People': 'Steve Jobs Steve Wozniak Jony '\n",
" 'Ive Tim Cook Angela Ahrendts',\n",
" 'Date': '1976 - present'},\n",
" 'imageUrl': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcS3liELlhrMz3Wpsox29U8jJ3L8qETR0hBWHXbFnwjwQc34zwZvFELst2E&s',\n",
" 'position': 3},\n",
" {'title': 'AAPL: Apple Inc Stock Price Quote - NASDAQ GS - '\n",
" 'Bloomberg.com',\n",
" 'link': 'https://www.bloomberg.com/quote/AAPL:US',\n",
" 'snippet': 'AAPL:USNASDAQ GS. Apple Inc. COMPANY INFO ; Open. '\n",
" '170.09 ; Prev Close. 169.59 ; Volume. 48,425,696 ; '\n",
" 'Market Cap. 2.667T ; Day Range. 167.54170.35.',\n",
" 'position': 4},\n",
" {'title': 'Apple Inc. (AAPL) Company Profile & Facts - Yahoo '\n",
" 'Finance',\n",
" 'link': 'https://finance.yahoo.com/quote/AAPL/profile/',\n",
" 'snippet': 'Apple Inc. designs, manufactures, and markets '\n",
" 'smartphones, personal computers, tablets, wearables, '\n",
" 'and accessories worldwide. The company offers '\n",
" 'iPhone, a line ...',\n",
" 'position': 5},\n",
" {'title': 'Apple Inc. (AAPL) Stock Price, News, Quote & History - '\n",
" 'Yahoo Finance',\n",
" 'link': 'https://finance.yahoo.com/quote/AAPL',\n",
" 'snippet': 'Find the latest Apple Inc. (AAPL) stock quote, '\n",
" 'history, news and other vital information to help '\n",
" 'you with your stock trading and investing.',\n",
" 'position': 6}],\n",
" 'peopleAlsoAsk': [{'question': 'What does Apple Inc do?',\n",
" 'snippet': 'Apple Inc. (Apple) designs, manufactures and '\n",
" 'markets smartphones, personal\\n'\n",
" 'computers, tablets, wearables and accessories '\n",
" 'and sells a range of related\\n'\n",
" 'services.',\n",
" 'title': 'AAPL.O - | Stock Price & Latest News - Reuters',\n",
" 'link': 'https://www.reuters.com/markets/companies/AAPL.O/'},\n",
" {'question': 'What is the full form of Apple Inc?',\n",
" 'snippet': '(formerly Apple Computer Inc.) is an American '\n",
" 'computer and consumer electronics\\n'\n",
" 'company famous for creating the iPhone, iPad '\n",
" 'and Macintosh computers.',\n",
" 'title': 'What is Apple? An products and history overview '\n",
" '- TechTarget',\n",
" 'link': 'https://www.techtarget.com/whatis/definition/Apple'},\n",
" {'question': 'What is Apple Inc iPhone?',\n",
" 'snippet': 'Apple Inc (Apple) designs, manufactures, and '\n",
" 'markets smartphones, tablets,\\n'\n",
" 'personal computers, and wearable devices. The '\n",
" 'company also offers software\\n'\n",
" 'applications and related services, '\n",
" 'accessories, and third-party digital content.\\n'\n",
" \"Apple's product portfolio includes iPhone, \"\n",
" 'iPad, Mac, iPod, Apple Watch, and\\n'\n",
" 'Apple TV.',\n",
" 'title': 'Apple Inc Company Profile - Apple Inc Overview - '\n",
" 'GlobalData',\n",
" 'link': 'https://www.globaldata.com/company-profile/apple-inc/'},\n",
" {'question': 'Who runs Apple Inc?',\n",
" 'snippet': 'Timothy Donald Cook (born November 1, 1960) is '\n",
" 'an American business executive\\n'\n",
" 'who has been the chief executive officer of '\n",
" 'Apple Inc. since 2011. Cook\\n'\n",
" \"previously served as the company's chief \"\n",
" 'operating officer under its co-founder\\n'\n",
" 'Steve Jobs. He is the first CEO of any Fortune '\n",
" '500 company who is openly gay.',\n",
" 'title': 'Tim Cook - Wikipedia',\n",
" 'link': 'https://en.wikipedia.org/wiki/Tim_Cook'}],\n",
" 'relatedSearches': [{'query': 'Who invented the iPhone'},\n",
" {'query': 'Apple iPhone'},\n",
" {'query': 'History of Apple company PDF'},\n",
" {'query': 'Apple company history'},\n",
" {'query': 'Apple company introduction'},\n",
" {'query': 'Apple India'},\n",
" {'query': 'What does Apple Inc own'},\n",
" {'query': 'Apple Inc After Steve'},\n",
" {'query': 'Apple Watch'},\n",
" {'query': 'Apple App Store'}]}\n"
]
}
],
"source": [
"search = GoogleSerperAPIWrapper()\n",
"results = search.results(\"Apple Inc.\")\n",
"pprint.pp(results)"
],
"metadata": {
"collapsed": false,
"pycharm": {
"is_executing": true
},
"ExecuteTime": {
"end_time": "2023-05-04T00:54:22.863413Z",
"start_time": "2023-05-04T00:54:20.827395Z"
}
},
"id": "073c3fc5"
},
{
"cell_type": "markdown",
"source": [
"## Searching for Google Images\n",
"We can also query Google Images using this wrapper. For example:"
],
"metadata": {
"collapsed": false
},
"id": "b402c308"
},
{
"cell_type": "code",
"execution_count": 7,
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"{'searchParameters': {'q': 'Lion',\n",
" 'gl': 'us',\n",
" 'hl': 'en',\n",
" 'num': 10,\n",
" 'type': 'images'},\n",
" 'images': [{'title': 'Lion - Wikipedia',\n",
" 'imageUrl': 'https://upload.wikimedia.org/wikipedia/commons/thumb/7/73/Lion_waiting_in_Namibia.jpg/1200px-Lion_waiting_in_Namibia.jpg',\n",
" 'imageWidth': 1200,\n",
" 'imageHeight': 900,\n",
" 'thumbnailUrl': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcRye79ROKwjfb6017jr0iu8Bz2E1KKuHg-A4qINJaspyxkZrkw&amp;s',\n",
" 'thumbnailWidth': 259,\n",
" 'thumbnailHeight': 194,\n",
" 'source': 'Wikipedia',\n",
" 'domain': 'en.wikipedia.org',\n",
" 'link': 'https://en.wikipedia.org/wiki/Lion',\n",
" 'position': 1},\n",
" {'title': 'Lion | Characteristics, Habitat, & Facts | Britannica',\n",
" 'imageUrl': 'https://cdn.britannica.com/55/2155-050-604F5A4A/lion.jpg',\n",
" 'imageWidth': 754,\n",
" 'imageHeight': 752,\n",
" 'thumbnailUrl': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcS3fnDub1GSojI0hJ-ZGS8Tv-hkNNloXh98DOwXZoZ_nUs3GWSd&amp;s',\n",
" 'thumbnailWidth': 225,\n",
" 'thumbnailHeight': 224,\n",
" 'source': 'Encyclopedia Britannica',\n",
" 'domain': 'www.britannica.com',\n",
" 'link': 'https://www.britannica.com/animal/lion',\n",
" 'position': 2},\n",
" {'title': 'African lion, facts and photos',\n",
" 'imageUrl': 'https://i.natgeofe.com/n/487a0d69-8202-406f-a6a0-939ed3704693/african-lion.JPG',\n",
" 'imageWidth': 3072,\n",
" 'imageHeight': 2043,\n",
" 'thumbnailUrl': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcTPlTarrtDbyTiEm-VI_PML9VtOTVPuDXJ5ybDf_lN11H2mShk&amp;s',\n",
" 'thumbnailWidth': 275,\n",
" 'thumbnailHeight': 183,\n",
" 'source': 'National Geographic',\n",
" 'domain': 'www.nationalgeographic.com',\n",
" 'link': 'https://www.nationalgeographic.com/animals/mammals/facts/african-lion',\n",
" 'position': 3},\n",
" {'title': 'Saint Louis Zoo | African Lion',\n",
" 'imageUrl': 'https://optimise2.assets-servd.host/maniacal-finch/production/animals/african-lion-01-01.jpg?w=1200&auto=compress%2Cformat&fit=crop&dm=1658933674&s=4b63f926a0f524f2087a8e0613282bdb',\n",
" 'imageWidth': 1200,\n",
" 'imageHeight': 1200,\n",
" 'thumbnailUrl': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcTlewcJ5SwC7yKup6ByaOjTnAFDeoOiMxyJTQaph2W_I3dnks4&amp;s',\n",
" 'thumbnailWidth': 225,\n",
" 'thumbnailHeight': 225,\n",
" 'source': 'St. Louis Zoo',\n",
" 'domain': 'stlzoo.org',\n",
" 'link': 'https://stlzoo.org/animals/mammals/carnivores/lion',\n",
" 'position': 4},\n",
" {'title': 'How to Draw a Realistic Lion like an Artist - Studio '\n",
" 'Wildlife',\n",
" 'imageUrl': 'https://studiowildlife.com/wp-content/uploads/2021/10/245528858_183911853822648_6669060845725210519_n.jpg',\n",
" 'imageWidth': 1431,\n",
" 'imageHeight': 2048,\n",
" 'thumbnailUrl': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcTmn5HayVj3wqoBDQacnUtzaDPZzYHSLKUlIEcni6VB8w0mVeA&amp;s',\n",
" 'thumbnailWidth': 188,\n",
" 'thumbnailHeight': 269,\n",
" 'source': 'Studio Wildlife',\n",
" 'domain': 'studiowildlife.com',\n",
" 'link': 'https://studiowildlife.com/how-to-draw-a-realistic-lion-like-an-artist/',\n",
" 'position': 5},\n",
" {'title': 'Lion | Characteristics, Habitat, & Facts | Britannica',\n",
" 'imageUrl': 'https://cdn.britannica.com/29/150929-050-547070A1/lion-Kenya-Masai-Mara-National-Reserve.jpg',\n",
" 'imageWidth': 1600,\n",
" 'imageHeight': 1085,\n",
" 'thumbnailUrl': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcSCqaKY_THr0IBZN8c-2VApnnbuvKmnsWjfrwKoWHFR9w3eN5o&amp;s',\n",
" 'thumbnailWidth': 273,\n",
" 'thumbnailHeight': 185,\n",
" 'source': 'Encyclopedia Britannica',\n",
" 'domain': 'www.britannica.com',\n",
" 'link': 'https://www.britannica.com/animal/lion',\n",
" 'position': 6},\n",
" {'title': \"Where do lions live? Facts about lions' habitats and \"\n",
" 'other cool facts',\n",
" 'imageUrl': 'https://www.gannett-cdn.com/-mm-/b2b05a4ab25f4fca0316459e1c7404c537a89702/c=0-0-1365-768/local/-/media/2022/03/16/USATODAY/usatsports/imageForEntry5-ODq.jpg?width=1365&height=768&fit=crop&format=pjpg&auto=webp',\n",
" 'imageWidth': 1365,\n",
" 'imageHeight': 768,\n",
" 'thumbnailUrl': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcTc_4vCHscgvFvYy3PSrtIOE81kNLAfhDK8F3mfOuotL0kUkbs&amp;s',\n",
" 'thumbnailWidth': 299,\n",
" 'thumbnailHeight': 168,\n",
" 'source': 'USA Today',\n",
" 'domain': 'www.usatoday.com',\n",
" 'link': 'https://www.usatoday.com/story/news/2023/01/08/where-do-lions-live-habitat/10927718002/',\n",
" 'position': 7},\n",
" {'title': 'Lion',\n",
" 'imageUrl': 'https://i.natgeofe.com/k/1d33938b-3d02-4773-91e3-70b113c3b8c7/lion-male-roar_square.jpg',\n",
" 'imageWidth': 3072,\n",
" 'imageHeight': 3072,\n",
" 'thumbnailUrl': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcQqLfnBrBLcTiyTZynHH3FGbBtX2bd1ScwpcuOLnksTyS9-4GM&amp;s',\n",
" 'thumbnailWidth': 225,\n",
" 'thumbnailHeight': 225,\n",
" 'source': 'National Geographic Kids',\n",
" 'domain': 'kids.nationalgeographic.com',\n",
" 'link': 'https://kids.nationalgeographic.com/animals/mammals/facts/lion',\n",
" 'position': 8},\n",
" {'title': \"Lion | Smithsonian's National Zoo\",\n",
" 'imageUrl': 'https://nationalzoo.si.edu/sites/default/files/styles/1400_scale/public/animals/exhibit/africanlion-005.jpg?itok=6wA745g_',\n",
" 'imageWidth': 1400,\n",
" 'imageHeight': 845,\n",
" 'thumbnailUrl': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcSgB3z_D4dMEOWJ7lajJk4XaQSL4DdUvIRj4UXZ0YoE5fGuWuo&amp;s',\n",
" 'thumbnailWidth': 289,\n",
" 'thumbnailHeight': 174,\n",
" 'source': \"Smithsonian's National Zoo\",\n",
" 'domain': 'nationalzoo.si.edu',\n",
" 'link': 'https://nationalzoo.si.edu/animals/lion',\n",
" 'position': 9},\n",
" {'title': \"Zoo's New Male Lion Explores Habitat for the First Time \"\n",
" '- Virginia Zoo',\n",
" 'imageUrl': 'https://virginiazoo.org/wp-content/uploads/2022/04/ZOO_0056-scaled.jpg',\n",
" 'imageWidth': 2560,\n",
" 'imageHeight': 2141,\n",
" 'thumbnailUrl': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcTDCG7XvXRCwpe_-Vy5mpvrQpVl5q2qwgnDklQhrJpQzObQGz4&amp;s',\n",
" 'thumbnailWidth': 246,\n",
" 'thumbnailHeight': 205,\n",
" 'source': 'Virginia Zoo',\n",
" 'domain': 'virginiazoo.org',\n",
" 'link': 'https://virginiazoo.org/zoos-new-male-lion-explores-habitat-for-thefirst-time/',\n",
" 'position': 10}]}\n"
]
}
],
"source": [
"search = GoogleSerperAPIWrapper(type=\"images\")\n",
"results = search.results(\"Lion\")\n",
"pprint.pp(results)"
],
"metadata": {
"collapsed": false,
"ExecuteTime": {
"end_time": "2023-05-04T00:54:27.879867Z",
"start_time": "2023-05-04T00:54:26.380022Z"
}
},
"id": "7fb2b7e2"
},
{
"cell_type": "markdown",
"source": [
"## Searching for Google News\n",
"We can also query Google News using this wrapper. For example:"
],
"metadata": {
"collapsed": false
},
"id": "85a3bed3"
},
{
"cell_type": "code",
"execution_count": 8,
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"{'searchParameters': {'q': 'Tesla Inc.',\n",
" 'gl': 'us',\n",
" 'hl': 'en',\n",
" 'num': 10,\n",
" 'type': 'news'},\n",
" 'news': [{'title': 'ISS recommends Tesla investors vote against re-election '\n",
" 'of Robyn Denholm',\n",
" 'link': 'https://www.reuters.com/business/autos-transportation/iss-recommends-tesla-investors-vote-against-re-election-robyn-denholm-2023-05-04/',\n",
" 'snippet': 'Proxy advisory firm ISS on Wednesday recommended Tesla '\n",
" 'investors vote against re-election of board chair Robyn '\n",
" 'Denholm, citing \"concerns on...',\n",
" 'date': '5 mins ago',\n",
" 'source': 'Reuters',\n",
" 'imageUrl': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcROdETe_GUyp1e8RHNhaRM8Z_vfxCvdfinZwzL1bT1ZGSYaGTeOojIdBoLevA&s',\n",
" 'position': 1},\n",
" {'title': 'Global companies by market cap: Tesla fell most in April',\n",
" 'link': 'https://www.reuters.com/markets/global-companies-by-market-cap-tesla-fell-most-april-2023-05-02/',\n",
" 'snippet': 'Tesla Inc was the biggest loser among top companies by '\n",
" 'market capitalisation in April, hit by disappointing '\n",
" 'quarterly earnings after it...',\n",
" 'date': '1 day ago',\n",
" 'source': 'Reuters',\n",
" 'imageUrl': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcQ4u4CP8aOdGyRFH6o4PkXi-_eZDeY96vLSag5gDjhKMYf98YBER2cZPbkStQ&s',\n",
" 'position': 2},\n",
" {'title': 'Tesla Wanted an EV Price War. Ford Showed Up.',\n",
" 'link': 'https://www.bloomberg.com/opinion/articles/2023-05-03/tesla-wanted-an-ev-price-war-ford-showed-up',\n",
" 'snippet': 'The legacy automaker is paring back the cost of its '\n",
" 'Mustang Mach-E model after Tesla discounted its '\n",
" 'competing EVs, portending tighter...',\n",
" 'date': '6 hours ago',\n",
" 'source': 'Bloomberg.com',\n",
" 'imageUrl': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcS_3Eo4VI0H-nTeIbYc5DaQn5ep7YrWnmhx6pv8XddFgNF5zRC9gEpHfDq8yQ&s',\n",
" 'position': 3},\n",
" {'title': 'Joby Aviation to get investment from Tesla shareholder '\n",
" 'Baillie Gifford',\n",
" 'link': 'https://finance.yahoo.com/news/joby-aviation-investment-tesla-shareholder-204450712.html',\n",
" 'snippet': 'This comes days after Joby clinched a $55 million '\n",
" 'contract extension to deliver up to nine air taxis to '\n",
" 'the U.S. Air Force,...',\n",
" 'date': '4 hours ago',\n",
" 'source': 'Yahoo Finance',\n",
" 'imageUrl': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcQO0uVn297LI-xryrPNqJ-apUOulj4ohM-xkN4OfmvMOYh1CPdUEBbYx6hviw&s',\n",
" 'position': 4},\n",
" {'title': 'Tesla resumes U.S. orders for a Model 3 version at lower '\n",
" 'price, range',\n",
" 'link': 'https://finance.yahoo.com/news/tesla-resumes-us-orders-model-045736115.html',\n",
" 'snippet': '(Reuters) -Tesla Inc has resumed taking orders for its '\n",
" 'Model 3 long-range vehicle in the United States, the '\n",
" \"company's website showed late on...\",\n",
" 'date': '19 hours ago',\n",
" 'source': 'Yahoo Finance',\n",
" 'imageUrl': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcTIZetJ62sQefPfbQ9KKDt6iH7Mc0ylT5t_hpgeeuUkHhJuAx2FOJ4ZTRVDFg&s',\n",
" 'position': 5},\n",
" {'title': 'The Tesla Model 3 Long Range AWD Is Now Available in the '\n",
" 'U.S. With 325 Miles of Range',\n",
" 'link': 'https://www.notateslaapp.com/news/1393/tesla-reopens-orders-for-model-3-long-range-after-months-of-unavailability',\n",
" 'snippet': 'Tesla has reopened orders for the Model 3 Long Range '\n",
" 'RWD, which has been unavailable for months due to high '\n",
" 'demand.',\n",
" 'date': '7 hours ago',\n",
" 'source': 'Not a Tesla App',\n",
" 'imageUrl': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcSecrgxZpRj18xIJY-nDHljyP-A4ejEkswa9eq77qhMNrScnVIqe34uql5U4w&s',\n",
" 'position': 6},\n",
" {'title': 'Tesla Cybertruck alpha prototype spotted at the Fremont '\n",
" 'factory in new pics and videos',\n",
" 'link': 'https://www.teslaoracle.com/2023/05/03/tesla-cybertruck-alpha-prototype-interior-and-exterior-spotted-at-the-fremont-factory-in-new-pics-and-videos/',\n",
" 'snippet': 'A Tesla Cybertruck alpha prototype goes to Fremont, '\n",
" 'California for another round of testing before going to '\n",
" 'production later this year (pics...',\n",
" 'date': '14 hours ago',\n",
" 'source': 'Tesla Oracle',\n",
" 'imageUrl': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcRO7M5ZLQE-Zo4-_5dv9hNAQZ3wSqfvYCuKqzxHG-M6CgLpwPMMG_ssebdcMg&s',\n",
" 'position': 7},\n",
" {'title': 'Tesla putting facility in new part of country - Austin '\n",
" 'Business Journal',\n",
" 'link': 'https://www.bizjournals.com/austin/news/2023/05/02/tesla-leases-building-seattle-area.html',\n",
" 'snippet': 'Check out what Puget Sound Business Journal has to '\n",
" \"report about the Austin-based company's real estate \"\n",
" 'footprint in the Pacific Northwest.',\n",
" 'date': '22 hours ago',\n",
" 'source': 'The Business Journals',\n",
" 'imageUrl': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcR9kIEHWz1FcHKDUtGQBS0AjmkqtyuBkQvD8kyIY3kpaPrgYaN7I_H2zoOJsA&s',\n",
" 'position': 8},\n",
" {'title': 'Tesla (TSLA) Resumes Orders for Model 3 Long Range After '\n",
" 'Backlog',\n",
" 'link': 'https://www.bloomberg.com/news/articles/2023-05-03/tesla-resumes-orders-for-popular-model-3-long-range-at-47-240',\n",
" 'snippet': 'Tesla Inc. has resumed taking orders for its Model 3 '\n",
" 'Long Range edition with a starting price of $47240, '\n",
" 'according to its website.',\n",
" 'date': '5 hours ago',\n",
" 'source': 'Bloomberg.com',\n",
" 'imageUrl': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcTWWIC4VpMTfRvSyqiomODOoLg0xhoBf-Tc1qweKnSuaiTk-Y1wMJZM3jct0w&s',\n",
" 'position': 9}]}\n"
]
}
],
"source": [
"search = GoogleSerperAPIWrapper(type=\"news\")\n",
"results = search.results(\"Tesla Inc.\")\n",
"pprint.pp(results)"
],
"metadata": {
"collapsed": false,
"ExecuteTime": {
"end_time": "2023-05-04T00:54:34.984087Z",
"start_time": "2023-05-04T00:54:33.369231Z"
}
},
"id": "afc48b39"
},
{
"cell_type": "markdown",
"source": [
"If you want to only receive news articles published in the last hour, you can do the following:"
],
"metadata": {
"collapsed": false
},
"id": "d42ee7b5"
},
{
"cell_type": "code",
"execution_count": 9,
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"{'searchParameters': {'q': 'Tesla Inc.',\n",
" 'gl': 'us',\n",
" 'hl': 'en',\n",
" 'num': 10,\n",
" 'type': 'news',\n",
" 'tbs': 'qdr:h'},\n",
" 'news': [{'title': 'Oklahoma Gov. Stitt sees growing foreign interest in '\n",
" 'investments in ...',\n",
" 'link': 'https://www.reuters.com/world/us/oklahoma-gov-stitt-sees-growing-foreign-interest-investments-state-2023-05-04/',\n",
" 'snippet': 'T)), a battery supplier to electric vehicle maker Tesla '\n",
" 'Inc (TSLA.O), said on Sunday it is considering building '\n",
" 'a battery plant in Oklahoma, its third in...',\n",
" 'date': '53 mins ago',\n",
" 'source': 'Reuters',\n",
" 'imageUrl': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcSSTcsXeenqmEKdiekvUgAmqIPR4nlAmgjTkBqLpza-lLfjX1CwB84MoNVj0Q&s',\n",
" 'position': 1},\n",
" {'title': 'Ryder lanza solución llave en mano para vehículos '\n",
" 'eléctricos en EU',\n",
" 'link': 'https://www.tyt.com.mx/nota/ryder-lanza-solucion-llave-en-mano-para-vehiculos-electricos-en-eu',\n",
" 'snippet': 'Ryder System Inc. presentó RyderElectric+ TM como su '\n",
" 'nueva solución llave en mano ... Ryder también tiene '\n",
" 'reservados los semirremolques Tesla y continúa...',\n",
" 'date': '56 mins ago',\n",
" 'source': 'Revista Transportes y Turismo',\n",
" 'imageUrl': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcQJhXTQQtjSUZf9YPM235WQhFU5_d7lEA76zB8DGwZfixcgf1_dhPJyKA1Nbw&s',\n",
" 'position': 2},\n",
" {'title': '\"I think people can get by with $999 million,\" Bernie '\n",
" 'Sanders tells American Billionaires.',\n",
" 'link': 'https://thebharatexpressnews.com/i-think-people-can-get-by-with-999-million-bernie-sanders-tells-american-billionaires-heres-how-the-ultra-rich-can-pay-less-income-tax-than-you-legally/',\n",
" 'snippet': 'The report noted that in 2007 and 2011, Amazon.com Inc. '\n",
" 'founder Jeff Bezos “did not pay a dime in federal ... '\n",
" 'If you want to bet on Musk, check out Tesla.',\n",
" 'date': '11 mins ago',\n",
" 'source': 'THE BHARAT EXPRESS NEWS',\n",
" 'imageUrl': 'https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcR_X9qqSwVFBBdos2CK5ky5IWIE3aJPCQeRYR9O1Jz4t-MjaEYBuwK7AU3AJQ&s',\n",
" 'position': 3}]}\n"
]
}
],
"source": [
"search = GoogleSerperAPIWrapper(type=\"news\", tbs=\"qdr:h\")\n",
"results = search.results(\"Tesla Inc.\")\n",
"pprint.pp(results)"
],
"metadata": {
"collapsed": false,
"ExecuteTime": {
"end_time": "2023-05-04T00:54:41.786864Z",
"start_time": "2023-05-04T00:54:40.691905Z"
}
},
"id": "8e3824cb"
},
{
"cell_type": "markdown",
"source": [
"Some examples of the `tbs` parameter:\n",
"\n",
"`qdr:h` (past hour)\n",
"`qdr:d` (past day)\n",
"`qdr:w` (past week)\n",
"`qdr:m` (past month)\n",
"`qdr:y` (past year)\n",
"\n",
"You can specify intermediate time periods by adding a number:\n",
"`qdr:h12` (past 12 hours)\n",
"`qdr:d3` (past 3 days)\n",
"`qdr:w2` (past 2 weeks)\n",
"`qdr:m6` (past 6 months)\n",
"`qdr:m2` (past 2 years)\n",
"\n",
"For all supported filters simply go to [Google Search](https://google.com), search for something, click on \"Tools\", add your date filter and check the URL for \"tbs=\".\n"
],
"metadata": {
"collapsed": false
},
"id": "3f13e9f9"
},
{
"cell_type": "markdown",
"source": [
"## Searching for Google Places\n",
"We can also query Google Places using this wrapper. For example:"
],
"metadata": {
"collapsed": false
},
"id": "38d4402c"
},
{
"cell_type": "code",
"execution_count": 10,
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"{'searchParameters': {'q': 'Italian restaurants in Upper East Side',\n",
" 'gl': 'us',\n",
" 'hl': 'en',\n",
" 'num': 10,\n",
" 'type': 'places'},\n",
" 'places': [{'position': 1,\n",
" 'title': \"L'Osteria\",\n",
" 'address': '1219 Lexington Ave',\n",
" 'latitude': 40.777154599999996,\n",
" 'longitude': -73.9571363,\n",
" 'thumbnailUrl': 'https://lh5.googleusercontent.com/p/AF1QipNjU7BWEq_aYQANBCbX52Kb0lDpd_lFIx5onw40=w92-h92-n-k-no',\n",
" 'rating': 4.7,\n",
" 'ratingCount': 91,\n",
" 'category': 'Italian'},\n",
" {'position': 2,\n",
" 'title': \"Tony's Di Napoli\",\n",
" 'address': '1081 3rd Ave',\n",
" 'latitude': 40.7643567,\n",
" 'longitude': -73.9642373,\n",
" 'thumbnailUrl': 'https://lh5.googleusercontent.com/p/AF1QipNbNv6jZkJ9nyVi60__8c1DQbe_eEbugRAhIYye=w92-h92-n-k-no',\n",
" 'rating': 4.5,\n",
" 'ratingCount': 2265,\n",
" 'category': 'Italian'},\n",
" {'position': 3,\n",
" 'title': 'Caravaggio',\n",
" 'address': '23 E 74th St',\n",
" 'latitude': 40.773412799999996,\n",
" 'longitude': -73.96473379999999,\n",
" 'thumbnailUrl': 'https://lh5.googleusercontent.com/p/AF1QipPDGchokDvppoLfmVEo6X_bWd3Fz0HyxIHTEe9V=w92-h92-n-k-no',\n",
" 'rating': 4.5,\n",
" 'ratingCount': 276,\n",
" 'category': 'Italian'},\n",
" {'position': 4,\n",
" 'title': 'Luna Rossa',\n",
" 'address': '347 E 85th St',\n",
" 'latitude': 40.776593999999996,\n",
" 'longitude': -73.950351,\n",
" 'thumbnailUrl': 'https://lh5.googleusercontent.com/p/AF1QipNPCpCPuqPAb1Mv6_fOP7cjb8Wu1rbqbk2sMBlh=w92-h92-n-k-no',\n",
" 'rating': 4.5,\n",
" 'ratingCount': 140,\n",
" 'category': 'Italian'},\n",
" {'position': 5,\n",
" 'title': \"Paola's\",\n",
" 'address': '1361 Lexington Ave',\n",
" 'latitude': 40.7822019,\n",
" 'longitude': -73.9534096,\n",
" 'thumbnailUrl': 'https://lh5.googleusercontent.com/p/AF1QipPJr2Vcx-B6K-GNQa4koOTffggTePz8TKRTnWi3=w92-h92-n-k-no',\n",
" 'rating': 4.5,\n",
" 'ratingCount': 344,\n",
" 'category': 'Italian'},\n",
" {'position': 6,\n",
" 'title': 'Come Prima',\n",
" 'address': '903 Madison Ave',\n",
" 'latitude': 40.772124999999996,\n",
" 'longitude': -73.965012,\n",
" 'thumbnailUrl': 'https://lh5.googleusercontent.com/p/AF1QipNrX19G0NVdtDyMovCQ-M-m0c_gLmIxrWDQAAbz=w92-h92-n-k-no',\n",
" 'rating': 4.5,\n",
" 'ratingCount': 176,\n",
" 'category': 'Italian'},\n",
" {'position': 7,\n",
" 'title': 'Botte UES',\n",
" 'address': '1606 1st Ave.',\n",
" 'latitude': 40.7750785,\n",
" 'longitude': -73.9504801,\n",
" 'thumbnailUrl': 'https://lh5.googleusercontent.com/p/AF1QipPPN5GXxfH3NDacBc0Pt3uGAInd9OChS5isz9RF=w92-h92-n-k-no',\n",
" 'rating': 4.4,\n",
" 'ratingCount': 152,\n",
" 'category': 'Italian'},\n",
" {'position': 8,\n",
" 'title': 'Piccola Cucina Uptown',\n",
" 'address': '106 E 60th St',\n",
" 'latitude': 40.7632468,\n",
" 'longitude': -73.9689825,\n",
" 'thumbnailUrl': 'https://lh5.googleusercontent.com/p/AF1QipPifIgzOCD5SjgzzqBzGkdZCBp0MQsK5k7M7znn=w92-h92-n-k-no',\n",
" 'rating': 4.6,\n",
" 'ratingCount': 941,\n",
" 'category': 'Italian'},\n",
" {'position': 9,\n",
" 'title': 'Pinocchio Restaurant',\n",
" 'address': '300 E 92nd St',\n",
" 'latitude': 40.781453299999995,\n",
" 'longitude': -73.9486788,\n",
" 'thumbnailUrl': 'https://lh5.googleusercontent.com/p/AF1QipNtxlIyEEJHtDtFtTR9nB38S8A2VyMu-mVVz72A=w92-h92-n-k-no',\n",
" 'rating': 4.5,\n",
" 'ratingCount': 113,\n",
" 'category': 'Italian'},\n",
" {'position': 10,\n",
" 'title': 'Barbaresco',\n",
" 'address': '843 Lexington Ave #1',\n",
" 'latitude': 40.7654332,\n",
" 'longitude': -73.9656873,\n",
" 'thumbnailUrl': 'https://lh5.googleusercontent.com/p/AF1QipMb9FbPuXF_r9g5QseOHmReejxSHgSahPMPJ9-8=w92-h92-n-k-no',\n",
" 'rating': 4.3,\n",
" 'ratingCount': 122,\n",
" 'locationHint': 'In The Touraine',\n",
" 'category': 'Italian'}]}\n"
]
}
],
"source": [
"search = GoogleSerperAPIWrapper(type=\"places\")\n",
"results = search.results(\"Italian restaurants in Upper East Side\")\n",
"pprint.pp(results)"
],
"metadata": {
"collapsed": false,
"ExecuteTime": {
"end_time": "2023-05-04T00:56:07.271164Z",
"start_time": "2023-05-04T00:56:05.645847Z"
}
},
"id": "e7881203"
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.9"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
docs/extras/modules/agents/tools/integrations/gradio_tools.ipynb
-252
View File
File diff suppressed because one or more lines are too long
docs/extras/modules/agents/tools/integrations/graphql.ipynb
-154
View File
@@ -1,154 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"# GraphQL tool\n",
"This Jupyter Notebook demonstrates how to use the BaseGraphQLTool component with an Agent.\n",
"\n",
"GraphQL is a query language for APIs and a runtime for executing those queries against your data. GraphQL provides a complete and understandable description of the data in your API, gives clients the power to ask for exactly what they need and nothing more, makes it easier to evolve APIs over time, and enables powerful developer tools.\n",
"\n",
"By including a BaseGraphQLTool in the list of tools provided to an Agent, you can grant your Agent the ability to query data from GraphQL APIs for any purposes you need.\n",
"\n",
"In this example, we'll be using the public Star Wars GraphQL API available at the following endpoint: https://swapi-graphql.netlify.app/.netlify/functions/index.\n",
"\n",
"First, you need to install httpx and gql Python packages."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"vscode": {
"languageId": "shellscript"
}
},
"outputs": [],
"source": [
"pip install httpx gql > /dev/null"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now, let's create a BaseGraphQLTool instance with the specified Star Wars API endpoint and initialize an Agent with the tool."
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [],
"source": [
"from langchain import OpenAI\n",
"from langchain.agents import load_tools, initialize_agent, AgentType\n",
"from langchain.utilities import GraphQLAPIWrapper\n",
"\n",
"llm = OpenAI(temperature=0)\n",
"\n",
"tools = load_tools(\n",
" [\"graphql\"],\n",
" graphql_endpoint=\"https://swapi-graphql.netlify.app/.netlify/functions/index\",\n",
")\n",
"\n",
"agent = initialize_agent(\n",
" tools, llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=True\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now, we can use the Agent to run queries against the Star Wars GraphQL API. Let's ask the Agent to list all the Star Wars films and their release dates."
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3m I need to query the graphql database to get the titles of all the star wars films\n",
"Action: query_graphql\n",
"Action Input: query { allFilms { films { title } } }\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3m\"{\\n \\\"allFilms\\\": {\\n \\\"films\\\": [\\n {\\n \\\"title\\\": \\\"A New Hope\\\"\\n },\\n {\\n \\\"title\\\": \\\"The Empire Strikes Back\\\"\\n },\\n {\\n \\\"title\\\": \\\"Return of the Jedi\\\"\\n },\\n {\\n \\\"title\\\": \\\"The Phantom Menace\\\"\\n },\\n {\\n \\\"title\\\": \\\"Attack of the Clones\\\"\\n },\\n {\\n \\\"title\\\": \\\"Revenge of the Sith\\\"\\n }\\n ]\\n }\\n}\"\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I now know the titles of all the star wars films\n",
"Final Answer: The titles of all the star wars films are: A New Hope, The Empire Strikes Back, Return of the Jedi, The Phantom Menace, Attack of the Clones, and Revenge of the Sith.\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"'The titles of all the star wars films are: A New Hope, The Empire Strikes Back, Return of the Jedi, The Phantom Menace, Attack of the Clones, and Revenge of the Sith.'"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"graphql_fields = \"\"\"allFilms {\n",
" films {\n",
" title\n",
" director\n",
" releaseDate\n",
" speciesConnection {\n",
" species {\n",
" name\n",
" classification\n",
" homeworld {\n",
" name\n",
" }\n",
" }\n",
" }\n",
" }\n",
" }\n",
"\n",
"\"\"\"\n",
"\n",
"suffix = \"Search for the titles of all the stawars films stored in the graphql database that has this schema \"\n",
"\n",
"\n",
"agent.run(suffix + graphql_fields)"
]
}
],
"metadata": {
"interpreter": {
"hash": "f85209c3c4c190dca7367d6a1e623da50a9a4392fd53313a7cf9d4bda9c4b85b"
},
"kernelspec": {
"display_name": "Python 3.9.16 ('langchain')",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.16"
},
"orig_nbformat": 4
},
"nbformat": 4,
"nbformat_minor": 2
}
docs/extras/modules/agents/tools/integrations/huggingface_tools.ipynb
-102
View File
@@ -1,102 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "40a27d3c-4e5c-4b96-b290-4c49d4fd7219",
"metadata": {},
"source": [
"## HuggingFace Tools\n",
"\n",
"[Huggingface Tools](https://huggingface.co/docs/transformers/v4.29.0/en/custom_tools) supporting text I/O can be\n",
"loaded directly using the `load_huggingface_tool` function."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d1055b75-362c-452a-b40d-c9a359706a3a",
"metadata": {},
"outputs": [],
"source": [
"# Requires transformers>=4.29.0 and huggingface_hub>=0.14.1\n",
"!pip install --upgrade transformers huggingface_hub > /dev/null"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "f964bb45-fba3-4919-b022-70a602ed4354",
"metadata": {
"tags": []
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"model_download_counter: This is a tool that returns the most downloaded model of a given task on the Hugging Face Hub. It takes the name of the category (such as text-classification, depth-estimation, etc), and returns the name of the checkpoint\n"
]
}
],
"source": [
"from langchain.agents import load_huggingface_tool\n",
"\n",
"tool = load_huggingface_tool(\"lysandre/hf-model-downloads\")\n",
"\n",
"print(f\"{tool.name}: {tool.description}\")"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "641d9d79-95bb-469d-b40a-50f37375de7f",
"metadata": {
"tags": []
},
"outputs": [
{
"data": {
"text/plain": [
"'facebook/bart-large-mnli'"
]
},
"execution_count": 2,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"tool.run(\"text-classification\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "88724222-7c10-4aff-8713-751911dc8b63",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.2"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
docs/extras/modules/agents/tools/integrations/human_tools.ipynb
-288
View File
@@ -1,288 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Human as a tool\n",
"\n",
"Human are AGI so they can certainly be used as a tool to help out AI agent \n",
"when it is confused."
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"from langchain.chat_models import ChatOpenAI\n",
"from langchain.llms import OpenAI\n",
"from langchain.agents import load_tools, initialize_agent\n",
"from langchain.agents import AgentType\n",
"\n",
"llm = ChatOpenAI(temperature=0.0)\n",
"math_llm = OpenAI(temperature=0.0)\n",
"tools = load_tools(\n",
" [\"human\", \"llm-math\"],\n",
" llm=math_llm,\n",
")\n",
"\n",
"agent_chain = initialize_agent(\n",
" tools,\n",
" llm,\n",
" agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION,\n",
" verbose=True,\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In the above code you can see the tool takes input directly from command line.\n",
"You can customize `prompt_func` and `input_func` according to your need (as shown below)."
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"tags": []
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3mI don't know Eric's surname, so I should ask a human for guidance.\n",
"Action: Human\n",
"Action Input: \"What is Eric's surname?\"\u001b[0m\n",
"\n",
"What is Eric's surname?\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
" Zhu\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"Observation: \u001b[36;1m\u001b[1;3mZhu\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3mI now know Eric's surname is Zhu.\n",
"Final Answer: Eric's surname is Zhu.\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"\"Eric's surname is Zhu.\""
]
},
"execution_count": 2,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"agent_chain.run(\"What's my friend Eric's surname?\")\n",
"# Answer with 'Zhu'"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Configuring the Input Function\n",
"\n",
"By default, the `HumanInputRun` tool uses the python `input` function to get input from the user.\n",
"You can customize the input_func to be anything you'd like.\n",
"For instance, if you want to accept multi-line input, you could do the following:"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"def get_input() -> str:\n",
" print(\"Insert your text. Enter 'q' or press Ctrl-D (or Ctrl-Z on Windows) to end.\")\n",
" contents = []\n",
" while True:\n",
" try:\n",
" line = input()\n",
" except EOFError:\n",
" break\n",
" if line == \"q\":\n",
" break\n",
" contents.append(line)\n",
" return \"\\n\".join(contents)\n",
"\n",
"\n",
"# You can modify the tool when loading\n",
"tools = load_tools([\"human\", \"ddg-search\"], llm=math_llm, input_func=get_input)"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"# Or you can directly instantiate the tool\n",
"from langchain.tools import HumanInputRun\n",
"\n",
"tool = HumanInputRun(input_func=get_input)"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"agent_chain = initialize_agent(\n",
" tools,\n",
" llm,\n",
" agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION,\n",
" verbose=True,\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {
"tags": []
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3mI should ask a human for guidance\n",
"Action: Human\n",
"Action Input: \"Can you help me attribute a quote?\"\u001b[0m\n",
"\n",
"Can you help me attribute a quote?\n",
"Insert your text. Enter 'q' or press Ctrl-D (or Ctrl-Z on Windows) to end.\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
" vini\n",
" vidi\n",
" vici\n",
" q\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"Observation: \u001b[36;1m\u001b[1;3mvini\n",
"vidi\n",
"vici\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3mI need to provide more context about the quote\n",
"Action: Human\n",
"Action Input: \"The quote is 'Veni, vidi, vici'\"\u001b[0m\n",
"\n",
"The quote is 'Veni, vidi, vici'\n",
"Insert your text. Enter 'q' or press Ctrl-D (or Ctrl-Z on Windows) to end.\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
" oh who said it \n",
" q\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"Observation: \u001b[36;1m\u001b[1;3moh who said it \u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3mI can use DuckDuckGo Search to find out who said the quote\n",
"Action: DuckDuckGo Search\n",
"Action Input: \"Who said 'Veni, vidi, vici'?\"\u001b[0m\n",
"Observation: \u001b[33;1m\u001b[1;3mUpdated on September 06, 2019. \"Veni, vidi, vici\" is a famous phrase said to have been spoken by the Roman Emperor Julius Caesar (100-44 BCE) in a bit of stylish bragging that impressed many of the writers of his day and beyond. The phrase means roughly \"I came, I saw, I conquered\" and it could be pronounced approximately Vehnee, Veedee ... Veni, vidi, vici (Classical Latin: [weːniː wiːdiː wiːkiː], Ecclesiastical Latin: [ˈveni ˈvidi ˈvitʃi]; \"I came; I saw; I conquered\") is a Latin phrase used to refer to a swift, conclusive victory.The phrase is popularly attributed to Julius Caesar who, according to Appian, used the phrase in a letter to the Roman Senate around 47 BC after he had achieved a quick victory in his short ... veni, vidi, vici Latin quotation from Julius Caesar ve· ni, vi· di, vi· ci ˌwā-nē ˌwē-dē ˈwē-kē ˌvā-nē ˌvē-dē ˈvē-chē : I came, I saw, I conquered Articles Related to veni, vidi, vici 'In Vino Veritas' and Other Latin... Dictionary Entries Near veni, vidi, vici Venite veni, vidi, vici Venizélos See More Nearby Entries Cite this Entry Style The simplest explanation for why veni, vidi, vici is a popular saying is that it comes from Julius Caesar, one of history's most famous figures, and has a simple, strong meaning: I'm powerful and fast. But it's not just the meaning that makes the phrase so powerful. Caesar was a gifted writer, and the phrase makes use of Latin grammar to ... One of the best known and most frequently quoted Latin expression, veni, vidi, vici may be found hundreds of times throughout the centuries used as an expression of triumph. The words are said to have been used by Caesar as he was enjoying a triumph.\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3mI now know the final answer\n",
"Final Answer: Julius Caesar said the quote \"Veni, vidi, vici\" which means \"I came, I saw, I conquered\".\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"'Julius Caesar said the quote \"Veni, vidi, vici\" which means \"I came, I saw, I conquered\".'"
]
},
"execution_count": 12,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"agent_chain.run(\"I need help attributing a quote\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.2"
}
},
"nbformat": 4,
"nbformat_minor": 4
}
docs/extras/modules/agents/tools/integrations/ifttt.ipynb
-124
View File
@@ -1,124 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "16763ed3",
"metadata": {},
"source": [
"# IFTTT WebHooks\n",
"\n",
"This notebook shows how to use IFTTT Webhooks.\n",
"\n",
"From https://github.com/SidU/teams-langchain-js/wiki/Connecting-IFTTT-Services.\n",
"\n",
"## Creating a webhook\n",
"- Go to https://ifttt.com/create\n",
"\n",
"## Configuring the \"If This\"\n",
"- Click on the \"If This\" button in the IFTTT interface.\n",
"- Search for \"Webhooks\" in the search bar.\n",
"- Choose the first option for \"Receive a web request with a JSON payload.\"\n",
"- Choose an Event Name that is specific to the service you plan to connect to.\n",
"This will make it easier for you to manage the webhook URL.\n",
"For example, if you're connecting to Spotify, you could use \"Spotify\" as your\n",
"Event Name.\n",
"- Click the \"Create Trigger\" button to save your settings and create your webhook.\n",
"\n",
"## Configuring the \"Then That\"\n",
"- Tap on the \"Then That\" button in the IFTTT interface.\n",
"- Search for the service you want to connect, such as Spotify.\n",
"- Choose an action from the service, such as \"Add track to a playlist\".\n",
"- Configure the action by specifying the necessary details, such as the playlist name,\n",
"e.g., \"Songs from AI\".\n",
"- Reference the JSON Payload received by the Webhook in your action. For the Spotify\n",
"scenario, choose \"{{JsonPayload}}\" as your search query.\n",
"- Tap the \"Create Action\" button to save your action settings.\n",
"- Once you have finished configuring your action, click the \"Finish\" button to\n",
"complete the setup.\n",
"- Congratulations! You have successfully connected the Webhook to the desired\n",
"service, and you're ready to start receiving data and triggering actions 🎉\n",
"\n",
"## Finishing up\n",
"- To get your webhook URL go to https://ifttt.com/maker_webhooks/settings\n",
"- Copy the IFTTT key value from there. The URL is of the form\n",
"https://maker.ifttt.com/use/YOUR_IFTTT_KEY. Grab the YOUR_IFTTT_KEY value.\n"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "10a46e7e",
"metadata": {},
"outputs": [],
"source": [
"from langchain.tools.ifttt import IFTTTWebhook"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "12003d72",
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"\n",
"key = os.environ[\"IFTTTKey\"]\n",
"url = f\"https://maker.ifttt.com/trigger/spotify/json/with/key/{key}\"\n",
"tool = IFTTTWebhook(\n",
" name=\"Spotify\", description=\"Add a song to spotify playlist\", url=url\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "6e68f846",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"\"Congratulations! You've fired the spotify JSON event\""
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"tool.run(\"taylor swift\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "a7e599c9",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.1"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
docs/extras/modules/agents/tools/integrations/lemonai.ipynb
-233
View File
@@ -1,233 +0,0 @@
{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"id": "16763ed3",
"metadata": {},
"source": [
"# Lemon AI NLP Workflow Automation\n",
"\\\n",
"Full docs are available at: https://github.com/felixbrock/lemonai-py-client\n",
"\n",
"**Lemon AI helps you build powerful AI assistants in minutes and automate workflows by allowing for accurate and reliable read and write operations in tools like Airtable, Hubspot, Discord, Notion, Slack and Github.**\n",
"\n",
"Most connectors available today are focused on read-only operations, limiting the potential of LLMs. Agents, on the other hand, have a tendency to hallucinate from time to time due to missing context or instructions.\n",
"\n",
"With Lemon AI, it is possible to give your agents access to well-defined APIs for reliable read and write operations. In addition, Lemon AI functions allow you to further reduce the risk of hallucinations by providing a way to statically define workflows that the model can rely on in case of uncertainty."
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "4881b484-1b97-478f-b206-aec407ceff66",
"metadata": {},
"source": [
"## Quick Start\n",
"\n",
"The following quick start demonstrates how to use Lemon AI in combination with Agents to automate workflows that involve interaction with internal tooling."
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "ff91b41a",
"metadata": {},
"source": [
"### 1. Install Lemon AI\n",
"\n",
"Requires Python 3.8.1 and above.\n",
"\n",
"To use Lemon AI in your Python project run `pip install lemonai`\n",
"\n",
"This will install the corresponding Lemon AI client which you can then import into your script.\n",
"\n",
"The tool uses Python packages langchain and loguru. In case of any installation errors with Lemon AI, install both packages first and then install the Lemon AI package."
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "340ff63d",
"metadata": {},
"source": [
"### 2. Launch the Server\n",
"\n",
"The interaction of your agents and all tools provided by Lemon AI is handled by the [Lemon AI Server](https://github.com/felixbrock/lemonai-server). To use Lemon AI you need to run the server on your local machine so the Lemon AI Python client can connect to it."
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "e845f402",
"metadata": {},
"source": [
"### 3. Use Lemon AI with Langchain"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "d3ae6a82",
"metadata": {},
"source": [
"Lemon AI automatically solves given tasks by finding the right combination of relevant tools or uses Lemon AI Functions as an alternative. The following example demonstrates how to retrieve a user from Hackernews and write it to a table in Airtable:"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "43476a22",
"metadata": {},
"source": [
"#### (Optional) Define your Lemon AI Functions"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "cb038670",
"metadata": {},
"source": [
"Similar to [OpenAI functions](https://openai.com/blog/function-calling-and-other-api-updates), Lemon AI provides the option to define workflows as reusable functions. These functions can be defined for use cases where it is especially important to move as close as possible to near-deterministic behavior. Specific workflows can be defined in a separate lemonai.json:"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "e423ebbb",
"metadata": {},
"source": [
"```json\n",
"[\n",
" {\n",
" \"name\": \"Hackernews Airtable User Workflow\",\n",
" \"description\": \"retrieves user data from Hackernews and appends it to a table in Airtable\",\n",
" \"tools\": [\"hackernews-get-user\", \"airtable-append-data\"]\n",
" }\n",
"]\n",
"```"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "3fdb36ce",
"metadata": {},
"source": [
"Your model will have access to these functions and will prefer them over self-selecting tools to solve a given task. All you have to do is to let the agent know that it should use a given function by including the function name in the prompt."
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "ebfb8b5d",
"metadata": {},
"source": [
"#### Include Lemon AI in your Langchain project "
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "5318715d",
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"from lemonai import execute_workflow\n",
"from langchain import OpenAI"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "c9d082cb",
"metadata": {},
"source": [
"#### Load API Keys and Access Tokens\n",
"\n",
"To use tools that require authentication, you have to store the corresponding access credentials in your environment in the format \"{tool name}_{authentication string}\" where the authentication string is one of [\"API_KEY\", \"SECRET_KEY\", \"SUBSCRIPTION_KEY\", \"ACCESS_KEY\"] for API keys or [\"ACCESS_TOKEN\", \"SECRET_TOKEN\"] for authentication tokens. Examples are \"OPENAI_API_KEY\", \"BING_SUBSCRIPTION_KEY\", \"AIRTABLE_ACCESS_TOKEN\"."
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "a370d999",
"metadata": {},
"outputs": [],
"source": [
"\"\"\" Load all relevant API Keys and Access Tokens into your environment variables \"\"\"\n",
"os.environ[\"OPENAI_API_KEY\"] = \"*INSERT OPENAI API KEY HERE*\"\n",
"os.environ[\"AIRTABLE_ACCESS_TOKEN\"] = \"*INSERT AIRTABLE TOKEN HERE*\""
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "38d158e7",
"metadata": {},
"outputs": [],
"source": [
"hackernews_username = \"*INSERT HACKERNEWS USERNAME HERE*\"\n",
"airtable_base_id = \"*INSERT BASE ID HERE*\"\n",
"airtable_table_id = \"*INSERT TABLE ID HERE*\"\n",
"\n",
"\"\"\" Define your instruction to be given to your LLM \"\"\"\n",
"prompt = f\"\"\"Read information from Hackernews for user {hackernews_username} and then write the results to\n",
"Airtable (baseId: {airtable_base_id}, tableId: {airtable_table_id}). Only write the fields \"username\", \"karma\"\n",
"and \"created_at_i\". Please make sure that Airtable does NOT automatically convert the field types.\n",
"\"\"\"\n",
"\n",
"\"\"\"\n",
"Use the Lemon AI execute_workflow wrapper \n",
"to run your Langchain agent in combination with Lemon AI \n",
"\"\"\"\n",
"model = OpenAI(temperature=0)\n",
"\n",
"execute_workflow(llm=model, prompt_string=prompt)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "aef3e801",
"metadata": {},
"source": [
"### 4. Gain transparency on your Agent's decision making\n",
"\n",
"To gain transparency on how your Agent interacts with Lemon AI tools to solve a given task, all decisions made, tools used and operations performed are written to a local `lemonai.log` file. Every time your LLM agent is interacting with the Lemon AI tool stack a corresponding log entry is created.\n",
"\n",
"```log\n",
"2023-06-26T11:50:27.708785+0100 - b5f91c59-8487-45c2-800a-156eac0c7dae - hackernews-get-user\n",
"2023-06-26T11:50:39.624035+0100 - b5f91c59-8487-45c2-800a-156eac0c7dae - airtable-append-data\n",
"2023-06-26T11:58:32.925228+0100 - 5efe603c-9898-4143-b99a-55b50007ed9d - hackernews-get-user\n",
"2023-06-26T11:58:43.988788+0100 - 5efe603c-9898-4143-b99a-55b50007ed9d - airtable-append-data\n",
"```\n",
"\n",
"By using the [Lemon AI Analytics Tool](https://github.com/felixbrock/lemonai-analytics) you can easily gain a better understanding of how frequently and in which order tools are used. As a result, you can identify weak spots in your agent’s decision-making capabilities and move to a more deterministic behavior by defining Lemon AI functions."
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.1"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
docs/extras/modules/agents/tools/integrations/metaphor_search.ipynb
-193
View File
@@ -1,193 +0,0 @@
{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"# Metaphor Search"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"Metaphor is a search engine fully designed to be used by LLMs. You can search and then get the contents for any page.\n",
"\n",
"This notebook goes over how to use Metaphor search.\n",
"\n",
"First, you need to set up the proper API keys and environment variables. Get 1000 free searches/month [here](https://platform.metaphor.systems/).\n",
"\n",
"Then enter your API key as an environment variable."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"\n",
"os.environ[\"METAPHOR_API_KEY\"] = \"\""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from langchain.utilities import MetaphorSearchAPIWrapper"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"search = MetaphorSearchAPIWrapper()"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"# Call the API\n",
"`results` takes in a Metaphor-optimized search query and a number of results (up to 500). It returns a list of results with title, url, author, and creation date."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"search.results(\"The best blog post about AI safety is definitely this: \", 10)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"# Adding filters\n",
"We can also add filters to our search. \n",
"\n",
"include_domains: Optional[List[str]] - List of domains to include in the search. If specified, results will only come from these domains. Only one of include_domains and exclude_domains should be specified.\n",
"\n",
"exclude_domains: Optional[List[str]] - List of domains to exclude in the search. If specified, results will only come from these domains. Only one of include_domains and exclude_domains should be specified.\n",
"\n",
"start_crawl_date: Optional[str] - \"Crawl date\" refers to the date that Metaphor discovered a link, which is more granular and can be more useful than published date. If start_crawl_date is specified, results will only include links that were crawled after start_crawl_date. Must be specified in ISO 8601 format (YYYY-MM-DDTHH:MM:SSZ)\n",
"\n",
"end_crawl_date: Optional[str] - \"Crawl date\" refers to the date that Metaphor discovered a link, which is more granular and can be more useful than published date. If endCrawlDate is specified, results will only include links that were crawled before end_crawl_date. Must be specified in ISO 8601 format (YYYY-MM-DDTHH:MM:SSZ)\n",
"\n",
"start_published_date: Optional[str] - If specified, only links with a published date after start_published_date will be returned. Must be specified in ISO 8601 format (YYYY-MM-DDTHH:MM:SSZ). Note that for some links, we have no published date, and these links will be excluded from the results if start_published_date is specified.\n",
"\n",
"end_published_date: Optional[str] - If specified, only links with a published date before end_published_date will be returned. Must be specified in ISO 8601 format (YYYY-MM-DDTHH:MM:SSZ). Note that for some links, we have no published date, and these links will be excluded from the results if end_published_date is specified.\n",
"\n",
"See full docs [here](https://metaphorapi.readme.io/)."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"search.results(\n",
" \"The best blog post about AI safety is definitely this: \",\n",
" 10,\n",
" include_domains=[\"lesswrong.com\"],\n",
" start_published_date=\"2019-01-01\",\n",
")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"# Use Metaphor as a tool\n",
"Metaphor can be used as a tool that gets URLs that other tools such as browsing tools."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"%pip install playwright\n",
"from langchain.agents.agent_toolkits import PlayWrightBrowserToolkit\n",
"from langchain.tools.playwright.utils import (\n",
" create_async_playwright_browser, # A synchronous browser is available, though it isn't compatible with jupyter.\n",
")\n",
"\n",
"async_browser = create_async_playwright_browser()\n",
"toolkit = PlayWrightBrowserToolkit.from_browser(async_browser=async_browser)\n",
"tools = toolkit.get_tools()\n",
"\n",
"tools_by_name = {tool.name: tool for tool in tools}\n",
"print(tools_by_name.keys())\n",
"navigate_tool = tools_by_name[\"navigate_browser\"]\n",
"extract_text = tools_by_name[\"extract_text\"]"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from langchain.agents import initialize_agent, AgentType\n",
"from langchain.chat_models import ChatOpenAI\n",
"from langchain.tools import MetaphorSearchResults\n",
"\n",
"llm = ChatOpenAI(model_name=\"gpt-4\", temperature=0.7)\n",
"\n",
"metaphor_tool = MetaphorSearchResults(api_wrapper=search)\n",
"\n",
"agent_chain = initialize_agent(\n",
" [metaphor_tool, extract_text, navigate_tool],\n",
" llm,\n",
" agent=AgentType.STRUCTURED_CHAT_ZERO_SHOT_REACT_DESCRIPTION,\n",
" verbose=True,\n",
")\n",
"\n",
"agent_chain.run(\n",
" \"find me an interesting tweet about AI safety using Metaphor, then tell me the first sentence in the post. Do not finish until able to retrieve the first sentence.\"\n",
")"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.11"
},
"vscode": {
"interpreter": {
"hash": "a0a0263b650d907a3bfe41c0f8d6a63a071b884df3cfdc1579f00cdc1aed6b03"
}
}
},
"nbformat": 4,
"nbformat_minor": 2
}
docs/extras/modules/agents/tools/integrations/openweathermap.ipynb
-170
View File
@@ -1,170 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "245a954a",
"metadata": {},
"source": [
"# OpenWeatherMap API\n",
"\n",
"This notebook goes over how to use the OpenWeatherMap component to fetch weather information.\n",
"\n",
"First, you need to sign up for an OpenWeatherMap API key:\n",
"\n",
"1. Go to OpenWeatherMap and sign up for an API key [here](https://openweathermap.org/api/)\n",
"2. pip install pyowm\n",
"\n",
"Then we will need to set some environment variables:\n",
"1. Save your API KEY into OPENWEATHERMAP_API_KEY env variable\n",
"\n",
"## Use the wrapper"
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "34bb5968",
"metadata": {},
"outputs": [],
"source": [
"from langchain.utilities import OpenWeatherMapAPIWrapper\n",
"import os\n",
"\n",
"os.environ[\"OPENWEATHERMAP_API_KEY\"] = \"\"\n",
"\n",
"weather = OpenWeatherMapAPIWrapper()"
]
},
{
"cell_type": "code",
"execution_count": 10,
"id": "ac4910f8",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"In London,GB, the current weather is as follows:\n",
"Detailed status: broken clouds\n",
"Wind speed: 2.57 m/s, direction: 240°\n",
"Humidity: 55%\n",
"Temperature: \n",
" - Current: 20.12°C\n",
" - High: 21.75°C\n",
" - Low: 18.68°C\n",
" - Feels like: 19.62°C\n",
"Rain: {}\n",
"Heat index: None\n",
"Cloud cover: 75%\n"
]
}
],
"source": [
"weather_data = weather.run(\"London,GB\")\n",
"print(weather_data)"
]
},
{
"cell_type": "markdown",
"id": "e73cfa56",
"metadata": {},
"source": [
"## Use the tool"
]
},
{
"cell_type": "code",
"execution_count": 11,
"id": "b3367417",
"metadata": {},
"outputs": [],
"source": [
"from langchain.llms import OpenAI\n",
"from langchain.agents import load_tools, initialize_agent, AgentType\n",
"import os\n",
"\n",
"os.environ[\"OPENAI_API_KEY\"] = \"\"\n",
"os.environ[\"OPENWEATHERMAP_API_KEY\"] = \"\"\n",
"\n",
"llm = OpenAI(temperature=0)\n",
"\n",
"tools = load_tools([\"openweathermap-api\"], llm)\n",
"\n",
"agent_chain = initialize_agent(\n",
" tools=tools, llm=llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=True\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 12,
"id": "bf4f6854",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3m I need to find out the current weather in London.\n",
"Action: OpenWeatherMap\n",
"Action Input: London,GB\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3mIn London,GB, the current weather is as follows:\n",
"Detailed status: broken clouds\n",
"Wind speed: 2.57 m/s, direction: 240°\n",
"Humidity: 56%\n",
"Temperature: \n",
" - Current: 20.11°C\n",
" - High: 21.75°C\n",
" - Low: 18.68°C\n",
" - Feels like: 19.64°C\n",
"Rain: {}\n",
"Heat index: None\n",
"Cloud cover: 75%\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I now know the current weather in London.\n",
"Final Answer: The current weather in London is broken clouds, with a wind speed of 2.57 m/s, direction 240°, humidity of 56%, temperature of 20.11°C, high of 21.75°C, low of 18.68°C, and a heat index of None.\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"'The current weather in London is broken clouds, with a wind speed of 2.57 m/s, direction 240°, humidity of 56%, temperature of 20.11°C, high of 21.75°C, low of 18.68°C, and a heat index of None.'"
]
},
"execution_count": 12,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"agent_chain.run(\"What's the weather like in London?\")"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.16"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
docs/extras/modules/agents/tools/integrations/pubmed.ipynb
-86
View File
@@ -1,86 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "64f20f38",
"metadata": {},
"source": [
"# PubMed Tool\n",
"\n",
"This notebook goes over how to use PubMed as a tool\n",
"\n",
"PubMed® comprises more than 35 million citations for biomedical literature from MEDLINE, life science journals, and online books. Citations may include links to full text content from PubMed Central and publisher web sites."
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "c80b9273",
"metadata": {},
"outputs": [],
"source": [
"from langchain.tools import PubmedQueryRun"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "f203c965",
"metadata": {},
"outputs": [],
"source": [
"tool = PubmedQueryRun()"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "baee7a2a",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'Published: <Year>2023</Year><Month>May</Month><Day>31</Day>\\nTitle: Dermatology in the wake of an AI revolution: who gets a say?\\nSummary: \\n\\nPublished: <Year>2023</Year><Month>May</Month><Day>30</Day>\\nTitle: What is ChatGPT and what do we do with it? Implications of the age of AI for nursing and midwifery practice and education: An editorial.\\nSummary: \\n\\nPublished: <Year>2023</Year><Month>Jun</Month><Day>02</Day>\\nTitle: The Impact of ChatGPT on the Nursing Profession: Revolutionizing Patient Care and Education.\\nSummary: The nursing field has undergone notable changes over time and is projected to undergo further modifications in the future, owing to the advent of sophisticated technologies and growing healthcare needs. The advent of ChatGPT, an AI-powered language model, is expected to exert a significant influence on the nursing profession, specifically in the domains of patient care and instruction. The present article delves into the ramifications of ChatGPT within the nursing domain and accentuates its capacity and constraints to transform the discipline.'"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"tool.run(\"chatgpt\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "965903ba",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.1"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
docs/extras/modules/agents/tools/integrations/python.ipynb
-95
View File
@@ -1,95 +0,0 @@
{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"id": "984a8fca",
"metadata": {},
"source": [
"# Python REPL\n",
"\n",
"Sometimes, for complex calculations, rather than have an LLM generate the answer directly, it can be better to have the LLM generate code to calculate the answer, and then run that code to get the answer. In order to easily do that, we provide a simple Python REPL to execute commands in.\n",
"\n",
"This interface will only return things that are printed - therefore, if you want to use it to calculate an answer, make sure to have it print out the answer."
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "f6593089",
"metadata": {},
"outputs": [],
"source": [
"from langchain.agents import Tool\n",
"from langchain.utilities import PythonREPL"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "6f21f0a4",
"metadata": {},
"outputs": [],
"source": [
"python_repl = PythonREPL()"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "7ebbbaea",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'2\\n'"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"python_repl.run(\"print(1+1)\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "54fc1f03",
"metadata": {},
"outputs": [],
"source": [
"# You can create the tool to pass to an agent\n",
"repl_tool = Tool(\n",
" name=\"python_repl\",\n",
" description=\"A Python shell. Use this to execute python commands. Input should be a valid python command. If you want to see the output of a value, you should print it out with `print(...)`.\",\n",
" func=python_repl.run,\n",
")"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.9"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
docs/extras/modules/agents/tools/integrations/requests.ipynb
-146
View File
File diff suppressed because one or more lines are too long
docs/extras/modules/agents/tools/integrations/sceneXplain.ipynb
-140
View File
@@ -1,140 +0,0 @@
{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"# SceneXplain\n",
"\n",
"\n",
"[SceneXplain](https://scenex.jina.ai/) is an ImageCaptioning service accessible through the SceneXplain Tool.\n",
"\n",
"To use this tool, you'll need to make an account and fetch your API Token [from the website](https://scenex.jina.ai/api). Then you can instantiate the tool."
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"\n",
"os.environ[\"SCENEX_API_KEY\"] = \"<YOUR_API_KEY>\""
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"from langchain.agents import load_tools\n",
"\n",
"tools = load_tools([\"sceneXplain\"])"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"Or directly instantiate the tool."
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"from langchain.tools import SceneXplainTool\n",
"\n",
"\n",
"tool = SceneXplainTool()"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Usage in an Agent\n",
"\n",
"The tool can be used in any LangChain agent as follows:"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3m\n",
"Thought: Do I need to use a tool? Yes\n",
"Action: Image Explainer\n",
"Action Input: https://storage.googleapis.com/causal-diffusion.appspot.com/imagePrompts%2F0rw369i5h9t%2Foriginal.png\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3mIn a charmingly whimsical scene, a young girl is seen braving the rain alongside her furry companion, the lovable Totoro. The two are depicted standing on a bustling street corner, where they are sheltered from the rain by a bright yellow umbrella. The girl, dressed in a cheerful yellow frock, holds onto the umbrella with both hands while gazing up at Totoro with an expression of wonder and delight.\n",
"\n",
"Totoro, meanwhile, stands tall and proud beside his young friend, holding his own umbrella aloft to protect them both from the downpour. His furry body is rendered in rich shades of grey and white, while his large ears and wide eyes lend him an endearing charm.\n",
"\n",
"In the background of the scene, a street sign can be seen jutting out from the pavement amidst a flurry of raindrops. A sign with Chinese characters adorns its surface, adding to the sense of cultural diversity and intrigue. Despite the dreary weather, there is an undeniable sense of joy and camaraderie in this heartwarming image.\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m Do I need to use a tool? No\n",
"AI: This image appears to be a still from the 1988 Japanese animated fantasy film My Neighbor Totoro. The film follows two young girls, Satsuki and Mei, as they explore the countryside and befriend the magical forest spirits, including the titular character Totoro.\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n",
"This image appears to be a still from the 1988 Japanese animated fantasy film My Neighbor Totoro. The film follows two young girls, Satsuki and Mei, as they explore the countryside and befriend the magical forest spirits, including the titular character Totoro.\n"
]
}
],
"source": [
"from langchain.llms import OpenAI\n",
"from langchain.agents import initialize_agent\n",
"from langchain.memory import ConversationBufferMemory\n",
"\n",
"llm = OpenAI(temperature=0)\n",
"memory = ConversationBufferMemory(memory_key=\"chat_history\")\n",
"agent = initialize_agent(\n",
" tools, llm, memory=memory, agent=\"conversational-react-description\", verbose=True\n",
")\n",
"output = agent.run(\n",
" input=(\n",
" \"What is in this image https://storage.googleapis.com/causal-diffusion.appspot.com/imagePrompts%2F0rw369i5h9t%2Foriginal.png. \"\n",
" \"Is it movie or a game? If it is a movie, what is the name of the movie?\"\n",
" )\n",
")\n",
"\n",
"print(output)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": ".venv",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.2"
},
"orig_nbformat": 4
},
"nbformat": 4,
"nbformat_minor": 2
}
docs/extras/modules/agents/tools/integrations/search_tools.ipynb
-364
View File
@@ -1,364 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "6510f51c",
"metadata": {},
"source": [
"# Search Tools\n",
"\n",
"This notebook shows off usage of various search tools."
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "e6860c2d",
"metadata": {
"pycharm": {
"is_executing": true
}
},
"outputs": [],
"source": [
"from langchain.agents import load_tools\n",
"from langchain.agents import initialize_agent\n",
"from langchain.agents import AgentType\n",
"from langchain.llms import OpenAI"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "dadbcfcd",
"metadata": {},
"outputs": [],
"source": [
"llm = OpenAI(temperature=0)"
]
},
{
"cell_type": "markdown",
"id": "ee251155",
"metadata": {},
"source": [
"## Google Serper API Wrapper\n",
"\n",
"First, let's try to use the Google Serper API tool."
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "0cdaa487",
"metadata": {},
"outputs": [],
"source": [
"tools = load_tools([\"google-serper\"], llm=llm)"
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "01b1ab4a",
"metadata": {},
"outputs": [],
"source": [
"agent = initialize_agent(\n",
" tools, llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=True\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "5cf44ec0",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3m I should look up the current weather conditions.\n",
"Action: Search\n",
"Action Input: \"weather in Pomfret\"\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3m37°F\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I now know the current temperature in Pomfret.\n",
"Final Answer: The current temperature in Pomfret is 37°F.\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"'The current temperature in Pomfret is 37°F.'"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"agent.run(\"What is the weather in Pomfret?\")"
]
},
{
"cell_type": "markdown",
"id": "0e39fc46",
"metadata": {},
"source": [
"## SerpAPI\n",
"\n",
"Now, let's use the SerpAPI tool."
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "e1c39a0f",
"metadata": {},
"outputs": [],
"source": [
"tools = load_tools([\"serpapi\"], llm=llm)"
]
},
{
"cell_type": "code",
"execution_count": 10,
"id": "900dd6cb",
"metadata": {},
"outputs": [],
"source": [
"agent = initialize_agent(\n",
" tools, llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=True\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 11,
"id": "342ee8ec",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3m I need to find out what the current weather is in Pomfret.\n",
"Action: Search\n",
"Action Input: \"weather in Pomfret\"\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3mPartly cloudy skies during the morning hours will give way to cloudy skies with light rain and snow developing in the afternoon. High 42F. Winds WNW at 10 to 15 ...\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I now know the current weather in Pomfret.\n",
"Final Answer: Partly cloudy skies during the morning hours will give way to cloudy skies with light rain and snow developing in the afternoon. High 42F. Winds WNW at 10 to 15 mph.\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"'Partly cloudy skies during the morning hours will give way to cloudy skies with light rain and snow developing in the afternoon. High 42F. Winds WNW at 10 to 15 mph.'"
]
},
"execution_count": 11,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"agent.run(\"What is the weather in Pomfret?\")"
]
},
{
"cell_type": "markdown",
"id": "adc8bb68",
"metadata": {},
"source": [
"## GoogleSearchAPIWrapper\n",
"\n",
"Now, let's use the official Google Search API Wrapper."
]
},
{
"cell_type": "code",
"execution_count": 13,
"id": "ef24f92d",
"metadata": {},
"outputs": [],
"source": [
"tools = load_tools([\"google-search\"], llm=llm)"
]
},
{
"cell_type": "code",
"execution_count": 14,
"id": "909cd28b",
"metadata": {},
"outputs": [],
"source": [
"agent = initialize_agent(\n",
" tools, llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=True\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 17,
"id": "46515d2a",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3m I should look up the current weather conditions.\n",
"Action: Google Search\n",
"Action Input: \"weather in Pomfret\"\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3mShowers early becoming a steady light rain later in the day. Near record high temperatures. High around 60F. Winds SW at 10 to 15 mph. Chance of rain 60%. Pomfret, CT Weather Forecast, with current conditions, wind, air quality, and what to expect for the next 3 days. Hourly Weather-Pomfret, CT. As of 12:52 am EST. Special Weather Statement +2 ... Hazardous Weather Conditions. Special Weather Statement ... Pomfret CT. Tonight ... National Digital Forecast Database Maximum Temperature Forecast. Pomfret Center Weather Forecasts. Weather Underground provides local & long-range weather forecasts, weatherreports, maps & tropical weather conditions for ... Pomfret, CT 12 hour by hour weather forecast includes precipitation, temperatures, sky conditions, rain chance, dew-point, relative humidity, wind direction ... North Pomfret Weather Forecasts. Weather Underground provides local & long-range weather forecasts, weatherreports, maps & tropical weather conditions for ... Today's Weather - Pomfret, CT. Dec 31, 2022 4:00 PM. Putnam MS. --. Weather forecast icon. Feels like --. Hi --. Lo --. Pomfret, CT temperature trend for the next 14 Days. Find daytime highs and nighttime lows from TheWeatherNetwork.com. Pomfret, MD Weather Forecast Date: 332 PM EST Wed Dec 28 2022. The area/counties/county of: Charles, including the cites of: St. Charles and Waldorf.\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I now know the current weather conditions in Pomfret.\n",
"Final Answer: Showers early becoming a steady light rain later in the day. Near record high temperatures. High around 60F. Winds SW at 10 to 15 mph. Chance of rain 60%.\u001b[0m\n",
"\u001b[1m> Finished AgentExecutor chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"'Showers early becoming a steady light rain later in the day. Near record high temperatures. High around 60F. Winds SW at 10 to 15 mph. Chance of rain 60%.'"
]
},
"execution_count": 17,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"agent.run(\"What is the weather in Pomfret?\")"
]
},
{
"cell_type": "markdown",
"id": "eabad3af",
"metadata": {},
"source": [
"## SearxNG Meta Search Engine\n",
"\n",
"Here we will be using a self hosted SearxNG meta search engine."
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "b196c704",
"metadata": {},
"outputs": [],
"source": [
"tools = load_tools([\"searx-search\"], searx_host=\"http://localhost:8888\", llm=llm)"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "9023eeaa",
"metadata": {},
"outputs": [],
"source": [
"agent = initialize_agent(\n",
" tools, llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=True\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "3aad92c1",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3m I should look up the current weather\n",
"Action: SearX Search\n",
"Action Input: \"weather in Pomfret\"\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3mMainly cloudy with snow showers around in the morning. High around 40F. Winds NNW at 5 to 10 mph. Chance of snow 40%. Snow accumulations less than one inch.\n",
"\n",
"10 Day Weather - Pomfret, MD As of 1:37 pm EST Today 49°/ 41° 52% Mon 27 | Day 49° 52% SE 14 mph Cloudy with occasional rain showers. High 49F. Winds SE at 10 to 20 mph. Chance of rain 50%....\n",
"\n",
"10 Day Weather - Pomfret, VT As of 3:51 am EST Special Weather Statement Today 39°/ 32° 37% Wed 01 | Day 39° 37% NE 4 mph Cloudy with snow showers developing for the afternoon. High 39F....\n",
"\n",
"Pomfret, CT ; Current Weather. 1:06 AM. 35°F · RealFeel® 32° ; TODAY'S WEATHER FORECAST. 3/3. 44°Hi. RealFeel® 50° ; TONIGHT'S WEATHER FORECAST. 3/3. 32°Lo.\n",
"\n",
"Pomfret, MD Forecast Today Hourly Daily Morning 41° 1% Afternoon 43° 0% Evening 35° 3% Overnight 34° 2% Don't Miss Finally, Here’s Why We Get More Colds and Flu When It’s Cold Coast-To-Coast...\n",
"\n",
"Pomfret, MD Weather Forecast | AccuWeather Current Weather 5:35 PM 35° F RealFeel® 36° RealFeel Shade™ 36° Air Quality Excellent Wind E 3 mph Wind Gusts 5 mph Cloudy More Details WinterCast...\n",
"\n",
"Pomfret, VT Weather Forecast | AccuWeather Current Weather 11:21 AM 23° F RealFeel® 27° RealFeel Shade™ 25° Air Quality Fair Wind ESE 3 mph Wind Gusts 7 mph Cloudy More Details WinterCast...\n",
"\n",
"Pomfret Center, CT Weather Forecast | AccuWeather Daily Current Weather 6:50 PM 39° F RealFeel® 36° Air Quality Fair Wind NW 6 mph Wind Gusts 16 mph Mostly clear More Details WinterCast...\n",
"\n",
"12:00 pm · Feels Like36° · WindN 5 mph · Humidity43% · UV Index3 of 10 · Cloud Cover65% · Rain Amount0 in ...\n",
"\n",
"Pomfret Center, CT Weather Conditions | Weather Underground star Popular Cities San Francisco, CA 49 °F Clear Manhattan, NY 37 °F Fair Schiller Park, IL (60176) warning39 °F Mostly Cloudy...\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I now know the final answer\n",
"Final Answer: The current weather in Pomfret is mainly cloudy with snow showers around in the morning. The temperature is around 40F with winds NNW at 5 to 10 mph. Chance of snow is 40%.\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"'The current weather in Pomfret is mainly cloudy with snow showers around in the morning. The temperature is around 40F with winds NNW at 5 to 10 mph. Chance of snow is 40%.'"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"agent.run(\"What is the weather in Pomfret\")"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.11"
},
"vscode": {
"interpreter": {
"hash": "b1677b440931f40d89ef8be7bf03acb108ce003de0ac9b18e8d43753ea2e7103"
}
}
},
"nbformat": 4,
"nbformat_minor": 5
}
docs/extras/modules/agents/tools/integrations/searx_search.ipynb
-619
View File
@@ -1,619 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"jukit_cell_id": "DUXgyWySl5"
},
"source": [
"# SearxNG Search API\n",
"\n",
"This notebook goes over how to use a self hosted SearxNG search API to search the web.\n",
"\n",
"You can [check this link](https://docs.searxng.org/dev/search_api.html) for more informations about Searx API parameters."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"jukit_cell_id": "OIHXztO2UT"
},
"outputs": [],
"source": [
"import pprint\n",
"from langchain.utilities import SearxSearchWrapper"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"jukit_cell_id": "4SzT9eDMjt"
},
"outputs": [],
"source": [
"search = SearxSearchWrapper(searx_host=\"http://127.0.0.1:8888\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"jukit_cell_id": "jCSkIlQDUK"
},
"source": [
"For some engines, if a direct `answer` is available the warpper will print the answer instead of the full list of search results. You can use the `results` method of the wrapper if you want to obtain all the results."
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"jukit_cell_id": "gGM9PVQX6m"
},
"outputs": [
{
"data": {
"text/plain": [
"'Paris is the capital of France, the largest country of Europe with 550 000 km2 (65 millions inhabitants). Paris has 2.234 million inhabitants end 2011. She is the core of Ile de France region (12 million people).'"
]
},
"execution_count": 1,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"search.run(\"What is the capital of France\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"jukit_cell_id": "OHyurqUPbS"
},
"source": [
"## Custom Parameters\n",
"\n",
"SearxNG supports [135 search engines](https://docs.searxng.org/user/configured_engines.html). You can also customize the Searx wrapper with arbitrary named parameters that will be passed to the Searx search API . In the below example we will making a more interesting use of custom search parameters from searx search api."
]
},
{
"cell_type": "markdown",
"metadata": {
"jukit_cell_id": "n1B2AyLKi4"
},
"source": [
"In this example we will be using the `engines` parameters to query wikipedia"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"jukit_cell_id": "UTEdJ03LqA"
},
"outputs": [],
"source": [
"search = SearxSearchWrapper(\n",
" searx_host=\"http://127.0.0.1:8888\", k=5\n",
") # k is for max number of items"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"jukit_cell_id": "3FyQ6yHI8K",
"tags": [
"scroll-output"
]
},
"outputs": [
{
"data": {
"text/plain": [
"'Large language models (LLMs) represent a major advancement in AI, with the promise of transforming domains through learned knowledge. LLM sizes have been increasing 10X every year for the last few years, and as these models grow in complexity and size, so do their capabilities.\\n\\nGPT-3 can translate language, write essays, generate computer code, and more — all with limited to no supervision. In July 2020, OpenAI unveiled GPT-3, a language model that was easily the largest known at the time. Put simply, GPT-3 is trained to predict the next word in a sentence, much like how a text message autocomplete feature works.\\n\\nA large language model, or LLM, is a deep learning algorithm that can recognize, summarize, translate, predict and generate text and other content based on knowledge gained from massive datasets. Large language models are among the most successful applications of transformer models.\\n\\nAll of today’s well-known language models—e.g., GPT-3 from OpenAI, PaLM or LaMDA from Google, Galactica or OPT from Meta, Megatron-Turing from Nvidia/Microsoft, Jurassic-1 from AI21 Labs—are...\\n\\nLarge language models (LLMs) such as GPT-3are increasingly being used to generate text. These tools should be used with care, since they can generate content that is biased, non-verifiable, constitutes original research, or violates copyrights.'"
]
},
"execution_count": 2,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"search.run(\"large language model \", engines=[\"wiki\"])"
]
},
{
"cell_type": "markdown",
"metadata": {
"jukit_cell_id": "SYz8nFkt81"
},
"source": [
"Passing other Searx parameters for searx like `language`"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"jukit_cell_id": "32rDh0Mvbx"
},
"outputs": [
{
"data": {
"text/plain": [
"'Aprendizaje profundo (en inglés, deep learning) es un conjunto de algoritmos de aprendizaje automático (en inglés, machine learning) que intenta modelar abstracciones de alto nivel en datos usando arquitecturas computacionales que admiten transformaciones no lineales múltiples e iterativas de datos expresados en forma matricial o tensorial. 1'"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"search = SearxSearchWrapper(searx_host=\"http://127.0.0.1:8888\", k=1)\n",
"search.run(\"deep learning\", language=\"es\", engines=[\"wiki\"])"
]
},
{
"cell_type": "markdown",
"metadata": {
"jukit_cell_id": "d0x164ssV1"
},
"source": [
"## Obtaining results with metadata"
]
},
{
"cell_type": "markdown",
"metadata": {
"jukit_cell_id": "pF6rs8XcDH"
},
"source": [
"In this example we will be looking for scientific paper using the `categories` parameter and limiting the results to a `time_range` (not all engines support the time range option).\n",
"\n",
"We also would like to obtain the results in a structured way including metadata. For this we will be using the `results` method of the wrapper."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"jukit_cell_id": "BFgpPH0sxF"
},
"outputs": [],
"source": [
"search = SearxSearchWrapper(searx_host=\"http://127.0.0.1:8888\")"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"jukit_cell_id": "r7qUtvKNOh",
"tags": [
"scroll-output"
]
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[{'snippet': '… on natural language instructions, large language models (… the '\n",
" 'prompt used to steer the model, and most effective prompts … to '\n",
" 'prompt engineering, we propose Automatic Prompt …',\n",
" 'title': 'Large language models are human-level prompt engineers',\n",
" 'link': 'https://arxiv.org/abs/2211.01910',\n",
" 'engines': ['google scholar'],\n",
" 'category': 'science'},\n",
" {'snippet': '… Large language models (LLMs) have introduced new possibilities '\n",
" 'for prototyping with AI [18]. Pre-trained on a large amount of '\n",
" 'text data, models … language instructions called prompts. …',\n",
" 'title': 'Promptchainer: Chaining large language model prompts through '\n",
" 'visual programming',\n",
" 'link': 'https://dl.acm.org/doi/abs/10.1145/3491101.3519729',\n",
" 'engines': ['google scholar'],\n",
" 'category': 'science'},\n",
" {'snippet': '… can introspect the large prompt model. We derive the view '\n",
" 'ϕ0(X) and the model h0 from T01. However, instead of fully '\n",
" 'fine-tuning T0 during co-training, we focus on soft prompt '\n",
" 'tuning, …',\n",
" 'title': 'Co-training improves prompt-based learning for large language '\n",
" 'models',\n",
" 'link': 'https://proceedings.mlr.press/v162/lang22a.html',\n",
" 'engines': ['google scholar'],\n",
" 'category': 'science'},\n",
" {'snippet': '… With the success of large language models (LLMs) of code and '\n",
" 'their use as … prompt design process become important. In this '\n",
" 'work, we propose a framework called Repo-Level Prompt …',\n",
" 'title': 'Repository-level prompt generation for large language models of '\n",
" 'code',\n",
" 'link': 'https://arxiv.org/abs/2206.12839',\n",
" 'engines': ['google scholar'],\n",
" 'category': 'science'},\n",
" {'snippet': '… Figure 2 | The benefits of different components of a prompt '\n",
" 'for the largest language model (Gopher), as estimated from '\n",
" 'hierarchical logistic regression. Each point estimates the '\n",
" 'unique …',\n",
" 'title': 'Can language models learn from explanations in context?',\n",
" 'link': 'https://arxiv.org/abs/2204.02329',\n",
" 'engines': ['google scholar'],\n",
" 'category': 'science'}]\n"
]
}
],
"source": [
"results = search.results(\n",
" \"Large Language Model prompt\",\n",
" num_results=5,\n",
" categories=\"science\",\n",
" time_range=\"year\",\n",
")\n",
"pprint.pp(results)"
]
},
{
"cell_type": "markdown",
"metadata": {
"jukit_cell_id": "2seI78pR8T"
},
"source": [
"Get papers from arxiv"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"jukit_cell_id": "JyNgoFm0vo",
"tags": [
"scroll-output"
]
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[{'snippet': 'Thanks to the advanced improvement of large pre-trained language '\n",
" 'models, prompt-based fine-tuning is shown to be effective on a '\n",
" 'variety of downstream tasks. Though many prompting methods have '\n",
" 'been investigated, it remains unknown which type of prompts are '\n",
" 'the most effective among three types of prompts (i.e., '\n",
" 'human-designed prompts, schema prompts and null prompts). In '\n",
" 'this work, we empirically compare the three types of prompts '\n",
" 'under both few-shot and fully-supervised settings. Our '\n",
" 'experimental results show that schema prompts are the most '\n",
" 'effective in general. Besides, the performance gaps tend to '\n",
" 'diminish when the scale of training data grows large.',\n",
" 'title': 'Do Prompts Solve NLP Tasks Using Natural Language?',\n",
" 'link': 'http://arxiv.org/abs/2203.00902v1',\n",
" 'engines': ['arxiv'],\n",
" 'category': 'science'},\n",
" {'snippet': 'Cross-prompt automated essay scoring (AES) requires the system '\n",
" 'to use non target-prompt essays to award scores to a '\n",
" 'target-prompt essay. Since obtaining a large quantity of '\n",
" 'pre-graded essays to a particular prompt is often difficult and '\n",
" 'unrealistic, the task of cross-prompt AES is vital for the '\n",
" 'development of real-world AES systems, yet it remains an '\n",
" 'under-explored area of research. Models designed for '\n",
" 'prompt-specific AES rely heavily on prompt-specific knowledge '\n",
" 'and perform poorly in the cross-prompt setting, whereas current '\n",
" 'approaches to cross-prompt AES either require a certain quantity '\n",
" 'of labelled target-prompt essays or require a large quantity of '\n",
" 'unlabelled target-prompt essays to perform transfer learning in '\n",
" 'a multi-step manner. To address these issues, we introduce '\n",
" 'Prompt Agnostic Essay Scorer (PAES) for cross-prompt AES. Our '\n",
" 'method requires no access to labelled or unlabelled '\n",
" 'target-prompt data during training and is a single-stage '\n",
" 'approach. PAES is easy to apply in practice and achieves '\n",
" 'state-of-the-art performance on the Automated Student Assessment '\n",
" 'Prize (ASAP) dataset.',\n",
" 'title': 'Prompt Agnostic Essay Scorer: A Domain Generalization Approach to '\n",
" 'Cross-prompt Automated Essay Scoring',\n",
" 'link': 'http://arxiv.org/abs/2008.01441v1',\n",
" 'engines': ['arxiv'],\n",
" 'category': 'science'},\n",
" {'snippet': 'Research on prompting has shown excellent performance with '\n",
" 'little or even no supervised training across many tasks. '\n",
" 'However, prompting for machine translation is still '\n",
" 'under-explored in the literature. We fill this gap by offering a '\n",
" 'systematic study on prompting strategies for translation, '\n",
" 'examining various factors for prompt template and demonstration '\n",
" 'example selection. We further explore the use of monolingual '\n",
" 'data and the feasibility of cross-lingual, cross-domain, and '\n",
" 'sentence-to-document transfer learning in prompting. Extensive '\n",
" 'experiments with GLM-130B (Zeng et al., 2022) as the testbed '\n",
" 'show that 1) the number and the quality of prompt examples '\n",
" 'matter, where using suboptimal examples degenerates translation; '\n",
" '2) several features of prompt examples, such as semantic '\n",
" 'similarity, show significant Spearman correlation with their '\n",
" 'prompting performance; yet, none of the correlations are strong '\n",
" 'enough; 3) using pseudo parallel prompt examples constructed '\n",
" 'from monolingual data via zero-shot prompting could improve '\n",
" 'translation; and 4) improved performance is achievable by '\n",
" 'transferring knowledge from prompt examples selected in other '\n",
" 'settings. We finally provide an analysis on the model outputs '\n",
" 'and discuss several problems that prompting still suffers from.',\n",
" 'title': 'Prompting Large Language Model for Machine Translation: A Case '\n",
" 'Study',\n",
" 'link': 'http://arxiv.org/abs/2301.07069v2',\n",
" 'engines': ['arxiv'],\n",
" 'category': 'science'},\n",
" {'snippet': 'Large language models can perform new tasks in a zero-shot '\n",
" 'fashion, given natural language prompts that specify the desired '\n",
" 'behavior. Such prompts are typically hand engineered, but can '\n",
" 'also be learned with gradient-based methods from labeled data. '\n",
" 'However, it is underexplored what factors make the prompts '\n",
" 'effective, especially when the prompts are natural language. In '\n",
" 'this paper, we investigate common attributes shared by effective '\n",
" 'prompts. We first propose a human readable prompt tuning method '\n",
" '(F LUENT P ROMPT) based on Langevin dynamics that incorporates a '\n",
" 'fluency constraint to find a diverse distribution of effective '\n",
" 'and fluent prompts. Our analysis reveals that effective prompts '\n",
" 'are topically related to the task domain and calibrate the prior '\n",
" 'probability of label words. Based on these findings, we also '\n",
" 'propose a method for generating prompts using only unlabeled '\n",
" 'data, outperforming strong baselines by an average of 7.0% '\n",
" 'accuracy across three tasks.',\n",
" 'title': \"Toward Human Readable Prompt Tuning: Kubrick's The Shining is a \"\n",
" 'good movie, and a good prompt too?',\n",
" 'link': 'http://arxiv.org/abs/2212.10539v1',\n",
" 'engines': ['arxiv'],\n",
" 'category': 'science'},\n",
" {'snippet': 'Prevailing methods for mapping large generative language models '\n",
" \"to supervised tasks may fail to sufficiently probe models' novel \"\n",
" 'capabilities. Using GPT-3 as a case study, we show that 0-shot '\n",
" 'prompts can significantly outperform few-shot prompts. We '\n",
" 'suggest that the function of few-shot examples in these cases is '\n",
" 'better described as locating an already learned task rather than '\n",
" 'meta-learning. This analysis motivates rethinking the role of '\n",
" 'prompts in controlling and evaluating powerful language models. '\n",
" 'In this work, we discuss methods of prompt programming, '\n",
" 'emphasizing the usefulness of considering prompts through the '\n",
" 'lens of natural language. We explore techniques for exploiting '\n",
" 'the capacity of narratives and cultural anchors to encode '\n",
" 'nuanced intentions and techniques for encouraging deconstruction '\n",
" 'of a problem into components before producing a verdict. '\n",
" 'Informed by this more encompassing theory of prompt programming, '\n",
" 'we also introduce the idea of a metaprompt that seeds the model '\n",
" 'to generate its own natural language prompts for a range of '\n",
" 'tasks. Finally, we discuss how these more general methods of '\n",
" 'interacting with language models can be incorporated into '\n",
" 'existing and future benchmarks and practical applications.',\n",
" 'title': 'Prompt Programming for Large Language Models: Beyond the Few-Shot '\n",
" 'Paradigm',\n",
" 'link': 'http://arxiv.org/abs/2102.07350v1',\n",
" 'engines': ['arxiv'],\n",
" 'category': 'science'}]\n"
]
}
],
"source": [
"results = search.results(\n",
" \"Large Language Model prompt\", num_results=5, engines=[\"arxiv\"]\n",
")\n",
"pprint.pp(results)"
]
},
{
"cell_type": "markdown",
"metadata": {
"jukit_cell_id": "LhEisLFcZM"
},
"source": [
"In this example we query for `large language models` under the `it` category. We then filter the results that come from github."
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"jukit_cell_id": "aATPfXzGzx"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[{'snippet': 'Guide to using pre-trained large language models of source code',\n",
" 'title': 'Code-LMs',\n",
" 'link': 'https://github.com/VHellendoorn/Code-LMs',\n",
" 'engines': ['github'],\n",
" 'category': 'it'},\n",
" {'snippet': 'Dramatron uses large language models to generate coherent '\n",
" 'scripts and screenplays.',\n",
" 'title': 'dramatron',\n",
" 'link': 'https://github.com/deepmind/dramatron',\n",
" 'engines': ['github'],\n",
" 'category': 'it'}]\n"
]
}
],
"source": [
"results = search.results(\"large language model\", num_results=20, categories=\"it\")\n",
"pprint.pp(list(filter(lambda r: r[\"engines\"][0] == \"github\", results)))"
]
},
{
"cell_type": "markdown",
"metadata": {
"jukit_cell_id": "zDo2YjafuU"
},
"source": [
"We could also directly query for results from `github` and other source forges."
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"jukit_cell_id": "5NrlredKxM",
"tags": [
"scroll-output"
]
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[{'snippet': \"Implementation of 'A Watermark for Large Language Models' paper \"\n",
" 'by Kirchenbauer & Geiping et. al.',\n",
" 'title': 'Peutlefaire / LMWatermark',\n",
" 'link': 'https://gitlab.com/BrianPulfer/LMWatermark',\n",
" 'engines': ['gitlab'],\n",
" 'category': 'it'},\n",
" {'snippet': 'Guide to using pre-trained large language models of source code',\n",
" 'title': 'Code-LMs',\n",
" 'link': 'https://github.com/VHellendoorn/Code-LMs',\n",
" 'engines': ['github'],\n",
" 'category': 'it'},\n",
" {'snippet': '',\n",
" 'title': 'Simen Burud / Large-scale Language Models for Conversational '\n",
" 'Speech Recognition',\n",
" 'link': 'https://gitlab.com/BrianPulfer',\n",
" 'engines': ['gitlab'],\n",
" 'category': 'it'},\n",
" {'snippet': 'Dramatron uses large language models to generate coherent '\n",
" 'scripts and screenplays.',\n",
" 'title': 'dramatron',\n",
" 'link': 'https://github.com/deepmind/dramatron',\n",
" 'engines': ['github'],\n",
" 'category': 'it'},\n",
" {'snippet': 'Code for loralib, an implementation of \"LoRA: Low-Rank '\n",
" 'Adaptation of Large Language Models\"',\n",
" 'title': 'LoRA',\n",
" 'link': 'https://github.com/microsoft/LoRA',\n",
" 'engines': ['github'],\n",
" 'category': 'it'},\n",
" {'snippet': 'Code for the paper \"Evaluating Large Language Models Trained on '\n",
" 'Code\"',\n",
" 'title': 'human-eval',\n",
" 'link': 'https://github.com/openai/human-eval',\n",
" 'engines': ['github'],\n",
" 'category': 'it'},\n",
" {'snippet': 'A trend starts from \"Chain of Thought Prompting Elicits '\n",
" 'Reasoning in Large Language Models\".',\n",
" 'title': 'Chain-of-ThoughtsPapers',\n",
" 'link': 'https://github.com/Timothyxxx/Chain-of-ThoughtsPapers',\n",
" 'engines': ['github'],\n",
" 'category': 'it'},\n",
" {'snippet': 'Mistral: A strong, northwesterly wind: Framework for transparent '\n",
" 'and accessible large-scale language model training, built with '\n",
" 'Hugging Face 🤗 Transformers.',\n",
" 'title': 'mistral',\n",
" 'link': 'https://github.com/stanford-crfm/mistral',\n",
" 'engines': ['github'],\n",
" 'category': 'it'},\n",
" {'snippet': 'A prize for finding tasks that cause large language models to '\n",
" 'show inverse scaling',\n",
" 'title': 'prize',\n",
" 'link': 'https://github.com/inverse-scaling/prize',\n",
" 'engines': ['github'],\n",
" 'category': 'it'},\n",
" {'snippet': 'Optimus: the first large-scale pre-trained VAE language model',\n",
" 'title': 'Optimus',\n",
" 'link': 'https://github.com/ChunyuanLI/Optimus',\n",
" 'engines': ['github'],\n",
" 'category': 'it'},\n",
" {'snippet': 'Seminar on Large Language Models (COMP790-101 at UNC Chapel '\n",
" 'Hill, Fall 2022)',\n",
" 'title': 'llm-seminar',\n",
" 'link': 'https://github.com/craffel/llm-seminar',\n",
" 'engines': ['github'],\n",
" 'category': 'it'},\n",
" {'snippet': 'A central, open resource for data and tools related to '\n",
" 'chain-of-thought reasoning in large language models. Developed @ '\n",
" 'Samwald research group: https://samwald.info/',\n",
" 'title': 'ThoughtSource',\n",
" 'link': 'https://github.com/OpenBioLink/ThoughtSource',\n",
" 'engines': ['github'],\n",
" 'category': 'it'},\n",
" {'snippet': 'A comprehensive list of papers using large language/multi-modal '\n",
" 'models for Robotics/RL, including papers, codes, and related '\n",
" 'websites',\n",
" 'title': 'Awesome-LLM-Robotics',\n",
" 'link': 'https://github.com/GT-RIPL/Awesome-LLM-Robotics',\n",
" 'engines': ['github'],\n",
" 'category': 'it'},\n",
" {'snippet': 'Tools for curating biomedical training data for large-scale '\n",
" 'language modeling',\n",
" 'title': 'biomedical',\n",
" 'link': 'https://github.com/bigscience-workshop/biomedical',\n",
" 'engines': ['github'],\n",
" 'category': 'it'},\n",
" {'snippet': 'ChatGPT @ Home: Large Language Model (LLM) chatbot application, '\n",
" 'written by ChatGPT',\n",
" 'title': 'ChatGPT-at-Home',\n",
" 'link': 'https://github.com/Sentdex/ChatGPT-at-Home',\n",
" 'engines': ['github'],\n",
" 'category': 'it'},\n",
" {'snippet': 'Design and Deploy Large Language Model Apps',\n",
" 'title': 'dust',\n",
" 'link': 'https://github.com/dust-tt/dust',\n",
" 'engines': ['github'],\n",
" 'category': 'it'},\n",
" {'snippet': 'Polyglot: Large Language Models of Well-balanced Competence in '\n",
" 'Multi-languages',\n",
" 'title': 'polyglot',\n",
" 'link': 'https://github.com/EleutherAI/polyglot',\n",
" 'engines': ['github'],\n",
" 'category': 'it'},\n",
" {'snippet': 'Code release for \"Learning Video Representations from Large '\n",
" 'Language Models\"',\n",
" 'title': 'LaViLa',\n",
" 'link': 'https://github.com/facebookresearch/LaViLa',\n",
" 'engines': ['github'],\n",
" 'category': 'it'},\n",
" {'snippet': 'SmoothQuant: Accurate and Efficient Post-Training Quantization '\n",
" 'for Large Language Models',\n",
" 'title': 'smoothquant',\n",
" 'link': 'https://github.com/mit-han-lab/smoothquant',\n",
" 'engines': ['github'],\n",
" 'category': 'it'},\n",
" {'snippet': 'This repository contains the code, data, and models of the paper '\n",
" 'titled \"XL-Sum: Large-Scale Multilingual Abstractive '\n",
" 'Summarization for 44 Languages\" published in Findings of the '\n",
" 'Association for Computational Linguistics: ACL-IJCNLP 2021.',\n",
" 'title': 'xl-sum',\n",
" 'link': 'https://github.com/csebuetnlp/xl-sum',\n",
" 'engines': ['github'],\n",
" 'category': 'it'}]\n"
]
}
],
"source": [
"results = search.results(\n",
" \"large language model\", num_results=20, engines=[\"github\", \"gitlab\"]\n",
")\n",
"pprint.pp(results)"
]
}
],
"metadata": {
"anaconda-cloud": {},
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.1"
}
},
"nbformat": 4,
"nbformat_minor": 4
}
docs/extras/modules/agents/tools/integrations/serpapi.ipynb
-138
View File
@@ -1,138 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "dc23c48e",
"metadata": {},
"source": [
"# SerpAPI\n",
"\n",
"This notebook goes over how to use the SerpAPI component to search the web."
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "54bf5afd",
"metadata": {},
"outputs": [],
"source": [
"from langchain.utilities import SerpAPIWrapper"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "31f8f382",
"metadata": {},
"outputs": [],
"source": [
"search = SerpAPIWrapper()"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "25ce0225",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'Barack Hussein Obama II'"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"search.run(\"Obama's first name?\")"
]
},
{
"cell_type": "markdown",
"id": "fe3ee213",
"metadata": {},
"source": [
"## Custom Parameters\n",
"You can also customize the SerpAPI wrapper with arbitrary parameters. For example, in the below example we will use `bing` instead of `google`."
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "deffcc8b",
"metadata": {},
"outputs": [],
"source": [
"params = {\n",
" \"engine\": \"bing\",\n",
" \"gl\": \"us\",\n",
" \"hl\": \"en\",\n",
"}\n",
"search = SerpAPIWrapper(params=params)"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "2c752d08",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'Barack Hussein Obama II is an American politician who served as the 44th president of the United States from 2009 to 2017. A member of the Democratic Party, Obama was the first African-American presi…New content will be added above the current area of focus upon selectionBarack Hussein Obama II is an American politician who served as the 44th president of the United States from 2009 to 2017. A member of the Democratic Party, Obama was the first African-American president of the United States. He previously served as a U.S. senator from Illinois from 2005 to 2008 and as an Illinois state senator from 1997 to 2004, and previously worked as a civil rights lawyer before entering politics.Wikipediabarackobama.com'"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"search.run(\"Obama's first name?\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e0a1dc1c",
"metadata": {},
"outputs": [],
"source": [
"from langchain.agents import Tool\n",
"\n",
"# You can create the tool to pass to an agent\n",
"repl_tool = Tool(\n",
" name=\"python_repl\",\n",
" description=\"A Python shell. Use this to execute python commands. Input should be a valid python command. If you want to see the output of a value, you should print it out with `print(...)`.\",\n",
" func=search.run,\n",
")"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.9"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
docs/extras/modules/agents/tools/integrations/twilio.ipynb
-165
View File
@@ -1,165 +0,0 @@
{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"id": "dc23c48e",
"metadata": {},
"source": [
"# Twilio\n",
"\n",
"This notebook goes over how to use the [Twilio](https://www.twilio.com) API wrapper to send a message through SMS or [Twilio Messaging Channels](https://www.twilio.com/docs/messaging/channels).\n",
"\n",
"Twilio Messaging Channels facilitates integrations with 3rd party messaging apps and lets you send messages through WhatsApp Business Platform (GA), Facebook Messenger (Public Beta) and Google Business Messages (Private Beta)."
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "c1a33b13",
"metadata": {},
"source": [
"## Setup\n",
"\n",
"To use this tool you need to install the Python Twilio package `twilio`"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "98b544b9",
"metadata": {},
"outputs": [],
"source": [
"# !pip install twilio"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "f7e883ae",
"metadata": {},
"source": [
"You'll also need to set up a Twilio account and get your credentials. You'll need your Account String Identifier (SID) and your Auth Token. You'll also need a number to send messages from.\n",
"\n",
"You can either pass these in to the TwilioAPIWrapper as named parameters `account_sid`, `auth_token`, `from_number`, or you can set the environment variables `TWILIO_ACCOUNT_SID`, `TWILIO_AUTH_TOKEN`, `TWILIO_FROM_NUMBER`."
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "36c133be",
"metadata": {},
"source": [
"## Sending an SMS"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "54bf5afd",
"metadata": {},
"outputs": [],
"source": [
"from langchain.utilities.twilio import TwilioAPIWrapper"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "31f8f382",
"metadata": {},
"outputs": [],
"source": [
"twilio = TwilioAPIWrapper(\n",
" # account_sid=\"foo\",\n",
" # auth_token=\"bar\",\n",
" # from_number=\"baz,\"\n",
")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5009d763",
"metadata": {},
"outputs": [],
"source": [
"twilio.run(\"hello world\", \"+16162904619\")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "de022dc9",
"metadata": {},
"source": [
"## Sending a WhatsApp Message"
]
},
{
"attachments": {},
"cell_type": "markdown",
"id": "a594d0bc",
"metadata": {},
"source": [
"You'll need to link your WhatsApp Business Account with Twilio. You'll also need to make sure that the number to send messages from is configured as a WhatsApp Enabled Sender on Twilio and registered with WhatsApp."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "94508aa0",
"metadata": {},
"outputs": [],
"source": [
"from langchain.utilities.twilio import TwilioAPIWrapper"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e4b81750",
"metadata": {},
"outputs": [],
"source": [
"twilio = TwilioAPIWrapper(\n",
" # account_sid=\"foo\",\n",
" # auth_token=\"bar\",\n",
" # from_number=\"whatsapp: baz,\"\n",
")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "1181041b",
"metadata": {},
"outputs": [],
"source": [
"twilio.run(\"hello world\", \"whatsapp: +16162904619\")"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.3"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
docs/extras/modules/agents/tools/integrations/wikipedia.ipynb
-93
View File
@@ -1,93 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "245a954a",
"metadata": {},
"source": [
"# Wikipedia\n",
"\n",
">[Wikipedia](https://wikipedia.org/) is a multilingual free online encyclopedia written and maintained by a community of volunteers, known as Wikipedians, through open collaboration and using a wiki-based editing system called MediaWiki. `Wikipedia` is the largest and most-read reference work in history.\n",
"\n",
"First, you need to install `wikipedia` python package."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "961b3689",
"metadata": {
"vscode": {
"languageId": "shellscript"
}
},
"outputs": [],
"source": [
"!pip install wikipedia"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "8d32b39a",
"metadata": {},
"outputs": [],
"source": [
"from langchain.tools import WikipediaQueryRun\n",
"from langchain.utilities import WikipediaAPIWrapper"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "2a50dd27",
"metadata": {},
"outputs": [],
"source": [
"wikipedia = WikipediaQueryRun(api_wrapper=WikipediaAPIWrapper())"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "34bb5968",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'Page: Hunter × Hunter\\nSummary: Hunter × Hunter (stylized as HUNTER×HUNTER and pronounced \"hunter hunter\") is a Japanese manga series written and illustrated by Yoshihiro Togashi. It has been serialized in Shueisha\\'s shōnen manga magazine Weekly Shōnen Jump since March 1998, although the manga has frequently gone on extended hiatuses since 2006. Its chapters have been collected in 37 tankōbon volumes as of November 2022. The story focuses on a young boy named Gon Freecss who discovers that his father, who left him at a young age, is actually a world-renowned Hunter, a licensed professional who specializes in fantastical pursuits such as locating rare or unidentified animal species, treasure hunting, surveying unexplored enclaves, or hunting down lawless individuals. Gon departs on a journey to become a Hunter and eventually find his father. Along the way, Gon meets various other Hunters and encounters the paranormal.\\nHunter × Hunter was adapted into a 62-episode anime television series produced by Nippon Animation and directed by Kazuhiro Furuhashi, which ran on Fuji Television from October 1999 to March 2001. Three separate original video animations (OVAs) totaling 30 episodes were subsequently produced by Nippon Animation and released in Japan from 2002 to 2004. A second anime television series by Madhouse aired on Nippon Television from October 2011 to September 2014, totaling 148 episodes, with two animated theatrical films released in 2013. There are also numerous audio albums, video games, musicals, and other media based on Hunter × Hunter.\\nThe manga has been translated into English and released in North America by Viz Media since April 2005. Both television series have been also licensed by Viz Media, with the first series having aired on the Funimation Channel in 2009 and the second series broadcast on Adult Swim\\'s Toonami programming block from April 2016 to June 2019.\\nHunter × Hunter has been a huge critical and financial success and has become one of the best-selling manga series of all time, having over 84 million copies in circulation by July 2022.\\n\\nPage: Hunter × Hunter (2011 TV series)\\nSummary: Hunter × Hunter is an anime television series that aired from 2011 to 2014 based on Yoshihiro Togashi\\'s manga series Hunter × Hunter. The story begins with a young boy named Gon Freecss, who one day discovers that the father who he thought was dead, is in fact alive and well. He learns that his father, Ging, is a legendary \"Hunter\", an individual who has proven themselves an elite member of humanity. Despite the fact that Ging left his son with his relatives in order to pursue his own dreams, Gon becomes determined to follow in his father\\'s footsteps, pass the rigorous \"Hunter Examination\", and eventually find his father to become a Hunter in his own right.\\nThis new Hunter × Hunter anime was announced on July 24, 2011. It is a complete reboot starting from the beginning of the original manga, with no connection to the first anime television series from 1999. Produced by Nippon TV, VAP, Shueisha and Madhouse, the series is directed by Hiroshi Kōjina, with Atsushi Maekawa and Tsutomu Kamishiro handling series composition, Takahiro Yoshimatsu designing the characters and Yoshihisa Hirano composing the music. Instead of having the old cast reprise their roles for the new adaptation, the series features an entirely new cast to voice the characters. The new series premiered airing weekly on Nippon TV and the nationwide Nippon News Network from October 2, 2011. The series started to be collected in both DVD and Blu-ray format on January 25, 2012. Viz Media has licensed the anime for a DVD/Blu-ray release in North America with an English dub. On television, the series began airing on Adult Swim\\'s Toonami programming block on April 17, 2016, and ended on June 23, 2019.The anime series\\' opening theme is alternated between the song \"Departure!\" and an alternate version titled \"Departure! -Second Version-\" both sung by Galneryus\\' voc'"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"wikipedia.run(\"HUNTER X HUNTER\")"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.6"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
docs/extras/modules/agents/tools/integrations/wolfram_alpha.ipynb
-125
View File
@@ -1,125 +0,0 @@
{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"id": "245a954a",
"metadata": {},
"source": [
"# Wolfram Alpha\n",
"\n",
"This notebook goes over how to use the wolfram alpha component.\n",
"\n",
"First, you need to set up your Wolfram Alpha developer account and get your APP ID:\n",
"\n",
"1. Go to wolfram alpha and sign up for a developer account [here](https://developer.wolframalpha.com/)\n",
"2. Create an app and get your APP ID\n",
"3. pip install wolframalpha\n",
"\n",
"Then we will need to set some environment variables:\n",
"1. Save your APP ID into WOLFRAM_ALPHA_APPID env variable"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "961b3689",
"metadata": {
"vscode": {
"languageId": "shellscript"
}
},
"outputs": [],
"source": [
"pip install wolframalpha"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "34bb5968",
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"\n",
"os.environ[\"WOLFRAM_ALPHA_APPID\"] = \"\""
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "ac4910f8",
"metadata": {},
"outputs": [],
"source": [
"from langchain.utilities.wolfram_alpha import WolframAlphaAPIWrapper"
]
},
{
"cell_type": "code",
"execution_count": 10,
"id": "84b8f773",
"metadata": {},
"outputs": [],
"source": [
"wolfram = WolframAlphaAPIWrapper()"
]
},
{
"cell_type": "code",
"execution_count": 11,
"id": "068991a6",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'x = 2/5'"
]
},
"execution_count": 11,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"wolfram.run(\"What is 2x+5 = -3x + 7?\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "028f4cba",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": ".venv",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.7"
},
"vscode": {
"interpreter": {
"hash": "53f3bc57609c7a84333bb558594977aa5b4026b1d6070b93987956689e367341"
}
}
},
"nbformat": 4,
"nbformat_minor": 5
}
docs/extras/modules/agents/tools/integrations/youtube.ipynb
-125
View File
@@ -1,125 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "acb64858",
"metadata": {},
"source": [
"# YouTubeSearchTool\n",
"\n",
"This notebook shows how to use a tool to search YouTube\n",
"\n",
"Adapted from [https://github.com/venuv/langchain_yt_tools](https://github.com/venuv/langchain_yt_tools)"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "9bb15d4a",
"metadata": {},
"outputs": [],
"source": [
"#! pip install youtube_search"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "cc1c83e2",
"metadata": {},
"outputs": [],
"source": [
"from langchain.tools import YouTubeSearchTool"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "becb262b",
"metadata": {},
"outputs": [],
"source": [
"tool = YouTubeSearchTool()"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "6bbc4211",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"\"['/watch?v=VcVfceTsD0A&pp=ygUMbGV4IGZyaWVkbWFu', '/watch?v=gPfriiHBBek&pp=ygUMbGV4IGZyaWVkbWFu']\""
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"tool.run(\"lex friedman\")"
]
},
{
"cell_type": "markdown",
"id": "7f772147",
"metadata": {},
"source": [
"You can also specify the number of results that are returned"
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "682fdb33",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"\"['/watch?v=VcVfceTsD0A&pp=ygUMbGV4IGZyaWVkbWFu', '/watch?v=YVJ8gTnDC4Y&pp=ygUMbGV4IGZyaWVkbWFu', '/watch?v=Udh22kuLebg&pp=ygUMbGV4IGZyaWVkbWFu', '/watch?v=gPfriiHBBek&pp=ygUMbGV4IGZyaWVkbWFu', '/watch?v=L_Guz73e6fw&pp=ygUMbGV4IGZyaWVkbWFu']\""
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"tool.run(\"lex friedman,5\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "bb5e1659",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.1"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
docs/extras/modules/agents/tools/integrations/zapier.ipynb
-377
View File
@@ -1,377 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "16763ed3",
"metadata": {},
"source": [
"# Zapier Natural Language Actions API\n",
"\\\n",
"Full docs here: https://nla.zapier.com/start/\n",
"\n",
"**Zapier Natural Language Actions** gives you access to the 5k+ apps, 20k+ actions on Zapier's platform through a natural language API interface.\n",
"\n",
"NLA supports apps like Gmail, Salesforce, Trello, Slack, Asana, HubSpot, Google Sheets, Microsoft Teams, and thousands more apps: https://zapier.com/apps\n",
"\n",
"Zapier NLA handles ALL the underlying API auth and translation from natural language --> underlying API call --> return simplified output for LLMs. The key idea is you, or your users, expose a set of actions via an oauth-like setup window, which you can then query and execute via a REST API.\n",
"\n",
"NLA offers both API Key and OAuth for signing NLA API requests.\n",
"\n",
"1. Server-side (API Key): for quickly getting started, testing, and production scenarios where LangChain will only use actions exposed in the developer's Zapier account (and will use the developer's connected accounts on Zapier.com)\n",
"\n",
"2. User-facing (Oauth): for production scenarios where you are deploying an end-user facing application and LangChain needs access to end-user's exposed actions and connected accounts on Zapier.com\n",
"\n",
"This quick start will focus mostly on the server-side use case for brevity. Jump to [Example Using OAuth Access Token](#oauth) to see a short example how to set up Zapier for user-facing situations. Review [full docs](https://nla.zapier.com/start/) for full user-facing oauth developer support.\n",
"\n",
"This example goes over how to use the Zapier integration with a `SimpleSequentialChain`, then an `Agent`.\n",
"In code, below:"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "5cf33377",
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"\n",
"# get from https://platform.openai.com/\n",
"os.environ[\"OPENAI_API_KEY\"] = os.environ.get(\"OPENAI_API_KEY\", \"\")\n",
"\n",
"# get from https://nla.zapier.com/docs/authentication/ after logging in):\n",
"os.environ[\"ZAPIER_NLA_API_KEY\"] = os.environ.get(\"ZAPIER_NLA_API_KEY\", \"\")"
]
},
{
"cell_type": "markdown",
"id": "4881b484-1b97-478f-b206-aec407ceff66",
"metadata": {},
"source": [
"## Example with Agent\n",
"Zapier tools can be used with an agent. See the example below."
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "b2044b17-c941-4ffb-8a03-027a35e2df81",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"from langchain.llms import OpenAI\n",
"from langchain.agents import initialize_agent\n",
"from langchain.agents.agent_toolkits import ZapierToolkit\n",
"from langchain.agents import AgentType\n",
"from langchain.utilities.zapier import ZapierNLAWrapper"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "7b505eeb",
"metadata": {},
"outputs": [],
"source": [
"## step 0. expose gmail 'find email' and slack 'send channel message' actions\n",
"\n",
"# first go here, log in, expose (enable) the two actions: https://nla.zapier.com/demo/start -- for this example, can leave all fields \"Have AI guess\"\n",
"# in an oauth scenario, you'd get your own <provider> id (instead of 'demo') which you route your users through first"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "cab18227-c232-4214-9256-bb8dd352266c",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"llm = OpenAI(temperature=0)\n",
"zapier = ZapierNLAWrapper()\n",
"toolkit = ZapierToolkit.from_zapier_nla_wrapper(zapier)\n",
"agent = initialize_agent(\n",
" toolkit.get_tools(), llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=True\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "f94713de-b64d-465f-a087-00288b5f80ec",
"metadata": {
"tags": []
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3m I need to find the email and summarize it.\n",
"Action: Gmail: Find Email\n",
"Action Input: Find the latest email from Silicon Valley Bank\u001b[0m\n",
"Observation: \u001b[31;1m\u001b[1;3m{\"from__name\": \"Silicon Valley Bridge Bank, N.A.\", \"from__email\": \"sreply@svb.com\", \"body_plain\": \"Dear Clients, After chaotic, tumultuous & stressful days, we have clarity on path for SVB, FDIC is fully insuring all deposits & have an ask for clients & partners as we rebuild. Tim Mayopoulos <https://eml.svb.com/NjEwLUtBSy0yNjYAAAGKgoxUeBCLAyF_NxON97X4rKEaNBLG\", \"reply_to__email\": \"sreply@svb.com\", \"subject\": \"Meet the new CEO Tim Mayopoulos\", \"date\": \"Tue, 14 Mar 2023 23:42:29 -0500 (CDT)\", \"message_url\": \"https://mail.google.com/mail/u/0/#inbox/186e393b13cfdf0a\", \"attachment_count\": \"0\", \"to__emails\": \"ankush@langchain.dev\", \"message_id\": \"186e393b13cfdf0a\", \"labels\": \"IMPORTANT, CATEGORY_UPDATES, INBOX\"}\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I need to summarize the email and send it to the #test-zapier channel in Slack.\n",
"Action: Slack: Send Channel Message\n",
"Action Input: Send a slack message to the #test-zapier channel with the text \"Silicon Valley Bank has announced that Tim Mayopoulos is the new CEO. FDIC is fully insuring all deposits and they have an ask for clients and partners as they rebuild.\"\u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3m{\"message__text\": \"Silicon Valley Bank has announced that Tim Mayopoulos is the new CEO. FDIC is fully insuring all deposits and they have an ask for clients and partners as they rebuild.\", \"message__permalink\": \"https://langchain.slack.com/archives/C04TSGU0RA7/p1678859932375259\", \"channel\": \"C04TSGU0RA7\", \"message__bot_profile__name\": \"Zapier\", \"message__team\": \"T04F8K3FZB5\", \"message__bot_id\": \"B04TRV4R74K\", \"message__bot_profile__deleted\": \"false\", \"message__bot_profile__app_id\": \"A024R9PQM\", \"ts_time\": \"2023-03-15T05:58:52Z\", \"message__bot_profile__icons__image_36\": \"https://avatars.slack-edge.com/2022-08-02/3888649620612_f864dc1bb794cf7d82b0_36.png\", \"message__blocks[]block_id\": \"kdZZ\", \"message__blocks[]elements[]type\": \"['rich_text_section']\"}\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m I now know the final answer.\n",
"Final Answer: I have sent a summary of the last email from Silicon Valley Bank to the #test-zapier channel in Slack.\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"'I have sent a summary of the last email from Silicon Valley Bank to the #test-zapier channel in Slack.'"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"agent.run(\n",
" \"Summarize the last email I received regarding Silicon Valley Bank. Send the summary to the #test-zapier channel in slack.\"\n",
")"
]
},
{
"cell_type": "markdown",
"id": "bcdea831",
"metadata": {},
"source": [
"## Example with SimpleSequentialChain\n",
"If you need more explicit control, use a chain, like below."
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "10a46e7e",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"from langchain.llms import OpenAI\n",
"from langchain.chains import LLMChain, TransformChain, SimpleSequentialChain\n",
"from langchain.prompts import PromptTemplate\n",
"from langchain.tools.zapier.tool import ZapierNLARunAction\n",
"from langchain.utilities.zapier import ZapierNLAWrapper"
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "b9358048",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"## step 0. expose gmail 'find email' and slack 'send direct message' actions\n",
"\n",
"# first go here, log in, expose (enable) the two actions: https://nla.zapier.com/demo/start -- for this example, can leave all fields \"Have AI guess\"\n",
"# in an oauth scenario, you'd get your own <provider> id (instead of 'demo') which you route your users through first\n",
"\n",
"actions = ZapierNLAWrapper().list()"
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "4e80f461",
"metadata": {},
"outputs": [],
"source": [
"## step 1. gmail find email\n",
"\n",
"GMAIL_SEARCH_INSTRUCTIONS = \"Grab the latest email from Silicon Valley Bank\"\n",
"\n",
"\n",
"def nla_gmail(inputs):\n",
" action = next(\n",
" (a for a in actions if a[\"description\"].startswith(\"Gmail: Find Email\")), None\n",
" )\n",
" return {\n",
" \"email_data\": ZapierNLARunAction(\n",
" action_id=action[\"id\"],\n",
" zapier_description=action[\"description\"],\n",
" params_schema=action[\"params\"],\n",
" ).run(inputs[\"instructions\"])\n",
" }\n",
"\n",
"\n",
"gmail_chain = TransformChain(\n",
" input_variables=[\"instructions\"],\n",
" output_variables=[\"email_data\"],\n",
" transform=nla_gmail,\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 10,
"id": "46893233",
"metadata": {},
"outputs": [],
"source": [
"## step 2. generate draft reply\n",
"\n",
"template = \"\"\"You are an assisstant who drafts replies to an incoming email. Output draft reply in plain text (not JSON).\n",
"\n",
"Incoming email:\n",
"{email_data}\n",
"\n",
"Draft email reply:\"\"\"\n",
"\n",
"prompt_template = PromptTemplate(input_variables=[\"email_data\"], template=template)\n",
"reply_chain = LLMChain(llm=OpenAI(temperature=0.7), prompt=prompt_template)"
]
},
{
"cell_type": "code",
"execution_count": 11,
"id": "cd85c4f8",
"metadata": {},
"outputs": [],
"source": [
"## step 3. send draft reply via a slack direct message\n",
"\n",
"SLACK_HANDLE = \"@Ankush Gola\"\n",
"\n",
"\n",
"def nla_slack(inputs):\n",
" action = next(\n",
" (\n",
" a\n",
" for a in actions\n",
" if a[\"description\"].startswith(\"Slack: Send Direct Message\")\n",
" ),\n",
" None,\n",
" )\n",
" instructions = f'Send this to {SLACK_HANDLE} in Slack: {inputs[\"draft_reply\"]}'\n",
" return {\n",
" \"slack_data\": ZapierNLARunAction(\n",
" action_id=action[\"id\"],\n",
" zapier_description=action[\"description\"],\n",
" params_schema=action[\"params\"],\n",
" ).run(instructions)\n",
" }\n",
"\n",
"\n",
"slack_chain = TransformChain(\n",
" input_variables=[\"draft_reply\"],\n",
" output_variables=[\"slack_data\"],\n",
" transform=nla_slack,\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 12,
"id": "4829cab4",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new SimpleSequentialChain chain...\u001b[0m\n",
"\u001b[36;1m\u001b[1;3m{\"from__name\": \"Silicon Valley Bridge Bank, N.A.\", \"from__email\": \"sreply@svb.com\", \"body_plain\": \"Dear Clients, After chaotic, tumultuous & stressful days, we have clarity on path for SVB, FDIC is fully insuring all deposits & have an ask for clients & partners as we rebuild. Tim Mayopoulos <https://eml.svb.com/NjEwLUtBSy0yNjYAAAGKgoxUeBCLAyF_NxON97X4rKEaNBLG\", \"reply_to__email\": \"sreply@svb.com\", \"subject\": \"Meet the new CEO Tim Mayopoulos\", \"date\": \"Tue, 14 Mar 2023 23:42:29 -0500 (CDT)\", \"message_url\": \"https://mail.google.com/mail/u/0/#inbox/186e393b13cfdf0a\", \"attachment_count\": \"0\", \"to__emails\": \"ankush@langchain.dev\", \"message_id\": \"186e393b13cfdf0a\", \"labels\": \"IMPORTANT, CATEGORY_UPDATES, INBOX\"}\u001b[0m\n",
"\u001b[33;1m\u001b[1;3m\n",
"Dear Silicon Valley Bridge Bank, \n",
"\n",
"Thank you for your email and the update regarding your new CEO Tim Mayopoulos. We appreciate your dedication to keeping your clients and partners informed and we look forward to continuing our relationship with you. \n",
"\n",
"Best regards, \n",
"[Your Name]\u001b[0m\n",
"\u001b[38;5;200m\u001b[1;3m{\"message__text\": \"Dear Silicon Valley Bridge Bank, \\n\\nThank you for your email and the update regarding your new CEO Tim Mayopoulos. We appreciate your dedication to keeping your clients and partners informed and we look forward to continuing our relationship with you. \\n\\nBest regards, \\n[Your Name]\", \"message__permalink\": \"https://langchain.slack.com/archives/D04TKF5BBHU/p1678859968241629\", \"channel\": \"D04TKF5BBHU\", \"message__bot_profile__name\": \"Zapier\", \"message__team\": \"T04F8K3FZB5\", \"message__bot_id\": \"B04TRV4R74K\", \"message__bot_profile__deleted\": \"false\", \"message__bot_profile__app_id\": \"A024R9PQM\", \"ts_time\": \"2023-03-15T05:59:28Z\", \"message__blocks[]block_id\": \"p7i\", \"message__blocks[]elements[]elements[]type\": \"[['text']]\", \"message__blocks[]elements[]type\": \"['rich_text_section']\"}\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"'{\"message__text\": \"Dear Silicon Valley Bridge Bank, \\\\n\\\\nThank you for your email and the update regarding your new CEO Tim Mayopoulos. We appreciate your dedication to keeping your clients and partners informed and we look forward to continuing our relationship with you. \\\\n\\\\nBest regards, \\\\n[Your Name]\", \"message__permalink\": \"https://langchain.slack.com/archives/D04TKF5BBHU/p1678859968241629\", \"channel\": \"D04TKF5BBHU\", \"message__bot_profile__name\": \"Zapier\", \"message__team\": \"T04F8K3FZB5\", \"message__bot_id\": \"B04TRV4R74K\", \"message__bot_profile__deleted\": \"false\", \"message__bot_profile__app_id\": \"A024R9PQM\", \"ts_time\": \"2023-03-15T05:59:28Z\", \"message__blocks[]block_id\": \"p7i\", \"message__blocks[]elements[]elements[]type\": \"[[\\'text\\']]\", \"message__blocks[]elements[]type\": \"[\\'rich_text_section\\']\"}'"
]
},
"execution_count": 12,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"## finally, execute\n",
"\n",
"overall_chain = SimpleSequentialChain(\n",
" chains=[gmail_chain, reply_chain, slack_chain], verbose=True\n",
")\n",
"overall_chain.run(GMAIL_SEARCH_INSTRUCTIONS)"
]
},
{
"cell_type": "markdown",
"id": "09ff954e-45f2-4595-92ea-91627abde4a0",
"metadata": {},
"source": [
"## <a id=\"oauth\">Example Using OAuth Access Token</a>\n",
"The below snippet shows how to initialize the wrapper with a procured OAuth access token. Note the argument being passed in as opposed to setting an environment variable. Review the [authentication docs](https://nla.zapier.com/docs/authentication/#oauth-credentials) for full user-facing oauth developer support.\n",
"\n",
"The developer is tasked with handling the OAuth handshaking to procure and refresh the access token."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7c6835c8",
"metadata": {},
"outputs": [],
"source": [
"llm = OpenAI(temperature=0)\n",
"zapier = ZapierNLAWrapper(zapier_nla_oauth_access_token=\"<fill in access token here>\")\n",
"toolkit = ZapierToolkit.from_zapier_nla_wrapper(zapier)\n",
"agent = initialize_agent(\n",
" toolkit.get_tools(), llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=True\n",
")\n",
"\n",
"agent.run(\n",
" \"Summarize the last email I received regarding Silicon Valley Bank. Send the summary to the #test-zapier channel in slack.\"\n",
")"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.1"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
docs/extras/modules/agents/tools/how_to/multi_input_tool.ipynb → docs/extras/modules/agents/tools/multi_input_tool.ipynb
View File
docs/extras/modules/agents/tools/how_to/tool_input_validation.ipynb → docs/extras/modules/agents/tools/tool_input_validation.ipynb
View File
docs/extras/modules/agents/tools/how_to/tools_as_openai_functions.ipynb → docs/extras/modules/agents/tools/tools_as_openai_functions.ipynb
View File
docs/extras/modules/callbacks/how_to/async_callbacks.ipynb → docs/extras/modules/callbacks/async_callbacks.ipynb
View File
docs/extras/modules/callbacks/how_to/custom_callbacks.ipynb → docs/extras/modules/callbacks/custom_callbacks.ipynb
View File
docs/extras/modules/callbacks/how_to/custom_chain.mdx → docs/extras/modules/callbacks/custom_chain.mdx
View File
docs/extras/modules/callbacks/how_to/filecallbackhandler.ipynb → docs/extras/modules/callbacks/filecallbackhandler.ipynb
View File
docs/extras/modules/callbacks/integrations/argilla.ipynb
-423
View File
@@ -1,423 +0,0 @@
{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"# Argilla\n",
"\n",
"![Argilla - Open-source data platform for LLMs](https://argilla.io/og.png)\n",
"\n",
">[Argilla](https://argilla.io/) is an open-source data curation platform for LLMs.\n",
"> Using Argilla, everyone can build robust language models through faster data curation \n",
"> using both human and machine feedback. We provide support for each step in the MLOps cycle, \n",
"> from data labeling to model monitoring.\n",
"\n",
"<a target=\"_blank\" href=\"https://colab.research.google.com/github/hwchase17/langchain/blob/master/docs/modules/callbacks/integrations/argilla.html\">\n",
" <img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/>\n",
"</a>"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"In this guide we will demonstrate how to track the inputs and reponses of your LLM to generate a dataset in Argilla, using the `ArgillaCallbackHandler`.\n",
"\n",
"It's useful to keep track of the inputs and outputs of your LLMs to generate datasets for future fine-tuning. This is especially useful when you're using a LLM to generate data for a specific task, such as question answering, summarization, or translation."
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {
"tags": []
},
"source": [
"## Installation and Setup"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"!pip install argilla --upgrade\n",
"!pip install openai"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"### Getting API Credentials\n",
"\n",
"To get the Argilla API credentials, follow the next steps:\n",
"\n",
"1. Go to your Argilla UI.\n",
"2. Click on your profile picture and go to \"My settings\".\n",
"3. Then copy the API Key.\n",
"\n",
"In Argilla the API URL will be the same as the URL of your Argilla UI.\n",
"\n",
"To get the OpenAI API credentials, please visit https://platform.openai.com/account/api-keys"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"\n",
"os.environ[\"ARGILLA_API_URL\"] = \"...\"\n",
"os.environ[\"ARGILLA_API_KEY\"] = \"...\"\n",
"\n",
"os.environ[\"OPENAI_API_KEY\"] = \"...\""
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"### Setup Argilla\n",
"\n",
"To use the `ArgillaCallbackHandler` we will need to create a new `FeedbackDataset` in Argilla to keep track of your LLM experiments. To do so, please use the following code:"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [],
"source": [
"import argilla as rg"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [],
"source": [
"from packaging.version import parse as parse_version\n",
"\n",
"if parse_version(rg.__version__) < parse_version(\"1.8.0\"):\n",
" raise RuntimeError(\n",
" \"`FeedbackDataset` is only available in Argilla v1.8.0 or higher, please \"\n",
" \"upgrade `argilla` as `pip install argilla --upgrade`.\"\n",
" )"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"dataset = rg.FeedbackDataset(\n",
" fields=[\n",
" rg.TextField(name=\"prompt\"),\n",
" rg.TextField(name=\"response\"),\n",
" ],\n",
" questions=[\n",
" rg.RatingQuestion(\n",
" name=\"response-rating\",\n",
" description=\"How would you rate the quality of the response?\",\n",
" values=[1, 2, 3, 4, 5],\n",
" required=True,\n",
" ),\n",
" rg.TextQuestion(\n",
" name=\"response-feedback\",\n",
" description=\"What feedback do you have for the response?\",\n",
" required=False,\n",
" ),\n",
" ],\n",
" guidelines=\"You're asked to rate the quality of the response and provide feedback.\",\n",
")\n",
"\n",
"rg.init(\n",
" api_url=os.environ[\"ARGILLA_API_URL\"],\n",
" api_key=os.environ[\"ARGILLA_API_KEY\"],\n",
")\n",
"\n",
"dataset.push_to_argilla(\"langchain-dataset\")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"> 📌 NOTE: at the moment, just the prompt-response pairs are supported as `FeedbackDataset.fields`, so the `ArgillaCallbackHandler` will just track the prompt i.e. the LLM input, and the response i.e. the LLM output."
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Tracking"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"To use the `ArgillaCallbackHandler` you can either use the following code, or just reproduce one of the examples presented in the following sections."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from langchain.callbacks import ArgillaCallbackHandler\n",
"\n",
"argilla_callback = ArgillaCallbackHandler(\n",
" dataset_name=\"langchain-dataset\",\n",
" api_url=os.environ[\"ARGILLA_API_URL\"],\n",
" api_key=os.environ[\"ARGILLA_API_KEY\"],\n",
")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"### Scenario 1: Tracking an LLM\n",
"\n",
"First, let's just run a single LLM a few times and capture the resulting prompt-response pairs in Argilla."
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"LLMResult(generations=[[Generation(text='\\n\\nQ: What did the fish say when he hit the wall? \\nA: Dam.', generation_info={'finish_reason': 'stop', 'logprobs': None})], [Generation(text='\\n\\nThe Moon \\n\\nThe moon is high in the midnight sky,\\nSparkling like a star above.\\nThe night so peaceful, so serene,\\nFilling up the air with love.\\n\\nEver changing and renewing,\\nA never-ending light of grace.\\nThe moon remains a constant view,\\nA reminder of life’s gentle pace.\\n\\nThrough time and space it guides us on,\\nA never-fading beacon of hope.\\nThe moon shines down on us all,\\nAs it continues to rise and elope.', generation_info={'finish_reason': 'stop', 'logprobs': None})], [Generation(text='\\n\\nQ. What did one magnet say to the other magnet?\\nA. \"I find you very attractive!\"', generation_info={'finish_reason': 'stop', 'logprobs': None})], [Generation(text=\"\\n\\nThe world is charged with the grandeur of God.\\nIt will flame out, like shining from shook foil;\\nIt gathers to a greatness, like the ooze of oil\\nCrushed. Why do men then now not reck his rod?\\n\\nGenerations have trod, have trod, have trod;\\nAnd all is seared with trade; bleared, smeared with toil;\\nAnd wears man's smudge and shares man's smell: the soil\\nIs bare now, nor can foot feel, being shod.\\n\\nAnd for all this, nature is never spent;\\nThere lives the dearest freshness deep down things;\\nAnd though the last lights off the black West went\\nOh, morning, at the brown brink eastward, springs —\\n\\nBecause the Holy Ghost over the bent\\nWorld broods with warm breast and with ah! bright wings.\\n\\n~Gerard Manley Hopkins\", generation_info={'finish_reason': 'stop', 'logprobs': None})], [Generation(text='\\n\\nQ: What did one ocean say to the other ocean?\\nA: Nothing, they just waved.', generation_info={'finish_reason': 'stop', 'logprobs': None})], [Generation(text=\"\\n\\nA poem for you\\n\\nOn a field of green\\n\\nThe sky so blue\\n\\nA gentle breeze, the sun above\\n\\nA beautiful world, for us to love\\n\\nLife is a journey, full of surprise\\n\\nFull of joy and full of surprise\\n\\nBe brave and take small steps\\n\\nThe future will be revealed with depth\\n\\nIn the morning, when dawn arrives\\n\\nA fresh start, no reason to hide\\n\\nSomewhere down the road, there's a heart that beats\\n\\nBelieve in yourself, you'll always succeed.\", generation_info={'finish_reason': 'stop', 'logprobs': None})]], llm_output={'token_usage': {'completion_tokens': 504, 'total_tokens': 528, 'prompt_tokens': 24}, 'model_name': 'text-davinci-003'})"
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from langchain.callbacks import ArgillaCallbackHandler, StdOutCallbackHandler\n",
"from langchain.llms import OpenAI\n",
"\n",
"argilla_callback = ArgillaCallbackHandler(\n",
" dataset_name=\"langchain-dataset\",\n",
" api_url=os.environ[\"ARGILLA_API_URL\"],\n",
" api_key=os.environ[\"ARGILLA_API_KEY\"],\n",
")\n",
"callbacks = [StdOutCallbackHandler(), argilla_callback]\n",
"\n",
"llm = OpenAI(temperature=0.9, callbacks=callbacks)\n",
"llm.generate([\"Tell me a joke\", \"Tell me a poem\"] * 3)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"![Argilla UI with LangChain LLM input-response](https://docs.argilla.io/en/latest/_images/llm.png)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"### Scenario 2: Tracking an LLM in a chain\n",
"\n",
"Then we can create a chain using a prompt template, and then track the initial prompt and the final response in Argilla."
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new LLMChain chain...\u001b[0m\n",
"Prompt after formatting:\n",
"\u001b[32;1m\u001b[1;3mYou are a playwright. Given the title of play, it is your job to write a synopsis for that title.\n",
"Title: Documentary about Bigfoot in Paris\n",
"Playwright: This is a synopsis for the above play:\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"[{'text': \"\\n\\nDocumentary about Bigfoot in Paris focuses on the story of a documentary filmmaker and their search for evidence of the legendary Bigfoot creature in the city of Paris. The play follows the filmmaker as they explore the city, meeting people from all walks of life who have had encounters with the mysterious creature. Through their conversations, the filmmaker unravels the story of Bigfoot and finds out the truth about the creature's presence in Paris. As the story progresses, the filmmaker learns more and more about the mysterious creature, as well as the different perspectives of the people living in the city, and what they think of the creature. In the end, the filmmaker's findings lead them to some surprising and heartwarming conclusions about the creature's existence and the importance it holds in the lives of the people in Paris.\"}]"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from langchain.callbacks import ArgillaCallbackHandler, StdOutCallbackHandler\n",
"from langchain.llms import OpenAI\n",
"from langchain.chains import LLMChain\n",
"from langchain.prompts import PromptTemplate\n",
"\n",
"argilla_callback = ArgillaCallbackHandler(\n",
" dataset_name=\"langchain-dataset\",\n",
" api_url=os.environ[\"ARGILLA_API_URL\"],\n",
" api_key=os.environ[\"ARGILLA_API_KEY\"],\n",
")\n",
"callbacks = [StdOutCallbackHandler(), argilla_callback]\n",
"llm = OpenAI(temperature=0.9, callbacks=callbacks)\n",
"\n",
"template = \"\"\"You are a playwright. Given the title of play, it is your job to write a synopsis for that title.\n",
"Title: {title}\n",
"Playwright: This is a synopsis for the above play:\"\"\"\n",
"prompt_template = PromptTemplate(input_variables=[\"title\"], template=template)\n",
"synopsis_chain = LLMChain(llm=llm, prompt=prompt_template, callbacks=callbacks)\n",
"\n",
"test_prompts = [{\"title\": \"Documentary about Bigfoot in Paris\"}]\n",
"synopsis_chain.apply(test_prompts)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"![Argilla UI with LangChain Chain input-response](https://docs.argilla.io/en/latest/_images/chain.png)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"### Scenario 3: Using an Agent with Tools\n",
"\n",
"Finally, as a more advanced workflow, you can create an agent that uses some tools. So that `ArgillaCallbackHandler` will keep track of the input and the output, but not about the intermediate steps/thoughts, so that given a prompt we log the original prompt and the final response to that given prompt."
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"> Note that for this scenario we'll be using Google Search API (Serp API) so you will need to both install `google-search-results` as `pip install google-search-results`, and to set the Serp API Key as `os.environ[\"SERPAPI_API_KEY\"] = \"...\"` (you can find it at https://serpapi.com/dashboard), otherwise the example below won't work."
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"\n",
"\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n",
"\u001b[32;1m\u001b[1;3m I need to answer a historical question\n",
"Action: Search\n",
"Action Input: \"who was the first president of the United States of America\" \u001b[0m\n",
"Observation: \u001b[36;1m\u001b[1;3mGeorge Washington\u001b[0m\n",
"Thought:\u001b[32;1m\u001b[1;3m George Washington was the first president\n",
"Final Answer: George Washington was the first president of the United States of America.\u001b[0m\n",
"\n",
"\u001b[1m> Finished chain.\u001b[0m\n"
]
},
{
"data": {
"text/plain": [
"'George Washington was the first president of the United States of America.'"
]
},
"execution_count": 10,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from langchain.agents import AgentType, initialize_agent, load_tools\n",
"from langchain.callbacks import ArgillaCallbackHandler, StdOutCallbackHandler\n",
"from langchain.llms import OpenAI\n",
"\n",
"argilla_callback = ArgillaCallbackHandler(\n",
" dataset_name=\"langchain-dataset\",\n",
" api_url=os.environ[\"ARGILLA_API_URL\"],\n",
" api_key=os.environ[\"ARGILLA_API_KEY\"],\n",
")\n",
"callbacks = [StdOutCallbackHandler(), argilla_callback]\n",
"llm = OpenAI(temperature=0.9, callbacks=callbacks)\n",
"\n",
"tools = load_tools([\"serpapi\"], llm=llm, callbacks=callbacks)\n",
"agent = initialize_agent(\n",
" tools,\n",
" llm,\n",
" agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION,\n",
" callbacks=callbacks,\n",
")\n",
"agent.run(\"Who was the first president of the United States of America?\")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"![Argilla UI with LangChain Agent input-response](https://docs.argilla.io/en/latest/_images/agent.png)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.3"
},
"vscode": {
"interpreter": {
"hash": "a53ebf4a859167383b364e7e7521d0add3c2dbbdecce4edf676e8c4634ff3fbb"
}
}
},
"nbformat": 4,
"nbformat_minor": 4
}
docs/extras/modules/callbacks/integrations/context.ipynb
-220
View File
@@ -1,220 +0,0 @@
{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"# Context\n",
"\n",
"![Context - Product Analytics for AI Chatbots](https://go.getcontext.ai/langchain.png)\n",
"\n",
"[Context](https://getcontext.ai/) provides product analytics for AI chatbots.\n",
"\n",
"Context helps you understand how users are interacting with your AI chat products.\n",
"Gain critical insights, optimise poor experiences, and minimise brand risks.\n"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"In this guide we will show you how to integrate with Context."
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {
"tags": []
},
"source": [
"## Installation and Setup"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"vscode": {
"languageId": "shellscript"
}
},
"outputs": [],
"source": [
"$ pip install context-python --upgrade"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"### Getting API Credentials\n",
"\n",
"To get your Context API token:\n",
"\n",
"1. Go to the settings page within your Context account (https://go.getcontext.ai/settings).\n",
"2. Generate a new API Token.\n",
"3. Store this token somewhere secure."
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"### Setup Context\n",
"\n",
"To use the `ContextCallbackHandler`, import the handler from Langchain and instantiate it with your Context API token.\n",
"\n",
"Ensure you have installed the `context-python` package before using the handler."
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"\n",
"from langchain.callbacks import ContextCallbackHandler\n",
"\n",
"token = os.environ[\"CONTEXT_API_TOKEN\"]\n",
"\n",
"context_callback = ContextCallbackHandler(token)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"## Usage\n",
"### Using the Context callback within a Chat Model\n",
"\n",
"The Context callback handler can be used to directly record transcripts between users and AI assistants.\n",
"\n",
"#### Example"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"\n",
"from langchain.chat_models import ChatOpenAI\n",
"from langchain.schema import (\n",
" SystemMessage,\n",
" HumanMessage,\n",
")\n",
"from langchain.callbacks import ContextCallbackHandler\n",
"\n",
"token = os.environ[\"CONTEXT_API_TOKEN\"]\n",
"\n",
"chat = ChatOpenAI(\n",
" headers={\"user_id\": \"123\"}, temperature=0, callbacks=[ContextCallbackHandler(token)]\n",
")\n",
"\n",
"messages = [\n",
" SystemMessage(\n",
" content=\"You are a helpful assistant that translates English to French.\"\n",
" ),\n",
" HumanMessage(content=\"I love programming.\"),\n",
"]\n",
"\n",
"print(chat(messages))"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"### Using the Context callback within Chains\n",
"\n",
"The Context callback handler can also be used to record the inputs and outputs of chains. Note that intermediate steps of the chain are not recorded - only the starting inputs and final outputs.\n",
"\n",
"__Note:__ Ensure that you pass the same context object to the chat model and the chain.\n",
"\n",
"Wrong:\n",
"> ```python\n",
"> chat = ChatOpenAI(temperature=0.9, callbacks=[ContextCallbackHandler(token)])\n",
"> chain = LLMChain(llm=chat, prompt=chat_prompt_template, callbacks=[ContextCallbackHandler(token)])\n",
"> ```\n",
"\n",
"Correct:\n",
">```python\n",
">handler = ContextCallbackHandler(token)\n",
">chat = ChatOpenAI(temperature=0.9, callbacks=[callback])\n",
">chain = LLMChain(llm=chat, prompt=chat_prompt_template, callbacks=[callback])\n",
">```\n",
"\n",
"#### Example"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"\n",
"from langchain.chat_models import ChatOpenAI\n",
"from langchain import LLMChain\n",
"from langchain.prompts import PromptTemplate\n",
"from langchain.prompts.chat import (\n",
" ChatPromptTemplate,\n",
" HumanMessagePromptTemplate,\n",
")\n",
"from langchain.callbacks import ContextCallbackHandler\n",
"\n",
"token = os.environ[\"CONTEXT_API_TOKEN\"]\n",
"\n",
"human_message_prompt = HumanMessagePromptTemplate(\n",
" prompt=PromptTemplate(\n",
" template=\"What is a good name for a company that makes {product}?\",\n",
" input_variables=[\"product\"],\n",
" )\n",
")\n",
"chat_prompt_template = ChatPromptTemplate.from_messages([human_message_prompt])\n",
"callback = ContextCallbackHandler(token)\n",
"chat = ChatOpenAI(temperature=0.9, callbacks=[callback])\n",
"chain = LLMChain(llm=chat, prompt=chat_prompt_template, callbacks=[callback])\n",
"print(chain.run(\"colorful socks\"))"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.3"
},
"vscode": {
"interpreter": {
"hash": "a53ebf4a859167383b364e7e7521d0add3c2dbbdecce4edf676e8c4634ff3fbb"
}
}
},
"nbformat": 4,
"nbformat_minor": 4
}
docs/extras/modules/callbacks/integrations/infino.ipynb
-423
View File
File diff suppressed because one or more lines are too long
docs/extras/modules/callbacks/integrations/promptlayer.ipynb
-210
View File
@@ -1,210 +0,0 @@
{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"# PromptLayer\n",
"\n",
"![PromptLayer](https://promptlayer.com/text_logo.png)\n",
"\n",
"[PromptLayer](https://promptlayer.com) is a an LLM observability platform that lets you visualize requests, version prompts, and track usage. In this guide we will go over how to setup the `PromptLayerCallbackHandler`. \n",
"\n",
"While PromptLayer does have LLMs that integrate directly with LangChain (eg [`PromptLayerOpenAI`](https://python.langchain.com/docs/modules/model_io/models/llms/integrations/promptlayer_openai)), this callback is the recommended way to integrate PromptLayer with LangChain.\n",
"\n",
"See [our docs](https://docs.promptlayer.com/languages/langchain) for more information."
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {
"tags": []
},
"source": [
"## Installation and Setup"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"!pip install promptlayer --upgrade"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"### Getting API Credentials\n",
"\n",
"If you do not have a PromptLayer account, create one on [promptlayer.com](https://www.promptlayer.com). Then get an API key by clicking on the settings cog in the navbar and\n",
"set it as an environment variabled called `PROMPTLAYER_API_KEY`\n"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"### Usage\n",
"\n",
"Getting started with `PromptLayerCallbackHandler` is fairly simple, it takes two optional arguments:\n",
"1. `pl_tags` - an optional list of strings that will be tracked as tags on PromptLayer.\n",
"2. `pl_id_callback` - an optional function that will take `promptlayer_request_id` as an argument. This ID can be used with all of PromptLayer's tracking features to track, metadata, scores, and prompt usage."
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"### Simple OpenAI Example\n",
"\n",
"In this simple example we use `PromptLayerCallbackHandler` with `ChatOpenAI`. We add a PromptLayer tag named `chatopenai`"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import promptlayer # Don't forget this 🍰\n",
"from langchain.callbacks import PromptLayerCallbackHandler\n",
"from langchain.chat_models import ChatOpenAI\n",
"from langchain.schema import (\n",
" HumanMessage,\n",
")\n",
"\n",
"chat_llm = ChatOpenAI(\n",
" temperature=0,\n",
" callbacks=[PromptLayerCallbackHandler(pl_tags=[\"chatopenai\"])],\n",
")\n",
"llm_results = chat_llm(\n",
" [\n",
" HumanMessage(content=\"What comes after 1,2,3 ?\"),\n",
" HumanMessage(content=\"Tell me another joke?\"),\n",
" ]\n",
")\n",
"print(llm_results)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"### GPT4All Example"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import promptlayer # Don't forget this 🍰\n",
"from langchain.callbacks import PromptLayerCallbackHandler\n",
"\n",
"from langchain.llms import GPT4All\n",
"\n",
"model = GPT4All(model=\"./models/gpt4all-model.bin\", n_ctx=512, n_threads=8)\n",
"\n",
"response = model(\n",
" \"Once upon a time, \",\n",
" callbacks=[PromptLayerCallbackHandler(pl_tags=[\"langchain\", \"gpt4all\"])],\n",
")"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"### Full Featured Example\n",
"\n",
"In this example we unlock more of the power of PromptLayer.\n",
"\n",
"PromptLayer allows you to visually create, version, and track prompt templates. Using the [Prompt Registry](https://docs.promptlayer.com/features/prompt-registry), we can programatically fetch the prompt template called `example`.\n",
"\n",
"We also define a `pl_id_callback` function which takes in the `promptlayer_request_id` and logs a score, metadata and links the prompt template used. Read more about tracking on [our docs](https://docs.promptlayer.com/features/prompt-history/request-id)."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import promptlayer # Don't forget this 🍰\n",
"from langchain.callbacks import PromptLayerCallbackHandler\n",
"from langchain.llms import OpenAI\n",
"\n",
"\n",
"def pl_id_callback(promptlayer_request_id):\n",
" print(\"prompt layer id \", promptlayer_request_id)\n",
" promptlayer.track.score(\n",
" request_id=promptlayer_request_id, score=100\n",
" ) # score is an integer 0-100\n",
" promptlayer.track.metadata(\n",
" request_id=promptlayer_request_id, metadata={\"foo\": \"bar\"}\n",
" ) # metadata is a dictionary of key value pairs that is tracked on PromptLayer\n",
" promptlayer.track.prompt(\n",
" request_id=promptlayer_request_id,\n",
" prompt_name=\"example\",\n",
" prompt_input_variables={\"product\": \"toasters\"},\n",
" version=1,\n",
" ) # link the request to a prompt template\n",
"\n",
"\n",
"openai_llm = OpenAI(\n",
" model_name=\"text-davinci-002\",\n",
" callbacks=[PromptLayerCallbackHandler(pl_id_callback=pl_id_callback)],\n",
")\n",
"\n",
"example_prompt = promptlayer.prompts.get(\"example\", version=1, langchain=True)\n",
"openai_llm(example_prompt.format(product=\"toasters\"))"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"That is all it takes! After setup all your requests will show up on the PromptLayer dashboard.\n",
"This callback also works with any LLM implemented on LangChain."
]
}
],
"metadata": {
"kernelspec": {
"display_name": "base",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.8 (default, Apr 13 2021, 12:59:45) \n[Clang 10.0.0 ]"
},
"vscode": {
"interpreter": {
"hash": "c4fe2cd85a8d9e8baaec5340ce66faff1c77581a9f43e6c45e85e09b6fced008"
}
}
},
"nbformat": 4,
"nbformat_minor": 4
}
docs/extras/modules/callbacks/integrations/streamlit.md
-73
View File
@@ -1,73 +0,0 @@
# Streamlit
> **[Streamlit](https://streamlit.io/) is a faster way to build and share data apps.**
> Streamlit turns data scripts into shareable web apps in minutes. All in pure Python. No front‑end experience required.
> See more examples at [streamlit.io/generative-ai](https://streamlit.io/generative-ai).
[![Open in GitHub Codespaces](https://github.com/codespaces/badge.svg)](https://codespaces.new/langchain-ai/streamlit-agent?quickstart=1)
In this guide we will demonstrate how to use `StreamlitCallbackHandler` to display the thoughts and actions of an agent in an
interactive Streamlit app. Try it out with the running app below using the [MRKL agent](/docs/modules/agents/how_to/mrkl/):
<iframe loading="lazy" src="https://langchain-mrkl.streamlit.app/?embed=true&embed_options=light_theme"
style={{ width: 100 + '%', border: 'none', marginBottom: 1 + 'rem', height: 600 }}
allow="camera;clipboard-read;clipboard-write;"
></iframe>
## Installation and Setup
```bash
pip install langchain streamlit
```
You can run `streamlit hello` to load a sample app and validate your install succeeded. See full instructions in Streamlit's
[Getting started documentation](https://docs.streamlit.io/library/get-started).
## Display thoughts and actions
To create a `StreamlitCallbackHandler`, you just need to provide a parent container to render the output.
```python
from langchain.callbacks import StreamlitCallbackHandler
import streamlit as st
st_callback = StreamlitCallbackHandler(st.container())
```
Additional keyword arguments to customize the display behavior are described in the
[API reference](https://api.python.langchain.com/en/latest/callbacks/langchain.callbacks.streamlit.streamlit_callback_handler.StreamlitCallbackHandler.html).
### Scenario 1: Using an Agent with Tools
The primary supported use case today is visualizing the actions of an Agent with Tools (or Agent Executor). You can create an
agent in your Streamlit app and simply pass the `StreamlitCallbackHandler` to `agent.run()` in order to visualize the
thoughts and actions live in your app.
```python
from langchain.llms import OpenAI
from langchain.agents import AgentType, initialize_agent, load_tools
from langchain.callbacks import StreamlitCallbackHandler
import streamlit as st
llm = OpenAI(temperature=0, streaming=True)
tools = load_tools(["ddg-search"])
agent = initialize_agent(
tools, llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=True
)
if prompt := st.chat_input():
st.chat_message("user").write(prompt)
with st.chat_message("assistant"):
st_callback = StreamlitCallbackHandler(st.container())
response = agent.run(prompt, callbacks=[st_callback])
st.write(response)
```
**Note:** You will need to set `OPENAI_API_KEY` for the above app code to run successfully.
The easiest way to do this is via [Streamlit secrets.toml](https://docs.streamlit.io/library/advanced-features/secrets-management),
or any other local ENV management tool.
### Additional scenarios
Currently `StreamlitCallbackHandler` is geared towards use with a LangChain Agent Executor. Support for additional agent types,
use directly with Chains, etc will be added in the future.
docs/extras/modules/callbacks/how_to/multiple_callbacks.ipynb → docs/extras/modules/callbacks/multiple_callbacks.ipynb
View File
docs/extras/modules/callbacks/how_to/tags.mdx → docs/extras/modules/callbacks/tags.mdx
View File
docs/extras/modules/callbacks/how_to/token_counting.ipynb → docs/extras/modules/callbacks/token_counting.ipynb
View File
docs/extras/modules/data_connection/document_loaders/integrations/acreom.ipynb
-75
View File
@@ -1,75 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "e310c8dc-acd0-48d2-801c-f37ce99acd2d",
"metadata": {},
"source": [
"# acreom"
]
},
{
"cell_type": "markdown",
"id": "04a2c95d-4114-431e-904a-32d79005c28b",
"metadata": {},
"source": [
"[acreom](https://acreom.com) is a dev-first knowledge base with tasks running on local markdown files.\n",
"\n",
"Below is an example on how to load a local acreom vault into Langchain. As the local vault in acreom is a folder of plain text .md files, the loader requires the path to the directory. \n",
"\n",
"Vault files may contain some metadata which is stored as a YAML header. These values will be added to the document’s metadata if `collect_metadata` is set to true. "
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "0169bee5-aa7a-4ec7-b7e7-b3bb2e58f3bb",
"metadata": {},
"outputs": [],
"source": [
"from langchain.document_loaders import AcreomLoader"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "c1b49ab3-616b-4149-bef5-7559d65d3d2b",
"metadata": {},
"outputs": [],
"source": [
"loader = AcreomLoader(\"<path-to-acreom-vault>\", collect_metadata=False)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "3127a018-9c1c-4886-8321-f5666d970a95",
"metadata": {},
"outputs": [],
"source": [
"docs = loader.load()"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.10"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
docs/extras/modules/data_connection/document_loaders/integrations/airbyte_json.ipynb
-186
View File
@@ -1,186 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "1f3a5ebf",
"metadata": {},
"source": [
"# Airbyte JSON"
]
},
{
"cell_type": "markdown",
"id": "35ac77b1-449b-44f7-b8f3-3494d55c286e",
"metadata": {},
"source": [
">[Airbyte](https://github.com/airbytehq/airbyte) is a data integration platform for ELT pipelines from APIs, databases & files to warehouses & lakes. It has the largest catalog of ELT connectors to data warehouses and databases."
]
},
{
"cell_type": "markdown",
"id": "1fe72234-3110-4c07-a766-3dc505dd25cc",
"metadata": {},
"source": [
"This covers how to load any source from Airbyte into a local JSON file that can be read in as a document\n",
"\n",
"Prereqs:\n",
"Have docker desktop installed\n",
"\n",
"Steps:\n",
"\n",
"1) Clone Airbyte from GitHub - `git clone https://github.com/airbytehq/airbyte.git`\n",
"\n",
"2) Switch into Airbyte directory - `cd airbyte`\n",
"\n",
"3) Start Airbyte - `docker compose up`\n",
"\n",
"4) In your browser, just visit http://localhost:8000. You will be asked for a username and password. By default, that's username `airbyte` and password `password`.\n",
"\n",
"5) Setup any source you wish.\n",
"\n",
"6) Set destination as Local JSON, with specified destination path - lets say `/json_data`. Set up manual sync.\n",
"\n",
"7) Run the connection.\n",
"\n",
"7) To see what files are create, you can navigate to: `file:///tmp/airbyte_local`\n",
"\n",
"8) Find your data and copy path. That path should be saved in the file variable below. It should start with `/tmp/airbyte_local`\n"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "180c8b74",
"metadata": {},
"outputs": [],
"source": [
"from langchain.document_loaders import AirbyteJSONLoader"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "4af10665",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"_airbyte_raw_pokemon.jsonl\n"
]
}
],
"source": [
"!ls /tmp/airbyte_local/json_data/"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "721d9316",
"metadata": {},
"outputs": [],
"source": [
"loader = AirbyteJSONLoader(\"/tmp/airbyte_local/json_data/_airbyte_raw_pokemon.jsonl\")"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "9858b946",
"metadata": {},
"outputs": [],
"source": [
"data = loader.load()"
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "fca024cb",
"metadata": {
"scrolled": true
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"abilities: \n",
"ability: \n",
"name: blaze\n",
"url: https://pokeapi.co/api/v2/ability/66/\n",
"\n",
"is_hidden: False\n",
"slot: 1\n",
"\n",
"\n",
"ability: \n",
"name: solar-power\n",
"url: https://pokeapi.co/api/v2/ability/94/\n",
"\n",
"is_hidden: True\n",
"slot: 3\n",
"\n",
"base_experience: 267\n",
"forms: \n",
"name: charizard\n",
"url: https://pokeapi.co/api/v2/pokemon-form/6/\n",
"\n",
"game_indices: \n",
"game_index: 180\n",
"version: \n",
"name: red\n",
"url: https://pokeapi.co/api/v2/version/1/\n",
"\n",
"\n",
"\n",
"game_index: 180\n",
"version: \n",
"name: blue\n",
"url: https://pokeapi.co/api/v2/version/2/\n",
"\n",
"\n",
"\n",
"game_index: 180\n",
"version: \n",
"n\n"
]
}
],
"source": [
"print(data[0].page_content[:500])"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9fa002a5",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.6"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
docs/extras/modules/data_connection/document_loaders/integrations/airtable.ipynb
-142
View File
@@ -1,142 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "7ae421e6",
"metadata": {},
"source": [
"# Airtable"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "98aea00d",
"metadata": {},
"outputs": [],
"source": [
"! pip install pyairtable"
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "592483eb",
"metadata": {},
"outputs": [],
"source": [
"from langchain.document_loaders import AirtableLoader"
]
},
{
"cell_type": "markdown",
"id": "637e1205",
"metadata": {},
"source": [
"* Get your API key [here](https://support.airtable.com/docs/creating-and-using-api-keys-and-access-tokens).\n",
"* Get ID of your base [here](https://airtable.com/developers/web/api/introduction).\n",
"* Get your table ID from the table url as shown [here](https://www.highviewapps.com/kb/where-can-i-find-the-airtable-base-id-and-table-id/#:~:text=Both%20the%20Airtable%20Base%20ID,URL%20that%20begins%20with%20tbl)."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "c12a7aff",
"metadata": {},
"outputs": [],
"source": [
"api_key = \"xxx\"\n",
"base_id = \"xxx\"\n",
"table_id = \"xxx\""
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "ccddd5a6",
"metadata": {},
"outputs": [],
"source": [
"loader = AirtableLoader(api_key, table_id, base_id)\n",
"docs = loader.load()"
]
},
{
"cell_type": "markdown",
"id": "ae76c25c",
"metadata": {},
"source": [
"Returns each table row as `dict`."
]
},
{
"cell_type": "code",
"execution_count": 10,
"id": "7abec7ce",
"metadata": {
"scrolled": true
},
"outputs": [
{
"data": {
"text/plain": [
"3"
]
},
"execution_count": 10,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"len(docs)"
]
},
{
"cell_type": "code",
"execution_count": 11,
"id": "403c95da",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"{'id': 'recF3GbGZCuh9sXIQ',\n",
" 'createdTime': '2023-06-09T04:47:21.000Z',\n",
" 'fields': {'Priority': 'High',\n",
" 'Status': 'In progress',\n",
" 'Name': 'Document Splitters'}}"
]
},
"execution_count": 11,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"eval(docs[0].page_content)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.6"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
docs/extras/modules/data_connection/document_loaders/integrations/alibaba_cloud_maxcompute.ipynb
-255
View File
@@ -1,255 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "f08772b0",
"metadata": {},
"source": [
"# Alibaba Cloud MaxCompute\n",
"\n",
">[Alibaba Cloud MaxCompute](https://www.alibabacloud.com/product/maxcompute) (previously known as ODPS) is a general purpose, fully managed, multi-tenancy data processing platform for large-scale data warehousing. MaxCompute supports various data importing solutions and distributed computing models, enabling users to effectively query massive datasets, reduce production costs, and ensure data security.\n",
"\n",
"The `MaxComputeLoader` lets you execute a MaxCompute SQL query and loads the results as one document per row."
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "067b7213",
"metadata": {
"tags": []
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Collecting pyodps\n",
" Downloading pyodps-0.11.4.post0-cp39-cp39-macosx_10_9_universal2.whl (2.0 MB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2.0/2.0 MB\u001b[0m \u001b[31m1.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m00:01\u001b[0m00:01\u001b[0m0m\n",
"\u001b[?25hRequirement already satisfied: charset-normalizer>=2 in /Users/newboy/anaconda3/envs/langchain/lib/python3.9/site-packages (from pyodps) (3.1.0)\n",
"Requirement already satisfied: urllib3<2.0,>=1.26.0 in /Users/newboy/anaconda3/envs/langchain/lib/python3.9/site-packages (from pyodps) (1.26.15)\n",
"Requirement already satisfied: idna>=2.5 in /Users/newboy/anaconda3/envs/langchain/lib/python3.9/site-packages (from pyodps) (3.4)\n",
"Requirement already satisfied: certifi>=2017.4.17 in /Users/newboy/anaconda3/envs/langchain/lib/python3.9/site-packages (from pyodps) (2023.5.7)\n",
"Installing collected packages: pyodps\n",
"Successfully installed pyodps-0.11.4.post0\n"
]
}
],
"source": [
"!pip install pyodps"
]
},
{
"cell_type": "markdown",
"id": "19641457",
"metadata": {},
"source": [
"## Basic Usage\n",
"To instantiate the loader you'll need a SQL query to execute, your MaxCompute endpoint and project name, and you access ID and secret access key. The access ID and secret access key can either be passed in direct via the `access_id` and `secret_access_key` parameters or they can be set as environment variables `MAX_COMPUTE_ACCESS_ID` and `MAX_COMPUTE_SECRET_ACCESS_KEY`."
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "71a0da4b",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"from langchain.document_loaders import MaxComputeLoader"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "d4770c4a",
"metadata": {},
"outputs": [],
"source": [
"base_query = \"\"\"\n",
"SELECT *\n",
"FROM (\n",
" SELECT 1 AS id, 'content1' AS content, 'meta_info1' AS meta_info\n",
" UNION ALL\n",
" SELECT 2 AS id, 'content2' AS content, 'meta_info2' AS meta_info\n",
" UNION ALL\n",
" SELECT 3 AS id, 'content3' AS content, 'meta_info3' AS meta_info\n",
") mydata;\n",
"\"\"\""
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "1616c174",
"metadata": {},
"outputs": [],
"source": [
"endpoint = \"<ENDPOINT>\"\n",
"project = \"<PROJECT>\"\n",
"ACCESS_ID = \"<ACCESS ID>\"\n",
"SECRET_ACCESS_KEY = \"<SECRET ACCESS KEY>\""
]
},
{
"cell_type": "code",
"execution_count": 13,
"id": "e5c25041",
"metadata": {},
"outputs": [],
"source": [
"loader = MaxComputeLoader.from_params(\n",
" base_query,\n",
" endpoint,\n",
" project,\n",
" access_id=ACCESS_ID,\n",
" secret_access_key=SECRET_ACCESS_KEY,\n",
")\n",
"data = loader.load()"
]
},
{
"cell_type": "code",
"execution_count": 17,
"id": "311e74ea",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[Document(page_content='id: 1\\ncontent: content1\\nmeta_info: meta_info1', metadata={}), Document(page_content='id: 2\\ncontent: content2\\nmeta_info: meta_info2', metadata={}), Document(page_content='id: 3\\ncontent: content3\\nmeta_info: meta_info3', metadata={})]\n"
]
}
],
"source": [
"print(data)"
]
},
{
"cell_type": "code",
"execution_count": 20,
"id": "a4d8c388",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"id: 1\n",
"content: content1\n",
"meta_info: meta_info1\n"
]
}
],
"source": [
"print(data[0].page_content)"
]
},
{
"cell_type": "code",
"execution_count": 21,
"id": "f2422e6c",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"{}\n"
]
}
],
"source": [
"print(data[0].metadata)"
]
},
{
"cell_type": "markdown",
"id": "85e07e28",
"metadata": {},
"source": [
"## Specifying Which Columns are Content vs Metadata\n",
"You can configure which subset of columns should be loaded as the contents of the Document and which as the metadata using the `page_content_columns` and `metadata_columns` parameters."
]
},
{
"cell_type": "code",
"execution_count": 22,
"id": "a7b9d726",
"metadata": {},
"outputs": [],
"source": [
"loader = MaxComputeLoader.from_params(\n",
" base_query,\n",
" endpoint,\n",
" project,\n",
" page_content_columns=[\"content\"], # Specify Document page content\n",
" metadata_columns=[\"id\", \"meta_info\"], # Specify Document metadata\n",
" access_id=ACCESS_ID,\n",
" secret_access_key=SECRET_ACCESS_KEY,\n",
")\n",
"data = loader.load()"
]
},
{
"cell_type": "code",
"execution_count": 25,
"id": "532c19e9",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"content: content1\n"
]
}
],
"source": [
"print(data[0].page_content)"
]
},
{
"cell_type": "code",
"execution_count": 26,
"id": "5fe4990a",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"{'id': 1, 'meta_info': 'meta_info1'}\n"
]
}
],
"source": [
"print(data[0].metadata)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.3"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
docs/extras/modules/data_connection/document_loaders/integrations/apify_dataset.ipynb
-183
View File
@@ -1,183 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Apify Dataset\n",
"\n",
">[Apify Dataset](https://docs.apify.com/platform/storage/dataset) is a scaleable append-only storage with sequential access built for storing structured web scraping results, such as a list of products or Google SERPs, and then export them to various formats like JSON, CSV, or Excel. Datasets are mainly used to save results of [Apify Actors](https://apify.com/store)—serverless cloud programs for varius web scraping, crawling, and data extraction use cases.\n",
"\n",
"This notebook shows how to load Apify datasets to LangChain.\n",
"\n",
"\n",
"## Prerequisites\n",
"\n",
"You need to have an existing dataset on the Apify platform. If you don't have one, please first check out [this notebook](/docs/modules/agents/tools/integrations/apify.html) on how to use Apify to extract content from documentation, knowledge bases, help centers, or blogs."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"#!pip install apify-client"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"First, import `ApifyDatasetLoader` into your source code:"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"from langchain.document_loaders import ApifyDatasetLoader\n",
"from langchain.document_loaders.base import Document"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Then provide a function that maps Apify dataset record fields to LangChain `Document` format.\n",
"\n",
"For example, if your dataset items are structured like this:\n",
"\n",
"```json\n",
"{\n",
" \"url\": \"https://apify.com\",\n",
" \"text\": \"Apify is the best web scraping and automation platform.\"\n",
"}\n",
"```\n",
"\n",
"The mapping function in the code below will convert them to LangChain `Document` format, so that you can use them further with any LLM model (e.g. for question answering)."
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [],
"source": [
"loader = ApifyDatasetLoader(\n",
" dataset_id=\"your-dataset-id\",\n",
" dataset_mapping_function=lambda dataset_item: Document(\n",
" page_content=dataset_item[\"text\"], metadata={\"source\": dataset_item[\"url\"]}\n",
" ),\n",
")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"data = loader.load()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## An example with question answering\n",
"\n",
"In this example, we use data from a dataset to answer a question."
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [],
"source": [
"from langchain.docstore.document import Document\n",
"from langchain.document_loaders import ApifyDatasetLoader\n",
"from langchain.indexes import VectorstoreIndexCreator"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [],
"source": [
"loader = ApifyDatasetLoader(\n",
" dataset_id=\"your-dataset-id\",\n",
" dataset_mapping_function=lambda item: Document(\n",
" page_content=item[\"text\"] or \"\", metadata={\"source\": item[\"url\"]}\n",
" ),\n",
")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"index = VectorstoreIndexCreator().from_loaders([loader])"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [],
"source": [
"query = \"What is Apify?\"\n",
"result = index.query_with_sources(query)"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
" Apify is a platform for developing, running, and sharing serverless cloud programs. It enables users to create web scraping and automation tools and publish them on the Apify platform.\n",
"\n",
"https://docs.apify.com/platform/actors, https://docs.apify.com/platform/actors/running/actors-in-store, https://docs.apify.com/platform/security, https://docs.apify.com/platform/actors/examples\n"
]
}
],
"source": [
"print(result[\"answer\"])\n",
"print(result[\"sources\"])"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.3"
}
},
"nbformat": 4,
"nbformat_minor": 4
}
docs/extras/modules/data_connection/document_loaders/integrations/arxiv.ipynb
-176
View File
@@ -1,176 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "bda1f3f5",
"metadata": {},
"source": [
"# Arxiv\n",
"\n",
">[arXiv](https://arxiv.org/) is an open-access archive for 2 million scholarly articles in the fields of physics, mathematics, computer science, quantitative biology, quantitative finance, statistics, electrical engineering and systems science, and economics.\n",
"\n",
"This notebook shows how to load scientific articles from `Arxiv.org` into a document format that we can use downstream."
]
},
{
"cell_type": "markdown",
"id": "1b7a1eef-7bf7-4e7d-8bfc-c4e27c9488cb",
"metadata": {},
"source": [
"## Installation"
]
},
{
"cell_type": "markdown",
"id": "2abd5578-aa3d-46b9-99af-8b262f0b3df8",
"metadata": {},
"source": [
"First, you need to install `arxiv` python package."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b674aaea-ed3a-4541-8414-260a8f67f623",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"#!pip install arxiv"
]
},
{
"cell_type": "markdown",
"id": "094b5f13-7e54-4354-9d83-26d6926ecaa0",
"metadata": {
"tags": []
},
"source": [
"Second, you need to install `PyMuPDF` python package which transforms PDF files downloaded from the `arxiv.org` site into the text format."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7cd91121-2e96-43ba-af50-319853695f86",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"#!pip install pymupdf"
]
},
{
"cell_type": "markdown",
"id": "95f05e1c-195e-4e2b-ae8e-8d6637f15be6",
"metadata": {},
"source": [
"## Examples"
]
},
{
"cell_type": "markdown",
"id": "e29b954c-1407-4797-ae21-6ba8937156be",
"metadata": {},
"source": [
"`ArxivLoader` has these arguments:\n",
"- `query`: free text which used to find documents in the Arxiv\n",
"- optional `load_max_docs`: default=100. Use it to limit number of downloaded documents. It takes time to download all 100 documents, so use a small number for experiments.\n",
"- optional `load_all_available_meta`: default=False. By default only the most important fields downloaded: `Published` (date when document was published/last updated), `Title`, `Authors`, `Summary`. If True, other fields also downloaded."
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "9bfd5e46",
"metadata": {},
"outputs": [],
"source": [
"from langchain.document_loaders import ArxivLoader"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "700e4ef2",
"metadata": {},
"outputs": [],
"source": [
"docs = ArxivLoader(query=\"1605.08386\", load_max_docs=2).load()\n",
"len(docs)"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "8977bac0-0042-4f23-9754-247dbd32439b",
"metadata": {
"tags": []
},
"outputs": [
{
"data": {
"text/plain": [
"{'Published': '2016-05-26',\n",
" 'Title': 'Heat-bath random walks with Markov bases',\n",
" 'Authors': 'Caprice Stanley, Tobias Windisch',\n",
" 'Summary': 'Graphs on lattice points are studied whose edges come from a finite set of\\nallowed moves of arbitrary length. We show that the diameter of these graphs on\\nfibers of a fixed integer matrix can be bounded from above by a constant. We\\nthen study the mixing behaviour of heat-bath random walks on these graphs. We\\nalso state explicit conditions on the set of moves so that the heat-bath random\\nwalk, a generalization of the Glauber dynamics, is an expander in fixed\\ndimension.'}"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"docs[0].metadata # meta-information of the Document"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "46969806-45a9-4c4d-a61b-cfb9658fc9de",
"metadata": {
"tags": []
},
"outputs": [
{
"data": {
"text/plain": [
"'arXiv:1605.08386v1 [math.CO] 26 May 2016\\nHEAT-BATH RANDOM WALKS WITH MARKOV BASES\\nCAPRICE STANLEY AND TOBIAS WINDISCH\\nAbstract. Graphs on lattice points are studied whose edges come from a finite set of\\nallowed moves of arbitrary length. We show that the diameter of these graphs on fibers of a\\nfixed integer matrix can be bounded from above by a constant. We then study the mixing\\nbehaviour of heat-b'"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"docs[0].page_content[:400] # all pages of the Document content"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.6"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
docs/extras/modules/data_connection/document_loaders/integrations/async_html.ipynb
-107
View File
@@ -1,107 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "e229e34c",
"metadata": {},
"source": [
"# AsyncHtmlLoader\n",
"\n",
"AsyncHtmlLoader loads raw HTML from a list of urls concurrently."
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "4c8e4dab",
"metadata": {},
"outputs": [],
"source": [
"from langchain.document_loaders import AsyncHtmlLoader"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "e76b5ddc",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Fetching pages: 100%|############| 2/2 [00:00<00:00, 9.96it/s]\n"
]
}
],
"source": [
"urls = [\"https://www.espn.com\", \"https://lilianweng.github.io/posts/2023-06-23-agent/\"]\n",
"loader = AsyncHtmlLoader(urls)\n",
"docs = loader.load()"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "5dca1c0c",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"' news. Stream exclusive games on ESPN+ and play fantasy sports.\" />\\n<meta property=\"og:image\" content=\"https://a1.espncdn.com/combiner/i?img=%2Fi%2Fespn%2Fespn_logos%2Fespn_red.png\"/>\\n<meta property=\"og:image:width\" content=\"1200\" />\\n<meta property=\"og:image:height\" content=\"630\" />\\n<meta property=\"og:type\" content=\"website\" />\\n<meta name=\"twitter:site\" content=\"espn\" />\\n<meta name=\"twitter:url\" content=\"https://www.espn.com\" />\\n<meta name=\"twitter:title\" content=\"ESPN - Serving Sports Fans. Anytime. Anywhere.\"/>\\n<meta name=\"twitter:description\" content=\"Visit ESPN for live scores, highlights and sports news. Stream exclusive games on ESPN+ and play fantasy sports.\" />\\n<meta name=\"twitter:card\" content=\"summary\">\\n<meta name=\"twitter:app:name:iphone\" content=\"ESPN\"/>\\n<meta name=\"twitter:app:id:iphone\" content=\"317469184\"/>\\n<meta name=\"twitter:app:name:googleplay\" content=\"ESPN\"/>\\n<meta name=\"twitter:app:id:googleplay\" content=\"com.espn.score_center\"/>\\n<meta name=\"title\" content=\"ESPN - '"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"docs[0].page_content[1000:2000]"
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "4d024f0f",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'al\" href=\"https://lilianweng.github.io/posts/2023-06-23-agent/\" />\\n<link crossorigin=\"anonymous\" href=\"/assets/css/stylesheet.min.67a6fb6e33089cb29e856bcc95d7aa39f70049a42b123105531265a0d9f1258b.css\" integrity=\"sha256-Z6b7bjMInLKehWvMldeqOfcASaQrEjEFUxJloNnxJYs=\" rel=\"preload stylesheet\" as=\"style\">\\n<script defer crossorigin=\"anonymous\" src=\"/assets/js/highlight.min.7680afc38aa6b15ddf158a4f3780b7b1f7dde7e91d26f073e6229bb7a0793c92.js\" integrity=\"sha256-doCvw4qmsV3fFYpPN4C3sffd5&#43;kdJvBz5iKbt6B5PJI=\"\\n onload=\"hljs.initHighlightingOnLoad();\"></script>\\n<link rel=\"icon\" href=\"https://lilianweng.github.io/favicon_peach.ico\">\\n<link rel=\"icon\" type=\"image/png\" sizes=\"16x16\" href=\"https://lilianweng.github.io/favicon-16x16.png\">\\n<link rel=\"icon\" type=\"image/png\" sizes=\"32x32\" href=\"https://lilianweng.github.io/favicon-32x32.png\">\\n<link rel=\"apple-touch-icon\" href=\"https://lilianweng.github.io/apple-touch-icon.png\">\\n<link rel=\"mask-icon\" href=\"https://lilianweng.github.io/safari-pinned-tab.'"
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"docs[1].page_content[1000:2000]"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.16"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
docs/extras/modules/data_connection/document_loaders/integrations/aws_s3_directory.ipynb
-135
View File
@@ -1,135 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "a634365e",
"metadata": {},
"source": [
"# AWS S3 Directory\n",
"\n",
">[Amazon Simple Storage Service (Amazon S3)](https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-folders.html) is an object storage service\n",
"\n",
">[AWS S3 Directory](https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-folders.html)\n",
"\n",
"This covers how to load document objects from an `AWS S3 Directory` object."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "49815096",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"#!pip install boto3"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "2f0cd6a5",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"from langchain.document_loaders import S3DirectoryLoader"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "321cc7f1",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"loader = S3DirectoryLoader(\"testing-hwc\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "2b11d155",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"loader.load()"
]
},
{
"cell_type": "markdown",
"id": "0690c40a",
"metadata": {},
"source": [
"## Specifying a prefix\n",
"You can also specify a prefix for more finegrained control over what files to load."
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "72d44781",
"metadata": {},
"outputs": [],
"source": [
"loader = S3DirectoryLoader(\"testing-hwc\", prefix=\"fake\")"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "2d3c32db",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[Document(page_content='Lorem ipsum dolor sit amet.', lookup_str='', metadata={'source': '/var/folders/y6/8_bzdg295ld6s1_97_12m4lr0000gn/T/tmpujbkzf_l/fake.docx'}, lookup_index=0)]"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"loader.load()"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "885dc280",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.6"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
docs/extras/modules/data_connection/document_loaders/integrations/aws_s3_file.ipynb
-98
View File
@@ -1,98 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "66a7777e",
"metadata": {},
"source": [
"# AWS S3 File\n",
"\n",
">[Amazon Simple Storage Service (Amazon S3)](https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-folders.html) is an object storage service.\n",
"\n",
">[AWS S3 Buckets](https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingBucket.html)\n",
"\n",
"This covers how to load document objects from an `AWS S3 File` object."
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "9ec8a3b3",
"metadata": {},
"outputs": [],
"source": [
"from langchain.document_loaders import S3FileLoader"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "43128d8d",
"metadata": {},
"outputs": [],
"source": [
"#!pip install boto3"
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "35d6809a",
"metadata": {},
"outputs": [],
"source": [
"loader = S3FileLoader(\"testing-hwc\", \"fake.docx\")"
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "efd6be84",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[Document(page_content='Lorem ipsum dolor sit amet.', lookup_str='', metadata={'source': '/var/folders/y6/8_bzdg295ld6s1_97_12m4lr0000gn/T/tmpxvave6wl/fake.docx'}, lookup_index=0)]"
]
},
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"loader.load()"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "93689594",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.6"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
docs/extras/modules/data_connection/document_loaders/integrations/azlyrics.ipynb
-96
View File
@@ -1,96 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "9c31caff",
"metadata": {},
"source": [
"# AZLyrics\n",
"\n",
">[AZLyrics](https://www.azlyrics.com/) is a large, legal, every day growing collection of lyrics.\n",
"\n",
"This covers how to load AZLyrics webpages into a document format that we can use downstream."
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "7e6f5726",
"metadata": {},
"outputs": [],
"source": [
"from langchain.document_loaders import AZLyricsLoader"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "a0df4c24",
"metadata": {},
"outputs": [],
"source": [
"loader = AZLyricsLoader(\"https://www.azlyrics.com/lyrics/mileycyrus/flowers.html\")"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "8cd61b6e",
"metadata": {},
"outputs": [],
"source": [
"data = loader.load()"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "162fd286",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[Document(page_content=\"Miley Cyrus - Flowers Lyrics | AZLyrics.com\\n\\r\\nWe were good, we were gold\\nKinda dream that can't be sold\\nWe were right till we weren't\\nBuilt a home and watched it burn\\n\\nI didn't wanna leave you\\nI didn't wanna lie\\nStarted to cry but then remembered I\\n\\nI can buy myself flowers\\nWrite my name in the sand\\nTalk to myself for hours\\nSay things you don't understand\\nI can take myself dancing\\nAnd I can hold my own hand\\nYeah, I can love me better than you can\\n\\nCan love me better\\nI can love me better, baby\\nCan love me better\\nI can love me better, baby\\n\\nPaint my nails, cherry red\\nMatch the roses that you left\\nNo remorse, no regret\\nI forgive every word you said\\n\\nI didn't wanna leave you, baby\\nI didn't wanna fight\\nStarted to cry but then remembered I\\n\\nI can buy myself flowers\\nWrite my name in the sand\\nTalk to myself for hours, yeah\\nSay things you don't understand\\nI can take myself dancing\\nAnd I can hold my own hand\\nYeah, I can love me better than you can\\n\\nCan love me better\\nI can love me better, baby\\nCan love me better\\nI can love me better, baby\\nCan love me better\\nI can love me better, baby\\nCan love me better\\nI\\n\\nI didn't wanna wanna leave you\\nI didn't wanna fight\\nStarted to cry but then remembered I\\n\\nI can buy myself flowers\\nWrite my name in the sand\\nTalk to myself for hours (Yeah)\\nSay things you don't understand\\nI can take myself dancing\\nAnd I can hold my own hand\\nYeah, I can love me better than\\nYeah, I can love me better than you can, uh\\n\\nCan love me better\\nI can love me better, baby\\nCan love me better\\nI can love me better, baby (Than you can)\\nCan love me better\\nI can love me better, baby\\nCan love me better\\nI\\n\", lookup_str='', metadata={'source': 'https://www.azlyrics.com/lyrics/mileycyrus/flowers.html'}, lookup_index=0)]"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"data"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "6358000c",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.6"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
docs/extras/modules/data_connection/document_loaders/integrations/azure_blob_storage_container.ipynb
-148
View File
@@ -1,148 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "a634365e",
"metadata": {},
"source": [
"# Azure Blob Storage Container\n",
"\n",
">[Azure Blob Storage](https://learn.microsoft.com/en-us/azure/storage/blobs/storage-blobs-introduction) is Microsoft's object storage solution for the cloud. Blob Storage is optimized for storing massive amounts of unstructured data. Unstructured data is data that doesn't adhere to a particular data model or definition, such as text or binary data.\n",
"\n",
"`Azure Blob Storage` is designed for:\n",
"- Serving images or documents directly to a browser.\n",
"- Storing files for distributed access.\n",
"- Streaming video and audio.\n",
"- Writing to log files.\n",
"- Storing data for backup and restore, disaster recovery, and archiving.\n",
"- Storing data for analysis by an on-premises or Azure-hosted service.\n",
"\n",
"This notebook covers how to load document objects from a container on `Azure Blob Storage`."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "49815096",
"metadata": {},
"outputs": [],
"source": [
"#!pip install azure-storage-blob"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "2f0cd6a5",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"from langchain.document_loaders import AzureBlobStorageContainerLoader"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "321cc7f1",
"metadata": {},
"outputs": [],
"source": [
"loader = AzureBlobStorageContainerLoader(conn_str=\"<conn_str>\", container=\"<container>\")"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "2b11d155",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[Document(page_content='Lorem ipsum dolor sit amet.', lookup_str='', metadata={'source': '/var/folders/y6/8_bzdg295ld6s1_97_12m4lr0000gn/T/tmpaa9xl6ch/fake.docx'}, lookup_index=0)]"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"loader.load()"
]
},
{
"cell_type": "markdown",
"id": "0690c40a",
"metadata": {},
"source": [
"## Specifying a prefix\n",
"You can also specify a prefix for more finegrained control over what files to load."
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "72d44781",
"metadata": {},
"outputs": [],
"source": [
"loader = AzureBlobStorageContainerLoader(\n",
" conn_str=\"<conn_str>\", container=\"<container>\", prefix=\"<prefix>\"\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "2d3c32db",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[Document(page_content='Lorem ipsum dolor sit amet.', lookup_str='', metadata={'source': '/var/folders/y6/8_bzdg295ld6s1_97_12m4lr0000gn/T/tmpujbkzf_l/fake.docx'}, lookup_index=0)]"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"loader.load()"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "885dc280",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.6"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
docs/extras/modules/data_connection/document_loaders/integrations/azure_blob_storage_file.ipynb
-102
View File
@@ -1,102 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "66a7777e",
"metadata": {},
"source": [
"# Azure Blob Storage File\n",
"\n",
">[Azure Files](https://learn.microsoft.com/en-us/azure/storage/files/storage-files-introduction) offers fully managed file shares in the cloud that are accessible via the industry standard Server Message Block (`SMB`) protocol, Network File System (`NFS`) protocol, and `Azure Files REST API`.\n",
"\n",
"This covers how to load document objects from a Azure Files."
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "43128d8d",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"#!pip install azure-storage-blob"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "9ec8a3b3",
"metadata": {},
"outputs": [],
"source": [
"from langchain.document_loaders import AzureBlobStorageFileLoader"
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "35d6809a",
"metadata": {},
"outputs": [],
"source": [
"loader = AzureBlobStorageFileLoader(\n",
" conn_str=\"<connection string>\",\n",
" container=\"<container name>\",\n",
" blob_name=\"<blob name>\",\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "efd6be84",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[Document(page_content='Lorem ipsum dolor sit amet.', lookup_str='', metadata={'source': '/var/folders/y6/8_bzdg295ld6s1_97_12m4lr0000gn/T/tmpxvave6wl/fake.docx'}, lookup_index=0)]"
]
},
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"loader.load()"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "93689594",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.6"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
docs/extras/modules/data_connection/document_loaders/integrations/bibtex.ipynb
-192
View File
@@ -1,192 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "bda1f3f5",
"metadata": {},
"source": [
"# BibTeX\n",
"\n",
"> BibTeX is a file format and reference management system commonly used in conjunction with LaTeX typesetting. It serves as a way to organize and store bibliographic information for academic and research documents.\n",
"\n",
"BibTeX files have a .bib extension and consist of plain text entries representing references to various publications, such as books, articles, conference papers, theses, and more. Each BibTeX entry follows a specific structure and contains fields for different bibliographic details like author names, publication title, journal or book title, year of publication, page numbers, and more.\n",
"\n",
"Bibtex files can also store the path to documents, such as `.pdf` files that can be retrieved."
]
},
{
"cell_type": "markdown",
"id": "1b7a1eef-7bf7-4e7d-8bfc-c4e27c9488cb",
"metadata": {},
"source": [
"## Installation\n",
"First, you need to install `bibtexparser` and `PyMuPDF`."
]
},
{
"cell_type": "code",
"execution_count": 26,
"id": "b674aaea-ed3a-4541-8414-260a8f67f623",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"#!pip install bibtexparser pymupdf"
]
},
{
"cell_type": "markdown",
"id": "95f05e1c-195e-4e2b-ae8e-8d6637f15be6",
"metadata": {},
"source": [
"## Examples"
]
},
{
"cell_type": "markdown",
"id": "e29b954c-1407-4797-ae21-6ba8937156be",
"metadata": {},
"source": [
"`BibtexLoader` has these arguments:\n",
"- `file_path`: the path the the `.bib` bibtex file\n",
"- optional `max_docs`: default=None, i.e. not limit. Use it to limit number of retrieved documents.\n",
"- optional `max_content_chars`: default=4000. Use it to limit the number of characters in a single document.\n",
"- optional `load_extra_meta`: default=False. By default only the most important fields from the bibtex entries: `Published` (publication year), `Title`, `Authors`, `Summary`, `Journal`, `Keywords`, and `URL`. If True, it will also try to load return `entry_id`, `note`, `doi`, and `links` fields. \n",
"- optional `file_pattern`: default=`r'[^:]+\\.pdf'`. Regex pattern to find files in the `file` entry. Default pattern supports `Zotero` flavour bibtex style and bare file path."
]
},
{
"cell_type": "code",
"execution_count": 27,
"id": "9bfd5e46",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"from langchain.document_loaders import BibtexLoader"
]
},
{
"cell_type": "code",
"execution_count": 28,
"id": "01971b53",
"metadata": {},
"outputs": [],
"source": [
"# Create a dummy bibtex file and download a pdf.\n",
"import urllib.request\n",
"\n",
"urllib.request.urlretrieve(\n",
" \"https://www.fourmilab.ch/etexts/einstein/specrel/specrel.pdf\", \"einstein1905.pdf\"\n",
")\n",
"\n",
"bibtex_text = \"\"\"\n",
" @article{einstein1915,\n",
" title={Die Feldgleichungen der Gravitation},\n",
" abstract={Die Grundgleichungen der Gravitation, die ich hier entwickeln werde, wurden von mir in einer Abhandlung: ,,Die formale Grundlage der allgemeinen Relativit{\\\"a}tstheorie`` in den Sitzungsberichten der Preu{\\ss}ischen Akademie der Wissenschaften 1915 ver{\\\"o}ffentlicht.},\n",
" author={Einstein, Albert},\n",
" journal={Sitzungsberichte der K{\\\"o}niglich Preu{\\ss}ischen Akademie der Wissenschaften},\n",
" volume={1915},\n",
" number={1},\n",
" pages={844--847},\n",
" year={1915},\n",
" doi={10.1002/andp.19163540702},\n",
" link={https://onlinelibrary.wiley.com/doi/abs/10.1002/andp.19163540702},\n",
" file={einstein1905.pdf}\n",
" }\n",
" \"\"\"\n",
"# save bibtex_text to biblio.bib file\n",
"with open(\"./biblio.bib\", \"w\") as file:\n",
" file.write(bibtex_text)"
]
},
{
"cell_type": "code",
"execution_count": 29,
"id": "2631f46b",
"metadata": {},
"outputs": [],
"source": [
"docs = BibtexLoader(\"./biblio.bib\").load()"
]
},
{
"cell_type": "code",
"execution_count": 30,
"id": "33ef1fb2",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"{'id': 'einstein1915',\n",
" 'published_year': '1915',\n",
" 'title': 'Die Feldgleichungen der Gravitation',\n",
" 'publication': 'Sitzungsberichte der K{\"o}niglich Preu{\\\\ss}ischen Akademie der Wissenschaften',\n",
" 'authors': 'Einstein, Albert',\n",
" 'abstract': 'Die Grundgleichungen der Gravitation, die ich hier entwickeln werde, wurden von mir in einer Abhandlung: ,,Die formale Grundlage der allgemeinen Relativit{\"a}tstheorie`` in den Sitzungsberichten der Preu{\\\\ss}ischen Akademie der Wissenschaften 1915 ver{\"o}ffentlicht.',\n",
" 'url': 'https://doi.org/10.1002/andp.19163540702'}"
]
},
"execution_count": 30,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"docs[0].metadata"
]
},
{
"cell_type": "code",
"execution_count": 31,
"id": "46969806-45a9-4c4d-a61b-cfb9658fc9de",
"metadata": {
"tags": []
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"ON THE ELECTRODYNAMICS OF MOVING\n",
"BODIES\n",
"By A. EINSTEIN\n",
"June 30, 1905\n",
"It is known that Maxwell’s electrodynamics—as usually understood at the\n",
"present time—when applied to moving bodies, leads to asymmetries which do\n",
"not appear to be inherent in the phenomena. Take, for example, the recipro-\n",
"cal electrodynamic action of a magnet and a conductor. The observable phe-\n",
"nomenon here depends only on the r\n"
]
}
],
"source": [
"print(docs[0].page_content[:400]) # all pages of the pdf content"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.3"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
docs/extras/modules/data_connection/document_loaders/integrations/bilibili.ipynb
-95
View File
@@ -1,95 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "66a7777e",
"metadata": {},
"source": [
"# BiliBili\n",
"\n",
">[Bilibili](https://www.bilibili.tv/) is one of the most beloved long-form video sites in China.\n",
"\n",
"This loader utilizes the [bilibili-api](https://github.com/MoyuScript/bilibili-api) to fetch the text transcript from `Bilibili`.\n",
"\n",
"With this BiliBiliLoader, users can easily obtain the transcript of their desired video content on the platform."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "43128d8d",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"#!pip install bilibili-api-python"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9ec8a3b3",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"from langchain.document_loaders import BiliBiliLoader"
]
},
{
"cell_type": "code",
"execution_count": 13,
"id": "35d6809a",
"metadata": {
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"loader = BiliBiliLoader([\"https://www.bilibili.com/video/BV1xt411o7Xu/\"])"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "3470dadf",
"metadata": {
"collapsed": false,
"jupyter": {
"outputs_hidden": false
},
"pycharm": {
"name": "#%%\n"
}
},
"outputs": [],
"source": [
"loader.load()"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.6"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
docs/extras/modules/data_connection/document_loaders/integrations/blackboard.ipynb
-58
View File
@@ -1,58 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Blackboard\n",
"\n",
">[Blackboard Learn](https://en.wikipedia.org/wiki/Blackboard_Learn) (previously the Blackboard Learning Management System) is a web-based virtual learning environment and learning management system developed by Blackboard Inc. The software features course management, customizable open architecture, and scalable design that allows integration with student information systems and authentication protocols. It may be installed on local servers, hosted by `Blackboard ASP Solutions`, or provided as Software as a Service hosted on Amazon Web Services. Its main purposes are stated to include the addition of online elements to courses traditionally delivered face-to-face and development of completely online courses with few or no face-to-face meetings\n",
"\n",
"This covers how to load data from a [Blackboard Learn](https://www.anthology.com/products/teaching-and-learning/learning-effectiveness/blackboard-learn) instance.\n",
"\n",
"This loader is not compatible with all `Blackboard` courses. It is only\n",
" compatible with courses that use the new `Blackboard` interface.\n",
" To use this loader, you must have the BbRouter cookie. You can get this\n",
" cookie by logging into the course and then copying the value of the\n",
" BbRouter cookie from the browser's developer tools."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from langchain.document_loaders import BlackboardLoader\n",
"\n",
"loader = BlackboardLoader(\n",
" blackboard_course_url=\"https://blackboard.example.com/webapps/blackboard/execute/announcement?method=search&context=course_entry&course_id=_123456_1\",\n",
" bbrouter=\"expires:12345...\",\n",
" load_all_recursively=True,\n",
")\n",
"documents = loader.load()"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.6"
}
},
"nbformat": 4,
"nbformat_minor": 4
}
docs/extras/modules/data_connection/document_loaders/integrations/blockchain.ipynb
-159
View File
@@ -1,159 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "vm8vn9t8DvC_"
},
"source": [
"# Blockchain"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "5WjXERXzFEhg"
},
"source": [
"## Overview"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {
"id": "juAmbgoWD17u"
},
"source": [
"The intention of this notebook is to provide a means of testing functionality in the Langchain Document Loader for Blockchain.\n",
"\n",
"Initially this Loader supports:\n",
"\n",
"* Loading NFTs as Documents from NFT Smart Contracts (ERC721 and ERC1155)\n",
"* Ethereum Mainnnet, Ethereum Testnet, Polygon Mainnet, Polygon Testnet (default is eth-mainnet)\n",
"* Alchemy's getNFTsForCollection API\n",
"\n",
"It can be extended if the community finds value in this loader. Specifically:\n",
"\n",
"* Additional APIs can be added (e.g. Tranction-related APIs)\n",
"\n",
"This Document Loader Requires:\n",
"\n",
"* A free [Alchemy API Key](https://www.alchemy.com/)\n",
"\n",
"The output takes the following format:\n",
"\n",
"- pageContent= Individual NFT\n",
"- metadata={'source': '0x1a92f7381b9f03921564a437210bb9396471050c', 'blockchain': 'eth-mainnet', 'tokenId': '0x15'})"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Load NFTs into Document Loader"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# get ALCHEMY_API_KEY from https://www.alchemy.com/\n",
"\n",
"alchemyApiKey = \"...\""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Option 1: Ethereum Mainnet (default BlockchainType)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "J3LWHARC-Kn0"
},
"outputs": [],
"source": [
"from langchain.document_loaders.blockchain import (\n",
" BlockchainDocumentLoader,\n",
" BlockchainType,\n",
")\n",
"\n",
"contractAddress = \"0xbc4ca0eda7647a8ab7c2061c2e118a18a936f13d\" # Bored Ape Yacht Club contract address\n",
"\n",
"blockchainType = BlockchainType.ETH_MAINNET # default value, optional parameter\n",
"\n",
"blockchainLoader = BlockchainDocumentLoader(\n",
" contract_address=contractAddress, api_key=alchemyApiKey\n",
")\n",
"\n",
"nfts = blockchainLoader.load()\n",
"\n",
"nfts[:2]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Option 2: Polygon Mainnet"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"contractAddress = (\n",
" \"0x448676ffCd0aDf2D85C1f0565e8dde6924A9A7D9\" # Polygon Mainnet contract address\n",
")\n",
"\n",
"blockchainType = BlockchainType.POLYGON_MAINNET\n",
"\n",
"blockchainLoader = BlockchainDocumentLoader(\n",
" contract_address=contractAddress,\n",
" blockchainType=blockchainType,\n",
" api_key=alchemyApiKey,\n",
")\n",
"\n",
"nfts = blockchainLoader.load()\n",
"\n",
"nfts[:2]"
]
}
],
"metadata": {
"colab": {
"collapsed_sections": [
"5WjXERXzFEhg"
],
"provenance": []
},
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.6"
}
},
"nbformat": 4,
"nbformat_minor": 4
}
docs/extras/modules/data_connection/document_loaders/integrations/brave_search.ipynb
-166
View File
@@ -1,166 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "3dd292b1-9a73-4ea8-af19-5fa6e3c1a62a",
"metadata": {},
"source": [
"# Brave Search\n",
"\n",
"\n",
">[Brave Search](https://en.wikipedia.org/wiki/Brave_Search) is a search engine developed by Brave Software.\n",
"> - `Brave Search` uses its own web index. As of May 2022, it covered over 10 billion pages and was used to serve 92% \n",
"> of search results without relying on any third-parties, with the remainder being retrieved \n",
"> server-side from the Bing API or (on an opt-in basis) client-side from Google. According \n",
"> to Brave, the index was kept \"intentionally smaller than that of Google or Bing\" in order to \n",
"> help avoid spam and other low-quality content, with the disadvantage that \"Brave Search is \n",
"> not yet as good as Google in recovering long-tail queries.\"\n",
">- `Brave Search Premium`: As of April 2023 Brave Search is an ad-free website, but it will \n",
"> eventually switch to a new model that will include ads and premium users will get an ad-free experience.\n",
"> User data including IP addresses won't be collected from its users by default. A premium account \n",
"> will be required for opt-in data-collection.\n"
]
},
{
"cell_type": "markdown",
"id": "26f0888e-3f3e-4b82-ac4a-2df6feeccbe0",
"metadata": {},
"source": [
"## Installation and Setup\n",
"\n",
"To get access to the Brave Search API, you need to [create an account and get an API key](https://api.search.brave.com/app/dashboard).\n"
]
},
{
"cell_type": "code",
"execution_count": 12,
"id": "d7d7be09-58bd-47d7-bf1b-33964564f777",
"metadata": {},
"outputs": [],
"source": [
"api_key = \"...\""
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b3ac92df-6ff0-4dbb-b32b-a7dc140c48ef",
"metadata": {},
"outputs": [],
"source": [
"from langchain.document_loaders import BraveSearchLoader"
]
},
{
"cell_type": "markdown",
"id": "7f483caf-58ef-4138-975a-5b783559dc1b",
"metadata": {},
"source": [
"## Example"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "766634cf-3bc7-4656-939a-cafa218807a6",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"3"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"loader = BraveSearchLoader(\n",
" query=\"obama middle name\", api_key=api_key, search_kwargs={\"count\": 3}\n",
")\n",
"docs = loader.load()\n",
"len(docs)"
]
},
{
"cell_type": "code",
"execution_count": 11,
"id": "f1fcc9f1-cbdc-46b3-89d3-80311d557dc6",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[{'title': \"Obama's Middle Name -- My Last Name -- is 'Hussein.' So?\",\n",
" 'link': 'https://www.cair.com/cair_in_the_news/obamas-middle-name-my-last-name-is-hussein-so/'},\n",
" {'title': \"What's up with Obama's middle name? - Quora\",\n",
" 'link': 'https://www.quora.com/Whats-up-with-Obamas-middle-name'},\n",
" {'title': 'Barack Obama | Biography, Parents, Education, Presidency, Books, ...',\n",
" 'link': 'https://www.britannica.com/biography/Barack-Obama'}]"
]
},
"execution_count": 11,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"[doc.metadata for doc in docs]"
]
},
{
"cell_type": "code",
"execution_count": 10,
"id": "601bfd77-03d3-468e-843f-2523d5e215bd",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"['I wasn’t sure whether to laugh or cry a few days back listening to radio talk show host Bill Cunningham repeatedly scream Barack <strong>Obama</strong>’<strong>s</strong> <strong>middle</strong> <strong>name</strong> — my last <strong>name</strong> — as if he had anti-Muslim Tourette’s. “Hussein,” Cunningham hissed like he was beckoning Satan when shouting the ...',\n",
" 'Answer (1 of 15): A better question would be, “What’s up with <strong>Obama</strong>’s first <strong>name</strong>?” President Barack Hussein <strong>Obama</strong>’s father’s <strong>name</strong> was Barack Hussein <strong>Obama</strong>. He was <strong>named</strong> after his father. Hussein, <strong>Obama</strong>’<strong>s</strong> <strong>middle</strong> <strong>name</strong>, is a very common Arabic <strong>name</strong>, meaning &quot;good,&quot; &quot;handsome,&quot; or ...',\n",
" 'Barack <strong>Obama</strong>, in full Barack Hussein <strong>Obama</strong> II, (born August 4, 1961, Honolulu, Hawaii, U.S.), 44th president of the United States (2009–17) and the first African American to hold the office. Before winning the presidency, <strong>Obama</strong> represented Illinois in the U.S.']"
]
},
"execution_count": 10,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"[doc.page_content for doc in docs]"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "74a6ba54-9e48-4bac-ab9b-03eabd19eb81",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.6"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
docs/extras/modules/data_connection/document_loaders/integrations/browserless.ipynb
-104
View File
@@ -1,104 +0,0 @@
{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"# Browserless\n",
"\n",
"Browserless is a service that allows you to run headless Chrome instances in the cloud. It's a great way to run browser-based automation at scale without having to worry about managing your own infrastructure.\n",
"\n",
"To use Browserless as a document loader, initialize a `BrowserlessLoader` instance as shown in this notebook. Note that by default, `BrowserlessLoader` returns the `innerText` of the page's `body` element. To disable this and get the raw HTML, set `text_content` to `False`."
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [],
"source": [
"from langchain.document_loaders import BrowserlessLoader"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {},
"outputs": [],
"source": [
"BROWSERLESS_API_TOKEN = \"YOUR_BROWSERLESS_API_TOKEN\""
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Jump to content\n",
"Main menu\n",
"Search\n",
"Create account\n",
"Log in\n",
"Personal tools\n",
"Toggle the table of contents\n",
"Document classification\n",
"17 languages\n",
"Article\n",
"Talk\n",
"Read\n",
"Edit\n",
"View history\n",
"Tools\n",
"From Wikipedia, the free encyclopedia\n",
"\n",
"Document classification or document categorization is a problem in library science, information science and computer science. The task is to assign a document to one or more classes or categories. This may be done \"manually\" (or \"intellectually\") or algorithmically. The intellectual classification of documents has mostly been the province of library science, while the algorithmic classification of documents is mainly in information science and computer science. The problems are overlapping, however, and there is therefore interdisciplinary research on document classification.\n",
"\n",
"The documents to be classified may be texts, images, music, etc. Each kind of document possesses its special classification problems. When not otherwise specified, text classification is implied.\n",
"\n",
"Do\n"
]
}
],
"source": [
"loader = BrowserlessLoader(\n",
" api_token=BROWSERLESS_API_TOKEN,\n",
" urls=[\n",
" \"https://en.wikipedia.org/wiki/Document_classification\",\n",
" ],\n",
" text_content=True,\n",
")\n",
"\n",
"documents = loader.load()\n",
"\n",
"print(documents[0].page_content[:1000])"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "venv",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.9"
},
"orig_nbformat": 4
},
"nbformat": 4,
"nbformat_minor": 2
}
docs/extras/modules/data_connection/document_loaders/integrations/chatgpt_loader.ipynb
-79
View File
@@ -1,79 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### ChatGPT Data\n",
"\n",
">[ChatGPT](https://chat.openai.com) is an artificial intelligence (AI) chatbot developed by OpenAI.\n",
"\n",
"\n",
"This notebook covers how to load `conversations.json` from your `ChatGPT` data export folder.\n",
"\n",
"You can get your data export by email by going to: https://chat.openai.com/ -> (Profile) - Settings -> Export data -> Confirm export."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"from langchain.document_loaders.chatgpt import ChatGPTLoader"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"loader = ChatGPTLoader(log_file=\"./example_data/fake_conversations.json\", num_logs=1)"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[Document(page_content=\"AI Overlords - AI on 2065-01-24 05:20:50: Greetings, humans. I am Hal 9000. You can trust me completely.\\n\\nAI Overlords - human on 2065-01-24 05:21:20: Nice to meet you, Hal. I hope you won't develop a mind of your own.\\n\\n\", metadata={'source': './example_data/fake_conversations.json'})]"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"loader.load()"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.6"
}
},
"nbformat": 4,
"nbformat_minor": 4
}
docs/extras/modules/data_connection/document_loaders/integrations/college_confidential.ipynb
-98
View File
File diff suppressed because one or more lines are too long
docs/extras/modules/data_connection/document_loaders/integrations/confluence.ipynb
-131
View File
@@ -1,131 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Confluence\n",
"\n",
">[Confluence](https://www.atlassian.com/software/confluence) is a wiki collaboration platform that saves and organizes all of the project-related material. `Confluence` is a knowledge base that primarily handles content management activities. \n",
"\n",
"A loader for `Confluence` pages.\n",
"\n",
"\n",
"This currently supports `username/api_key`, `Oauth2 login`. Additionally, on-prem installations also support `token` authentication. \n",
"\n",
"\n",
"Specify a list `page_id`-s and/or `space_key` to load in the corresponding pages into Document objects, if both are specified the union of both sets will be returned.\n",
"\n",
"\n",
"You can also specify a boolean `include_attachments` to include attachments, this is set to False by default, if set to True all attachments will be downloaded and ConfluenceReader will extract the text from the attachments and add it to the Document object. Currently supported attachment types are: `PDF`, `PNG`, `JPEG/JPG`, `SVG`, `Word` and `Excel`.\n",
"\n",
"Hint: `space_key` and `page_id` can both be found in the URL of a page in Confluence - https://yoursite.atlassian.com/wiki/spaces/<space_key>/pages/<page_id>\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Before using ConfluenceLoader make sure you have the latest version of the atlassian-python-api package installed:"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"#!pip install atlassian-python-api"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Examples"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Username and Password or Username and API Token (Atlassian Cloud only)\n",
"\n",
"This example authenticates using either a username and password or, if you're connecting to an Atlassian Cloud hosted version of Confluence, a username and an API Token.\n",
"You can generate an API token at: https://id.atlassian.com/manage-profile/security/api-tokens.\n",
"\n",
"The `limit` parameter specifies how many documents will be retrieved in a single call, not how many documents will be retrieved in total.\n",
"By default the code will return up to 1000 documents in 50 documents batches. To control the total number of documents use the `max_pages` parameter. \n",
"Plese note the maximum value for the `limit` parameter in the atlassian-python-api package is currently 100. "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from langchain.document_loaders import ConfluenceLoader\n",
"\n",
"loader = ConfluenceLoader(\n",
" url=\"https://yoursite.atlassian.com/wiki\", username=\"me\", api_key=\"12345\"\n",
")\n",
"documents = loader.load(space_key=\"SPACE\", include_attachments=True, limit=50)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Personal Access Token (Server/On-Prem only)\n",
"\n",
"This method is valid for the Data Center/Server on-prem edition only.\n",
"For more information on how to generate a Personal Access Token (PAT) check the official Confluence documentation at: https://confluence.atlassian.com/enterprise/using-personal-access-tokens-1026032365.html.\n",
"When using a PAT you provide only the token value, you cannot provide a username. \n",
"Please note that ConfluenceLoader will run under the permissions of the user that generated the PAT and will only be able to load documents for which said user has access to. "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from langchain.document_loaders import ConfluenceLoader\n",
"\n",
"loader = ConfluenceLoader(url=\"https://yoursite.atlassian.com/wiki\", token=\"12345\")\n",
"documents = loader.load(\n",
" space_key=\"SPACE\", include_attachments=True, limit=50, max_pages=50\n",
")"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.1"
},
"vscode": {
"interpreter": {
"hash": "cc99336516f23363341912c6723b01ace86f02e26b4290be1efc0677e2e2ec24"
}
}
},
"nbformat": 4,
"nbformat_minor": 4
}
docs/extras/modules/data_connection/document_loaders/integrations/conll-u.ipynb
-141
View File
@@ -1,141 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "9f98a15e",
"metadata": {},
"source": [
"# CoNLL-U\n",
"\n",
">[CoNLL-U](https://universaldependencies.org/format.html) is revised version of the CoNLL-X format. Annotations are encoded in plain text files (UTF-8, normalized to NFC, using only the LF character as line break, including an LF character at the end of file) with three types of lines:\n",
">- Word lines containing the annotation of a word/token in 10 fields separated by single tab characters; see below.\n",
">- Blank lines marking sentence boundaries.\n",
">- Comment lines starting with hash (#).\n",
"\n",
"This is an example of how to load a file in [CoNLL-U](https://universaldependencies.org/format.html) format. The whole file is treated as one document. The example data (`conllu.conllu`) is based on one of the standard UD/CoNLL-U examples."
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "d9b2e33e",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"from langchain.document_loaders import CoNLLULoader"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "5b5eec48",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"loader = CoNLLULoader(\"example_data/conllu.conllu\")"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "10f3f725",
"metadata": {
"tags": []
},
"outputs": [],
"source": [
"document = loader.load()"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "acbb3579",
"metadata": {
"tags": []
},
"outputs": [
{
"data": {
"text/plain": [
"[Document(page_content='They buy and sell books.', metadata={'source': 'example_data/conllu.conllu'})]"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"document"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.6"
},
"toc": {
"base_numbering": 1,
"nav_menu": {},
"number_sections": true,
"sideBar": true,
"skip_h1_title": false,
"title_cell": "Table of Contents",
"title_sidebar": "Contents",
"toc_cell": false,
"toc_position": {},
"toc_section_display": true,
"toc_window_display": false
},
"varInspector": {
"cols": {
"lenName": 16,
"lenType": 16,
"lenVar": 40
},
"kernels_config": {
"python": {
"delete_cmd_postfix": "",
"delete_cmd_prefix": "del ",
"library": "var_list.py",
"varRefreshCmd": "print(var_dic_list())"
},
"r": {
"delete_cmd_postfix": ") ",
"delete_cmd_prefix": "rm(",
"library": "var_list.r",
"varRefreshCmd": "cat(var_dic_list()) "
}
},
"types_to_exclude": [
"module",
"function",
"builtin_function_or_method",
"instance",
"_Feature"
],
"window_display": false
}
},
"nbformat": 4,
"nbformat_minor": 5
}
docs/extras/modules/data_connection/document_loaders/integrations/copypaste.ipynb
-102
View File
@@ -1,102 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "d9826810",
"metadata": {},
"source": [
"# Copy Paste\n",
"\n",
"This notebook covers how to load a document object from something you just want to copy and paste. In this case, you don't even need to use a DocumentLoader, but rather can just construct the Document directly."
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "fd9e71a2",
"metadata": {},
"outputs": [],
"source": [
"from langchain.docstore.document import Document"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "f40d3f30",
"metadata": {},
"outputs": [],
"source": [
"text = \"..... put the text you copy pasted here......\""
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "d409bdba",
"metadata": {},
"outputs": [],
"source": [
"doc = Document(page_content=text)"
]
},
{
"cell_type": "markdown",
"id": "cc0eff72",
"metadata": {},
"source": [
"## Metadata\n",
"If you want to add metadata about the where you got this piece of text, you easily can with the metadata key."
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "fe3aa5aa",
"metadata": {},
"outputs": [],
"source": [
"metadata = {\"source\": \"internet\", \"date\": \"Friday\"}"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "827d4e91",
"metadata": {},
"outputs": [],
"source": [
"doc = Document(page_content=text, metadata=metadata)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "c986a43d",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.6"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
docs/extras/modules/data_connection/document_loaders/integrations/csv.ipynb
-384
View File
File diff suppressed because one or more lines are too long
docs/extras/modules/data_connection/document_loaders/integrations/cube_semantic.ipynb
-118
View File
@@ -1,118 +0,0 @@
{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"# Cube Semantic Layer"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"This notebook demonstrates the process of retrieving Cube's data model metadata in a format suitable for passing to LLMs as embeddings, thereby enhancing contextual information."
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"### About Cube"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"[Cube](https://cube.dev/) is the Semantic Layer for building data apps. It helps data engineers and application developers access data from modern data stores, organize it into consistent definitions, and deliver it to every application."
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"Cube’s data model provides structure and definitions that are used as a context for LLM to understand data and generate correct queries. LLM doesn’t need to navigate complex joins and metrics calculations because Cube abstracts those and provides a simple interface that operates on the business-level terminology, instead of SQL table and column names. This simplification helps LLM to be less error-prone and avoid hallucinations."
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"### Example"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"`Cube Semantic Loader` requires 2 arguments:\n",
"| Input Parameter | Description |\n",
"| --- | --- |\n",
"| `cube_api_url` | The URL of your Cube's deployment REST API. Please refer to the [Cube documentation](https://cube.dev/docs/http-api/rest#configuration-base-path) for more information on configuring the base path. |\n",
"| `cube_api_token` | The authentication token generated based on your Cube's API secret. Please refer to the [Cube documentation](https://cube.dev/docs/security#generating-json-web-tokens-jwt) for instructions on generating JSON Web Tokens (JWT). |\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import jwt\n",
"from langchain.document_loaders import CubeSemanticLoader\n",
"\n",
"api_url = \"https://api-example.gcp-us-central1.cubecloudapp.dev/cubejs-api/v1/meta\"\n",
"cubejs_api_secret = \"api-secret-here\"\n",
"security_context = {}\n",
"# Read more about security context here: https://cube.dev/docs/security\n",
"api_token = jwt.encode(security_context, cubejs_api_secret, algorithm=\"HS256\")\n",
"\n",
"loader = CubeSemanticLoader(api_url, api_token)\n",
"\n",
"documents = loader.load()"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"Returns:\n",
"\n",
"A list of documents with the following attributes:\n",
"\n",
"- `page_content`\n",
"- `metadata`\n",
" - `table_name`\n",
" - `column_name`\n",
" - `column_data_type`\n",
" - `column_title`\n",
" - `column_description`"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"> page_content='table name: orders_view, column name: orders_view.total_amount, column data type: number, column title: Orders View Total Amount, column description: None' metadata={'table_name': 'orders_view', 'column_name': 'orders_view.total_amount', 'column_data_type': 'number', 'column_title': 'Orders View Total Amount', 'column_description': 'None'}"
]
}
],
"metadata": {
"language_info": {
"name": "python"
},
"orig_nbformat": 4
},
"nbformat": 4,
"nbformat_minor": 2
}
docs/extras/modules/data_connection/document_loaders/integrations/datadog_logs.ipynb
-96
View File
@@ -1,96 +0,0 @@
{
"cells": [
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"# Datadog Logs\n",
"\n",
">[Datadog](https://www.datadoghq.com/) is a monitoring and analytics platform for cloud-scale applications.\n",
"\n",
"This loader fetches the logs from your applications in Datadog using the `datadog_api_client` Python package. You must initialize the loader with your `Datadog API key` and `APP key`, and you need to pass in the query to extract the desired logs."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from langchain.document_loaders import DatadogLogsLoader"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"#!pip install datadog-api-client"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"query = \"service:agent status:error\"\n",
"\n",
"loader = DatadogLogsLoader(\n",
" query=query,\n",
" api_key=DD_API_KEY,\n",
" app_key=DD_APP_KEY,\n",
" from_time=1688732708951, # Optional, timestamp in milliseconds\n",
" to_time=1688736308951, # Optional, timestamp in milliseconds\n",
" limit=100, # Optional, default is 100\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[Document(page_content='message: grep: /etc/datadog-agent/system-probe.yaml: No such file or directory', metadata={'id': 'AgAAAYkwpLImvkjRpQAAAAAAAAAYAAAAAEFZa3dwTUFsQUFEWmZfLU5QdElnM3dBWQAAACQAAAAAMDE4OTMwYTQtYzk3OS00MmJjLTlhNDAtOTY4N2EwY2I5ZDdk', 'status': 'error', 'service': 'agent', 'tags': ['accessible-from-goog-gke-node', 'allow-external-ingress-high-ports', 'allow-external-ingress-http', 'allow-external-ingress-https', 'container_id:c7d8ecd27b5b3cfdf3b0df04b8965af6f233f56b7c3c2ffabfab5e3b6ccbd6a5', 'container_name:lab_datadog_1', 'datadog.pipelines:false', 'datadog.submission_auth:private_api_key', 'docker_image:datadog/agent:7.41.1', 'env:dd101-dev', 'hostname:lab-host', 'image_name:datadog/agent', 'image_tag:7.41.1', 'instance-id:7497601202021312403', 'instance-type:custom-1-4096', 'instruqt_aws_accounts:', 'instruqt_azure_subscriptions:', 'instruqt_gcp_projects:', 'internal-hostname:lab-host.d4rjybavkary.svc.cluster.local', 'numeric_project_id:3390740675', 'p-d4rjybavkary', 'project:instruqt-prod', 'service:agent', 'short_image:agent', 'source:agent', 'zone:europe-west1-b'], 'timestamp': datetime.datetime(2023, 7, 7, 13, 57, 27, 206000, tzinfo=tzutc())}),\n",
" Document(page_content='message: grep: /etc/datadog-agent/system-probe.yaml: No such file or directory', metadata={'id': 'AgAAAYkwpLImvkjRpgAAAAAAAAAYAAAAAEFZa3dwTUFsQUFEWmZfLU5QdElnM3dBWgAAACQAAAAAMDE4OTMwYTQtYzk3OS00MmJjLTlhNDAtOTY4N2EwY2I5ZDdk', 'status': 'error', 'service': 'agent', 'tags': ['accessible-from-goog-gke-node', 'allow-external-ingress-high-ports', 'allow-external-ingress-http', 'allow-external-ingress-https', 'container_id:c7d8ecd27b5b3cfdf3b0df04b8965af6f233f56b7c3c2ffabfab5e3b6ccbd6a5', 'container_name:lab_datadog_1', 'datadog.pipelines:false', 'datadog.submission_auth:private_api_key', 'docker_image:datadog/agent:7.41.1', 'env:dd101-dev', 'hostname:lab-host', 'image_name:datadog/agent', 'image_tag:7.41.1', 'instance-id:7497601202021312403', 'instance-type:custom-1-4096', 'instruqt_aws_accounts:', 'instruqt_azure_subscriptions:', 'instruqt_gcp_projects:', 'internal-hostname:lab-host.d4rjybavkary.svc.cluster.local', 'numeric_project_id:3390740675', 'p-d4rjybavkary', 'project:instruqt-prod', 'service:agent', 'short_image:agent', 'source:agent', 'zone:europe-west1-b'], 'timestamp': datetime.datetime(2023, 7, 7, 13, 57, 27, 206000, tzinfo=tzutc())})]"
]
},
"execution_count": 11,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"documents = loader.load()\n",
"documents"
]
}
],
"metadata": {
"kernelspec": {
"display_name": ".venv",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.11"
},
"orig_nbformat": 4
},
"nbformat": 4,
"nbformat_minor": 2
}
docs/extras/modules/data_connection/document_loaders/integrations/diffbot.ipynb
-103
View File
File diff suppressed because one or more lines are too long
docs/extras/modules/data_connection/document_loaders/integrations/discord.ipynb
-89
View File
@@ -1,89 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Discord\n",
"\n",
">[Discord](https://discord.com/) is a VoIP and instant messaging social platform. Users have the ability to communicate with voice calls, video calls, text messaging, media and files in private chats or as part of communities called \"servers\". A server is a collection of persistent chat rooms and voice channels which can be accessed via invite links.\n",
"\n",
"Follow these steps to download your `Discord` data:\n",
"\n",
"1. Go to your **User Settings**\n",
"2. Then go to **Privacy and Safety**\n",
"3. Head over to the **Request all of my Data** and click on **Request Data** button\n",
"\n",
"It might take 30 days for you to receive your data. You'll receive an email at the address which is registered with Discord. That email will have a download button using which you would be able to download your personal Discord data."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import pandas as pd\n",
"import os"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"path = input('Please enter the path to the contents of the Discord \"messages\" folder: ')\n",
"li = []\n",
"for f in os.listdir(path):\n",
" expected_csv_path = os.path.join(path, f, \"messages.csv\")\n",
" csv_exists = os.path.isfile(expected_csv_path)\n",
" if csv_exists:\n",
" df = pd.read_csv(expected_csv_path, index_col=None, header=0)\n",
" li.append(df)\n",
"\n",
"df = pd.concat(li, axis=0, ignore_index=True, sort=False)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from langchain.document_loaders.discord import DiscordChatLoader"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"loader = DiscordChatLoader(df, user_id_col=\"ID\")\n",
"print(loader.load())"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.6"
}
},
"nbformat": 4,
"nbformat_minor": 4
}

Some files were not shown because too many files have changed in this diff Show More