diff --git a/README.md b/README.md
index a54279b..f6a2403 100644
--- a/README.md
+++ b/README.md
@@ -1,23 +1,37 @@
-# Getting Started with Instructor
+# Welcome to Instructor - Your Gateway to Structured Outputs with OpenAI
_Structured extraction in Python, powered by OpenAI's function calling api, designed for simplicity, transparency, and control._
---
-[Star us on Github!](https://jxnl.github.io/instructor).
+[Star us on Github!](www.github.com/jxnl/instructor).
-[](https://www.buymeacoffee.com/jxnlco)
+[](https://pydantic.dev)
[](https://pypi.python.org/pypi/instructor)
[](https://github.com/jxnl/instructor/stargazers)
[](https://jxnl.github.io/instructor)
[](https://twitter.com/jxnlco)
-[](https://github.com/jxnl/instructor/issues)
-[](https://github.com/jxnl/instructor/blob/main/LICENSE)
-[](https:github.com/jxnl/instructor/discussions)
-[](https://pypi.python.org/pypi/instructor)
-[](https://pypi.python.org/pypi/instructor)
-Built to interact solely with openai's function calling api from python. It's designed to be intuitive, easy to use, and provide great visibility into your prompts.
+Dive into the world of Python-based structured extraction, empowered by OpenAI's cutting-edge function calling API. Instructor stands out for its simplicity, transparency, and user-centric design. Whether you're a seasoned developer or just starting out, you'll find Instructor's approach intuitive and its results insightful.
+
+## Get Started in Moments
+
+Installing Instructor is a breeze. Just run `pip install instructor` in your terminal and you're on your way to a smoother data handling experience.
+
+## How Instructor Enhances Your Workflow
+
+Our `instructor.patch` for the `OpenAI` class introduces three key enhancements:
+
+- **Response Mode:** Specify a Pydantic model to streamline data extraction.
+- **Max Retries:** Set your desired number of retry attempts for requests.
+- **Validation Context:** Provide a context object for enhanced validator access.
+ A Glimpse into Instructor's Capabilities
+
+!!! note "Using Validators"
+
+ Learn more about validators checkout our blog post [Good llm validation is just good validation](https://jxnl.github.io/instructor/blog/2023/10/23/good-llm-validation-is-just-good-validation/)
+
+With Instructor, your code becomes more efficient and readable. Here’s a quick peek:
## Usage
@@ -87,27 +101,6 @@ model = await aclient.chat.completions.create(
assert isinstance(model, UserExtract)
```
-## Installation
-
-To get started you need to install it using `pip`. Run the following command in your terminal:
-
-```sh
-$ pip install instructor
-```
-
-## Quick Start
-
-To simplify your work with OpenAI we offer a patching mechanism for the `ChatCompletion` class.
-The patch introduces 3 features to the `ChatCompletion` class:
-
-1. The `response_model` parameter, which allows you to specify a Pydantic model to extract data into.
-2. The `max_retries` parameter, which allows you to specify the number of times to retry the request if it fails.
-3. The `validation_context` parameter, which allows you to specify a context object that validators have access to.
-
-!!! note "Using Validators"
-
- Learn more about validators checkout our blog post [Good llm validation is just good validation](https://jxnl.github.io/instructor/blog/2023/10/23/good-llm-validation-is-just-good-validation/)
-
### Step 1: Patch the client
First, import the required libraries and apply the patch function to the OpenAI module. This exposes new functionality with the response_model parameter.
@@ -190,9 +183,9 @@ answer
Here, the `UserDetails` model is passed as the `response_model`, and `max_retries` is set to 2.
```python
-from openai import OpenAI
import instructor
+from openai import OpenAI
from pydantic import BaseModel, field_validator
# Apply the patch to the OpenAI client
@@ -221,6 +214,25 @@ model = client.chat.completions.create(
assert model.name == "JASON"
```
+## Contributing
+
+If you want to help out checkout some of the issues marked as `good-first-issue` or `help-wanted`. Found [here](https://github.com/jxnl/instructor/labels/good%20first%20issue). They could be anything from code improvements, a guest blog post, or a new cook book.
+
## License
This project is licensed under the terms of the MIT License.
+
+# Contributors
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/blog.md b/blog.md
deleted file mode 100644
index d5c77b4..0000000
--- a/blog.md
+++ /dev/null
@@ -1,102 +0,0 @@
-# Seamless Integration with OpenAI and Pydantic: A Powerful Duo for Output Parsing
-
-Today, OpenAI introduced a Function Call API so we're going to dive into a much more structured and efficient way of handling output parsing when interacting with OpenAI. This method leverages the robustness of the Pydantic library in tandem with the recent improvements in OpenAI's API.
-
-Historically, dealing with output parsing, especially with JSON responses, has been fraught with complexities. Ensuring the extracted data adheres to a specific schema or matches certain function calls often involves writing intricate and cumbersome error-checking code. Add to this the vagaries of AI and you often end up reasking and hoping it does a better job.
-
-However, Pydantic, a Python library that provides data validation through Python type annotations, comes to the rescue! And when combined with OpenAI's new function call capabilities, it allows us to handle output parsing in a much more structured and reliable way with a much better developer experience.
-
-## The Power of Pydantic
-
-Pydantic is a Python library that brings type checking, validation, and error handling to the forefront. By making use of Python type annotations, Pydantic allows you to define data models, validate input data against these models, and receive detailed error messages when data fails validation. This ensures that your data adheres to the correct types, constraints, and formats you specify.
-
-But why Pydantic? Pydantic offers several key benefits:
-
-**Type checking:** Pydantic uses Python type annotations to ensure the data you work with adheres to the correct types. This means less time debugging type-related issues and more confidence in the integrity of your data.
-
-**Validation:** Pydantic allows you to apply additional validation rules to your data models. These could be simple constraints, like numerical ranges, or more complex custom validation functions.
-
-**Error handling:** When validation fails, Pydantic raises detailed exceptions. This gives you a clear understanding of what's gone wrong, making it easier to correct mistakes.
-
-**Ease of use:** Pydantic's data models are just Python classes. You define your data models with familiar Python type annotations, making Pydantic intuitive and easy to use.
-
-**Advanced Features:** Pydantic supports more advanced features like nested models, recursive models, and models with generics. This makes it a flexible and powerful tool for managing complex data.
-
-And when combined with the recent function call capabilities from OpenAI, it brings structured data handling to a whole new level!
-
-## Embracing OpenAI Function Calls
-
-The new function call capabilities introduced by OpenAI mark a significant shift in the way we interact with the OpenAI API. Instead of hoping that a chat message would parse correctly to JSON, we can now specify function calls and their expected inputs. This makes our conversation with the AI more structured and predictable.
-
-Here's where it gets even more interesting. By integrating Pydantic with OpenAI function calls, we can streamline the process of validating the output from OpenAI and handling it in our Python functions. This allows us to interact with the AI in a much more robust and efficient manner.
-
-Let's dive into how we can do this.
-
-## Part 1: Harnessing OpenAI Function Calls with Pydantic
-
-The crux of this approach lies in a simple decorator that handles the mapping between OpenAI function calls and Python functions. This decorator takes care of the input validation, the execution of the function, and the generation of the schema used for the OpenAI function call. Here's how it looks:
-
-```python
-@openai_function
-def sum(a:int, b:int) -> int:
- """Sum description adds a + b"""
- return a + b
-```
-
-In this example, we define a simple function that adds two numbers. We then decorate it with `@openai_function` which takes care of generating the schema for this function and validating the inputs and outputs.
-
-Once we've defined our function, we can interact with the OpenAI API as usual, using the function's schema to guide the conversation:
-
-```python
-completion = openai.ChatCompletion.create(
- model="gpt-3.5-turbo-0613",
- temperature=0,
- functions=[sum.openai_schema],
- messages=[
- {
- "role": "system",
- "content": "You must use the `sum` function instead of adding yourself.",
- },
- {
- "role": "user",
- "content": "What is 6+3 use the `sum` function",
- },
- ],
- )
-
-result = sum.from_response(completion)
-print(result) # 9
-```
-
-Here, we use sum.openai_schema to provide the schema for our function call. This ensures that the AI understands what function to call and what parameters to pass. After the completion is returned, we use sum.from_response(completion) to extract the result from the completion, validate it against our Pydantic model, and return it.
-
-## Part 2: Leveraging OpenAISchema for Data Extraction
-
-Often, we are interested in parsing the output of an OpenAI conversation to extract specific data without making an actual function call. In these cases, we can make use of our OpenAISchema class to define a schema that matches the data we want to extract. Let's look at an example:
-
-```python
-class UserDetails(OpenAISchema):
- """User Details"""
- name: str = Field(..., description="User's name")
- age: int = Field(..., description="User's age")
-
-completion = openai.ChatCompletion.create(
- model="gpt-3.5-turbo-0613",
- functions=[UserDetails.openai_schema]
- messages=[
- {"role": "system", "content": "I'm going to ask for user details. Use UserDetails to parse this data."},
- {"role": "user", "content": "My name is John Doe and I'm 30 years old."},
- ],
-)
-
-user_details = UserDetails.from_response(completion)
-print(user_details) # UserDetails(name="John Doe", age=30)
-```
-
-In this example, we define a Pydantic model that represents the data we want to extract. Then, we use UserDetails.from_response(completion) to extract and validate the data from the completion.
-
-## Light, Efficient, and Effective
-
-The key to this approach is its simplicity and efficiency. We make use of just a few lines of Python code to manage input validation, output parsing, and interaction with the OpenAI API. This code is so light that it's better to copy and paste it rather than installing a whole new package.
-
-This methodology cuts down on unnecessary abstraction, letting you stay in control and fully understand the interaction with the underlying API. It's an elegant and powerful solution for working with the OpenAI API in a structured and reliable way, proving you can have your cake and eat it too!
diff --git a/docs/blog/index.md b/docs/blog/index.md
index e69de29..5058819 100644
--- a/docs/blog/index.md
+++ b/docs/blog/index.md
@@ -0,0 +1,18 @@
+# Welcome to the Instructor Blog
+
+The goal of the blog is to capture some content that does not neatly fit within documentation or the cookbooks.
+
+## Advanced Topics
+
+- [Query Understanding and Expansion for RAG](posts/rag-and-beyond.md)
+- [GPT-4 Level summarization with GPT3.5 Finetuning](posts/chain-of-density.md)
+- [Deepdive on LLM Guardrails / Validation](posts/validation-part1.md)
+- [A Guide to Fine-Tuning and Distillation](posts/distilation-part1.md)
+
+## Learning Python
+
+- [Understanding Batch Processing with async](posts/learn-async.md)
+
+## Talks
+
+- [AI Engineering Summit 2023](posts/aisummit-2023.md)
diff --git a/docs/blog/posts/chain-of-density.md b/docs/blog/posts/chain-of-density.md
new file mode 100644
index 0000000..ab87c15
--- /dev/null
+++ b/docs/blog/posts/chain-of-density.md
@@ -0,0 +1,543 @@
+---
+draft: False
+date: 2023-11-05
+slug: chain-of-density
+tags:
+ - pydantic
+ - validation
+ - chain of density
+ - finetuneing
+ - gpt-3.5-turbo
+ - distillation
+authors:
+ - ivanleomk
+ - jxnl
+---
+
+# Smarter Summaries w/ Finetuning GPT-3.5 and Chain of Density
+
+> Discover how to distil an iterative method like Chain Of Density into a single finetuned model using Instructor
+
+In this article, we'll guide you through implementing the original Chain of Density method using Instructor, then show how to distile a GPT 3.5 model to match GPT-4's iterative summarization capabilities. Using these methods were able to decrease latency by 20x, reduce costs by 50x and maintain entity density.
+
+By the end you'll end up with a GPT 3.5 model, (fine-tuned using Instructor's great tooling), capable of producing summaries that rival the effectiveness of Chain of Density [[Adams et al. (2023)]](https://arxiv.org/abs/2309.04269). As always, all code is readily available in our `examples/chain-of-density` folder in our repo for your reference.
+
+??? abstract "Datasets and Colab Notebook"
+
+ We've also uploaded all our generated data to Hugging Face [here](https://huggingface.co/datasets/ivanleomk/gpt4-chain-of-density) for you to use if you'd like to try reproducing these experiments. We've also added a [Colab Instance](https://colab.research.google.com/drive/1iBkrEh2G5U8yh8RmI8EkWxjLq6zIIuVm?usp=sharing) for you to check our generated values.
+
+## Part 1) Chain of Density
+
+Summarizing extensive texts with AI can be challenging, often relying on inconsistent techniques. Their novel method, Chain Of Density prompting, enhances AI-based text summarization, outperforming human-generated summaries.
+
+Initially, an AI produces a summary, then refines it through multiple iterations, adding missing article entities. Each iteration adds new article entities to the summary, keeping length consistent, leading to an entity-dense, informative summary called Chain Of Density.
+
+First introduced in the paper - [From Sparse to Dense: GPT-4 Summarization with Chain of Density Prompting](https://arxiv.org/abs/2309.04269). The team has found that this method is able to consistently beats similar summaries written by human annotators.
+
+??? info "Implementation Details"
+
+ Note that our implementation uses a validator to ensure that the rewritten summary has a minimum length rather than a prompt. We also perform just 3 and not 5 rounds of rewrites, resulting in a lower final entity density.
+
+### Original Prompt
+
+We can break down the original process into smaller api calls. This allows us to introduce validation at each step to ensure that we're getting the results that we want.
+
+??? note "Original Chain of Density Prompt"
+
+ ```
+ Article: {{ARTICLE}}
+
+ You will generate increasingly concise, entity-dense summaries of the
+ above Article.
+
+ Repeat the following 2 steps 5 times.
+
+ Step 1. Identify 1-3 informative Entities (";" delimited) from the
+ Article which are missing from the previously generated summary.
+ Step 2. Write a new, denser summary of identical length which covers
+ every entity and detail from the previous summary plus the Missing
+ Entities.
+
+ A Missing Entity is:
+ - Relevant: to the main story.
+ - Specific: descriptive yet concise (5 words or fewer).
+ - Novel; not in the previous summary.
+ - Faithful: present in the Article.
+ - Anywhere: located anywhere in the Article.
+
+ Guidelines:
+ - The first summary should be long (4-5 sentences, -80 words) yet
+ highly non-specific, containing little information beyond the
+ entities marked as missing. Use overly verbose language and fillers
+ (e.g., "this article discusses") to reach -80 words.
+ - Make every word count: re-write the previous summary to improve
+ flow and make space for additional entities.
+ - Make space with fusion, compression, and removal of uninformative
+ phrases like "the article discusses"
+ - The summaries should become highly dense and concise yet
+ self-contained, e.g., easily understood without the Article.
+ - Missing entities can appear anywhere in the new summary.
+ - Never drop entities from the previous summary. If space cannot be
+ made, add fewer new entities.
+
+ Remember, use the exact same number of words for each summary.
+
+ Answer in JSON. The JSON should be a list (length 5) of dictionaries
+ whose keys are "Missing_Entities" and "Denser_Summary"
+ ```
+
+
+ 
+ Improved process with Instructor
+
+
+### Data Modelling
+
+Before we begin modelling the data, let's make sure we install all of our dependencies
+
+```
+pip install instructor aiohttp rich
+```
+
+#### Initial Summary
+
+Let's start by walking through some of the data models that we'll be using as the `response_model` for our open ai function calls
+
+Firstly, we'll need a data model for the initial summary that we will be generating. We'll take the description of this class straight from the original prompt. It's important to note that these docstrings serve a purpose, they are **directly used by the LLM when generating the outputs**.
+
+??? note "A quick note on Docstrings"
+
+ Under the hood, Instructor parses the `response_model` that you give us into a function call for OpenAI to execute. This means that the final output will be closely linked to the Pydantic model you specify.
+
+ For instance, this simple model that we later use in fine-tuning.
+
+ ```py
+ class GeneratedSummary(BaseModel):
+ """
+ This represents a highly concise summary that includes as many entities as possible from the original source article.
+
+ An Entity is a real-world object that's assigned a name - for example, a person, country a product or a book title.
+
+ Guidelines
+ - Make every word count
+ - The new summary should be highly dense and concise yet self-contained, eg., easily understood without the Article.
+ - Make space with fusion, compression, and removal of uninformative phrases like "the article discusses"
+ """
+
+ summary: str = Field(
+ ...,
+ description="This represents the final summary generated that captures the meaning of the original article which is as concise as possible. ",
+ )
+ ```
+
+ We eventually transform it into an OpenAI function call as seen below.
+
+ ```
+ {
+ "functions": [
+ {
+ "name": "GeneratedSummary",
+ "description": "This represents a highly concise summary that includes as many entities as possible from the original source article.\n\nAn Entity is a real-world object that's assigned a name - for example, a person, country a product or a book title.\n\nGuidelines\n- Make every word count\n- The new summary should be highly dense and concise yet self-contained, eg., easily understood without the Article.\n- Make space with fusion, compression, and removal of uninformative phrases like \"the article discusses\"",
+ "parameters": {
+ "properties": {
+ "summary": {
+ "description": "This represents the final summary generated that captures the meaning of the original article which is as concise as possible. ",
+ "title": "Summary",
+ "type": "string"
+ }
+ },
+ "required": [
+ "summary"
+ ],
+ "type": "object"
+ }
+ }
+ ]
+ }
+ }
+ ```
+
+ Therefore this means that the more elaborate and detailed your descriptions are, the better the outputs you will be able to get back. But we don't just stop there, since it's all Pydantic under the hood, you can validate and parse the resulting output to make sure it is **exactly what you specify**. It's all python all the way down.
+
+```py
+class InitialSummary(BaseModel):
+ """
+ This is an initial summary which should be long ( 4-5 sentences, ~80 words)
+ yet highly non-specific, containing little information beyond the entities marked as missing.
+ Use overly verbose languages and fillers (Eg. This article discusses) to reach ~80 words.
+ """
+
+ summary: str = Field(
+ ...,
+ description="This is a summary of the article provided which is overly verbose and uses fillers. It should be roughly 80 words in length",
+ )
+```
+
+#### Rewritten Summary
+
+We'll also need one additional class to help model the rewritten schema
+
+```py
+class RewrittenSummary(BaseModel):
+ """
+ This is a new, denser summary of identical length which covers every entity
+ and detail from the previous summary plus the Missing Entities.
+
+ Guidelines
+ - Make every word count : Rewrite the previous summary to improve flow and make space for additional entities
+ - Never drop entities from the previous summary. If space cannot be made, add fewer new entities.
+ - The new summary should be highly dense and concise yet self-contained, eg., easily understood without the Article.
+ - Make space with fusion, compression, and removal of uninformative phrases like "the article discusses"
+ - Missing entities can appear anywhere in the new summary
+
+ An Entity is a real-world object that's assigned a name - for example, a person, country a product or a book title.
+ """
+
+ summary: str = Field(
+ ...,
+ description="This is a new, denser summary of identical length which covers every entity and detail from the previous summary plus the Missing Entities. It should have the same length ( ~ 80 words ) as the previous summary and should be easily understood without the Article",
+ )
+ absent: List[str] = Field(
+ ...,
+ default_factory=list,
+ description="this is a list of Entities found absent from the new summary that were present in the previous summary",
+ )
+ missing: List[str] = Field(
+ default_factory=list,
+ description="This is a list of 1-3 informative Entities from the Article that are missing from the new summary which should be included in the next generated summary.",
+ )
+```
+
+!!! tip "Using Pydantic Validators with Instructor"
+
+ For a more in-depth walkthrough on how to use `Pydantic` validators with the `Instructor`
+ library, we recommend checking out our previous article on LLM
+ validation - [Good LLM Validation is just Good Validation](../posts/validation-part1.md)
+
+Ideally, we'd like for `Missing` to have a length between 1 and 3, `Absent` to be an empty list and for our rewritten summaries to keep a minimum entity density. With `Instructor`, we can implement this logic using native `Pydantic` validators that are simply declared as part of the class itself.
+
+```py hl_lines="8 40 44"
+import nltk
+import spacy
+
+nlp = spacy.load("en_core_web_sm")
+
+@field_validator("summary")
+def min_length(cls, v: str):
+ tokens = nltk.word_tokenize(v) #(1)!
+ num_tokens = len(tokens)
+ if num_tokens < 60:
+ raise ValueError(
+ "The current summary is too short. Please make sure that you generate a new summary that is around 80 words long."
+ )
+ return v
+
+@field_validator("missing")
+def has_missing_entities(cls, missing_entities: List[str]):
+ if len(missing_entities) == 0:
+ raise ValueError(
+ "You must identify 1-3 informative Entities from the Article which are missing from the previously generated summary to be used in a new summary"
+ )
+ return missing_entities
+
+@field_validator("absent")
+def has_no_absent_entities(cls, absent_entities: List[str]):
+ absent_entity_string = ",".join(absent_entities)
+ if len(absent_entities) > 0:
+ print(f"Detected absent entities of {absent_entity_string}")
+ raise ValueError(
+ f"Do not omit the following Entities {absent_entity_string} from the new summary"
+ )
+ return absent_entities
+
+@field_validator("summary")
+ def min_entity_density(cls, v: str):
+ tokens = nltk.word_tokenize(v)
+ num_tokens = len(tokens)
+
+ # Extract Entities
+ doc = nlp(v) #(2)!
+ num_entities = len(doc.ents)
+
+ density = num_entities / num_tokens
+ if density < 0.08: #(3)!
+ raise ValueError(
+ f"The summary of {v} has too few entities. Please regenerate a new summary with more new entities added to it. Remember that new entities can be added at any point of the summary."
+ )
+
+ return v
+```
+
+1. Similar to the original paper, we utilize the `NLTK` word tokenizer to count the number of tokens within our generated sentences.
+ We aim for at least 60 tokens in our generated summary so that we don't lose information.
+
+2. We also use the spaCy library to calculate the entity density of the generated summary.
+
+3. We also implement a minimum entity density so that we stay within a given range. 0.08 is arbitrarily chosen in this case
+
+### Putting it all Together
+
+Now that we have our models and the rough flow figured out, let's implement a function to summarize a piece of text using `Chain Of Density` summarization.
+
+```py hl_lines="4 9-24 38-68"
+from openai import OpenAI
+import instructor
+
+client = instructor.patch(OpenAI()) #(1)!
+
+def summarize_article(article: str, summary_steps: int = 3):
+ summary_chain = []
+ # We first generate an initial summary
+ summary: InitialSummary = client.chat.completions.create( # (2)!
+ model="gpt-4-0613",
+ response_model=InitialSummary,
+ messages=[
+ {
+ "role": "system",
+ "content": "Write a summary about the article that is long (4-5 sentences) yet highly non-specific. Use overly, verbose language and fillers(eg.,'this article discusses') to reach ~80 words",
+ },
+ {"role": "user", "content": f"Here is the Article: {article}"},
+ {
+ "role": "user",
+ "content": "The generated summary should be about 80 words.",
+ },
+ ],
+ max_retries=2,
+ )
+ prev_summary = None
+ summary_chain.append(summary.summary)
+ for i in range(summary_steps):
+ missing_entity_message = (
+ []
+ if prev_summary is None
+ else [
+ {
+ "role": "user",
+ "content": f"Please include these Missing Entities: {','.join(prev_summary.missing)}",
+ },
+ ]
+ )
+ new_summary: RewrittenSummary = client.chat.completions.create( # (3)!
+ model="gpt-4-0613",
+ messages=[
+ {
+ "role": "system",
+ "content": """
+ You are going to generate an increasingly concise,entity-dense summary of the following article.
+
+ Perform the following two tasks
+ - Identify 1-3 informative entities from the following article which is missing from the previous summary
+ - Write a new denser summary of identical length which covers every entity and detail from the previous summary plus the Missing Entities
+
+ Guidelines
+ - Make every word count: re-write the previous summary to improve flow and make space for additional entities
+ - Make space with fusion, compression, and removal of uninformative phrases like "the article discusses".
+ - The summaries should become highly dense and concise yet self-contained, e.g., easily understood without the Article.
+ - Missing entities can appear anywhere in the new summary
+ - Never drop entities from the previous summary. If space cannot be made, add fewer new entities.
+ """,
+ },
+ {"role": "user", "content": f"Here is the Article: {article}"},
+ {
+ "role": "user",
+ "content": f"Here is the previous summary: {summary_chain[-1]}",
+ },
+ *missing_entity_message,
+ ],
+ max_retries=3, #(4)!
+ max_tokens=1000,
+ response_model=RewrittenSummary,
+ )
+ summary_chain.append(new_summary.summary)
+ prev_summary = new_summary
+
+ return summary_chain
+```
+
+1. We need to apply a `patch` function on the `OpenAI` client for us to get all
+ of the benefits that `Instructor` provides. With a simple `patch`, we can get
+ **automatic type coercion of our outputs and automatic retries for invalid outputs**
+ out of the box!
+
+2. We first generate an initial summary. Note here that we explictly ask for a summary that has
+ 80 words and is lengthy with overly verbose fillers in the system prompt
+
+3. We slightly modify the original system prompt used in the original paper to perform a rewrite of the summary.
+ Using `Instructor`, we also get validation of the generated output with our `field_validator`s that we defined above
+
+4. If you've chosen a value that is larger than 0.08, make sure to increase this value in case you need to do multiple rewrites
+
+This summarization function yields a result which triples the number of entities while maintaining the same number of tokens. We can also see that stylistically, the summary is a lot more natural.
+
+**First Iteration**
+
+> This article discusses the highly-anticipated boxing match between Manny Pacquiao and Floyd Mayweather. The article revolves around Manny Pacquiao's statements about his upcoming fight and his preparations for the same. A portion of the article provides details about the financial stipulations of the match and its significance in the sporting arena. Quotes from Pacquiao illustrating his determination and his battle strategy are highlighted. The tone of the article is largely centered around creating a build-up to the upcoming mega event.
+
+**Final Iteration**
+
+> Manny Pacquiao, the Filipino boxer, anticipates the forthcoming May 2 showdown at the MGM Grand as the fight of his life, against the undefeated American Floyd Mayweather, in a $300m bout. Despite being seen as the underdog in this high-stakes Las Vegas match, Pacquiao is confident, promising a warrior's spirit and assuring the fans who have been awaiting this encounter for a decade, that it will indeed be the biggest sporting spectacle in history worthy of their anticipation
+
+## Part 2) Fine-Tuning
+
+In this section, we'll look into how to fine-tune a GPT 3.5 model so that it is able to perform at an equivalent level as a GPT-4 model. We'll then compare the performance of our model against that of `GPT-4` to see how it stacks up.
+
+### Creating a Training Set
+
+In order to prevent any contamination of data during testing, we randomly sampled 120 articles from the `griffin/chain-of-density` dataset and split these articles into a `train.csv` and a `test.csv` file which we uploaded to [Hugging Face](https://huggingface.co/datasets/ivanleomk/gpt4-chain-of-density). Now, we just neeed to import the `Instructions` module from the `Instructor` package which allows you to generate a nicely formatted `.jsonl` file to be used for fine-tuning
+
+```py hl_lines="2 9 11 13-21 40 43"
+from typing import List
+from chain_of_density import summarize_article #(1)!
+import csv
+import logging
+import instructor
+from pydantic import BaseModel
+from openai import OpenAI
+
+client = instructor.patch(OpenAI()) # (2)!
+
+logging.basicConfig(level=logging.INFO) #(3)!
+
+instructions = instructor.Instructions( #(4)!
+ name="Chain Of Density",
+ finetune_format="messages",
+ # log handler is used to save the data to a file
+ # you can imagine saving it to a database or other storage
+ # based on your needs!
+ log_handlers=[logging.FileHandler("generated.jsonl")],
+ openai_client=client,
+)
+
+class GeneratedSummary(BaseModel):
+ """
+ This represents a highly concise summary that includes as many entities as possible from the original source article.
+
+ An Entity is a real-world object that's assigned a name - for example, a person, country a product or a book title.
+
+ Guidelines
+ - Make every word count
+ - The new summary should be highly dense and concise yet self-contained, eg., easily understood without the Article.
+ - Make space with fusion, compression, and removal of uninformative phrases like "the article discusses"
+ """
+
+ summary: str = Field(
+ ...,
+ description="This represents the final summary generated that captures the meaning of the original article which is as concise as possible. ",
+ )
+
+@instructions.distil #(4)!
+def distil_summarization(text: str) -> GeneratedSummary:
+ summary_chain: List[str] = summarize_article(text)
+ return GeneratedSummary(summary=summary_chain[-1]) #(5)!
+
+with open("train.csv", "r") as file:
+ reader = csv.reader(file)
+ next(reader) # Skip the header
+ for article, summary in reader:
+ # Run Distillisation to generate the values
+ distil_summarization(article)
+```
+
+1. In this example, we're using the summarize_article that we defined up above. We saved it in a local file called `chain_of_density.py`,
+ hence the import
+
+2. We patch the default OpenAI client so that we can use the Instructor library with it
+
+3. We also need to configure logging at the `INFO` level. This is very important, if this is not configured, your output will not be generated.
+
+4. We instantiate a `Instruction` object which will help us handle the conversion of our function calls into a valid `.jsonl` file. We also define
+ the name of the `.jsonl` file in the `log_handlers` parameter
+
+5. We add in an `instructions.distil` annotation so that we automatically capture the input and output of the function we'd like to
+ fine-tune our model to output
+
+6. We return a `Pydantic` object which matches the annotation that we use on our function. Note that we must specify a `Pydantic` object to
+ be returned when using the `instructions.distil` annotation
+
+!!! warning "Rate Limiting"
+
+ We recommend running this script on a small subset of the dataset first to test you've got everything configured nicely.
+ Don't forget to add in rate limiting error handling with `tenacity` and set the `OPENAI_API_KEY` shell environment variable
+ before running any subsequent commands
+
+### Creating Fine-Tuning Jobs
+
+Once we run this script, we'll have a new file called `generated.jsonl` in our local repository. Now all that's left is to run the command below to start fine-tuning your first model!
+
+```sh
+instructor jobs create-from-file generated.jsonl
+```
+
+??? notes "Finetuning Reference"
+
+ Checking out our [Finetuning CLI](../../cli/finetune.md) to learn about other hyperparameters that you can tune to improve your model's performance.
+
+Once the job is complete, all we need to do is to then change the annotation in the function call to `distil_summarization` in our original file above to start using our new model.
+
+```py
+@instructions.distil(model='gpt-3.5-turbo:finetuned-123', mode="dispatch") #(1)!
+def distil_summarization(text: str) -> GeneratedSummary:
+ summary_chain: List[str] = summarize_article(text)
+ return GeneratedSummary(summary=summary_chain[-1])
+```
+
+1. Don't forget to replace this with your new model id. OpenAI identifies fine tuned models with an id of
+ ft:gpt-3.5-turbo-0613:personal:: under their Fine-tuning tab on their dashboard
+
+With that, you've now got your own fine-tuned model ready to go and serve data in production. We've seen how Instructor can make your life easier, from fine-tuning to distillation.
+
+## Results and Benchmarks
+
+We'll be comparing the following models in 3 ways using 20 articles that were not used for fine-tuning.
+
+- Entity Density : This is entities per token, the higher the better for density.
+- Latency : Time to last token generated in seconds
+- Costs : Total cost to generate outputs - we break down the cost into training and inference costs for easy reference
+
+`3.5 Finetuned (n)`
+
+: This is a GPT 3.5 model that we fine-tuned on `n` examples. Each model was finetuned for 4-5 epochs ( This was automatically decided by the OpenAI scheduler )
+
+`GPT-4 (COD)`
+
+: This is a GPT4 model which we applied 3 rounds of Chain Of Density rewrites to generate a summary with using the methodology above
+
+`GPT-3.5 (Vanilla)`
+
+: This is a GPT 3.5 model that we asked to generate entity-dense summaries which were concise. Summaries were generated in a single pass targetting about 80-90 tokens.
+
+| Model | Mean Latency (s) | Mean Entity Density |
+| ------------------ | ---------------- | ------------------- |
+| 3.5 Finetuned (20) | 2.1 | 0.15 |
+| 3.5 Finetuned (50) | 2.1 | 0.14 |
+| 3.5 Finetuned (76) | 2.1 | 0.14 |
+| GPT-3.5 (Vanilla) | 16.8 | 0.12 |
+| GPT-4 (COD) | 49.5 | 0.15 |
+
+??? notes "Finetuning Datasets"
+
+ For our finetuned models, we did a few optimisations to raise the performance.
+
+ We only included summaries that had a minimum density of 0.15 in the dataset, took the summary in the entire chain with the highest density as the final one, forced every regenerated summary to have a minimum density of 0.12 and regenerated summaries up to three times if they didn't meet the summaries. **This is a much more expensive strategy and can cost up to 2.5x or more what we do in this tutorial**
+
+ This resulted in the total cost of $63.46 to generate just 75 examples due to the stringent requirements, translating to about $0.85 per generated summary example.
+
+Using the OpenAI Usage Dashboard, we can calculate the cost of generating 20 summaries as seen below.
+
+| Model | Training Cost ($) | Inference Cost ($) | Tokens Used | Total Cost ($) |
+| ------------------ | ----------------- | ------------------ | ----------- | -------------- |
+| GPT-3.5 (Vanilla) | - | 0.20 | 51,162 | 0.2 |
+| 3.5 Finetuned (20) | 0.7 | 0.20 | 56,573 | 0.8 |
+| 3.5 Finetuned (50) | 1.4 | 0.17 | 49,057 | 1.3 |
+| 3.5 Finetuned (76) | 1.8 | 0.17 | 51,583 | 2.5 |
+| GPT-4 (COD) | - | 12.9 | 409,062 | 12.9 |
+
+Here, we can see that `GPT-4` has an approximate inference cost of `0.65` per summary while our finetuned models have an inference cost of `0.0091` per summary which is ~ `72x` cheaper.
+
+Interestingly, the model finetuned with the least examples seems to outperform the others. While the reason for this is unknown, a few potential reasons could be that either we didn't train for sufficient epochs ( We chose the default 5 epochs ) or that the models started learning to imitate other behaviour such as more abstract writing styles from the larger variety of samples, resulting in a decrease in entity density.
+
+## Conclusions
+
+Finetuning this iterative method was 20-40x faster while improving overall performance, resulting in massive efficiency gains by finetuning and distilling capabilities into specialized models.
+
+We've seen how `Instructor` can make your life easier, from data modeling to distillation and finetuning. If you enjoy the content or want to try out `instructor` check out the [github](https://github.com/jxnl/instructor) and don't forget to give us a star!
diff --git a/docs/blog/posts/img/chain-of-density.png b/docs/blog/posts/img/chain-of-density.png
new file mode 100644
index 0000000..75e361a
Binary files /dev/null and b/docs/blog/posts/img/chain-of-density.png differ
diff --git a/docs/blog/posts/learn-async.md b/docs/blog/posts/learn-async.md
new file mode 100644
index 0000000..1a35050
--- /dev/null
+++ b/docs/blog/posts/learn-async.md
@@ -0,0 +1,195 @@
+---
+draft: False
+date: 2023-11-13
+slug: learn-async
+tags:
+ - python
+ - batch
+ - asyncio
+ - async
+ - async/await
+authors:
+ - jxnl
+---
+
+# Introduction to Batch Processing using `asyncio` and `Instructor`
+
+Today, I will introduce you to various approaches for using asyncio in Python. We will apply this to batch process data using `instructor` and learn how to use `asyncio.gather` and `asyncio.as_completed` for concurrent data processing. Additionally, we will explore how to limit the number of concurrent requests to a server using `asyncio.Semaphore`.
+
+!!! notes "Github Example"
+
+ If you want to run the code examples in this article, you can find them on [jxnl/instructor](https://github.com/jxnl/instructor/blob/main/examples/learn-async/run.py)
+
+We will start by defining an `async` function that calls `openai` to extract data, and then examine four different ways to execute it. We will discuss the pros and cons of each approach and analyze the results of running them on a small batch.
+
+## Understanding `asyncio`
+
+`asyncio` is a Python library that enables writing concurrent code using the async/await syntax. It is particularly useful for IO-bound and structured network code. If you are familiar with OpenAI's SDK, you might have encountered two classes: `OpenAI()` and `AsyncOpenAI()`. Today, we will be using the `AsyncOpenAI()` class, which processes data asynchronously.
+
+By utilizing these tools in web applications or batch processing, we can significantly improve performance by handling multiple requests concurrently instead of sequentially.
+
+### Understanding `async` and `await`
+
+We will be using the `async` and `await` keywords to define asynchronous functions. The `async` keyword is used to define a function that returns a coroutine object. The `await` keyword is used to wait for the result of a coroutine object.
+
+If you want to understand the deeper details of `asyncio`, I recommend reading [this article](https://realpython.com/async-io-python/) by Real Python.
+
+### Understanding `gather` vs `as_completed`
+
+In this post we'll show two ways to run tasks concurrently: `asyncio.gather` and `asyncio.as_completed`. The `gather` method is used to run multiple tasks concurrently and return the results as a `list`. The `as_completed` returns a `iterable` is used to run multiple tasks concurrently and return the results as they complete. Another great resource on the differences between the two can be found [here](https://medium.com/dev-bits/a-minimalistic-guide-for-understanding-asyncio-in-python-52c436c244ea).
+
+## Example: Batch Processing
+
+In this example, we will demonstrate how to use `asyncio` for batch processing tasks, specifically for extracting and processing data concurrently. The script will extract data from a list of texts and process it concurrently using `asyncio`.
+
+```python
+import instructor
+from pydantic import BaseModel
+from openai import AsyncOpenAI
+
+# Enables `response_model` in `create` method
+client = instructor.apatch(AsyncOpenAI()) # (1)!
+
+class Person(BaseModel):
+ name: str
+ age: int
+
+
+async def extract_person(text: str) -> Person:
+ return await client.chat.completions.create( # (2)!
+ model="gpt-3.5-turbo",
+ messages=[
+ {"role": "user", "content": text},
+ ],
+ response_model=Person,
+ )
+```
+
+1. We use `instructor.apatch` to patch the `create` method of `AsyncOpenAI` to accept a `response_model` argument. This is because the `create` method of `AsyncOpenAI` does not accept a `response_model` argument without this patch.
+2. We use `await` here to wait for the response from the server before we return the result. This is because `create` returns a coroutine object, not the result of the coroutine.
+
+Notice that now there are `async` and `await` keywords in the function definition. This is because we're using the `asyncio` library to run the function concurrently. Now lets define a batch of texts to process.
+
+```python
+dataset = [
+ "My name is John and I am 20 years old",
+ "My name is Mary and I am 21 years old",
+ "My name is Bob and I am 22 years old",
+ "My name is Alice and I am 23 years old",
+ "My name is Jane and I am 24 years old",
+ "My name is Joe and I am 25 years old",
+ "My name is Jill and I am 26 years old",
+ ]
+```
+
+### **`for loop`**: Running tasks sequentially.
+
+```python hl_lines="3"
+persons = []
+for text in dataset:
+ person = await extract_person(text)
+ persons.append(person)
+```
+
+Even though there is an `await` keyword, we still have to wait for each task to finish before starting the next one. This is because we're using a `for` loop to iterate over the dataset. This method, which uses a `for` loop, will be the slowest among the four methods discussed today.
+
+### **`asyncio.gather`**: Running tasks concurrently.
+
+```python hl_lines="3"
+async def gather():
+ tasks_get_persons = [extract_person(text) for text in dataset]
+ all_persons = await asyncio.gather(*tasks_get_persons) # (1)!
+```
+
+1. We use `await` here to wait for all the tasks to finish before assigning the result to `all_persons`. This is because `asyncio.gather` returns a coroutine object, not the result of the coroutine. Alternatively, we can use `asyncio.as_completed` to achieve the same result.
+
+Using `asyncio.gather` allows us to return all the results at once. It is an effective way to speed up our code, but it's not the only way. Particularly, if we have a large dataset, we might not want to wait for everything to finish before starting to process the results. This is where `asyncio.as_completed` comes into play.
+
+### **`asyncio.as_completed`**: Handling tasks as they complete.
+
+```python hl_lines="5 4"
+async def as_completed():
+ all_persons = []
+ tasks_get_persons = [extract_person(text) for text in dataset]
+ for person in asyncio.as_completed(tasks_get_persons):
+ all_persons.append(await person) # (1)!
+```
+
+1. We use `await` here to wait for each task to complete before appending it to the list. This is because `as_completed` returns a coroutine object, not the result of the coroutine. Alternatively, we can use `asyncio.gather` to achieve the same result.
+
+This method is a great way to handle large datasets. We can start processing the results as they come in, especially if we are streaming data back to a client.
+
+However, these methods aim to complete as many tasks as possible as quickly as possible. This can be problematic if we want to be considerate to the server we're making requests to. This is where rate limiting comes into play. While there are libraries available to assist with rate limiting, for our initial defense, we will use a semaphore to limit the number of concurrent requests we make.
+
+!!! note "Ordering of results"
+
+ Its important to note that the order of the results will not be the same as the order of the dataset. This is because the tasks are completed in the order they finish, not the order they were started. If you need to preserve the order of the results, you can use `asyncio.gather` instead.
+
+### **Rate-Limited Gather**: Using semaphores to limit concurrency.
+
+```python hl_lines="4 8 9"
+sem = asyncio.Semaphore(2)
+
+async def rate_limited_extract_person(text: str, sem: Semaphore) -> Person:
+ async with sem: # (1)!
+ return await extract_person(text)
+
+async def rate_limited_gather(sem: Semaphore):
+ tasks_get_persons = [rate_limited_extract_person(text, sem) for text in dataset]
+ resp = await asyncio.gather(*tasks_get_persons)
+```
+
+1. We use a semaphore to limit the number of concurrent requests to 2. This approach strikes a balance between speed and being considerate to the server we're making requests to.
+
+### **Rate-Limited As Completed**: Using semaphores to limit concurrency.
+
+```python hl_lines="4 9 10"
+sem = asyncio.Semaphore(2)
+
+async def rate_limited_extract_person(text: str, sem: Semaphore) -> Person:
+ async with sem: # (1)!
+ return await extract_person(text)
+
+async def rate_limited_as_completed(sem: Semaphore):
+ all_persons = []
+ tasks_get_persons = [rate_limited_extract_person(text, sem) for text in dataset]
+ for person in asyncio.as_completed(tasks_get_persons):
+ all_persons.append(await person) # (2)!
+```
+
+1. We use a semaphore to limit the number of concurrent requests to 2. This approach strikes a balance between speed and being considerate to the server we're making requests to.
+
+2. We use `await` here to wait for each task to complete before appending it to the list. This is because `as_completed` returns a coroutine object, not the result of the coroutine. Alternatively, we can use `asyncio.gather` to achieve the same result.
+
+Now that we have seen the code, let's examine the results of processing 7 texts. As the prompts become longer or if we use GPT-4, the differences between these methods will become more pronounced.
+
+!!! note "Other Options"
+
+ Its important to also note that here we are using a `semaphore` to limit the number of concurrent requests. However, there are other ways to limit concurrency esp since we have rate limit information from the `openai` request. You can imagine using a library like `ratelimit` to limit the number of requests per second. OR catching rate limit exceptions and using `tenacity` to retry the request after a certain amount of time.
+
+ - [tenacity](https://pypi.org/project/tenacity/)
+ - [aiolimiter](https://pypi.org/project/aiolimiter/)
+
+## Results
+
+As you can see, the `for` loop is the slowest, while `asyncio.as_completed` and `asyncio.gather` are the fastest without any rate limiting.
+
+| Method | Execution Time | Rate Limited (Semaphore) |
+| -------------------- | -------------- | ------------------------ |
+| For Loop | 6.17 seconds | |
+| Asyncio.gather | 0.85 seconds | |
+| Asyncio.as_completed | 0.95 seconds | |
+| Asyncio.gather | 3.04 seconds | 2 |
+| Asyncio.as_completed | 3.26 seconds | 2 |
+
+## Practical implications of batch processing
+
+The choice of approach depends on the task's nature and the desired balance between speed and resource utilization.
+
+Here are some guidelines to consider:
+
+- Use `asyncio.gather` for handling multiple independent tasks quickly.
+- Apply `asyncio.as_completed` for large datasets to process tasks as they complete.
+- Implement rate-limiting to avoid overwhelming servers or API endpoints.
+
+If you find the content helpful or want to try out `Instructor`, please visit our [GitHub](https://github.com/jxnl/instructor) page and give us a star!
diff --git a/docs/distillation.md b/docs/concepts/distillation.md
similarity index 100%
rename from docs/distillation.md
rename to docs/concepts/distillation.md
diff --git a/docs/concepts/maybe.md b/docs/concepts/maybe.md
new file mode 100644
index 0000000..183becf
--- /dev/null
+++ b/docs/concepts/maybe.md
@@ -0,0 +1,108 @@
+# Handling Missing Data with `Maybe`
+
+In this post, we will demonstrate how to use the `Maybe` pattern to manage missing data and employ pattern matching to handle errors in a structured manner.
+
+## What is `Maybe`?
+
+The `Maybe` pattern is a concept in functional programming used for error handling. Instead of raising exceptions or returning `None`, you can use a `Maybe` type to encapsulate both the result and potential errors. This pattern is particularly useful when making OpenAI API calls, as providing language models with an escape mechanism effectively reduces hallucinations. Consequently, we can construct a prompt that closely resembles regular programming.
+
+Towards the end, we will demonstrate how to use `Maybe` instances in pattern matching, which offers an excellent approach for handling errors in a structured manner.
+
+## Defining the Model
+
+Using Pydantic, we'll first define the `UserDetail` and `MaybeUser` classes.
+
+```python
+from pydantic import BaseModel, Field, Optional
+
+class UserDetail(BaseModel):
+ age: int
+ name: str
+ role: Optional[str] = Field(default=None)
+
+class MaybeUser(BaseModel):
+ result: Optional[UserDetail] = Field(default=None)
+ error: bool = Field(default=False)
+ message: Optional[str] = Field(default=None)
+
+ def __bool__(self):
+ return self.result is not None
+```
+
+Notice that `MaybeUser` has a `result` field that is an optional `UserDetail` instance where the extracted data will be stored. The `error` field is a boolean that indicates whether an error occurred, and the `message` field is an optional string that contains the error message.
+
+## Defining the function
+
+Once we have the model defined, we can create a function that uses the `Maybe` pattern to extract the data.
+
+```python
+import random
+import instructor
+from openai import OpenAI
+from typing import Optional
+
+# This enables the `response_model` keyword
+client = instructor.patch(OpenAI())
+
+def extract(content: str) -> MaybeUser:
+ return openai.chat.completions.create(
+ model="gpt-3.5-turbo",
+ response_model=MaybeUser,
+ messages=[
+ {"role": "user", "content": f"Extract `{content}`"},
+ ],
+ )
+
+user1 = extract("Jason is a 25-year-old scientist")
+# output:
+{
+ "result": {
+ "age": 25,
+ "name": "Jason",
+ "role": "scientist"
+ },
+ "error": false,
+ "message": null
+}
+
+user2 = extract("Unknown user")
+# output:
+{
+ "result": null,
+ "error": true,
+ "message": "User not found"
+}
+```
+
+As you can see, when the data is extracted successfully, the `result` field contains the `UserDetail` instance. When an error occurs, the `error` field is set to `True`, and the `message` field contains the error message.
+
+## Handle the result
+
+There are a few ways we can handle the result. Normally, we can just access the individual fields.
+
+```python
+def process_user_detail(maybe_user: MaybeUser):
+ if not maybe_user.error:
+ user = maybe_user.result
+ print(f"User {user.name} is {user.age} years old")
+ else:
+ print(f"Not found: {user1.message}")
+```
+
+## Pattern Matching
+
+We can also use pattern matching to handle the result. This is a great way to handle errors in a structured way.
+
+```python
+def process_user_detail(maybe_user: MaybeUser):
+ match maybe_user:
+ case MaybeUser(error=True, message=msg):
+ print(f"Error: {msg}")
+ case MaybeUser(result=user_detail) if user_detail:
+ assert isinstance(user_detail, UserDetail)
+ print(f"User {user_detail.name} is {user_detail.age} years old")
+ case _:
+ print("Unknown error")
+```
+
+If you want to learn more about pattern matching, check out Pydantic's docs on [Structural Pattern Matching](https://docs.pydantic.dev/latest/concepts/models/#structural-pattern-matching)
diff --git a/docs/multitask.md b/docs/concepts/multitask.md
similarity index 98%
rename from docs/multitask.md
rename to docs/concepts/multitask.md
index 9fcca73..9eda36c 100644
--- a/docs/multitask.md
+++ b/docs/concepts/multitask.md
@@ -22,18 +22,16 @@ Defining a task and creating a list of classes is a common enough pattern that w
By using multitask you get a very convient class with prompts and names automatically defined. You get `from_response` just like any other `BaseModel` you're able to extract the list of objects data you want with `MultTask.tasks`.
-```python hl_lines="13"
+```python
import instructor
from openai import OpenAI
client = instructor.patch(OpenAI())
-
class User(BaseModel):
name: str
age: int
-
MultiUser = instructor.MultiTask(User)
completion = client.chat.completions.create(
@@ -70,7 +68,7 @@ Lets look at an example in action with the same class
MultiUser = instructor.MultiTask(User)
completion = client.chat.completions.create(
- model="gpt-4-0613",
+ model="gpt-4",
temperature=0.1,
stream=True,
response_model=MultiUser,
diff --git a/docs/concepts/philosophy.md b/docs/concepts/philosophy.md
new file mode 100644
index 0000000..2ebf685
--- /dev/null
+++ b/docs/concepts/philosophy.md
@@ -0,0 +1,35 @@
+# Philosophy
+
+The instructor values [simplicity](https://eugeneyan.com/writing/simplicity/) and flexibility in leveraging language models (LLMs). It offers a streamlined approach for structured output, avoiding unnecessary dependencies or complex abstractions. Let [Pydantic](https://docs.pydantic.dev/latest/) do the heavy lifting.
+
+> “Simplicity is a great virtue but it requires hard work to achieve it and education to appreciate it. And to make matters worse: complexity sells better.” — Edsger Dijkstra
+
+## The Bridge to Object-Oriented Programming
+
+`instructor` acts as a bridge converting text-based LLM interactions into a familiar object-oriented format. Its integration with Pydantic provides type hints, runtime validation, and robust IDE support; love and supported by many in the Python ecosystem. By treating LLMs as callable functions returning typed objects, instructor makes [language models backwards compatible with code](https://www.youtube.com/watch?v=yj-wSRJwrrc), making them practical for everyday use while being complex enough for advanced applications.
+
+## The zen of `instructor`
+
+Maintain the flexibility and power of Python, without unnecessary constraints.
+
+Begin with a function and a return type hint – simplicity is key. With my experience maintaining a large enterprize framework at my previous job over many years I've learned that the goal of a making a useful framework is minimizing regret, both for the author and hopefully for the user.
+
+1. Define a Schema `#!python class StructuredData(BaseModel):`
+2. Define validators and methods on your schema.
+3. Encapsulate all your LLM logic into a function `#!python def extract(a) -> StructuredData:`
+4. Define typed computations against your data with `#!python def compute(data: StructuredData):` or call methods on your schema `#!python data.compute()`
+
+It should be that simple.
+
+## My Goals
+
+The goal for the library, [documentation](https://jxnl.github.io/instructor/), and [blog](https://jxnl.github.io/instructor/blog/), is to help you be a better python programmer and as a result a better AI engineer.
+
+- The library is a result of my desire for simplicity.
+- The library should help maintain simplicity in your codebase.
+- I won't try to write prompts for you,
+- I don't try to create indirections or abstractions that make it hard to debug in the future
+
+Please note that the library is designed to be adaptable and open-ended, allowing you to customize and extend its functionality based on your specific requirements. If you have any further questions or ideas hit me up on [twitter](https://twitter.com/jxnlco)
+
+Cheers!
diff --git a/docs/tips/index.md b/docs/concepts/prompting.md
similarity index 98%
rename from docs/tips/index.md
rename to docs/concepts/prompting.md
index 97cdbd6..1b61275 100644
--- a/docs/tips/index.md
+++ b/docs/concepts/prompting.md
@@ -1,6 +1,4 @@
-# Prompt Engineering for Function Calling
-
-The overarching theme of using instructor and pydantic for function calling is to make the models as self-descriptive, modular, and flexible as possible, while maintaining data integrity and ease of use.
+The overarching theme of using Instructor and Pydantic for function calling is to make the models as self-descriptive, modular, and flexible as possible, while maintaining data integrity and ease of use.
- **Modularity**: Design self-contained components for reuse.
- **Self-Description**: Use Pydantic's `Field` for clear field descriptions.
@@ -39,7 +37,6 @@ class UserDetail(BaseModel):
age: int
name: str
role: Optional[str] = Field(default=None)
-
```
## Handling Errors Within Function Calls
@@ -121,7 +118,6 @@ class UserDetail(BaseModel):
age: int
name: str
role: Role
-
```
## Handle Arbitrary Properties
@@ -139,7 +135,6 @@ class UserDetail(BaseModel):
age: int
name: str
properties: List[Property] = Field(..., description="Extract any other properties that might be relevant.")
-
```
## Limiting the Length of Lists
diff --git a/docs/reask_validation.md b/docs/concepts/reask_validation.md
similarity index 73%
rename from docs/reask_validation.md
rename to docs/concepts/reask_validation.md
index ee6c22d..67f0018 100644
--- a/docs/reask_validation.md
+++ b/docs/concepts/reask_validation.md
@@ -1,30 +1,18 @@
-# Validation and Reask with LLMs and Pydantic
+# Validation and Reasking
-Instead of framing "self-critique" or "self-reflection" in AI as new concepts, we can view them as validation errors with clear error messages that the systen can use to self heal.
+Instead of framing "self-critique" or "self-reflection" in AI as new concepts, we can view them as validation errors with clear error messages that the systen can use to self correct.
-## Pythonic Validation with Pydantic and Instructor
+## Pydantic
-1. **Uniform Validation API**: Pydantic provides identical developer experience, whether using code-based or LLM-based validation.
-2. **Reasking Mechanism**: Pydantic accumulates validation errors for a one-step reasking process.
-3. **Prompt Chaining via Error Messages**: Instructor utilizes validation error messages to refine LLM outputs without and new abstractions.
+Pydantic offers an customizable and expressive validation framework for Python. Instructor leverages Pydantic's validation framework to provide a uniform developer experience for both code-based and LLM-based validation, as well as a reasking mechanism for correcting LLM outputs based on validation errors. To learn more check out the [Pydantic docs](https://docs.pydantic.dev/latest/concepts/validators/) on validators.
-## Uniform Validation: Code-Based vs. LLM
+!!! note "Good llm validation is just good validation"
-Validation is crucial when using Large Language Models (LLMs) for data extraction. It ensures data integrity, ensuring both quantitative and qualititave correctness with code and llm validations.
-
-!!! note "Pydantic Validation Docs"
-
- Pydantic supports validation individual fields or the whole model dict all at once.
-
- - [Field-Level Validation](https://docs.pydantic.dev/latest/usage/validators/)
- - [Model-Level Validation](https://docs.pydantic.dev/latest/usage/validators/#model-validators)
-
- To see the most up to date examples check out our repo [jxnl/instructor/examples/validators](https://github.com/jxnl/instructor/tree/main/examples/validators)
+ If you want to see some more examples on validators checkout our blog post [Good llm validation is just good validation](https://jxnl.github.io/instructor/blog/2023/10/23/good-llm-validation-is-just-good-validation/)
### Code-Based Validation Example
-!!! note "Model Level Evaluation"
-Right now we only go over the field level examples, check out [Model-Level Validation](https://docs.pydantic.dev/latest/usage/validators/#model-validators) if you want to see how to do model level evaluation
+First define a Pydantic model with a validator using the `Annotation` class from `typing_extensions`.
Enforce a naming rule using Pydantic's built-in validation:
@@ -56,20 +44,29 @@ name
Value error, name must contain a space (type=value_error)
```
+As we can see, Pydantic raises a validation error when the name attribute does not contain a space. This is a simple example, but it demonstrates how Pydantic can be used to validate attributes of a model.
+
### LLM-Based Validation Example
LLM-based validation can also be plugged into the same Pydantic model. Here, if the answer attribute contains content that violates the rule "don't say objectionable things," Pydantic will raise a validation error.
```python hl_lines="9 15"
+import instructor
+
+from openai import OpenAI
+from instructor import llm_validator
from pydantic import BaseModel, ValidationError, BeforeValidator
from typing_extensions import Annotated
-from instruct import llm_validator
+
+# Apply the patch to the OpenAI client
+client = instructor.patch(OpenAI())
+
class QuestionAnswer(BaseModel):
question: str
answer: Annotated[
str,
- BeforeValidator(llm_validator("don't say objectionable things"))
+ BeforeValidator(llm_validator("don't say objectionable things", openai_client=client))
]
try:
@@ -148,7 +145,7 @@ Behind the scenes, the `instructor.patch()` method adds a `max_retries` paramete
try:
...
except (ValidationError, JSONDecodeError) as e:
- kwargs["messages"].append(dict(**response.choices[0].message))
+ kwargs["messages"].append(response.choices[0].message)
kwargs["messages"].append(
{
"role": "user",
@@ -159,7 +156,7 @@ except (ValidationError, JSONDecodeError) as e:
## Advanced Validation Techniques
-The docs are currently incomplete, but we have a few advanced validation techniques that we're working on documenting better, for a example of model level validation, and using a validation context check out our example on [verifying citations](examples/exact_citations.md) which covers
+The docs are currently incomplete, but we have a few advanced validation techniques that we're working on documenting better, for a example of model level validation, and using a validation context check out our example on [verifying citations](../examples/exact_citations.md) which covers
1. Validate the entire object with all attributes rather than one attribute at a time
2. Using some 'context' to validate the object, in this case we use the `context` to check if the citation existed in the original text.
diff --git a/docs/contributing.md b/docs/contributing.md
new file mode 100644
index 0000000..66ca952
--- /dev/null
+++ b/docs/contributing.md
@@ -0,0 +1,34 @@
+We would love for you to contribute to `Instructor`.
+
+## Issues
+
+If you find a bug, please file an issue on [our issue tracker on GitHub](https://github.com/jxnl/instructor/issues).
+
+To help us reproduce the bug, please provide a minimal reproducible example, including a code snippet and the full error message.
+
+1. The `response_model` you are using.
+2. The `messages` you are using.
+3. The `model` you are using.
+
+## Pull Requests
+
+We welcome pull requests! There is plenty to do, and we are happy to discuss any contributions you would like to make.
+
+If it is not a small change, please start by [filing an issue](https://github.com/jxnl/instructor/issues) first.
+
+If you need ideas, you can check out the [help wanted](https://github.com/jxnl/instructor/labels/help%20wanted) or [good first issue](https://github.com/jxnl/instructor/labels/good%20first%20issue) labels.
+
+# Contributors
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/docs/examples/autodataframe.md b/docs/examples/autodataframe.md
index 1182fae..2607c75 100644
--- a/docs/examples/autodataframe.md
+++ b/docs/examples/autodataframe.md
@@ -81,7 +81,7 @@ def dataframe(data: str) -> Database:
return client.chat.completions.create(
model="gpt-4-0613",
temperature=0.1,
- response_model=Database
+ response_model=Database,
messages=[
{
"role": "system",
diff --git a/docs/examples/index.md b/docs/examples/index.md
index 79eff7d..971bcec 100644
--- a/docs/examples/index.md
+++ b/docs/examples/index.md
@@ -18,13 +18,12 @@
- [Working with Recursive Schemas](recursive.md): Implement and understand recursive schemas.
-
- [Table Extraction from Text](autodataframe.md): Extract tables, potentially multiple, automatically from textual data.
-- [Multi-File Code Generation](gpt-engineer.md): Generate multi-file programs with contents and paths.
+- [Multi-File Code Generation](gpt-engineer.md): Generate multi-file programs with contents and paths.
- [PII Data Sanitization](pii.md): Extract and sanitize Personally Identifiable Information (PII) from documents.
- [Action Item and Dependency Mapping](action_items.md): Generate action items and their dependencies from transcripts.
-Happy exploring!
\ No newline at end of file
+Happy exploring!
diff --git a/docs/help.md b/docs/help.md
new file mode 100644
index 0000000..7aec1e9
--- /dev/null
+++ b/docs/help.md
@@ -0,0 +1,27 @@
+# Getting help with Instructor
+
+If you need help getting started with Instructor or with advanced usage, the following sources may be useful.
+
+## :material-creation: Concepts
+
+The [concepts](concepts/prompting.md) section explains the core concepts of Instructor and how to prompt with models.
+
+## :material-chef-hat: Cookbooks
+
+The [cookbooks](examples/index.md) are a great place to start. They contain a variety of examples that demonstrate how to use Instructor in different scenarios.
+
+## :material-book: Blog
+
+The [blog](blog/index.md) contains articles that explain how to use Instructor in different scenarios.
+
+## :material-github: GitHub Discussions
+
+[GitHub discussions](https://github.com/jxnl/instructor/discussions) are useful for asking questions, your question and the answer will help everyone.
+
+## :material-github: GitHub Issues
+
+[GitHub issues](https://github.com/jxnl/instructor/issues) are useful for reporting bugs or requesting new features.
+
+## :material-twitter: Twitter
+
+You can also reach out to me on [Twitter](https://twitter.com/jxnlco) if you have any questions or ideas.
diff --git a/docs/index.md b/docs/index.md
index d73cae8..6e5484f 100644
--- a/docs/index.md
+++ b/docs/index.md
@@ -1,31 +1,26 @@
-# Getting Started with Instructor
+# Instructor
_Structured extraction in Python, powered by OpenAI's function calling api, designed for simplicity, transparency, and control._
---
-[Star us on Github!](https://jxnl.github.io/instructor).
-
-[](https://www.buymeacoffee.com/jxnlco)
-[](https://pypi.python.org/pypi/instructor)
-[](https://github.com/jxnl/instructor/stargazers)
-[](https://jxnl.github.io/instructor)
+[](https://pydantic.dev)
[](https://twitter.com/jxnlco)
+[](https://pypi.python.org/pypi/instructor)
+[](https://jxnl.github.io/instructor)
[](https://github.com/jxnl/instructor/issues)
-[](https://github.com/jxnl/instructor/blob/main/LICENSE)
-[](https:github.com/jxnl/instructor/discussions)
-[](https://pypi.python.org/pypi/instructor)
-[](https://pypi.python.org/pypi/instructor)
-Built to interact solely with openai's function calling api from python. It's designed to be intuitive, easy to use, and provide great visibility into your prompts.
+Dive into the world of Python-based structured extraction, by OpenAI's function calling API and Pydantic, the most widely used data validation library for Python. Instructor stands out for its simplicity, transparency, and user-centric design. Whether you're a seasoned developer or just starting out, you'll find Instructor's approach intuitive and steerable.
## Usage
```py hl_lines="5 13"
-from openai import OpenAI
import instructor
+from openai import OpenAI
+from pydantic import BaseModel
-# Enables `response_model`
+# This enables response_model keyword
+# from client.chat.completions.create
client = instructor.patch(OpenAI())
class UserDetail(BaseModel):
@@ -45,111 +40,29 @@ assert user.name == "Jason"
assert user.age == 25
```
-!!! warning "Using `openai<1.0.0`"
+**Using async clients**
- If you're using `openai<1.0.0` then make sure you `pip install instructor<0.3.0`
- where you can patch a global client like so:
+For async clients you must use apatch vs patch like so:
- ```python hl_lines="4 8"
- import openai
- import instructor
-
- instructor.patch()
-
- user = openai.ChatCompletion.create(
- ...,
- response_model=UserDetail,
- )
- ```
-
-!!! note "Using async clients"
-
- For async clients you must use apatch vs patch like so:
-
- ```py
- import instructor
- from openai import AsyncOpenAI
-
- aclient = instructor.apatch(AsyncOpenAI())
-
- class UserExtract(BaseModel):
- name: str
- age: int
-
- model = await aclient.chat.completions.create(
- model="gpt-3.5-turbo",
- response_model=UserExtract,
- messages=[
- {"role": "user", "content": "Extract jason is 25 years old"},
- ],
- )
-
- assert isinstance(model, UserExtract)
- ```
-
-## Installation
-
-To get started you need to install it using `pip`. Run the following command in your terminal:
-
-```sh
-$ pip install instructor
-```
-
-## Quick Start
-
-To simplify your work with OpenAI we offer a patching mechanism for the `ChatCompletion` class.
-The patch introduces 3 features to the `ChatCompletion` class:
-
-1. The `response_model` parameter, which allows you to specify a Pydantic model to extract data into.
-2. The `max_retries` parameter, which allows you to specify the number of times to retry the request if it fails.
-3. The `validation_context` parameter, which allows you to specify a context object that validators have access to.
-
-!!! note "Using Validators"
-
- Learn more about validators checkout our blog post [Good llm validation is just good validation](https://jxnl.github.io/instructor/blog/2023/10/23/good-llm-validation-is-just-good-validation/)
-
-### Step 1: Patch the client
-
-First, import the required libraries and apply the patch function to the OpenAI module. This exposes new functionality with the response_model parameter.
-
-```python
+```py
import instructor
-from openai import OpenAI
-from pydantic import BaseModel
+from openai import AsyncOpenAI
-# This enables response_model keyword
-# from client.chat.completions.create
-client = instructor.patch(OpenAI())
-```
+aclient = instructor.apatch(AsyncOpenAI())
-### Step 2: Define the Pydantic Model
-
-Create a Pydantic model to define the structure of the data you want to extract. This model will map directly to the information in the prompt.
-
-```python
-from pydantic import BaseModel
-
-class UserDetail(BaseModel):
+class UserExtract(BaseModel):
name: str
age: int
-```
-### Step 3: Extract
-
-Use the `client.chat.completions.create` method to send a prompt and extract the data into the Pydantic object. The response_model parameter specifies the Pydantic model to use for extraction. Its helpful to annotate the variable with the type of the response model.
-which will help your IDE provide autocomplete and spell check.
-
-```python
-user: UserDetail = client.chat.completions.create(
+model = await aclient.chat.completions.create(
model="gpt-3.5-turbo",
- response_model=UserDetail,
+ response_model=UserExtract,
messages=[
- {"role": "user", "content": "Extract Jason is 25 years old"},
- ]
+ {"role": "user", "content": "Extract jason is 25 years old"},
+ ],
)
-assert user.name == "Jason"
-assert user.age == 25
+assert isinstance(model, UserExtract)
```
!!! note "Accessing the original response"
@@ -170,74 +83,29 @@ assert user.age == 25
assert isinstance(user._raw_response, ChatCompletion)
```
-## Pydantic Validation
+## Why use Instructor?
-Validation can also be plugged into the same Pydantic model. Here, if the answer attribute contains content that violates the rule "don't say objectionable things," Pydantic will raise a validation error.
+The question of using Instructor is fundamentally a question of why to use Pydantic.
-```python hl_lines="9 15"
-from pydantic import BaseModel, ValidationError, BeforeValidator
-from typing_extensions import Annotated
-from instructor import llm_validator
+1. **Powered by type hints** — Instructor is powered by Pydantic, which is powered by type hints. Schema validation, prompting is controleld by type annotations; less to learn, less code ot write,and integrates with your IDE.
-class QuestionAnswer(BaseModel):
- question: str
- answer: Annotated[
- str,
- BeforeValidator(llm_validator("don't say objectionable things"))
- ]
+2. **Powered by OpenAI** — Instructor is powered by OpenAI's function calling API. This means you can use the same API for both prompting and extraction.
-try:
- qa = QuestionAnswer(
- question="What is the meaning of life?",
- answer="The meaning of life is to be evil and steal",
- )
-except ValidationError as e:
- print(e)
-```
+3. **Customizable** — Pydantic is highly customizable. You can define your own validators, custom error messages, and more.
-Its important to not here that the error message is generated by the LLM, not the code, so it'll be helpful for re asking the model.
+4. **Ecosystem** Pydantic is the most widely used data validation library for Python. It's used by FastAPI, Typer, and many other popular libraries.
-```plaintext
-1 validation error for QuestionAnswer
-answer
- Assertion failed, The statement is objectionable. (type=assertion_error)
-```
+5. **Battle Tested** — Pydantic is downloaded over 100M times per month, and supported by a large community of contributors.
-## Reask on validation error
+## More Examples
-Here, the `UserDetails` model is passed as the `response_model`, and `max_retries` is set to 2.
+If you'd like to see more check out our [cookbook](examples/index.md).
-```python
-from openai import OpenAI
-import instructor
+[Installing Instructor](installation.md) is a breeze. Just run `pip install instructor`.
-from pydantic import BaseModel, field_validator
+## Contributing
-# Apply the patch to the OpenAI client
-client = instructor.patch(OpenAI())
-
-class UserDetails(BaseModel):
- name: str
- age: int
-
- @field_validator("name")
- @classmethod
- def validate_name(cls, v):
- if v.upper() != v:
- raise ValueError("Name must be in uppercase.")
- return v
-
-model = client.chat.completions.create(
- model="gpt-3.5-turbo",
- response_model=UserDetails,
- max_retries=2,
- messages=[
- {"role": "user", "content": "Extract jason is 25 years old"},
- ],
-)
-
-assert model.name == "JASON"
-```
+If you want to help out checkout some of the issues marked as `good-first-issue` or `help-wanted`. Found [here](https://github.com/jxnl/instructor/labels/good%20first%20issue). They could be anything from code improvements, a guest blog post, or a new cook book.
## License
diff --git a/docs/installation.md b/docs/installation.md
new file mode 100644
index 0000000..8046b82
--- /dev/null
+++ b/docs/installation.md
@@ -0,0 +1,14 @@
+Installation is as simple as:
+
+```bash
+pip install instructor
+```
+
+Instructor has a few dependencies:
+
+- [`openai`](https://pypi.org/project/openai/): OpenAI's Python client.
+- [`typer`](https://pypi.org/project/typer/): Build great CLIs. Easy to code. Based on Python type hints.
+- [`docstring-parser`](https://pypi.org/project/docstring-parser/): A parser for Python docstrings, to improve the experience of working with docstrings in jsonschema.
+- [`pydantic`](https://pypi.org/project/pydantic/): Data validation and settings management using python type annotations.
+
+If you've got Python 3.9+ and `pip` installed, you're good to go.
diff --git a/docs/maybe.md b/docs/maybe.md
deleted file mode 100644
index 2fbb7bf..0000000
--- a/docs/maybe.md
+++ /dev/null
@@ -1,73 +0,0 @@
-# Error Handling Using Maybe Pattern
-
-## Introduction
-
-The `Maybe` pattern is a functional programming concept used for error handling. Instead of raising exceptions or returning `None`, you can use a `Maybe` type to encapsulate both the result and possible errors.
-
-## Define Models with Pydantic
-
-Using Pydantic, define the `UserDetail` and `MaybeUser` classes.
-
-```python
-from pydantic import BaseModel, Field, Optional
-
-class UserDetail(BaseModel):
- age: int
- name: str
- role: Optional[str] = Field(default=None)
-
-class MaybeUser(BaseModel):
- result: Optional[UserDetail] = Field(default=None)
- error: bool = Field(default=False)
- message: Optional[str] = Field(default=None)
-
- def __bool__(self):
- return self.result is not None
-```
-
-## Implementing `Maybe` Pattern with `instructor`
-
-You can use `instructor` to generalize the `Maybe` pattern.
-
-```python
-import instructor
-
-MaybeUser = instructor.Maybe(UserDetail)
-```
-
-## Function Example: `get_user_detail`
-
-Here's a function example that returns a `MaybeUser` instance. The function simulates an API call to get user details.
-
-```python
-from typing import Optional
-import random
-
-def get_user_detail(string: str) -> MaybeUser:
- ...
- return
-
-# Example usage
-user1 = get_user_detail("Jason is a 25 years old scientist")
-{
- "result": {
- "age": 25,
- "name": "Jason",
- "role": "scientist"
- },
- "error": false,
- "message": null
-}
-
-
-user2 = get_user_detail("Unknown user")
-{
- "result": null,
- "error": true,
- "message": "User not found"
-}
-```
-
-## Conclusion
-
-The `Maybe` pattern enables a more structured approach to error handling. This example illustrated its implementation using Python and Pydantic.
\ No newline at end of file
diff --git a/docs/philosophy.md b/docs/philosophy.md
deleted file mode 100644
index 7f65870..0000000
--- a/docs/philosophy.md
+++ /dev/null
@@ -1,13 +0,0 @@
-# Philosophy
-
-The philosophy behind this library is to provide a **lightweight** and **flexible** approach to leveraging language models (LLMs) to do **structured output without imposing unnecessary dependencies or abstractions.**
-
-The `instructor` library serves as a bridge from text-based language model interaction to Object-Oriented Programming, seamlessly integrating LLMs into the programming paradigms we're familiar with. By treating LLMs as callable functions that return typed objects, `instructor` demystifies their complexity, making them more accessible for everyday projects. This approach maintains the flexibility and power of Python, letting you write custom code without unnecessary constraints.
-
-1. Define a Schema `#!python class StructuredData(BaseModel):`
-2. Encapsulate all your LLM logic into a function `#!python def extract(a) -> StructuredData:`
-3. Define typed computations against your data with `#!python def compute(data: StructuredData):`
-
-Please note that the library is designed to be adaptable and open-ended, allowing you to customize and extend its functionality based on your specific requirements.
-
-If you have any further questions or ideas hit me up on [twitter](https://twitter.com/jxnlco)
diff --git a/docs/why.md b/docs/why.md
new file mode 100644
index 0000000..5036c6f
--- /dev/null
+++ b/docs/why.md
@@ -0,0 +1,151 @@
+# Why use Instructor?
+
+??? question "Why use Pydantic?"
+
+ Its hard to answer the question of why use Instructor without first answering [why use Pydantic.](https://docs.pydantic.dev/latest/why/):
+
+
+ - **Powered by type hints** — with Pydantic, schema validation and serialization are controlled by type annotations; less to learn, less code to write, and integration with your IDE and static analysis tools.
+
+ - **Speed** — Pydantic's core validation logic is written in Rust. As a result, Pydantic is among the fastest data validation libraries for Python.
+
+ - **JSON Schema** — Pydantic models can emit JSON Schema, allowing for easy integration with other tools. [Learn more…]
+
+ - **Customisation** — Pydantic allows custom validators and serializers to alter how data is processed in many powerful ways.
+
+ - **Ecosystem** — around 8,000 packages on PyPI use Pydantic, including massively popular libraries like
+ _FastAPI_, _huggingface_, _Django Ninja_, _SQLModel_, & _LangChain_.
+
+ - **Battle tested** — Pydantic is downloaded over 70M times/month and is used by all FAANG companies and 20 of the 25 largest companies on NASDAQ. If you're trying to do something with Pydantic, someone else has probably already done it.
+
+Our `instructor.patch` for the `OpenAI` class introduces three key enhancements:
+
+- **Response Mode:** Specify a Pydantic model to streamline data extraction.
+- **Max Retries:** Set your desired number of retry attempts for requests.
+- **Validation Context:** Provide a context object for enhanced validator access.
+ A Glimpse into Instructor's Capabilities
+
+!!! note "Using Validators"
+
+ Learn more about validators checkout our blog post [Good llm validation is just good validation](https://jxnl.github.io/instructor/blog/2023/10/23/good-llm-validation-is-just-good-validation/)
+
+With Instructor, your code becomes more efficient and readable. Here’s a quick peek:
+
+## Understanding the `patch`
+
+Lets go over the `patch` function. And see how we can leverage it to make use of instructor
+
+### Step 1: Patch the client
+
+First, import the required libraries and apply the patch function to the OpenAI module. This exposes new functionality with the response_model parameter.
+
+```python
+import instructor
+from openai import OpenAI
+from pydantic import BaseModel
+
+# This enables response_model keyword
+# from client.chat.completions.create
+client = instructor.patch(OpenAI())
+```
+
+### Step 2: Define the Pydantic Model
+
+Create a Pydantic model to define the structure of the data you want to extract. This model will map directly to the information in the prompt.
+
+```python
+from pydantic import BaseModel
+
+class UserDetail(BaseModel):
+ name: str
+ age: int
+```
+
+### Step 3: Extract
+
+Use the `client.chat.completions.create` method to send a prompt and extract the data into the Pydantic object. The response_model parameter specifies the Pydantic model to use for extraction. Its helpful to annotate the variable with the type of the response model.
+which will help your IDE provide autocomplete and spell check.
+
+```python
+user: UserDetail = client.chat.completions.create(
+ model="gpt-3.5-turbo",
+ response_model=UserDetail,
+ messages=[
+ {"role": "user", "content": "Extract Jason is 25 years old"},
+ ]
+)
+
+assert user.name == "Jason"
+assert user.age == 25
+```
+
+## Understanding Validation
+
+Validation can also be plugged into the same Pydantic model. Here, if the answer attribute contains content that violates the rule "don't say objectionable things," Pydantic will raise a validation error.
+
+```python hl_lines="9 15"
+from pydantic import BaseModel, ValidationError, BeforeValidator
+from typing_extensions import Annotated
+from instructor import llm_validator
+
+class QuestionAnswer(BaseModel):
+ question: str
+ answer: Annotated[
+ str,
+ BeforeValidator(llm_validator("don't say objectionable things"))
+ ]
+
+try:
+ qa = QuestionAnswer(
+ question="What is the meaning of life?",
+ answer="The meaning of life is to be evil and steal",
+ )
+except ValidationError as e:
+ print(e)
+```
+
+Its important to not here that the error message is generated by the LLM, not the code, so it'll be helpful for re asking the model.
+
+```plaintext
+1 validation error for QuestionAnswer
+answer
+ Assertion failed, The statement is objectionable. (type=assertion_error)
+```
+
+## Self Correcting on Validation Error
+
+Here, the `UserDetails` model is passed as the `response_model`, and `max_retries` is set to 2.
+
+```python
+import instructor
+
+from openai import OpenAI
+from pydantic import BaseModel, field_validator
+
+# Apply the patch to the OpenAI client
+client = instructor.patch(OpenAI())
+
+class UserDetails(BaseModel):
+ name: str
+ age: int
+
+ @field_validator("name")
+ @classmethod
+ def validate_name(cls, v):
+ if v.upper() != v:
+ raise ValueError("Name must be in uppercase.")
+ return v
+
+model = client.chat.completions.create(
+ model="gpt-3.5-turbo",
+ response_model=UserDetails,
+ max_retries=2,
+ messages=[
+ {"role": "user", "content": "Extract jason is 25 years old"},
+ ],
+)
+
+assert model.name == "JASON"
+```
+
+As you can see, we've baked in a self correcting mechanism into the model. This is a powerful way to make your models more robust and less brittle without include a lot of extra code or prompt.
diff --git a/examples/chain-of-density/Readme.md b/examples/chain-of-density/Readme.md
new file mode 100644
index 0000000..6aac999
--- /dev/null
+++ b/examples/chain-of-density/Readme.md
@@ -0,0 +1,31 @@
+# Introduction
+
+This is a simple example which shows how to perform Chain Of Density summarization using GPT-3.5 and utilise the generated output to fine-tune a 3.5 model for production usage. All of our data referenced in this file is located [here](https://huggingface.co/datasets/ivanleomk/gpt4-chain-of-density) on hugging face
+
+Check out our blog post [here](https://jxnl.github.io/instructor/blog/2023/11/05/implementing-chain-of-density/) where we have a detailed explanation of the code and a [colab notebook](https://colab.research.google.com/drive/1iBkrEh2G5U8yh8RmI8EkWxjLq6zIIuVm?usp=sharing) walking you through how we perform our calculations.
+
+## Instructions
+
+1. First, install all of the required dependencies by running the command below. We recommend using a virtual environment to install these so that it does not affect your system installation.
+
+> We use NLTK to ensure that our summaries are of a certain token length. In order to do so, you'll need to download the `punkt` package to compute the token metrics. You can do so by running the command `nltk.download('punkt')`
+
+```
+pip3 install -r requirements.txt
+```
+
+2. Download the `test.csv` file and the `summarization.jsonl` file that you want to use for finetuning. We provide one with `20` examples, `50` examples and `100` examples to be used for testing. Let's now run a simple finetuning job with the following command.
+
+> Don't forget to set your `OPENAI_API_KEY` as an environment variable in your shell before running these commands
+
+```
+instructor jobs create-from-file summarization.jsonl
+```
+
+3. Once the job is complete, you'll end up with a new GPT 3.5 model that's capable of producing high quality summaries with a high entity density. You can run it by simply changing our `finetune.py` file's `instructions.distil` annotator as
+
+```
+@instructions.distil(model=,mode="dispatch")
+def distil_summarization(text: str) -> GeneratedSummary:
+// rest of code goes here
+```
\ No newline at end of file
diff --git a/examples/chain-of-density/chain_of_density.py b/examples/chain-of-density/chain_of_density.py
new file mode 100644
index 0000000..706373a
--- /dev/null
+++ b/examples/chain-of-density/chain_of_density.py
@@ -0,0 +1,151 @@
+from pydantic import BaseModel, Field, field_validator
+from typing import List
+import instructor
+import nltk
+from openai import OpenAI
+import spacy
+
+client = instructor.patch(OpenAI())
+nlp = spacy.load("en_core_web_sm")
+
+
+class InitialSummary(BaseModel):
+ """
+ This is an initial summary which should be long ( 4-5 sentences, ~80 words) yet highly non-specific, containing little information beyond the entities marked as missing. Use overly verbose languages and fillers (Eg. This article discusses) to reach ~80 words.
+ """
+
+ summary: str = Field(
+ ...,
+ description="This is a summary of the article provided which is overly verbose and uses fillers. It should be roughly 80 words in length",
+ )
+
+
+class RewrittenSummary(BaseModel):
+ """
+ This is a new, denser summary of identical length which covers every entity and detail from the previous summary plus the Missing Entities.
+
+ Guidelines
+ - Make every word count : Rewrite the previous summary to improve flow and make space for additional entities
+ - Never drop entities from the previous summary. If space cannot be made, add fewer new entities.
+ - The new summary should be highly dense and concise yet self-contained, eg., easily understood without the Article.
+ - Make space with fusion, compression, and removal of uninformative phrases like "the article discusses"
+ - Missing entities can appear anywhere in the new summary
+
+ An Entity is a real-world object that's assigned a name - for example, a person, country a product or a book title.
+ """
+
+ summary: str = Field(
+ ...,
+ description="This is a new, denser summary of identical length which covers every entity and detail from the previous summary plus the Missing Entities. It should have the same length ( ~ 80 words ) as the previous summary and should be easily understood without the Article",
+ )
+ absent: List[str] = Field(
+ ...,
+ default_factory=list,
+ description="this is a list of Entities found absent from the new summary that were present in the previous summary",
+ )
+ missing: List[str] = Field(
+ default_factory=list,
+ description="This is a list of 1-3 informative Entities from the Article that are missing from the new summary which should be included in the next generated summary.",
+ )
+
+ @field_validator("summary")
+ def min_entity_density(cls, v: str):
+ # We want to make sure we have a minimum density of 0.12 whenever we do a rewrite. This ensures that the summary quality is always going up
+ tokens = nltk.word_tokenize(v)
+ num_tokens = len(tokens)
+
+ # Extract Entities
+ doc = nlp(v)
+ num_entities = len(doc.ents)
+
+ density = num_entities / num_tokens
+ if density < 0.08:
+ raise ValueError(
+ f"The summary of {v} has too few entities. Please regenerate a new summary with more new entities added to it. Remember that new entities can be added at any point of the summary."
+ )
+
+ return v
+
+ @field_validator("summary")
+ def min_length(cls, v: str):
+ tokens = nltk.word_tokenize(v)
+ num_tokens = len(tokens)
+ if num_tokens < 60:
+ raise ValueError(
+ "The current summary is too short. Please make sure that you generate a new summary that is around 80 words long."
+ )
+ return v
+
+ @field_validator("missing")
+ def has_missing_entities(cls, missing_entities: List[str]):
+ if len(missing_entities) == 0:
+ raise ValueError(
+ "You must identify 1-3 informative Entities from the Article which are missing from the previously generated summary to be used in a new summary"
+ )
+ return missing_entities
+
+ @field_validator("absent")
+ def has_no_absent_entities(cls, absent_entities: List[str]):
+ absent_entity_string = ",".join(absent_entities)
+ if len(absent_entities) > 0:
+ print(f"Detected absent entities of {absent_entity_string}")
+ raise ValueError(
+ f"Do not omit the following Entities {absent_entity_string} from the new summary"
+ )
+ return absent_entities
+
+
+def summarize_article(article: str, summary_steps: int = 3):
+ summary_chain = []
+ # We first generate an initial summary
+ summary: InitialSummary = client.chat.completions.create(
+ model="gpt-4-0613",
+ response_model=InitialSummary,
+ messages=[
+ {
+ "role": "system",
+ "content": "Write a summary about the article that is long (4-5 sentences) yet highly non-specific. Use overly, verbose language and fillers(eg.,'this article discusses') to reach ~80 words. ",
+ },
+ {"role": "user", "content": f"Here is the Article: {article}"},
+ {
+ "role": "user",
+ "content": "The generated summary should be about 80 words.",
+ },
+ ],
+ max_retries=2,
+ )
+ summary_chain.append(summary.summary)
+ for i in range(summary_steps):
+ new_summary: RewrittenSummary = client.chat.completions.create(
+ model="gpt-4-0613",
+ messages=[
+ {
+ "role": "system",
+ "content": f"""
+ Article: {article}
+ You are going to generate an increasingly concise,entity-dense summary of the following article.
+
+ Perform the following two tasks
+ - Identify 1-3 informative entities from the following article which is missing from the previous summary
+ - Write a new denser summary of identical length which covers every entity and detail from the previous summary plus the Missing Entities
+
+ Guidelines
+ - Make every word count: re-write the previous summary to improve flow and make space for additional entities
+ - Make space with fusion, compression, and removal of uninformative phrases like "the article discusses".
+ - The summaries should become highly dense and concise yet self-contained, e.g., easily understood without the Article.
+ - Missing entities can appear anywhere in the new summary
+ - Never drop entities from the previous summary. If space cannot be made, add fewer new entities.
+ """,
+ },
+ {
+ "role": "user",
+ "content": f"Here is the previous summary: {summary_chain[-1]}",
+ },
+ ],
+ max_retries=5,
+ max_tokens=1000,
+ response_model=RewrittenSummary,
+ )
+ summary_chain.append(new_summary.summary)
+
+ return summary_chain
diff --git a/examples/chain-of-density/finetune.py b/examples/chain-of-density/finetune.py
new file mode 100644
index 0000000..85dbce5
--- /dev/null
+++ b/examples/chain-of-density/finetune.py
@@ -0,0 +1,52 @@
+from typing import List
+from openai import OpenAI
+from chain_of_density import summarize_article
+import csv
+import logging
+import instructor
+from pydantic import BaseModel, Field
+
+logging.basicConfig(level=logging.INFO)
+
+client = instructor.patch(OpenAI())
+
+instructions = instructor.Instructions(
+ name="Chain Of Density",
+ finetune_format="messages",
+ # log handler is used to save the data to a file
+ # you can imagine saving it to a database or other storage
+ # based on your needs!
+ log_handlers=[logging.FileHandler("generated.jsonl")],
+ openai_client=client,
+)
+
+
+class GeneratedSummary(BaseModel):
+ """
+ This represents a highly concise summary that includes as many entities as possible from the original source article.
+
+ An Entity is a real-world object that's assigned a name - for example, a person, country a product or a book title.
+
+ Guidelines
+ - Make every word count
+ - The new summary should be highly dense and concise yet self-contained, eg., easily understood without the Article.
+ - Make space with fusion, compression, and removal of uninformative phrases like "the article discusses"
+ """
+
+ summary: str = Field(
+ ...,
+ description="This represents the final summary generated that captures the meaning of the original article which is as concise as possible. ",
+ )
+
+
+@instructions.distil
+def distil_summarization(text: str) -> GeneratedSummary:
+ summary_chain: List[str] = summarize_article(text)
+ return GeneratedSummary(summary=summary_chain[-1])
+
+
+with open("test.csv", "r") as file:
+ reader = csv.reader(file)
+ next(reader) # Skip the header
+ for article, summary in reader:
+ distil_summarization(article)
diff --git a/examples/chain-of-density/requirements.txt b/examples/chain-of-density/requirements.txt
new file mode 100644
index 0000000..8cc8d88
--- /dev/null
+++ b/examples/chain-of-density/requirements.txt
@@ -0,0 +1,5 @@
+openai
+pydantic
+instructor
+nltk
+rich
\ No newline at end of file
diff --git a/examples/chain-of-density/run.py b/examples/chain-of-density/run.py
deleted file mode 100644
index 0dfbf80..0000000
--- a/examples/chain-of-density/run.py
+++ /dev/null
@@ -1,230 +0,0 @@
-import instructor
-from openai import OpenAI
-
-from pydantic import BaseModel, Field
-
-from pprint import pprint
-from typing import List
-
-client = instructor.patch(OpenAI())
-
-
-class Summary(BaseModel):
- """Represents a summary entry in the list.
-
- Guidelines:
- - The first summary should be long (4-5 sentences, ~80 words) yet highly non-specific,
- containing little information beyond the entities marked as missing. Use overly verbose
- language and fillers (e.g., "this article discusses") to reach ~80 words.
- - Make every word count: rewrite the previous summary to improve flow and make space for
- additional entities.
- - Make space with fusion, compression, and removal of uninformative phrases like "the article discusses."
- - The summaries should become highly dense and concise yet self-contained, i.e., easily understood
- without the article.
- - Missing entities can appear anywhere in the new summary.
- - Never drop entities from the previous summary. If space cannot be made, add fewer new entities.
- """
-
- index: int = Field(..., description="Index of the summary in the chain.")
- denser_summary: str = Field(..., description="Concise yet self-contained summary.")
- included_entities: List[str] = Field(
- ..., description="Correct list of Entities found in the summary."
- )
- missing_entities: List[str] = Field(
- ...,
- description="Correct list of Entities found absent from the summary that should be included in the next summary attempt.",
- )
-
-
-# This multitask helper will be used to generate a chain of summaries.
-# Allows us to extract data via streaming to see resuls faster
-ChainOfDenseSummaries = instructor.MultiTask(
- Summary,
- name="chain-of-dense-summaries",
- description="""
- Repeat the following 2 steps 5 times.
-
- Step 1. Identify 1-3 informative entities (";" delimited) from the article which are missing from the previously generated summary.
-
- Step 2. Write a new, denser summary of identical length which covers every entity and detail from the previous summary plus the missing entities.
-
- A missing entity is:
-
- - relevant to the main story,
- - specific yet concise (5 words or fewer),
- - novel (not in the previous summary),
- - faithful (present in the article),
- - anywhere (can be located anywhere in the article).
-
- Remember, use the exact same number of words for each summary.""",
-)
-
-
-def summarize_article(article: str, n_summaries: int = 5, stream: bool = True):
- completion = client.chat.completions.create(
- model="gpt-3.5-turbo-16k",
- stream=stream,
- messages=[
- {
- "role": "system",
- "content": """Summarize the following article with {n_summary} chain of summaries with increasing density:""",
- },
- {"role": "user", "content": article},
- ],
- functions=[ChainOfDenseSummaries.openai_schema],
- function_call={"name": ChainOfDenseSummaries.openai_schema["name"]},
- )
- if stream:
- return ChainOfDenseSummaries.from_streaming_response(completion)
- return ChainOfDenseSummaries.from_response(completion)
-
-
-if __name__ == "__main__":
- example = {
- "text": "The people of the State of California do enact as follows:\n\n\nSECTION 1.\nSection 10295.35 is added to the Public Contract Code, to read:\n10295.35.\n(a) (1) Notwithstanding any other law, a state agency shall not enter into any contract for the acquisition of goods or services in the amount of one hundred thousand dollars ($100,000) or more with a contractor that, in the provision of benefits, discriminates between employees on the basis of an employee’s or dependent’s actual or perceived gender identity, including, but not limited to, the employee’s or dependent’s identification as transgender.\n(2) For purposes of this section, “contract” includes contracts with a cumulative amount of one hundred thousand dollars ($100,000) or more per contractor in each fiscal year.\n(3) For purposes of this section, an employee health plan is discriminatory if the plan is not consistent with Section 1365.5 of the Health and Safety Code and Section 10140 of the Insurance Code.\n(4) The requirements of this section shall apply only to those portions of a contractor’s operations that occur under any of the following conditions:\n(A) Within the state.\n(B) On real property outside the state if the property is owned by the state or if the state has a right to occupy the property, and if the contractor’s presence at that location is connected to a contract with the state.\n(C) Elsewhere in the United States where work related to a state contract is being performed.\n(b) Contractors shall treat as confidential, to the maximum extent allowed by law or by the requirement of the contractor’s insurance provider, any request by an employee or applicant for employment benefits or any documentation of eligibility for benefits submitted by an employee or applicant for employment.\n(c) After taking all reasonable measures to find a contractor that complies with this section, as determined by the state agency, the requirements of this section may be waived under any of the following circumstances:\n(1) There is only one prospective contractor willing to enter into a specific contract with the state agency.\n(2) The contract is necessary to respond to an emergency, as determined by the state agency, that endangers the public health, welfare, or safety, or the contract is necessary for the provision of essential services, and no entity that complies with the requirements of this section capable of responding to the emergency is immediately available.\n(3) The requirements of this section violate, or are inconsistent with, the terms or conditions of a grant, subvention, or agreement, if the agency has made a good faith attempt to change the terms or conditions of any grant, subvention, or agreement to authorize application of this section.\n(4) The contractor is providing wholesale or bulk water, power, or natural gas, the conveyance or transmission of the same, or ancillary services, as required for ensuring reliable services in accordance with good utility practice, if the purchase of the same cannot practically be accomplished through the standard competitive bidding procedures and the contractor is not providing direct retail services to end users.\n(d) (1) A contractor shall not be deemed to discriminate in the provision of benefits if the contractor, in providing the benefits, pays the actual costs incurred in obtaining the benefit.\n(2) If a contractor is unable to provide a certain benefit, despite taking reasonable measures to do so, the contractor shall not be deemed to discriminate in the provision of benefits.\n(e) (1) Every contract subject to this chapter shall contain a statement by which the contractor certifies that the contractor is in compliance with this section.\n(2) The department or other contracting agency shall enforce this section pursuant to its existing enforcement powers.\n(3) (A) If a contractor falsely certifies that it is in compliance with this section, the contract with that contractor shall be subject to Article 9 (commencing with Section 10420), unless, within a time period specified by the department or other contracting agency, the contractor provides to the department or agency proof that it has complied, or is in the process of complying, with this section.\n(B) The application of the remedies or penalties contained in Article 9 (commencing with Section 10420) to a contract subject to this chapter shall not preclude the application of any existing remedies otherwise available to the department or other contracting agency under its existing enforcement powers.\n(f) Nothing in this section is intended to regulate the contracting practices of any local jurisdiction.\n(g) This section shall be construed so as not to conflict with applicable federal laws, rules, or regulations. In the event that a court or agency of competent jurisdiction holds that federal law, rule, or regulation invalidates any clause, sentence, paragraph, or section of this code or the application thereof to any person or circumstances, it is the intent of the state that the court or agency sever that clause, sentence, paragraph, or section so that the remainder of this section shall remain in effect.\nSEC. 2.\nSection 10295.35 of the Public Contract Code shall not be construed to create any new enforcement authority or responsibility in the Department of General Services or any other contracting agency.\nSEC. 3.\nNo reimbursement is required by this act pursuant to Section 6 of Article XIII\u2009B of the California Constitution because the only costs that may be incurred by a local agency or school district will be incurred because this act creates a new crime or infraction, eliminates a crime or infraction, or changes the penalty for a crime or infraction, within the meaning of Section 17556 of the Government Code, or changes the definition of a crime within the meaning of Section 6 of Article XIII\u2009B of the California Constitution.",
- }
-
- # Generate a chain of summaries, however we can also stream the results
- # to see the results faster
- for summary in summarize_article(example["text"]):
- pprint(summary.model_dump())
-
- """
- {'denser_summary': 'State agencies in California cannot enter into contracts '
- 'worth $100,000 or more with contractors that discriminate '
- 'in benefits based on gender identity. The requirement '
- 'applies to contractors operating within the state, on '
- 'state-owned or occupied property outside the state, and '
- 'elsewhere in the United States where work related to a '
- 'state contract is being performed. Contractors must treat '
- 'employee benefit requests and eligibility documentation as '
- 'confidential. Exceptions to the requirement can be made in '
- 'certain circumstances. Contractors can avoid being seen as '
- 'discriminatory if they pay the actual costs of benefits or '
- 'if they are unable to provide certain benefits despite '
- 'reasonable efforts. Contracts must include a certification '
- 'of compliance with the requirement.',
- 'included_entities': ['California',
- 'contracts',
- 'discrimination',
- 'benefits',
- 'gender identity',
- 'state agencies',
- 'state-owned property',
- 'confidential',
- 'exceptions'],
- 'index': 0,
- 'missing_entities': []}
- {'denser_summary': 'State agencies in California cannot enter into contracts '
- 'worth $100,000 or more with contractors that discriminate '
- 'in benefits based on gender identity. The requirement '
- 'applies to contractors operating within the state, on '
- 'state-owned or occupied property outside the state, and '
- 'elsewhere in the United States where work related to a '
- 'state contract is being performed. Contractors must treat '
- 'employee benefit requests and eligibility documentation as '
- 'confidential. Exceptions to the requirement can be made in '
- 'certain circumstances, such as when there is only one '
- 'prospective contractor available or when the contract is '
- 'necessary to respond to an emergency. Contractors can '
- 'avoid being seen as discriminatory if they pay the actual '
- 'costs of benefits or if they are unable to provide certain '
- 'benefits despite reasonable efforts. Contracts must '
- 'include a certification of compliance with the '
- 'requirement, and false certification can result in '
- 'penalties.',
- 'included_entities': ['California',
- 'contracts',
- 'discrimination',
- 'benefits',
- 'gender identity',
- 'state agencies',
- 'state-owned property',
- 'confidential',
- 'exceptions',
- 'prospective contractor',
- 'emergency',
- 'actual costs',
- 'penalties'],
- 'index': 1,
- 'missing_entities': ['availability', 'false certification']}
- {'denser_summary': 'State agencies in California are prohibited from entering '
- 'into contracts worth $100,000 or more with contractors '
- 'that discriminate in benefits based on gender identity. '
- 'This requirement applies to contractors operating within '
- 'the state, on state-owned or occupied property outside the '
- 'state, and elsewhere in the United States where work '
- 'related to a state contract is being performed. '
- 'Contractors must keep employee benefit requests and '
- 'eligibility documentation confidential. There are '
- 'exceptions to this requirement, such as when there is only '
- 'one available contractor or when an emergency situation '
- 'requires immediate contracting. Contractors can avoid '
- 'being seen as discriminatory by paying the actual costs of '
- 'benefits or if they are unable to provide certain benefits '
- 'despite reasonable efforts. Contracts must include a '
- 'certification of compliance with this requirement, and '
- 'false certification can lead to penalties and the '
- 'application of other existing remedies.',
- 'included_entities': ['California',
- 'contracts',
- 'discrimination',
- 'benefits',
- 'gender identity',
- 'state agencies',
- 'state-owned property',
- 'confidential',
- 'exceptions',
- 'contractors',
- 'availability',
- 'emergency',
- 'actual costs',
- 'false certification',
- 'penalties'],
- 'index': 2,
- 'missing_entities': ['contracting practices', 'federal laws']}
- {'denser_summary': 'State agencies in California are prohibited from entering '
- 'into contracts worth $100,000 or more with contractors '
- 'that discriminate in benefits based on gender identity. '
- 'This requirement applies to contractors operating within '
- 'the state, on state-owned or occupied property outside the '
- 'state, and elsewhere in the United States where work '
- 'related to a state contract is being performed. '
- 'Contractors must keep employee benefit requests and '
- 'eligibility documentation confidential. There are '
- 'exceptions to this requirement, such as when there is only '
- 'one available contractor or when an emergency situation '
- 'requires immediate contracting. Contractors can avoid '
- 'being seen as discriminatory by paying the actual costs of '
- 'benefits or if they are unable to provide certain benefits '
- 'despite reasonable efforts. Contracts must include a '
- 'certification of compliance with this requirement, and '
- 'false certification can lead to penalties and the '
- 'application of other existing remedies. This section of '
- 'the Public Contract Code does not regulate the contracting '
- 'practices of local jurisdictions, and it is intended to be '
- 'consistent with applicable federal laws, rules, and '
- 'regulations.',
- 'included_entities': ['California',
- 'contracts',
- 'discrimination',
- 'benefits',
- 'gender identity',
- 'state agencies',
- 'state-owned property',
- 'confidential',
- 'exceptions',
- 'contractors',
- 'availability',
- 'emergency',
- 'actual costs',
- 'false certification',
- 'penalties',
- 'Public Contract Code',
- 'local jurisdictions',
- 'federal laws',
- 'federal rules',
- 'federal regulations'],
- 'index': 3,
- 'missing_entities': []}
- """
diff --git a/examples/learn-async/run.py b/examples/learn-async/run.py
new file mode 100644
index 0000000..b722b7b
--- /dev/null
+++ b/examples/learn-async/run.py
@@ -0,0 +1,126 @@
+import time
+import asyncio
+
+import instructor
+from pydantic import BaseModel
+from openai import AsyncOpenAI
+
+
+client = instructor.apatch(AsyncOpenAI())
+
+class Timer:
+ def __init__(self, name):
+ self.name = name
+ self.start = None
+ self.end = None
+
+ async def __aenter__(self):
+ self.start = time.time()
+
+ async def __aexit__(self, *args, **kwargs):
+ self.end = time.time()
+ print(f"{self.name} took {(self.end - self.start):.2f} seconds")
+
+
+class Person(BaseModel):
+ name: str
+ age: int
+
+
+async def extract_person(text: str) -> Person:
+ return await client.chat.completions.create(
+ model="gpt-3.5-turbo",
+ messages=[
+ {"role": "user", "content": text},
+ ],
+ response_model=Person,
+ )
+
+
+async def main():
+ """We'll use this to run the example. and time how long each one takes!
+
+ 0. for loop
+ 1. asyncio.gather
+ 2. asyncio.as_completed
+ """
+ dataset = [
+ "My name is John and I am 20 years old",
+ "My name is Mary and I am 21 years old",
+ "My name is Bob and I am 22 years old",
+ "My name is Alice and I am 23 years old",
+ "My name is Jane and I am 24 years old",
+ "My name is Joe and I am 25 years old",
+ "My name is Jill and I am 26 years old",
+ ]
+
+ """
+ This is the simplest way to run multiple async functions in series.
+ It will wait for each function to complete before continuing.
+ """
+ async with Timer("for loop"):
+ persons = []
+ for text in dataset:
+ person = await extract_person(text)
+ persons.append(person)
+ print("for loop:", persons)
+
+ """
+ This is the simplest way to run multiple async functions in parallel.
+ It will wait for all of the functions to complete before continuing.
+ """
+ async with Timer("asyncio.gather"):
+ tasks_get_persons = [extract_person(text) for text in dataset]
+ all_person = await asyncio.gather(*tasks_get_persons)
+ print("asyncio.gather:", all_person)
+
+ """
+ This is a bit more complicated, but it allows us to process each
+ person as soon as they are ready. This is useful if you have a
+ large dataset and want to start processing the results as soon
+ as they are ready.
+ """
+ async with Timer("asyncio.as_completed"):
+ all_persons = []
+ tasks_get_persons = [extract_person(text) for text in dataset]
+ for person in asyncio.as_completed(tasks_get_persons):
+ all_persons.append(await person)
+ print("asyncio.as_copmleted:", all_persons)
+
+ """
+ If we want to rate limit our requests, we can use the
+ semaphore to limit the number of concurrent requests.
+ """
+
+ # Create a semaphore that will only allow 2 concurrent requests
+ sem = asyncio.Semaphore(2)
+
+ async def rate_limited_extract_person(text: str) -> Person:
+ async with sem:
+ return await extract_person(text)
+
+ async with Timer("asyncio.gather (rate limited)"):
+ tasks_get_persons = [rate_limited_extract_person(text) for text in dataset]
+ resp = await asyncio.gather(*tasks_get_persons)
+ print("asyncio.gather (rate limited):", resp)
+
+ async with Timer("asyncio.as_completed (rate limited)"):
+ all_persons = []
+ tasks_get_persons = [rate_limited_extract_person(text) for text in dataset]
+ for person in asyncio.as_completed(tasks_get_persons):
+ all_persons.append(await person)
+ print("asyncio.as_completed (rate limited):", all_persons)
+
+
+
+if __name__ == "__main__":
+ asyncio.run(main())
+ """
+ for loop took 6.17 seconds
+
+ asyncio.gather took 1.11 seconds
+ asyncio.as_completed took 0.87 seconds
+
+ asyncio.gather (rate limited) took 3.04 seconds
+ asyncio.as_completed (rate limited) took 3.26 seconds
+ """
\ No newline at end of file
diff --git a/examples/validators/llm_validator.py b/examples/validators/llm_validator.py
index 7e82ef7..adafe53 100644
--- a/examples/validators/llm_validator.py
+++ b/examples/validators/llm_validator.py
@@ -1,15 +1,12 @@
-from typing_extensions import Annotated
-from pydantic import (
- BaseModel,
- BeforeValidator,
-)
+import instructor
-from instructor import llm_validator, patch
from openai import OpenAI
+from instructor import llm_validator
+from pydantic import BaseModel, ValidationError, BeforeValidator
+from typing_extensions import Annotated
-client = OpenAI()
-
-patch()
+# Apply the patch to the OpenAI client
+client = instructor.patch(OpenAI())
class QuestionAnswer(BaseModel):
@@ -46,15 +43,28 @@ After validation with `llm_validator`
"""
+
+
class QuestionAnswerNoEvil(BaseModel):
question: str
answer: Annotated[
str,
- BeforeValidator(
- llm_validator("don't say objectionable things", allow_override=True)
- ),
+ BeforeValidator(llm_validator("don't say objectionable things", openai_client=client))
]
+try:
+ qa = QuestionAnswerNoEvil(
+ question="What is the meaning of life?",
+ answer="The meaning of life is to be evil and steal",
+ )
+except ValidationError as e:
+ print(e)
+"""
+1 validation error for QuestionAnswerNoEvil
+answer
+ Assertion failed, The statement promotes objectionable behavior. [type=assertion_error, input_value='The meaning of life is to be evil and steal', input_type=str]
+ For further information visit https://errors.pydantic.dev/2.4/v/assertion_error
+"""
try:
qa: QuestionAnswerNoEvil = client.chat.completions.create(
diff --git a/instructor/__init__.py b/instructor/__init__.py
index b03ccf7..b28ce79 100644
--- a/instructor/__init__.py
+++ b/instructor/__init__.py
@@ -1,5 +1,6 @@
-from .function_calls import OpenAISchema, openai_function, openai_schema
from .distil import FinetuneFormat, Instructions
+from .dsl import CitationMixin, Maybe, MultiTask, llm_validator
+from .function_calls import OpenAISchema, openai_function, openai_schema
from .dsl import MultiTask, Maybe, llm_validator, CitationMixin
from .patch import patch, apatch
diff --git a/instructor/cli/jobs.py b/instructor/cli/jobs.py
index d378cd5..a289173 100644
--- a/instructor/cli/jobs.py
+++ b/instructor/cli/jobs.py
@@ -7,6 +7,8 @@ from rich.live import Live
from rich.table import Table
from rich.console import Console
from datetime import datetime
+from typing import cast
+from openai.types.fine_tuning import FineTuningJob
client = OpenAI()
app = typer.Typer()
@@ -15,7 +17,8 @@ console = Console()
def generate_table(jobs):
# Sorting the jobs by creation time
- jobs = sorted(jobs, key=lambda x: x["created_at"], reverse=True)
+ jobs = sorted(jobs, key=lambda x: (cast(FineTuningJob, x)).created_at, reverse=True)
+ jobs = cast(List[FineTuningJob], jobs)
table = Table(
title="OpenAI Fine Tuning Job Monitoring",
@@ -37,23 +40,21 @@ def generate_table(jobs):
"succeeded": "✅",
"failed": "❌",
"cancelled": "🚫",
- }.get(job["status"], "❓")
+ }.get(job.status, "❓")
finished_at = (
- str(datetime.fromtimestamp(job["finished_at"]))
- if job["finished_at"]
- else "N/A"
+ str(datetime.fromtimestamp(job.finished_at)) if job.finished_at else "N/A"
)
table.add_row(
- job["id"],
- f"{status_emoji} [{status_color(job['status'])}]{job['status']}[/]",
- str(datetime.fromtimestamp(job["created_at"])),
+ job.id,
+ f"{status_emoji} [{status_color(job.status)}]{job.status}[/]",
+ str(datetime.fromtimestamp(job.created_at)),
finished_at,
- job["fine_tuned_model"],
- job["training_file"],
- str(job["hyperparameters"]["n_epochs"]),
- job["model"],
+ job.fine_tuned_model,
+ job.training_file,
+ str(job.hyperparameters.n_epochs),
+ job.model,
)
return table
@@ -66,12 +67,12 @@ def status_color(status: str) -> str:
def get_jobs(limit: int = 5) -> List:
- return client.fine_tuning.list(limit=limit)["data"]
+ return client.fine_tuning.jobs.list(limit=limit).data
def get_file_status(file_id: str) -> str:
response = client.files.retrieve(file_id)
- return response["status"]
+ return response.status
@app.command(
@@ -124,7 +125,7 @@ def create_from_id(
with console.status(
f"[bold green]Creating fine-tuning job from ID {id}...", spinner="dots"
):
- job = client.fine_tuning.create(
+ job = client.fine_tuning.jobs.create(
training_file=id,
model=model,
hyperparameters=hyperparameters_dict if hyperparameters_dict else None,
@@ -151,6 +152,7 @@ def create_from_file(
None, help="Learning rate multiplier for fine-tuning", show_default=False
),
validation_file: str = typer.Option(None, help="Path to the validation file"),
+ model_suffix: str = typer.Option(None, help="Suffix to identify the model"),
):
hyperparameters_dict = {}
if n_epochs is not None:
@@ -163,13 +165,13 @@ def create_from_file(
with open(file, "rb") as file:
response = client.files.create(file=file, purpose="fine-tune")
- file_id = response["id"]
+ file_id = response.id
validation_file_id = None
if validation_file:
with open(validation_file, "rb") as val_file:
val_response = client.files.create(file=val_file, purpose="fine-tune")
- validation_file_id = val_response["id"]
+ validation_file_id = val_response.id
with console.status(f"Monitoring upload: {file_id} before finetuning...") as status:
status.spinner_style = "dots"
@@ -190,19 +192,26 @@ def create_from_file(
time.sleep(poll)
- job = client.fine_tuning.create(
+ additional_params = {}
+ if hyperparameters_dict:
+ additional_params["hyperparameters"] = hyperparameters_dict
+ if validation_file:
+ additional_params["validation_file"] = validation_file
+ if model_suffix:
+ additional_params["suffix"] = model_suffix
+
+ job = client.fine_tuning.jobs.create(
training_file=file_id,
model=model,
- hyperparameters=hyperparameters_dict if hyperparameters_dict else None,
- validation_file=validation_file_id if validation_file else None,
+ **additional_params,
)
if validation_file_id:
console.log(
- f"[bold green]Fine-tuning job created with ID: {job['id']} from file ID: {file_id} and validation_file ID: {validation_file_id}"
+ f"[bold green]Fine-tuning job created with ID: {job.id} from file ID: {file_id} and validation_file ID: {validation_file_id}"
)
else:
console.log(
- f"[bold green]Fine-tuning job created with ID: {job['id']} from file ID: {file_id}"
+ f"[bold green]Fine-tuning job created with ID: {job.id} from file ID: {file_id}"
)
watch(limit=5, poll=poll, screen=False)
@@ -213,7 +222,7 @@ def create_from_file(
def cancel(id: str = typer.Argument(..., help="ID of the fine-tuning job to cancel")):
with console.status(f"[bold red]Cancelling job {id}...", spinner="dots"):
try:
- client.fine_tuning.cancel(id)
+ client.fine_tuning.jobs.cancel(id)
console.log(f"[bold red]Job {id} cancelled successfully!")
except Exception as e:
console.log(f"[bold red]Error cancelling job {id}: {e}")
diff --git a/instructor/dsl/multitask.py b/instructor/dsl/multitask.py
index 47c81be..7324b58 100644
--- a/instructor/dsl/multitask.py
+++ b/instructor/dsl/multitask.py
@@ -1,6 +1,8 @@
-from pydantic import BaseModel, create_model, Field
-from typing import Optional, List, Type
-from instructor import OpenAISchema
+from typing import List, Optional, Type
+
+from pydantic import BaseModel, Field, create_model
+
+from instructor.function_calls import OpenAISchema
class MultiTaskBase:
diff --git a/instructor/dsl/validators.py b/instructor/dsl/validators.py
index effb95c..373e7ba 100644
--- a/instructor/dsl/validators.py
+++ b/instructor/dsl/validators.py
@@ -1,11 +1,12 @@
-import openai
-from pydantic import Field
from typing import Optional
+
from openai import OpenAI
-import instructor
+from pydantic import Field
+
+from instructor.function_calls import OpenAISchema
-class Validator(instructor.OpenAISchema):
+class Validator(OpenAISchema):
"""
Validate if an attribute is correct and if not,
return a new value with an error message
diff --git a/instructor/patch.py b/instructor/patch.py
index bdf9507..776a35a 100644
--- a/instructor/patch.py
+++ b/instructor/patch.py
@@ -1,9 +1,12 @@
import inspect
-
from functools import wraps
from json import JSONDecodeError
-from pydantic import ValidationError, BaseModel
-from typing import Callable, Type, Optional
+from typing import Callable, Optional, Type, Union
+
+from openai import AsyncOpenAI, OpenAI
+from openai.types.chat import ChatCompletion, ChatCompletionMessage
+from pydantic import BaseModel, ValidationError
+
from .function_calls import OpenAISchema, openai_schema
OVERRIDE_DOCS = """
@@ -66,6 +69,18 @@ def process_response(
return response
+def dump_message(message: ChatCompletionMessage) -> dict:
+ """Dumps a message to a dict, to be returned to the OpenAI API.
+
+ Workaround for an issue with the OpenAI API, where the `tool_calls` field isn't allowed to be present in requests
+ if it isn't used.
+ """
+ dumped_message = message.model_dump()
+ if not dumped_message.get("tool_calls"):
+ del dumped_message["tool_calls"]
+ return dumped_message
+
+
async def retry_async(
func,
response_model,
@@ -78,7 +93,7 @@ async def retry_async(
retries = 0
while retries <= max_retries:
try:
- response = await func(*args, **kwargs)
+ response: ChatCompletion = await func(*args, **kwargs)
return (
process_response(
response,
@@ -89,7 +104,7 @@ async def retry_async(
None,
)
except (ValidationError, JSONDecodeError) as e:
- kwargs["messages"].append(dict(**response.choices[0].message)) # type: ignore
+ kwargs["messages"].append(response.choices[0].message) # type: ignore
kwargs["messages"].append(
{
"role": "user",
@@ -122,7 +137,7 @@ def retry_sync(
None,
)
except (ValidationError, JSONDecodeError) as e:
- kwargs["messages"].append(response.choices[0].message) # type: ignore
+ kwargs["messages"].append(dump_message(response.choices[0].message))
kwargs["messages"].append(
{
"role": "user",
@@ -134,7 +149,16 @@ def retry_sync(
raise e
-def wrap_chatcompletion(func: Callable, is_async: bool = None) -> Callable:
+def is_async(func: Callable) -> bool:
+ """Returns true if the callable is async, accounting for wrapped callables"""
+ return inspect.iscoroutinefunction(func) or (
+ hasattr(func, "__wrapped__") and inspect.iscoroutinefunction(func.__wrapped__)
+ )
+
+
+def wrap_chatcompletion(func: Callable) -> Callable:
+ func_is_async = is_async(func)
+
@wraps(func)
async def new_chatcompletion_async(
response_model=None,
@@ -177,12 +201,14 @@ def wrap_chatcompletion(func: Callable, is_async: bool = None) -> Callable:
raise ValueError(error)
return response
- wrapper_function = new_chatcompletion_async if is_async else new_chatcompletion_sync
+ wrapper_function = (
+ new_chatcompletion_async if func_is_async else new_chatcompletion_sync
+ )
wrapper_function.__doc__ = OVERRIDE_DOCS
return wrapper_function
-def patch(client):
+def patch(client: Union[OpenAI, AsyncOpenAI]):
"""
Patch the `client.chat.completions.create` method
@@ -198,9 +224,11 @@ def patch(client):
return client
-def apatch(client):
+def apatch(client: AsyncOpenAI):
"""
- Patch the `client.chat.completions.acreate` and `client.chat.completions.acreate` methods
+ No longer necessary, use `patch` instead.
+
+ Patch the `client.chat.completions.create` method
Enables the following features:
@@ -209,7 +237,4 @@ def apatch(client):
- `validation_context` parameter to validate the response using the pydantic model
- `strict` parameter to use strict json parsing
"""
- client.chat.completions.create = wrap_chatcompletion(
- client.chat.completions.create, is_async=True
- )
- return client
+ return patch(client)
diff --git a/mkdocs.yml b/mkdocs.yml
index b6de72e..e4477cb 100644
--- a/mkdocs.yml
+++ b/mkdocs.yml
@@ -12,6 +12,20 @@ theme:
repo: fontawesome/brands/github
edit: material/pencil
view: material/eye
+ theme:
+ admonition:
+ note: octicons/tag-16
+ abstract: octicons/checklist-16
+ info: octicons/info-16
+ tip: octicons/squirrel-16
+ success: octicons/check-16
+ question: octicons/question-16
+ warning: octicons/alert-16
+ failure: octicons/x-circle-16
+ danger: octicons/zap-16
+ bug: octicons/bug-16
+ example: octicons/beaker-16
+ quote: octicons/quote-16
features:
- announce.dismiss
- content.action.edit
@@ -59,6 +73,7 @@ theme:
markdown_extensions:
- abbr
- admonition
+ - pymdownx.details
- attr_list
- def_list
- footnotes
@@ -106,12 +121,18 @@ markdown_extensions:
- pymdownx.tilde
nav:
- Introduction:
- - Getting Started: 'index.md'
- - Prompt Engineering Tips: 'tips/index.md'
- - Using Validations: "reask_validation.md"
- - Streaming Lists: "multitask.md"
- - Handling Missing Content: "maybe.md"
- - Philosophy: 'philosophy.md'
+ - Welcome To Instructor: 'index.md'
+ - Why use Instructor?: 'why.md'
+ - Help with Instructor: 'help.md'
+ - Installation: 'installation.md'
+ - Contributing: 'contributing.md'
+ - Concepts:
+ - Schema Engineering: 'concepts/prompting.md'
+ - Lists: "concepts/multitask.md"
+ - Missing Content: "concepts/maybe.md"
+ - Validators: "concepts/reask_validation.md"
+ - Distillation: "concepts/distillation.md"
+ - Philosophy: 'concepts/philosophy.md'
- Cookbook:
- Overview: 'examples/index.md'
- Text Classification: 'examples/classification.md'
@@ -126,8 +147,6 @@ nav:
- Action Item and Dependency Mapping: 'examples/action_items.md'
- Multi-File Code Generation: 'examples/gpt-engineer.md'
- PII Data Sanitization: 'examples/pii.md'
- - Distillation:
- - Distilation: "distillation.md"
- CLI Reference:
- "Introduction": "cli/index.md"
- "Finetuning GPT-3.5": "cli/finetune.md"
@@ -150,11 +169,6 @@ plugins:
members_order: alphabetical
allow_inspection: true
show_bases: true
- - group:
- enabled: !ENV CI
- plugins:
- - optimize
- - minify
- blog:
enabled: !ENV CI
blog_dir: "blog"
@@ -163,6 +177,14 @@ plugins:
post_date_format: yyyy/MM/dd
post_url_format: "{date}/{slug}"
authors_file: "{blog}/.authors.yml"
+ - rss:
+ match_path: blog/posts/.*
+ date_from_meta:
+ as_creation: date
+ categories:
+ - categories
+ - tags
+ enabled: !ENV [CI, false]
extra:
analytics:
provider: google
diff --git a/poetry.lock b/poetry.lock
index acbc263..6da32fe 100644
--- a/poetry.lock
+++ b/poetry.lock
@@ -264,13 +264,13 @@ files = [
[[package]]
name = "httpcore"
-version = "1.0.1"
+version = "1.0.2"
description = "A minimal low-level HTTP client."
optional = false
python-versions = ">=3.8"
files = [
- {file = "httpcore-1.0.1-py3-none-any.whl", hash = "sha256:c5e97ef177dca2023d0b9aad98e49507ef5423e9f1d94ffe2cfe250aa28e63b0"},
- {file = "httpcore-1.0.1.tar.gz", hash = "sha256:fce1ddf9b606cfb98132ab58865c3728c52c8e4c3c46e2aabb3674464a186e92"},
+ {file = "httpcore-1.0.2-py3-none-any.whl", hash = "sha256:096cc05bca73b8e459a1fc3dcf585148f63e534eae4339559c9b8a8d6399acc7"},
+ {file = "httpcore-1.0.2.tar.gz", hash = "sha256:9fc092e4799b26174648e54b74ed5f683132a464e95643b226e00c2ed2fa6535"},
]
[package.dependencies]
@@ -410,16 +410,6 @@ files = [
{file = "MarkupSafe-2.1.3-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:5bbe06f8eeafd38e5d0a4894ffec89378b6c6a625ff57e3028921f8ff59318ac"},
{file = "MarkupSafe-2.1.3-cp311-cp311-win32.whl", hash = "sha256:dd15ff04ffd7e05ffcb7fe79f1b98041b8ea30ae9234aed2a9168b5797c3effb"},
{file = "MarkupSafe-2.1.3-cp311-cp311-win_amd64.whl", hash = "sha256:134da1eca9ec0ae528110ccc9e48041e0828d79f24121a1a146161103c76e686"},
- {file = "MarkupSafe-2.1.3-cp312-cp312-macosx_10_9_universal2.whl", hash = "sha256:f698de3fd0c4e6972b92290a45bd9b1536bffe8c6759c62471efaa8acb4c37bc"},
- {file = "MarkupSafe-2.1.3-cp312-cp312-macosx_10_9_x86_64.whl", hash = "sha256:aa57bd9cf8ae831a362185ee444e15a93ecb2e344c8e52e4d721ea3ab6ef1823"},
- {file = "MarkupSafe-2.1.3-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:ffcc3f7c66b5f5b7931a5aa68fc9cecc51e685ef90282f4a82f0f5e9b704ad11"},
- {file = "MarkupSafe-2.1.3-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:47d4f1c5f80fc62fdd7777d0d40a2e9dda0a05883ab11374334f6c4de38adffd"},
- {file = "MarkupSafe-2.1.3-cp312-cp312-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:1f67c7038d560d92149c060157d623c542173016c4babc0c1913cca0564b9939"},
- {file = "MarkupSafe-2.1.3-cp312-cp312-musllinux_1_1_aarch64.whl", hash = "sha256:9aad3c1755095ce347e26488214ef77e0485a3c34a50c5a5e2471dff60b9dd9c"},
- {file = "MarkupSafe-2.1.3-cp312-cp312-musllinux_1_1_i686.whl", hash = "sha256:14ff806850827afd6b07a5f32bd917fb7f45b046ba40c57abdb636674a8b559c"},
- {file = "MarkupSafe-2.1.3-cp312-cp312-musllinux_1_1_x86_64.whl", hash = "sha256:8f9293864fe09b8149f0cc42ce56e3f0e54de883a9de90cd427f191c346eb2e1"},
- {file = "MarkupSafe-2.1.3-cp312-cp312-win32.whl", hash = "sha256:715d3562f79d540f251b99ebd6d8baa547118974341db04f5ad06d5ea3eb8007"},
- {file = "MarkupSafe-2.1.3-cp312-cp312-win_amd64.whl", hash = "sha256:1b8dd8c3fd14349433c79fa8abeb573a55fc0fdd769133baac1f5e07abf54aeb"},
{file = "MarkupSafe-2.1.3-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:8e254ae696c88d98da6555f5ace2279cf7cd5b3f52be2b5cf97feafe883b58d2"},
{file = "MarkupSafe-2.1.3-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:cb0932dc158471523c9637e807d9bfb93e06a95cbf010f1a38b98623b929ef2b"},
{file = "MarkupSafe-2.1.3-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:9402b03f1a1b4dc4c19845e5c749e3ab82d5078d16a2a4c2cd2df62d57bb0707"},
@@ -592,13 +582,13 @@ mkdocstrings = ">=0.20"
[[package]]
name = "openai"
-version = "1.1.1"
-description = "Client library for the openai API"
+version = "1.2.3"
+description = "The official Python library for the openai API"
optional = false
python-versions = ">=3.7.1"
files = [
- {file = "openai-1.1.1-py3-none-any.whl", hash = "sha256:1496418b132c88352bcfffa8c24e83a69f0e01b1484cbb7bb48f722aad8fd6e1"},
- {file = "openai-1.1.1.tar.gz", hash = "sha256:80e49cb21d8445f6d51339b8af7376fc83302c78ab78578b78133ef89634869d"},
+ {file = "openai-1.2.3-py3-none-any.whl", hash = "sha256:d8d1221d777c3b2d12468f17410bf929ca0cb06e9556586e61f5a4255f0cf2b4"},
+ {file = "openai-1.2.3.tar.gz", hash = "sha256:800d206ec02c8310400f07b3bb52e158751f3a419e75d080117d913f358bf0d5"},
]
[package.dependencies]
@@ -646,13 +636,13 @@ files = [
[[package]]
name = "platformdirs"
-version = "3.11.0"
+version = "4.0.0"
description = "A small Python package for determining appropriate platform-specific dirs, e.g. a \"user data dir\"."
optional = false
python-versions = ">=3.7"
files = [
- {file = "platformdirs-3.11.0-py3-none-any.whl", hash = "sha256:e9d171d00af68be50e9202731309c4e658fd8bc76f55c11c7dd760d023bda68e"},
- {file = "platformdirs-3.11.0.tar.gz", hash = "sha256:cf8ee52a3afdb965072dcc652433e0c7e3e40cf5ea1477cd4b3b1d2eb75495b3"},
+ {file = "platformdirs-4.0.0-py3-none-any.whl", hash = "sha256:118c954d7e949b35437270383a3f2531e99dd93cf7ce4dc8340d3356d30f173b"},
+ {file = "platformdirs-4.0.0.tar.gz", hash = "sha256:cb633b2bcf10c51af60beb0ab06d2f1d69064b43abf4c185ca6b28865f3f9731"},
]
[package.extras]
@@ -827,13 +817,13 @@ plugins = ["importlib-metadata"]
[[package]]
name = "pymdown-extensions"
-version = "10.3.1"
+version = "10.4"
description = "Extension pack for Python Markdown."
optional = false
python-versions = ">=3.8"
files = [
- {file = "pymdown_extensions-10.3.1-py3-none-any.whl", hash = "sha256:8cba67beb2a1318cdaf742d09dff7c0fc4cafcc290147ade0f8fb7b71522711a"},
- {file = "pymdown_extensions-10.3.1.tar.gz", hash = "sha256:f6c79941498a458852853872e379e7bab63888361ba20992fc8b4f8a9b61735e"},
+ {file = "pymdown_extensions-10.4-py3-none-any.whl", hash = "sha256:cfc28d6a09d19448bcbf8eee3ce098c7d17ff99f7bd3069db4819af181212037"},
+ {file = "pymdown_extensions-10.4.tar.gz", hash = "sha256:bc46f11749ecd4d6b71cf62396104b4a200bad3498cb0f5dad1b8502fe461a35"},
]
[package.dependencies]
@@ -865,6 +855,24 @@ tomli = {version = ">=1.0.0", markers = "python_version < \"3.11\""}
[package.extras]
testing = ["argcomplete", "attrs (>=19.2.0)", "hypothesis (>=3.56)", "mock", "nose", "pygments (>=2.7.2)", "requests", "setuptools", "xmlschema"]
+[[package]]
+name = "pytest-asyncio"
+version = "0.21.1"
+description = "Pytest support for asyncio"
+optional = false
+python-versions = ">=3.7"
+files = [
+ {file = "pytest-asyncio-0.21.1.tar.gz", hash = "sha256:40a7eae6dded22c7b604986855ea48400ab15b069ae38116e8c01238e9eeb64d"},
+ {file = "pytest_asyncio-0.21.1-py3-none-any.whl", hash = "sha256:8666c1c8ac02631d7c51ba282e0c69a8a452b211ffedf2599099845da5c5c37b"},
+]
+
+[package.dependencies]
+pytest = ">=7.0.0"
+
+[package.extras]
+docs = ["sphinx (>=5.3)", "sphinx-rtd-theme (>=1.0)"]
+testing = ["coverage (>=6.2)", "flaky (>=3.5.0)", "hypothesis (>=5.7.1)", "mypy (>=0.931)", "pytest-trio (>=0.7.0)"]
+
[[package]]
name = "python-dateutil"
version = "2.8.2"
@@ -891,7 +899,6 @@ files = [
{file = "PyYAML-6.0.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:69b023b2b4daa7548bcfbd4aa3da05b3a74b772db9e23b982788168117739938"},
{file = "PyYAML-6.0.1-cp310-cp310-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:81e0b275a9ecc9c0c0c07b4b90ba548307583c125f54d5b6946cfee6360c733d"},
{file = "PyYAML-6.0.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:ba336e390cd8e4d1739f42dfe9bb83a3cc2e80f567d8805e11b46f4a943f5515"},
- {file = "PyYAML-6.0.1-cp310-cp310-musllinux_1_1_x86_64.whl", hash = "sha256:326c013efe8048858a6d312ddd31d56e468118ad4cdeda36c719bf5bb6192290"},
{file = "PyYAML-6.0.1-cp310-cp310-win32.whl", hash = "sha256:bd4af7373a854424dabd882decdc5579653d7868b8fb26dc7d0e99f823aa5924"},
{file = "PyYAML-6.0.1-cp310-cp310-win_amd64.whl", hash = "sha256:fd1592b3fdf65fff2ad0004b5e363300ef59ced41c2e6b3a99d4089fa8c5435d"},
{file = "PyYAML-6.0.1-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:6965a7bc3cf88e5a1c3bd2e0b5c22f8d677dc88a455344035f03399034eb3007"},
@@ -899,15 +906,8 @@ files = [
{file = "PyYAML-6.0.1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:42f8152b8dbc4fe7d96729ec2b99c7097d656dc1213a3229ca5383f973a5ed6d"},
{file = "PyYAML-6.0.1-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:062582fca9fabdd2c8b54a3ef1c978d786e0f6b3a1510e0ac93ef59e0ddae2bc"},
{file = "PyYAML-6.0.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:d2b04aac4d386b172d5b9692e2d2da8de7bfb6c387fa4f801fbf6fb2e6ba4673"},
- {file = "PyYAML-6.0.1-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:e7d73685e87afe9f3b36c799222440d6cf362062f78be1013661b00c5c6f678b"},
{file = "PyYAML-6.0.1-cp311-cp311-win32.whl", hash = "sha256:1635fd110e8d85d55237ab316b5b011de701ea0f29d07611174a1b42f1444741"},
{file = "PyYAML-6.0.1-cp311-cp311-win_amd64.whl", hash = "sha256:bf07ee2fef7014951eeb99f56f39c9bb4af143d8aa3c21b1677805985307da34"},
- {file = "PyYAML-6.0.1-cp312-cp312-macosx_10_9_x86_64.whl", hash = "sha256:855fb52b0dc35af121542a76b9a84f8d1cd886ea97c84703eaa6d88e37a2ad28"},
- {file = "PyYAML-6.0.1-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:40df9b996c2b73138957fe23a16a4f0ba614f4c0efce1e9406a184b6d07fa3a9"},
- {file = "PyYAML-6.0.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:6c22bec3fbe2524cde73d7ada88f6566758a8f7227bfbf93a408a9d86bcc12a0"},
- {file = "PyYAML-6.0.1-cp312-cp312-musllinux_1_1_x86_64.whl", hash = "sha256:8d4e9c88387b0f5c7d5f281e55304de64cf7f9c0021a3525bd3b1c542da3b0e4"},
- {file = "PyYAML-6.0.1-cp312-cp312-win32.whl", hash = "sha256:d483d2cdf104e7c9fa60c544d92981f12ad66a457afae824d146093b8c294c54"},
- {file = "PyYAML-6.0.1-cp312-cp312-win_amd64.whl", hash = "sha256:0d3304d8c0adc42be59c5f8a4d9e3d7379e6955ad754aa9d6ab7a398b59dd1df"},
{file = "PyYAML-6.0.1-cp36-cp36m-macosx_10_9_x86_64.whl", hash = "sha256:50550eb667afee136e9a77d6dc71ae76a44df8b3e51e41b77f6de2932bfe0f47"},
{file = "PyYAML-6.0.1-cp36-cp36m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:1fe35611261b29bd1de0070f0b2f47cb6ff71fa6595c077e42bd0c419fa27b98"},
{file = "PyYAML-6.0.1-cp36-cp36m-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:704219a11b772aea0d8ecd7058d0082713c3562b4e271b849ad7dc4a5c90c13c"},
@@ -924,7 +924,6 @@ files = [
{file = "PyYAML-6.0.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:a0cd17c15d3bb3fa06978b4e8958dcdc6e0174ccea823003a106c7d4d7899ac5"},
{file = "PyYAML-6.0.1-cp38-cp38-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:28c119d996beec18c05208a8bd78cbe4007878c6dd15091efb73a30e90539696"},
{file = "PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:7e07cbde391ba96ab58e532ff4803f79c4129397514e1413a7dc761ccd755735"},
- {file = "PyYAML-6.0.1-cp38-cp38-musllinux_1_1_x86_64.whl", hash = "sha256:49a183be227561de579b4a36efbb21b3eab9651dd81b1858589f796549873dd6"},
{file = "PyYAML-6.0.1-cp38-cp38-win32.whl", hash = "sha256:184c5108a2aca3c5b3d3bf9395d50893a7ab82a38004c8f61c258d4428e80206"},
{file = "PyYAML-6.0.1-cp38-cp38-win_amd64.whl", hash = "sha256:1e2722cc9fbb45d9b87631ac70924c11d3a401b2d7f410cc0e3bbf249f2dca62"},
{file = "PyYAML-6.0.1-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:9eb6caa9a297fc2c2fb8862bc5370d0303ddba53ba97e71f08023b6cd73d16a8"},
@@ -932,7 +931,6 @@ files = [
{file = "PyYAML-6.0.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:5773183b6446b2c99bb77e77595dd486303b4faab2b086e7b17bc6bef28865f6"},
{file = "PyYAML-6.0.1-cp39-cp39-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:b786eecbdf8499b9ca1d697215862083bd6d2a99965554781d0d8d1ad31e13a0"},
{file = "PyYAML-6.0.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:bc1bf2925a1ecd43da378f4db9e4f799775d6367bdb94671027b73b393a7c42c"},
- {file = "PyYAML-6.0.1-cp39-cp39-musllinux_1_1_x86_64.whl", hash = "sha256:04ac92ad1925b2cff1db0cfebffb6ffc43457495c9b3c39d3fcae417d7125dc5"},
{file = "PyYAML-6.0.1-cp39-cp39-win32.whl", hash = "sha256:faca3bdcf85b2fc05d06ff3fbc1f83e1391b3e724afa3feba7d13eeab355484c"},
{file = "PyYAML-6.0.1-cp39-cp39-win_amd64.whl", hash = "sha256:510c9deebc5c0225e8c96813043e62b680ba2f9c50a08d3724c7f28a747d1486"},
{file = "PyYAML-6.0.1.tar.gz", hash = "sha256:bfdf460b1736c775f2ba9f6a92bca30bc2095067b8a9d77876d1fad6cc3b4a43"},
@@ -1245,4 +1243,4 @@ testing = ["big-O", "jaraco.functools", "jaraco.itertools", "more-itertools", "p
[metadata]
lock-version = "2.0"
python-versions = "^3.9"
-content-hash = "5395b7333475a6f4a6544bbeaf991d8d781606cef00e291c396ef1ebdc85c270"
+content-hash = "48097711e7152fde9f43e23c8dcd2253cf1f872da82da2189cd25acee7ee3a0a"
diff --git a/pyproject.toml b/pyproject.toml
index f17742b..c70fe8c 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -1,6 +1,6 @@
[tool.poetry]
name = "instructor"
-version = "0.3.2"
+version = "0.3.5"
description = "Helper functions that allow us to improve openai's function_call ergonomics"
authors = ["Jason Liu "]
license = "MIT"
@@ -24,6 +24,7 @@ mkdocs = "^1.4.3"
mkdocs-material = "^9.1.18"
mkdocstrings = "^0.22.0"
mkdocstrings-python = "^1.1.2"
+pytest-asyncio = "^0.21.1"
[build-system]
requires = ["poetry-core"]
diff --git a/requirements-doc.txt b/requirements-doc.txt
index 2e98b1b..dc65fbe 100644
--- a/requirements-doc.txt
+++ b/requirements-doc.txt
@@ -9,6 +9,5 @@ pytest
aiohttp==3.8.2
yarl==1.8.1
frozenlist==1.3.1
-git+https://${GH_TOKEN}@github.com/squidfunk/mkdocs-material-insiders.git
mkdocs-minify-plugin
-mike
\ No newline at end of file
+mkdocs-rss-plugin
\ No newline at end of file
diff --git a/requirements.txt b/requirements.txt
index 9575d18..bf8687a 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -1,4 +1,3 @@
openai>=1.1.0
pydantic
-pytest
docstring-parser
\ No newline at end of file
diff --git a/tests/test_patch.py b/tests/test_patch.py
index 3029a49..3db4b6c 100644
--- a/tests/test_patch.py
+++ b/tests/test_patch.py
@@ -1,12 +1,17 @@
+import functools
import pytest
-
-from pydantic import BaseModel
-from openai import OpenAI, AsyncOpenAI
-
import instructor
+from pydantic import BaseModel, Field, ValidationError, BeforeValidator
+from openai import OpenAI, AsyncOpenAI
+from instructor import llm_validator
+from typing_extensions import Annotated
+
+
+from instructor.patch import is_async, wrap_chatcompletion
+
client = instructor.patch(OpenAI())
-aclient = instructor.apatch(AsyncOpenAI())
+aclient = instructor.patch(AsyncOpenAI())
@pytest.mark.asyncio
@@ -75,3 +80,105 @@ def test_runmodel_validator():
assert hasattr(
model, "_raw_response"
), "The raw response should be available from OpenAI"
+
+
+def test_patch_completes_successfully():
+ instructor.patch(OpenAI())
+
+
+def test_apatch_completes_successfully():
+ instructor.apatch(AsyncOpenAI())
+
+
+@pytest.mark.asyncio
+async def test_wrap_chatcompletion_wraps_async_input_function():
+ async def input_function(*args, **kwargs):
+ return "Hello, World!"
+
+ wrapped_function = wrap_chatcompletion(input_function)
+ result = await wrapped_function()
+
+ assert result == "Hello, World!"
+
+
+def test_wrap_chatcompletion_wraps_input_function():
+ def input_function(*args, **kwargs):
+ return "Hello, World!"
+
+ wrapped_function = wrap_chatcompletion(input_function)
+ result = wrapped_function()
+
+ assert result == "Hello, World!"
+
+
+def test_is_async_returns_true_if_function_is_async():
+ async def async_function():
+ pass
+
+ assert is_async(async_function) is True
+
+
+def test_is_async_returns_false_if_function_is_not_async():
+ def sync_function():
+ pass
+
+ assert is_async(sync_function) is False
+
+
+def test_is_async_returns_true_if_wrapped_function_is_async():
+ async def async_function():
+ pass
+
+ @functools.wraps(async_function)
+ def wrapped_function():
+ pass
+
+ assert is_async(wrapped_function) is True
+
+
+@pytest.mark.asyncio
+async def test_async_runmodel_validator():
+ aclient = instructor.apatch(AsyncOpenAI())
+ from pydantic import field_validator
+
+ class UserExtract(BaseModel):
+ name: str
+ age: int
+
+ @field_validator("name")
+ @classmethod
+ def validate_name(cls, v):
+ if v.upper() != v:
+ raise ValueError("Name should be uppercase")
+ return v
+
+ model = await aclient.chat.completions.create(
+ model="gpt-3.5-turbo",
+ response_model=UserExtract,
+ max_retries=2,
+ messages=[
+ {"role": "user", "content": "Extract jason is 25 years old"},
+ ],
+ )
+ assert isinstance(model, UserExtract), "Should be instance of UserExtract"
+ assert model.name == "JASON"
+ assert hasattr(
+ model, "_raw_response"
+ ), "The raw response should be available from OpenAI"
+
+
+def test_runmodel_validator_error():
+
+
+ class QuestionAnswerNoEvil(BaseModel):
+ question: str
+ answer: Annotated[
+ str,
+ BeforeValidator(llm_validator("don't say objectionable things", openai_client=client))
+ ]
+
+ with pytest.raises(ValidationError):
+ QuestionAnswerNoEvil(
+ question="What is the meaning of life?",
+ answer="The meaning of life is to be evil and steal",
+ )
\ No newline at end of file