kennethreitz/instructor
1
0
Fork 0
mirror of https://github.com/kennethreitz/instructor.git synced 2026-06-05 22:50:18 +00:00
Code Issues Packages Projects Releases Wiki Activity

doc: update why

Browse Source
This commit is contained in:
Jason Liu
2024-03-15 19:03:47 -04:00
parent f47830caa9
commit 8e6614d2c7
1 changed files with 227 additions and 180 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/why.md
+227 -180
View File
@@ -1,10 +1,11 @@
# Why use Instructor?
??? question "Why use Pydantic?"
This is a letter from the author [Jason Liu](https://twitter.com/jxnlco) of Instructor. I'm a big fan of Pydantic and I think it's the best way to handle data validation in Python. I've been using it for years and I'm excited to bring it to the OpenAI API.
??? note "Why use Pydantic?"
Its hard to answer the question of why use Instructor without first answering [why use Pydantic.](https://docs.pydantic.dev/latest/why/):
- **Powered by type hints** — with Pydantic, schema validation and serialization are controlled by type annotations; less to learn, less code to write, and integration with your IDE and static analysis tools.
- **Speed** — Pydantic's core validation logic is written in Rust. As a result, Pydantic is among the fastest data validation libraries for Python.
@@ -18,188 +19,163 @@
- **Battle tested** — Pydantic is downloaded over 70M times/month and is used by all FAANG companies and 20 of the 25 largest companies on NASDAQ. If you're trying to do something with Pydantic, someone else has probably already done it.
Our `instructor.patch` for the `OpenAI` class introduces three key enhancements:
## Pydantic over Raw Schema
- **Response Mode:** Specify a Pydantic model to streamline data extraction.
- **Max Retries:** Set your desired number of retry attempts for requests.
- **Validation Context:** Provide a context object for enhanced validator access.
A Glimpse into Instructor's Capabilities
I find many prompt building tools to be overly complex and difficult to use, they might be simple to get started with a trivial examples but once you need more control, you have to wish they were simpler. Instructor does the least amount of work to get the job done.
!!! note "Using Validators"
=== "Pydantic"
Learn more about validators checkout our blog post [Good llm validation is just good validation](https://jxnl.github.io/instructor/blog/2023/10/23/good-llm-validation-is-just-good-validation/)
Pydantic is more readable and definitions and reference values are handled automatically. This is a big win for Instructor, as it allows us to focus on the data extraction and not the schema.
With Instructor, your code becomes more efficient and readable. Here’s a quick peek:
## Understanding the `patch`
Lets go over the `patch` function. And see how we can leverage it to make use of instructor
### Step 1: Patch the client
First, import the required libraries and apply the `patch` function to the OpenAI module. This exposes new functionality with the `response_model` parameter.
```python
import instructor
from openai import OpenAI
# This enables response_model keyword
# from client.chat.completions.create
client = instructor.patch(OpenAI())
```
### Step 2: Define the Pydantic Model
Create a Pydantic model to define the structure of the data you want to extract. This model will map directly to the information in the prompt.
```python
from pydantic import BaseModel
```python
from typing import List, Literal
from pydantic import BaseModel, Field
class UserDetail(BaseModel):
name: str
age: int
```
class Property(BaseModel):
name: str = Field(description="name of property in snake case")
value: str
### Step 3: Extract
class Character(BaseModel):
"""
Any character in a fictional story
"""
name: str
age: int
properties: List[Property]
role: Literal['protagonist', 'antagonist', 'supporting']
Use the `client.chat.completions.create` method to send a prompt and extract the data into the Pydantic object. The `response_model` parameter specifies the Pydantic model to use for extraction. Its helpful to annotate the variable with the type of the response model, which will help your IDE provide autocomplete and spell check.
class AllCharacters(BaseModel):
characters: List[Character] = Field(description="A list of all characters in the story")
```
```python
user: UserDetail = client.chat.completions.create(
model="gpt-3.5-turbo",
response_model=UserDetail,
messages=[
{"role": "user", "content": "Extract Jason is 25 years old"},
],
)
=== "Json Schema"
assert user.name == "Jason"
assert user.age == 25
```
Would you Ever prefer to code review this? Where everything is a string, ripe for typos and errors in references? I know I wouldn't.
## Understanding Validation
Validation can also be plugged into the same Pydantic model. Here, if the answer attribute contains content that violates the rule "don't say objectionable things," Pydantic will raise a validation error.
```python hl_lines="9 15"
from pydantic import BaseModel, ValidationError, BeforeValidator
from typing_extensions import Annotated
from instructor import llm_validator
class QuestionAnswer(BaseModel):
question: str
answer: Annotated[
str, BeforeValidator(llm_validator("don't say objectionable things"))
]
try:
qa = QuestionAnswer(
question="What is the meaning of life?",
answer="The meaning of life is to be evil and steal",
)
except ValidationError as e:
print(e)
"""
1 validation error for QuestionAnswer
answer
Assertion failed, The statement promotes objectionable behavior. [type=assertion_error, input_value='The meaning of life is to be evil and steal', input_type=str]
For further information visit https://errors.pydantic.dev/2.6/v/assertion_error
"""
```
Its important to note here that the error message is generated by the LLM, not the code, so it'll be helpful for re-asking the model.
```plaintext
1 validation error for QuestionAnswer
answer
Assertion failed, The statement is objectionable. (type=assertion_error)
```
## Self Correcting on Validation Error
Here, the `UserDetails` model is passed as the `response_model`, and `max_retries` is set to 2.
```python
import instructor
from openai import OpenAI
from pydantic import BaseModel, field_validator
# Apply the patch to the OpenAI client
client = instructor.patch(OpenAI())
class UserDetails(BaseModel):
name: str
age: int
@field_validator("name")
@classmethod
def validate_name(cls, v):
if v.upper() != v:
raise ValueError("Name must be in uppercase.")
return v
model = client.chat.completions.create(
model="gpt-3.5-turbo",
response_model=UserDetails,
max_retries=2,
messages=[
{"role": "user", "content": "Extract jason is 25 years old"},
],
)
assert model.name == "JASON"
```
## Iterables and Lists
We can also generate tasks as the tokens are streamed in by defining an `Iterable[T]` type.
Lets look at an example in action with the same class
```python hl_lines="6 26"
from typing import Iterable
Users = Iterable[User]
users = client.chat.completions.create(
model="gpt-4",
temperature=0.1,
stream=True,
response_model=Users,
messages=[
{
"role": "system",
"content": "You are a perfect entity extraction system",
```python
var = {
"$defs": {
"Character": {
"description": "Any character in a fictional story",
"properties": {
"name": {"title": "Name", "type": "string"},
"age": {"title": "Age", "type": "integer"},
"properties": {
"type": "array",
"items": {"$ref": "#/$defs/Property"},
"title": "Properties",
},
"role": {
"enum": ["protagonist", "antagonist", "supporting"],
"title": "Role",
"type": "string",
},
},
"required": ["name", "age", "properties", "role"],
"title": "Character",
"type": "object",
},
"Property": {
"properties": {
"name": {
"description": "name of property in snake case",
"title": "Name",
"type": "string",
},
"value": {"title": "Value", "type": "string"},
},
"required": ["name", "value"],
"title": "Property",
"type": "object",
},
},
{
"role": "user",
"content": (
f"Consider the data below:\n{input}"
"Correctly segment it into entitites"
"Make sure the JSON is correct"
),
"properties": {
"characters": {
"description": "A list of all characters in the story",
"items": {"$ref": "#/$defs/Character"},
"title": "Characters",
"type": "array",
}
},
],
max_tokens=1000,
)
"required": ["characters"],
"title": "AllCharacters",
"type": "object",
}
```
for user in users:
assert isinstance(user, User)
print(user)
## Easy to try and install
#> name="Jason" "age"=10
#> name="John" "age"=10
```
The minimum viable api just adds `response_model` to the client, if you dont think you want a model its very easy to remove it and continue building your application
=== "Instructor"
```python
import instructor
from openai import OpenAI
from pydantic import BaseModel
# Patch the OpenAI client with Instructor
client = instructor.patch(OpenAI())
class UserDetail(BaseModel):
name: str
age: int
# Function to extract user details
def extract_user() -> UserDetail:
user = client.chat.completions.create(
model="gpt-4-turbo-preview",
response_model=UserDetail,
messages=[
{"role": "user", "content": "Extract Jason is 25 years old"},
]
)
return user
```
=== "OpenAI"
```python
import openai
import json
def extract_user() -> dict:
completion = client.chat.completions.create(
model="gpt-4-turbo-preview",
tools=[
{
"type": "function",
"function": {
"name": "ExtractUser",
"description": "Correctly extracted `ExtractUser` with all the required parameters with correct types",
"parameters": {
"properties": {
"name": {"title": "Name", "type": "string"},
"age": {"title": "Age", "type": "integer"},
},
"required": ["age", "name"],
"type": "object",
},
},
}
],
tool_choice={"type": "function", "function": {"name": "ExtractUser"}},
messages=[
{"role": "user", "content": "Extract Jason is 25 years old"},
],
) # type: ignore
user = json_loads(completion.choices[0].message.tool_calls[0].function.arguments)
assert "name" in user, "Name is not in the response"
assert "age" in user, "Age is not in the response"
user["age"] = int(user["age"])
return user
```
## Partial Extraction
We also support partial extraction, which is useful for streaming in data that is incomplete.
We also support [partial](./concepts/partial.md) extraction, which is useful for streaming in data that is incomplete.
```python
import instructor
@@ -212,20 +188,7 @@ from rich.console import Console
client = instructor.patch(OpenAI())
text_block = """
In our recent online meeting, participants from various backgrounds joined to discuss the upcoming tech conference. The names and contact details of the participants were as follows:
- Name: John Doe, Email: johndoe@email.com, Twitter: @TechGuru44
- Name: Jane Smith, Email: janesmith@email.com, Twitter: @DigitalDiva88
- Name: Alex Johnson, Email: alexj@email.com, Twitter: @CodeMaster2023
During the meeting, we agreed on several key points. The conference will be held on March 15th, 2024, at the Grand Tech Arena located at 4521 Innovation Drive. Dr. Emily Johnson, a renowned AI researcher, will be our keynote speaker.
The budget for the event is set at $50,000, covering venue costs, speaker fees, and promotional activities. Each participant is expected to contribute an article to the conference blog by February 20th.
A follow-up meetingis scheduled for January 25th at 3 PM GMT to finalize the agenda and confirm the list of speakers.
"""
text_block = "..."
class User(BaseModel):
name: str
@@ -267,3 +230,87 @@ This will output the following:
![Partial Streaming Gif](./img/partial.gif)
As you can see, we've baked in a self correcting mechanism into the model. This is a powerful way to make your models more robust and less brittle without including a lot of extra code or prompts.
## Iterables and Lists
We can also generate tasks as the tokens are streamed in by defining an [`Iterable[T]`](./concepts/lists.md) type.
Lets look at an example in action with the same class
```python hl_lines="6 26"
from typing import Iterable
Users = Iterable[User]
users = client.chat.completions.create(
model="gpt-4",
temperature=0.1,
stream=True,
response_model=Users,
messages=[
{
"role": "system",
"content": "You are a perfect entity extraction system",
},
{
"role": "user",
"content": (
f"Consider the data below:\n{input}"
"Correctly segment it into entitites"
"Make sure the JSON is correct"
),
},
],
max_tokens=1000,
)
for user in users:
assert isinstance(user, User)
print(user)
#> name="Jason" "age"=10
#> name="John" "age"=10
```
## Simple Types
We also support [simple types](./concepts/types.md), which are useful for extracting simple values like numbers, strings, and booleans.
## Self Correcting on Validation Error
Due to pydantic's very own validation model, easily add validators to the model to correct the data.
If we run this code, we will get a validation error because the name is not in uppercase. While we could have included a prompt to fix this, we can also just add a field validator to the model. This will result in two API calls, to make sure you do your best to prompt before adding validators.
```python
import instructor
from openai import OpenAI
from pydantic import BaseModel, field_validator
# Apply the patch to the OpenAI client
client = instructor.patch(OpenAI())
class UserDetails(BaseModel):
name: str
age: int
@field_validator("name")
@classmethod
def validate_name(cls, v):
if v.upper() != v:
raise ValueError("Name must be in uppercase.")
return v
model = client.chat.completions.create(
model="gpt-3.5-turbo",
response_model=UserDetails,
max_retries=2,
messages=[
{"role": "user", "content": "Extract jason is 25 years old"},
],
)
assert model.name == "JASON"
```