kennethreitz/instructor
1
0
Fork 0
mirror of https://github.com/kennethreitz/instructor.git synced 2026-06-05 22:50:18 +00:00
Code Issues Packages Projects Releases Wiki Activity

doc(patching): update patching docs

Browse Source
This commit is contained in:
Jason Liu
2024-02-13 18:02:27 -05:00
parent dc4e908106
commit fdc1fc2bb9
1 changed files with 44 additions and 60 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/concepts/patching.md
+44 -60
View File
@@ -1,4 +1,4 @@
# Patching the OpenAI Client
# Patching
Instructor enhances client functionality with three new keywords for backwards compatibility. This allows use of the enhanced client as usual, with structured output benefits.
@@ -6,53 +6,15 @@ Instructor enhances client functionality with three new keywords for backwards c
- `max_retries`: Determines retry attempts for failed `chat.completions.create` validations.
- `validation_context`: Provides extra context to the validation process.
There are two ways to patch the OpenAI client: patching the client itself, or patching a specific function. The former is more general, while the latter is more specific.
Then there are a handful of modes to choose from:
There are three methods for structured output:
1. **Function Calling**: The primary method. Use this for stability and testing.
2. **Tool Calling**: Useful in specific scenarios; lacks the reasking feature of OpenAI's tool calling API.
3. **JSON Mode**: Offers closer adherence to JSON but with more potential validation errors. Suitable for specific non-function calling clients.
4. **Markdown JSON Mode**: Experimental, not recommended.
5. **JSON Schema Mode**: Only available with the [Together](../blog/posts/together.md) and [Anyscale](../blog/posts/anyscale.md) patches.
## Patching Modes
## Tool Calling
We support two kinds of patches One that patches anything isomorphic to the OpenAI client and the other to a specific create call
### Patch Client
```python
import instructor
from openai import OpenAI
client = instructor.patch(OpenAI())
```
### Patch Create
This allows you to patch any function that is isomorphic to `chat.completions.create`. function with messages and other parameters.
```python
import instructor
from openai import OpenAI
create_fn = OpenAI().chat.completions.create
create = instructor.patch(create=create_fn)
```
## Patching Modes
### Function Calling
```python
import instructor
from openai import OpenAI
client = instructor.patch(OpenAI(), mode=instructor.Mode.FUNCTIONS)
```
### Tool Calling
This is the recommended method for OpenAI clients. It is the most stable as functions is being deprecated soon.
```python
import instructor
@@ -61,17 +23,52 @@ from openai import OpenAI
client = instructor.patch(OpenAI(), mode=instructor.Mode.TOOLS)
```
### JSON Mode
## Parallel Tool Calling
Parallel tool calling is also an option but you must set `response_model` to be `Iterable[Union[...]]` types since we expect an array of results. Check out [Parallel Tool Calling](./parallel.md) for more information.
```python
import instructor
from instructor import Mode
from openai import OpenAI
client = instructor.patch(OpenAI(), mode=Mode.JSON)
client = instructor.patch(OpenAI(), mode=instructor.Mode.PARALLEL_TOOLS)
```
### Markdown JSON Mode
## Function Calling
Note that function calling is soon to be deprecated in favor of TOOL mode for OpenAI. But will still be supported for other clients.
```python
import instructor
from openai import OpenAI
client = instructor.patch(OpenAI(), mode=instructor.Mode.FUNCTIONS)
```
## JSON Mode
JSON mode uses OpenAI's JSON fromat for responses. by setting `response_format={"type": "json_object"}` in the `chat.completions.create` method.
```python
import instructor
from openai import OpenAI
client = instructor.patch(OpenAI(), mode=instructor.Mode.JSON)
```
## JSON Schema Mode
JSON Schema mode uses OpenAI's JSON fromat for responses. by setting `response_format={"type": "json_object", schema:response_model.model_json_schema()}` in the `chat.completions.create` method. This is only available for select clients (e.g. llama-cpp-python, Anyscale, Together)
```python
import instructor
from openai import OpenAI
client = instructor.patch(OpenAI(), mode=instructor.Mode.JSON_SCHEMA)
```
## Markdown JSON Mode
This just asks for the response in JSON format, but it is not recommended, and may not be supported in the future, this is just left to support vision models and will not give you the full benefits of instructor.
!!! warning "Experimental"
@@ -83,16 +80,3 @@ from openai import OpenAI
client = instructor.patch(OpenAI(), mode=instructor.Mode.MD_JSON)
```
### JSON Schema Mode
This is only available with the [Together](../blog/posts/together.md) and [Anyscale](../blog/posts/anyscale.md) patches.
```python
import instructor
from openai import OpenAI
client = instructor.patch(OpenAI(
...
), mode=instructor.Mode.JSON_SCHEMA)
```