Clean up streaming code (#377)

This commit is contained in:
Jason Liu
2024-01-31 18:31:33 -05:00
committed by GitHub
parent eeb2fb2f3d
commit ef76cfe97b
12 changed files with 190 additions and 80 deletions
+17 -1
View File
@@ -99,4 +99,20 @@ for user in users:
>>> name="John" "age"=10
```
This streaming is still a prototype, but should work quite well for simple schemas.
## Asynchronous Streaming
I also just want to call out in this example that `instructor` also supports asynchronous streaming. This is useful when you want to stream a response model and process the results as they come in, but you'll need to use the `async for` syntax to iterate over the results.
```python
model = await client.chat.completions.create(
model="gpt-4",
response_model=Iterable[UserExtract],
max_retries=2,
stream=stream,
messages=[
{"role": "user", "content": "Make two up people"},
],
)
async for m in model:
assert isinstance(m, UserExtract)
```
@@ -103,3 +103,21 @@ for extraction in extraction_stream:
This will output the following:
![Partial Streaming Gif](../img/partial.gif)
## Asynchronous Streaming
I also just want to call out in this example that `instructor` also supports asynchronous streaming. This is useful when you want to stream a response model and process the results as they come in, but you'll need to use the `async for` syntax to iterate over the results.
```python
model = await client.chat.completions.create(
model="gpt-4",
response_model=Partial[UserExtract],
max_retries=2,
stream=True,
messages=[
{"role": "user", "content": "Jason Liu is 12 years old"},
],
)
async for m in model:
assert isinstance(m, UserExtract)
```