mirror of
https://github.com/kennethreitz/instructor.git
synced 2026-06-05 22:50:18 +00:00
Clean up streaming code (#377)
This commit is contained in:
+17
-1
@@ -99,4 +99,20 @@ for user in users:
|
||||
>>> name="John" "age"=10
|
||||
```
|
||||
|
||||
This streaming is still a prototype, but should work quite well for simple schemas.
|
||||
## Asynchronous Streaming
|
||||
|
||||
I also just want to call out in this example that `instructor` also supports asynchronous streaming. This is useful when you want to stream a response model and process the results as they come in, but you'll need to use the `async for` syntax to iterate over the results.
|
||||
|
||||
```python
|
||||
model = await client.chat.completions.create(
|
||||
model="gpt-4",
|
||||
response_model=Iterable[UserExtract],
|
||||
max_retries=2,
|
||||
stream=stream,
|
||||
messages=[
|
||||
{"role": "user", "content": "Make two up people"},
|
||||
],
|
||||
)
|
||||
async for m in model:
|
||||
assert isinstance(m, UserExtract)
|
||||
```
|
||||
|
||||
@@ -103,3 +103,21 @@ for extraction in extraction_stream:
|
||||
This will output the following:
|
||||
|
||||

|
||||
|
||||
## Asynchronous Streaming
|
||||
|
||||
I also just want to call out in this example that `instructor` also supports asynchronous streaming. This is useful when you want to stream a response model and process the results as they come in, but you'll need to use the `async for` syntax to iterate over the results.
|
||||
|
||||
```python
|
||||
model = await client.chat.completions.create(
|
||||
model="gpt-4",
|
||||
response_model=Partial[UserExtract],
|
||||
max_retries=2,
|
||||
stream=True,
|
||||
messages=[
|
||||
{"role": "user", "content": "Jason Liu is 12 years old"},
|
||||
],
|
||||
)
|
||||
async for m in model:
|
||||
assert isinstance(m, UserExtract)
|
||||
```
|
||||
Reference in New Issue
Block a user