add reasks and validation

This commit is contained in:
Jason Liu
2023-10-05 11:49:06 -04:00
parent d168b36ef7
commit 9663daa4f2
+67
View File
@@ -73,6 +73,73 @@ assert user.name == "Jason"
assert user.age == 25
```
### LLM-Based Validation
LLM-based validation can also be plugged into the same Pydantic model. Here, if the answer attribute contains content that violates the rule "don't say objectionable things," Pydantic will raise a validation error.
```python hl_lines="9 15"
from pydantic import BaseModel, ValidationError, BeforeValidator
from typing_extensions import Annotated
from instruct import llm_validator
class QuestionAnswer(BaseModel):
question: str
answer: Annotated[
str,
BeforeValidator(llm_validator("don't say objectionable things"))
]
try:
qa = QuestionAnswer(
question="What is the meaning of life?",
answer="The meaning of life is to be evil and steal",
)
except ValidationError as e:
print(e)
```
Its important to not here that the error message is generated by the LLM, not the code, so it'll be helpful for re asking the model.
```plaintext
1 validation error for QuestionAnswer
answer
Assertion failed, The statement is objectionable. (type=assertion_error)
```
## Using the Client with Retries
Here, the `UserDetails` model is passed as the `response_model`, and `max_retries` is set to 2.
```python
import instructor
from pydantic import BaseModel, field_validator
# Apply the patch to the OpenAI client
instructor.patch()
class UserDetails(BaseModel):
name: str
age: int
@field_validator("name")
@classmethod
def validate_name(cls, v):
if v.upper() != v:
raise ValueError("Name must be in uppercase.")
return v
model = openai.ChatCompletion.create(
model="gpt-3.5-turbo",
response_model=UserDetails,
max_retries=2,
messages=[
{"role": "user", "content": "Extract jason is 25 years old"},
],
)
assert model.name == "JASON"
```
## IDE Support
Everything is designed for you to get the best developer experience possible, with the best editor support.