Add streaming support and update documentation

This commit is contained in:
2024-11-02 16:54:36 -04:00
parent 4d2c81850e
commit a68bd74fd8
2 changed files with 8 additions and 1 deletions
+1 -1
View File
@@ -3,7 +3,7 @@ Release History
## 0.2.2 (2024-11-02)
- Add openai streaming support (set `stream=True` to `generate_text`).
- Add streaming support (set `stream=True` to `generate_text`).
- `conv.prepend_system_message` now uses system role by default.
- Add `provider.supports_streaming` property.
- Add `provider.supports_structured_response` property.
+7
View File
@@ -104,6 +104,13 @@ Generate a response from an AI model based on a given prompt:
"The meaning of life is a profound philosophical question that has been explored by cultures, religions, and philosophers for centuries. Different people and belief systems offer varying interpretations:\n\n1. **Religious Perspectives:** Many religions propose that the meaning of life is to fulfill a divine purpose, serve God, or reach an afterlife. For example, Christianity often emphasizes love, faith, and service to God and others as central to lifes meaning.\n\n2. **Philosophical Views:** Philosophers offer diverse answers. Existentialists like Jean-Paul Sartre argue that life has no inherent meaning, and it is up to individuals to create their own purpose. Others, like Aristotle, suggest that achieving eudaimonia (flourishing or happiness) through virtuous living is the key to a meaningful life.\n\n3. **Scientific and Secular Approaches:** Some people find meaning through understanding the natural world, contributing to human knowledge, or through personal accomplishments and happiness. They may view life's meaning as a product of connection, legacy, or the pursuit of knowledge and creativity.\n\n4. **Personal Perspective:** For many, the meaning of life is deeply personal, involving their relationships, passions, and goals. These individuals define life's purpose through experiences, connections, and the impact they have on others and the world.\n\nUltimately, the meaning of life is a subjective question, with each person finding their own answers based on their beliefs, experiences, and reflections."
```
### Streaming Text
```python
>>> for chunk in sm.generate_text("Write a poem about the moon", stream=True):
... print(chunk, end="", flush=True)
```
### Structured Data with Pydantic
You can use Pydantic models to structure the response from the LLM, if the LLM supports it.