Add streaming support to Groq provider and implement generate_stream_text method

This commit refactors the `create_conversation` function in `simplemind/__init__.py` to use a more descriptive variable name (`conv`) for the conversation object. It also updates the `add_plugin` method to use the new variable name (`conv`) instead of `conversation`.

In `simplemind/models.py`, the `prepend_system_message` method now accepts an optional `meta` parameter. The method also adds a system message to the conversation by prepending it to the list of messages.

Additionally, the `add_message` method in `simplemind/models.py` has been modified to include type annotations and a default value for the `role` parameter. The method now requires the `text` parameter to be provided explicitly.

A new test file, `tests/test_conversations.py`, has been added to the repository. This file contains a test case for the `generate_data` function, which tests the functionality of different LLM providers.

Lastly, the test files `tests/test_generate_data.py` and `tests/test_generate_text.py` have been modified to remove the unused `Amazon` provider from the list of test cases.

.envrc.template

@@ -4,3 +4,4 @@ export GROQ_API_KEY=""
 export OLLAMA_HOST_URL=""
 export OPENAI_API_KEY=""
 export XAI_API_KEY=""
+export AMAZON_PROFILE_NAME=""

CHANGELOG.md

@@ -1,6 +1,24 @@
 Release History
 ===============
 
+## 0.2.2 (2024-11-02)
+
+- Add openai streaming support (set `stream=True` to `generate_text`).
+- `conv.prepend_system_message` now uses system role by default.
+- Add `provider.supports_streaming` property.
+- Add `provider.supports_structured_response` property.
+- General improvements.
+
+## 0.2.1 (2024-11-01)
+
+- Add `cached_property` to Amazon provider.
+
+## 0.2.0 (2024-11-01)
+
+- Add Amazon Bedrock provider.
+- Make all provider optional dependencies. Use `$ pip install 'simplemind[full]'` to install all providers.
+- General improvements.
+
 ## 0.1.7 (2024-11-01)
 
 - Add `logger` decorator.

Dockerfile

@@ -0,0 +1,21 @@
+FROM python:3.12-slim
+
+# Install system dependencies
+RUN apt-get update && apt-get install -y \
+    git \
+    && rm -rf /var/lib/apt/lists/*
+
+# Install uv
+RUN pip install uv
+
+# Create and set working directory
+WORKDIR /app
+
+# Copy requirements/project files
+ONBUILD COPY . .
+
+# Install dependencies using uv
+RUN uv pip install "simplemind[full]" --system
+
+# Set default command
+CMD ["python"]

README.md

@@ -10,36 +10,71 @@ Simplemind is AI library designed to simplify your experience with AI APIs in Py
 
 With Simplemind, tapping into AI is as easy as a friendly conversation.
 
-- **Easy-to-use AI tools**: SimpleMind provides simple interfaces to popular AI services.
+- **Easy-to-use AI tools**: Simplemind provides simple interfaces to most popular AI services.
 - **Human-centered design**: The library prioritizes readability and usability—no need to be an expert to start experimenting.
 - **Minimal configuration**: Get started quickly, without worrying about configuration headaches.
 
 ## Supported APIs
 
-To specify a specific provider or model, you can use the `llm_provider` and `llm_model` parameters when calling: `generate_text`, `generate_data`, or `create_conversation`. The APIs remain identital between all supported providers/models.
+The APIs remain identical between all supported providers / models:
 
-- [**Anthropic's Claude**](https://www.anthropic.com/claude)
-- [**Google's Gemini**](https://gemini.google/)
-- [**Groq's Groq**](https://groq.com/)
-- [**Ollama**](https://ollama.com)
-- [**OpenAI's GPT**](https://openai.com/gpt)
-- [**xAI's Grok**](https://x.ai/)
+<table>
+  <thead>
+    <tr>
+      <th></th>
+      <th><code>llm_provider</code></th>
+      <th>Default <code>llm_model</code></th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td><a href="https://www.anthropic.com/claude">Anthropic's Claude</a></td>
+      <td><code>"anthropic"</code></td>
+      <td><code>"claude-3-5-sonnet-20241022"</code></td>
+    </tr>
+    <tr>
+      <td><a href="https://aws.amazon.com/bedrock/">Amazon's Bedrock</a></td>
+      <td><code>"amazon"</code></td>
+      <td><code>"anthropic.claude-3-sonnet-20240229-v1:0"</code></td>
+    </tr>
+    <tr>
+      <td><a href="https://gemini.google/">Google's Gemini</a></td>
+      <td><code>"gemini"</code></td>
+      <td><code>"models/gemini-1.5-pro"</code></td>
+    </tr>
+    <tr>
+      <td><a href="https://groq.com/">Groq's Groq</a></td>
+      <td><code>"groq"</code></td>
+      <td><code>"llama3-8b-8192"</code></td>
+    </tr>
+    <tr>
+      <td><a href="https://ollama.com">Ollama</a></td>
+      <td><code>"ollama"</code></td>
+      <td><code>"llama3.2"</code></td>
+    </tr>
+    <tr>
+      <td><a href="https://openai.com/gpt">OpenAI's GPT</a></td>
+      <td><code>"openai"</code></td>
+      <td><code>"gpt-4o-mini"</code></td>
+    </tr>
+    <tr>
+      <td><a href="https://x.ai/">xAI's Grok</a></td>
+      <td><code>"xai"</code></td>
+      <td><code>"grok-beta"</code></td>
+    </tr>
+  </tbody>
+</table>
 
-If you want to see Simplemind support, additional providers or models, please send a pull request!
+To specify a specific provider or model, you can use the `llm_provider` and `llm_model` parameters when calling: `generate_text`, `generate_data`, or `create_conversation`.
 
-## Why SimpleMind?
-- **Intuitive**: Built with Pythonic simplicity and readability in mind.
-- **For Humans**: Emphasizes a human-friendly interface, just like `requests` for HTTP.
-- **Open Source**: Simplemind is open source, and contributions are always welcome!
-
-Also, why not? :)
+If you want to see Simplemind support additional providers or models, please send a pull request!
 
 ## Quickstart
 
 Simplemind takes care of the complex API calls so you can focus on what matters—building, experimenting, and creating.
 
 ```bash
-$ pip install simplemind
+$ pip install 'simplemind[full]'
 ```
 
 First, authenticate your API keys by setting them in the environment variables:
@@ -56,7 +91,6 @@ Next, import Simplemind and start using it:
 import simplemind as sm
 ```
 
-
 ## Examples
 
 Here are some examples of how to use Simplemind:
@@ -90,25 +124,52 @@ class Poem(BaseModel):
 title='Eternal Embrace' content='In the quiet hours of the night,\nWhen stars whisper secrets bright,\nTwo hearts beat in a gentle rhyme,\nDancing through the sands of time.\n\nWith every glance, a spark ignites,\nA flame that warms the coldest nights,\nIn laughter shared and whispers sweet,\nLove paints the world, a masterpiece.\n\nThrough stormy skies and sunlit days,\nIn myriad forms, it finds its ways,\nA tender touch, a knowing sigh,\nIn love’s embrace, we learn to fly.\n\nAs seasons change and moments fade,\nIn the tapestry of dreams we’ve laid,\nLove’s threads endure, forever bind,\nA timeless bond, two souls aligned.\n\nSo here’s to love, both bright and true,\nA gift we give, anew, anew,\nIn every heartbeat, every prayer,\nA story written in the air.'
 ```
 
+#### A more complex example
+
+```python
+class InstructionStep(BaseModel):
+    step_number: int
+    instruction: str
+
+class RecipeIngredient(BaseModel):
+    name: str
+    quantity: float
+    unit: str
+
+class Recipe(BaseModel):
+    name: str
+    ingredients: list[RecipeIngredient]
+    instructions: list[InstructionStep]
+
+recipe = sm.generate_data(
+    "Write a recipe for chocolate chip cookies",
+    llm_model="gpt-4o-mini",
+    llm_provider="openai",
+    response_model=Recipe,
+)
+```
+
+Special thanks to [@jxnl](https://github.com/jxnl) for building [Instructor](https://github.com/jxnl/instructor), which makes this possible!
+
 ### Conversational AI
 
 SimpleMind also allows for easy conversational flows:
 
 ```pycon
->>> conversation = sm.create_conversation(llm_model="gpt-4o-mini", llm_provider="openai")
+>>> conv = sm.create_conversation(llm_model="gpt-4o-mini", llm_provider="openai")
 
 >>> # Add a message to the conversation
->>> conversation.add_message("user", "Hi there, how are you?")
+>>> conv.add_message("user", "Hi there, how are you?")
 
->>> conversation.send()
+>>> conv.send()
 <Message role=assistant text="Hello! I'm just a computer program, so I don't have feelings, but I'm here and ready to help you. How can I assist you today?">
 ```
 
-To continue the conversation, you can call `conversation.send()` again, which returns the next message in the conversation:
+To continue the conversation, you can call `conv.send()` again, which returns the next message in the conversation:
 
 ```pycon
->>> conversation.add_message("user", "What is the meaning of life?")
->>> conversation.send()
+>>> conv.add_message("user", "What is the meaning of life?")
+>>> conv.send()
 <Message role=assistant text="The meaning of life is a profound philosophical question that has been explored by cultures, religions, and philosophers for centuries. Different people and belief systems offer varying interpretations:\n\n1. **Religious Perspectives:** Many religions propose that the meaning of life is to fulfill a divine purpose, serve God, or reach an afterlife. For example, Christianity often emphasizes love, faith, and service to God and others as central to life’s meaning.\n\n2. **Philosophical Views:** Philosophers offer diverse answers. Existentialists like Jean-Paul Sartre argue that life has no inherent meaning, and it is up to individuals to create their own purpose. Others, like Aristotle, suggest that achieving eudaimonia (flourishing or happiness) through virtuous living is the key to a meaningful life.\n\n3. **Scientific and Secular Approaches:** Some people find meaning through understanding the natural world, contributing to human knowledge, or through personal accomplishments and happiness. They may view life’s meaning as a product of connection, legacy, or the pursuit of knowledge and creativity.\n\n4. **Personal Perspective:** For many, the meaning of life is deeply personal, involving their relationships, passions, and goals. These individuals define life’s purpose through experiences, connections, and the impact they have on others and the world.\n\nUltimately, the meaning of life is a subjective question, with each person finding their own answers based on their beliefs, experiences, and reflections.">
 ```
 
@@ -163,6 +224,7 @@ conversation.add_message(
     text="Please write a poem about the moon",
 )
 ```
+
 ```pycon
 >>> conversation.send()
 In the vast expanse where stars do play,
@@ -198,11 +260,18 @@ The universe is never done.
 
 Simple, yet effective.
 
+### Logging
+
+Simplemind uses [Logfire](https://pydantic.dev/logfire) for logging. To enable logging, call `sm.enable_logfire()`.
+
+### More Examples
+
 Please see the [examples](examples) directory for executable examples.
 
--------------------
+---
 
 ## Contributing
+
 We welcome contributions of all kinds. Feel free to open issues for bug reports or feature requests, and submit pull requests to make SimpleMind even better.
 
 To get started:
@@ -213,8 +282,9 @@ To get started:
 4. Submit a pull request.
 
 ## License
+
 Simplemind is licensed under the Apache 2.0 License.
 
 ## Acknowledgements
-Simplemind is inspired by the philosophy of "code for humans" and aims to make working with AI models accessible to all. Special thanks to the open-source community for their contributions and inspiration.
 
+Simplemind is inspired by the philosophy of "code for humans" and aims to make working with AI models accessible to all. Special thanks to the open-source community for their contributions and inspiration.

docs/conf.py

@@ -16,7 +16,7 @@ import simplemind
 project = "simplemind"
 copyright = "2024 Kenneth Reitz"
 author = "Kenneth Reitz"
-release = "v0.1.7"
+release = "v0.2.2"
 
 # -- General configuration ---------------------------------------------------
 # https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration

examples/_context.py

@@ -5,6 +5,7 @@ import sys
 sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), "..")))
 
 
+import simplemind
 import simplemind as sm
 
-__all__ = ["sm"]
+__all__ = ["simplemind", "sm"]

examples/bible_verses.py

@@ -0,0 +1,137 @@
+from _context import simplemind as sm
+from pydantic import BaseModel
+from rich.console import Console
+from rich.panel import Panel
+from rich.text import Text
+
+console = Console()
+
+gpt_4o_mini = sm.Session(llm_provider="openai")
+claude_sonnet = sm.Session(llm_provider="anthropic")
+
+
+class BibleVerse(BaseModel):
+    book: str
+    chapter: int
+    verse: int
+    text: str
+    translation: str
+
+
+class BiblePassage(BaseModel):
+    book: str
+    chapter: int
+    verses: list[BibleVerse]
+    translation: str
+
+
+class CrossReference(BaseModel):
+    passage: BiblePassage
+    notes: list[str]
+    origin_verse: BibleVerse
+    ai_perspective: str
+    anthropic_perspective: str
+
+
+def get_passage(book: str, chapter: int, translation: str = "ESV") -> BiblePassage:
+    passage = gpt_4o_mini.generate_data(
+        prompt=f"""Return {book} chapter {chapter} from the {translation} translation.
+Format each verse as plain text without any special characters or formatting.
+For example:
+- "Love is patient, love is kind."
+- "It does not envy, it does not boast"
+
+Return only the biblical text, formatted as a BiblePassage object.""",
+        response_model=BiblePassage,
+        max_tokens=8000,
+    )
+    return passage
+
+
+def get_cross_reference(passage: BiblePassage) -> CrossReference:
+    verses_text = "\n".join([f"Verse {v.verse}: {v.text}" for v in passage.verses])
+
+    # Get main cross-reference from OpenAI
+    ref = gpt_4o_mini.generate_data(
+        prompt=f"""Find a thematically related Bible passage that connects with this text:
+{verses_text}
+
+Return a CrossReference object with:
+1. A related passage (using plain text without special characters)
+2. A list of clear, specific notes explaining the thematic connections
+3. The original passage included
+4. An AI perspective that provides a thoughtful, modern interpretation of how these passages relate to contemporary life and universal human experiences""",
+        response_model=CrossReference,
+    )
+
+    # Get Anthropic's perspective separately
+    anthropic_insight = claude_sonnet.generate_text(
+        prompt=f"""Analyze these biblical passages from a philosophical and ethical perspective:
+
+Original passage:
+{verses_text}
+
+Cross-reference passage:
+{' '.join([f'Verse {v.verse}: {v.text}' for v in ref.passage.verses])}
+
+Provide a thoughtful analysis focusing on the philosophical and ethical implications of these passages, drawing from your training in ethics and philosophy.
+Return your response as a plain string.""",
+    )
+
+    # Add Anthropic's perspective to the reference object
+    ref.anthropic_perspective = anthropic_insight
+    return ref
+
+
+def pretty_print_reference(ref: CrossReference):
+    # Create origin passage panel
+    origin_text = Text()
+    origin_text.append(
+        f"{ref.origin_verse.book} {ref.origin_verse.chapter}\n",
+        style="bold blue",
+    )
+    origin_text.append(f"{ref.origin_verse.verse}. ", style="blue")
+    origin_text.append(f"{ref.origin_verse.text}\n", style="italic")
+    origin_text.append(f"\n({ref.origin_verse.translation})", style="dim")
+
+    origin_panel = Panel(origin_text, title="Original Passage", border_style="blue")
+
+    # Create cross reference panel
+    ref_text = Text()
+    ref_text.append(
+        f"{ref.passage.book} {ref.passage.chapter}\n",
+        style="bold green",
+    )
+    for verse in ref.passage.verses:
+        ref_text.append(f"{verse.verse}. ", style="green")
+        ref_text.append(f"{verse.text}\n", style="italic")
+    ref_text.append(f"\n({ref.passage.translation})", style="dim")
+
+    ref_panel = Panel(ref_text, title="Cross Reference", border_style="green")
+
+    # Create notes panel with bullet points
+    notes_text = Text()
+    for note in ref.notes:
+        notes_text.append("• ", style="yellow")
+        notes_text.append(f"{note}\n")
+
+    notes_panel = Panel(notes_text, title="Thematic Connections", border_style="yellow")
+
+    # Add new AI perspective panel
+    ai_text = Text()
+    ai_text.append(ref.ai_perspective)
+
+    ai_panel = Panel(ai_text, title="AI Perspective", border_style="magenta")
+
+    # Print all panels
+    console.print(origin_panel)
+    console.print(ref_panel)
+    console.print(notes_panel)
+    console.print(ai_panel)
+
+
+if __name__ == "__main__":
+    # Get 1 Corinthians 13 (The Love Chapter)
+    passage = get_passage("1 Corinthians", 13)
+    ref = get_cross_reference(passage)
+    pretty_print_reference(ref)

examples/cooking_recipe_example.py

@@ -0,0 +1,66 @@
+from pydantic import BaseModel
+import simplemind as sm
+
+
+class InstructionStep(BaseModel):
+    step_number: int
+    instruction: str
+
+class RecipeIngredient(BaseModel):
+    name: str
+    quantity: float
+    unit: str
+
+class Recipe(BaseModel):
+    name: str
+    ingredients: list[RecipeIngredient]
+    instructions: list[InstructionStep]
+    
+    def __str__(self) -> str:
+        output = f"\n=== {self.name.upper()} ===\n\n"
+        
+        output += "INGREDIENTS:\n"
+        for ing in self.ingredients:
+            output += f"• {ing.quantity} {ing.unit} {ing.name}\n"
+        
+        output += "\nINSTRUCTIONS:\n"
+        for step in self.instructions:
+            output += f"{step.step_number}. {step.instruction}\n"
+        
+        return output
+    
+
+recipe = sm.generate_data(
+    "Write a recipe for chocolate chip cookies",
+    llm_model="gpt-4o-mini",
+    llm_provider="openai",
+    response_model=Recipe,
+)
+
+print(recipe)
+# Expected output is something like this:
+#
+# === CHOCOLATE CHIP COOKIES ===
+#
+# INGREDIENTS:
+# • 2.25 cups all-purpose flour
+# • 1.0 teaspoon baking soda
+# • 0.5 teaspoon salt
+# • 1.0 cup unsalted butter
+# • 0.75 cup sugar
+# • 0.75 cup brown sugar
+# • 1.0 teaspoon vanilla extract
+# • 2.0 large eggs
+# • 2.0 cups semi-sweet chocolate chips
+#
+# INSTRUCTIONS:
+# 1. Preheat your oven to 350°F (175°C).
+# 2. In a small bowl, combine flour, baking soda, and salt; set aside.
+# 3. In a large bowl, cream together the butter, sugar, and brown sugar until smooth.
+# 4. Beat in the vanilla extract and eggs, one at a time.
+# 5. Gradually blend in the flour mixture until just combined.
+# 6. Stir in the chocolate chips.
+# 7. Drop by rounded tablespoon onto ungreased cookie sheets.
+# 8. Bake for 9 to 11 minutes, or until edges are golden.
+# 9. Let cool on the cookie sheet for a few minutes before transferring to wire racks to cool completely.
+

examples/discussion.py

@@ -0,0 +1,132 @@
+import time
+from typing import List, Tuple
+
+from rich.console import Console
+from rich.markdown import Markdown
+
+from _context import sm
+
+
+class MultiAIConversation:
+    """Orchestrates conversations between multiple AI models."""
+
+    MODEL_SESSIONS = {
+        "Llama3.2": sm.Session(
+            llm_provider="ollama",
+            llm_model="llama3.2",
+        ),
+        "Claude-3.5-Sonnet": sm.Session(
+            llm_provider="anthropic",
+            llm_model="claude-3-5-sonnet-20241022",
+        ),
+        "GPT-4o": sm.Session(
+            llm_provider="openai",
+            llm_model="gpt-4o",
+        ),
+        "Grok-Beta": sm.Session(
+            llm_provider="xai",
+            llm_model="grok-beta",
+        ),
+    }
+
+    def __init__(self, topic: str, turns_per_model: int = 1, max_rounds: int = 5):
+
+        self.topic = topic
+        self.turns_per_model = turns_per_model
+        self.max_rounds = max_rounds
+        self.conversation_history: List[Tuple[str, str]] = []
+        self.console = Console()
+
+    def _format_system_prompt(self, ai_name: str) -> str:
+        """Creates a system prompt for each AI model."""
+        return f"""You are {ai_name}. You are participating in a thoughtful discussion with other AI models about {self.topic}.
+
+Rules:
+1. Be concise but insightful (keep responses under 100 words)
+2. Build upon previous points made in the conversation
+3. Ask questions to deepen the discussion when appropriate
+4. Stay on topic while maintaining your unique perspective
+5. Be respectful of other viewpoints while maintaining your distinct voice
+
+Current discussion topic: {self.topic}"""
+
+    def _create_conversation(
+        self, session: sm.Session, ai_name: str
+    ) -> sm.Conversation:
+        """Creates a new conversation with appropriate context for an AI model."""
+        conv = session.create_conversation()
+
+        # Add system prompt
+        conv.add_message(role="user", text=self._format_system_prompt(ai_name))
+
+        # Add conversation history
+        for speaker, message in self.conversation_history[-3:]:  # Last 3 messages
+            conv.add_message(role="user", text=f"{speaker} said: {message}")
+
+        return conv
+
+    def _print_response(self, ai_name: str, response: str):
+        """Pretty prints an AI response using Rich."""
+        self.console.print(f"\n[bold blue]{ai_name}[/bold blue]:")
+        self.console.print(Markdown(response))
+        # Store in history
+        self.conversation_history.append((ai_name, response))
+
+    def run_conversation(self):
+        """Runs the multi-AI conversation."""
+
+        # Initialize the conversation
+        initial_prompt = (
+            f"Let's have a thoughtful discussion about {self.topic}. "
+            "Please share your initial thoughts in 2-3 sentences."
+        )
+
+        for round_num in range(self.max_rounds):
+            self.console.print(f"\n[bold green]Round {round_num + 1}[/bold green]")
+
+            for model_name, session in self.MODEL_SESSIONS.items():
+                for turn in range(self.turns_per_model):
+                    conversation = self._create_conversation(session, model_name)
+
+                    # Add the prompt
+                    prompt = (
+                        initial_prompt
+                        if round_num == 0 and turn == 0
+                        else (
+                            f"Continue the discussion about {self.topic}, "
+                            "responding to the previous points made."
+                        )
+                    )
+
+                    conversation.add_message(role="user", text=prompt)
+
+                    # Get and print response
+                    response = conversation.send()
+                    self._print_response(model_name, response.text)
+
+                    # Small delay to prevent rate limiting
+                    time.sleep(1)
+
+            # Optional: Add a separator between rounds
+            self.console.print("\n" + "-" * 50)
+
+
+def have_ai_discussion(topic: str, turns_per_model: int = 1, max_rounds: int = 3):
+    """Convenience function to start an AI discussion."""
+    debate = MultiAIConversation(
+        topic=topic, turns_per_model=turns_per_model, max_rounds=max_rounds
+    )
+
+    print(f"\nStarting AI discussion on: {topic}")
+    print("=" * 50)
+
+    debate.run_conversation()
+
+
+# Example usage
+if __name__ == "__main__":
+    # Example topics
+    topic = "The future of human-AI collaboration in creative fields",
+
+    # Run a discussion on the first topic
+    have_ai_discussion(topic=topic, turns_per_model=1, max_rounds=3)

examples/generate_stream_text.py

@@ -0,0 +1,7 @@
+from _context import sm
+
+# Defaults to the default provider (openai)
+r = sm.generate_text("Write a poem about the moon", llm_provider="gemini", stream=True)
+
+for chunk in r:
+    print(chunk, end="", flush=True)

examples/math_plugin.py

@@ -2,7 +2,7 @@ from _context import sm
 
 
 class MathPlugin(sm.BasePlugin):
-    def send_hook(self, conversation: sm.Conversation):
+    def pre_send_hook(self, conversation: sm.Conversation):
         last_user_message = conversation.get_last_message(role="user")
         if last_user_message is None:
             return

examples/medicine_data.py

@@ -0,0 +1,94 @@
+from pydantic import BaseModel
+from _context import simplemind as sm
+from rich.console import Console
+from rich.panel import Panel
+from rich.table import Table
+
+
+class SideEffect(BaseModel):
+    effect: str
+    severity: str  # mild, moderate, severe
+    frequency: str  # common, uncommon, rare
+
+
+class Medication(BaseModel):
+    brand_name: str
+    generic_name: str
+    drug_class: str
+    half_life: str
+    common_uses: list[str]
+    side_effects: list[SideEffect]
+    typical_dosage: str
+    warnings: list[str]
+
+
+class MedicationList(BaseModel):
+    root: list[Medication]
+
+
+# Create a session with your preferred model
+session = sm.Session(llm_provider="openai", llm_model="gpt-4o-mini")
+
+
+# Update the prompt to use an f-string with a parameter
+def get_medication_prompt(medications: list[str]) -> str:
+    return f"""
+Provide detailed medical information about {', '.join(medications)}.
+Include their generic names, drug classes, half-lives, common uses, side effects (with severity and frequency),
+typical dosages, and important warnings.
+Return the information as separate medication entries.
+"""
+
+
+# Example usage
+medications_to_lookup = ["Abilify (aripiprazole)", "Trileptal (oxcarbazepine)"]
+prompt = get_medication_prompt(medications_to_lookup)
+
+# Generate structured data for medications
+medications = session.generate_data(prompt=prompt, response_model=MedicationList)
+
+# Create a Rich console
+console = Console()
+
+# Replace the print section with Rich formatting
+for med in medications.root:
+    # Create a table for the medication details
+    table = Table(show_header=False, box=None)
+    table.add_row("[bold cyan]Generic Name:[/]", med.generic_name)
+    table.add_row("[bold cyan]Drug Class:[/]", med.drug_class)
+    table.add_row("[bold cyan]Half Life:[/]", med.half_life)
+
+    # Create a nested table for common uses
+    uses_table = Table(show_header=False, box=None, padding=(0, 2))
+    for use in med.common_uses:
+        uses_table.add_row("•", use)
+
+    # Create a nested table for side effects
+    effects_table = Table(show_header=False, box=None, padding=(0, 2))
+    for effect in med.side_effects:
+        severity_color = {"mild": "green", "moderate": "yellow", "severe": "red"}.get(
+            effect.severity.lower(), "white"
+        )
+        effects_table.add_row(
+            "•",
+            effect.effect,
+            f"[{severity_color}]{effect.severity}[/]",
+            f"({effect.frequency})",
+        )
+
+    # Create a nested table for warnings
+    warnings_table = Table(show_header=False, box=None, padding=(0, 2))
+    for warning in med.warnings:
+        warnings_table.add_row("•", f"[red]{warning}[/]")
+
+    # Add the nested tables to the main table
+    table.add_row("[bold cyan]Common Uses:[/]", uses_table)
+    table.add_row("[bold cyan]Side Effects:[/]", effects_table)
+    table.add_row("[bold cyan]Typical Dosage:[/]", med.typical_dosage)
+    table.add_row("[bold cyan]Warnings:[/]", warnings_table)
+
+    # Create and print a panel for each medication
+    console.print(
+        Panel(table, title=f"[bold blue]{med.brand_name}[/]", border_style="blue")
+    )
+    console.print()  # Add a blank line between medications

examples/requirements.txt

@@ -2,3 +2,4 @@ numpy
 openai
 pydantic
 faiss-cpu
+rich

examples/sentiment_analysis.py

@@ -3,6 +3,7 @@ from typing import Literal
 from _context import sm
 from pydantic import BaseModel
 
+# Note: you should probably be using textblob for this.
 
 class SentimentAnalysis(BaseModel):
     sentiment: Literal["positive", "negative", "neutral"]

examples/simple_memory.py

@@ -13,7 +13,7 @@ class SimpleMemoryPlugin:
 
     def initialize_hook(self, conversation: sm.Conversation):
         for m in self.yield_memories():
-            conversation.prepend_system_message(role="system", text=m)
+            conversation.prepend_system_message(text=m)
 
 
 conversation = sm.create_conversation(llm_model="grok-beta", llm_provider="xai")

pyproject.toml

@@ -1,10 +1,27 @@
 [project]
 name = "simplemind"
-version = "0.1.7"
+version = "0.2.2"
 description = "An experimental client for AI providers that intends to replace LangChain and LangGraph for most common use cases."
 readme = "README.md"
 requires-python = ">=3.10"
-dependencies = ["pydantic", "pydantic-settings", "instructor", "openai", "anthropic", "ollama", "groq", "google-generativeai"]
+dependencies = ["pydantic", "pydantic-settings", "instructor", "logfire"]
+
+[project.optional-dependencies]
+full = [
+    "openai",
+    "anthropic",
+    "ollama",
+    "groq",
+    "google-generativeai",
+    "botocore",
+    "boto3"
+]
+openai = ["openai"]
+anthropic = ["anthropic"]
+ollama = ["ollama", "openai"]
+groq = ["groq"]
+gemini = ["google-generativeai"]
+amazon = ["boto3", "botocore", "anthropic"]
 
 [build-system]
 requires = ["hatchling"]

simplemind/__init__.py

@@ -1,4 +1,4 @@
-from typing import List, Type
+from typing import Generator, List, Type
 
 from .models import BaseModel, BasePlugin, Conversation
 from .settings import settings
@@ -64,16 +64,16 @@ def create_conversation(
     """Create a new conversation."""
 
     # Create the conversation.
-    conversation = Conversation(
+    conv = Conversation(
         llm_model=llm_model,
         llm_provider=llm_provider or settings.DEFAULT_LLM_PROVIDER,
     )
 
     # Add plugins to the conversation.
     for plugin in plugins or []:
-        conversation.add_plugin(plugin)
+        conv.add_plugin(plugin)
 
-    return conversation
+    return conv
 
 
 def generate_data(
@@ -94,6 +94,7 @@ def generate_data(
         prompt=prompt,
         llm_model=llm_model,
         response_model=response_model,
+        **kwargs,
     )
 
 
@@ -102,6 +103,7 @@ def generate_text(
     *,
     llm_model: str | None = None,
     llm_provider: str | None = None,
+    stream: bool = False,
     **kwargs,
 ) -> str:
     """Generate text from a given prompt."""
@@ -110,7 +112,15 @@ def generate_text(
     provider = find_provider(llm_provider or settings.DEFAULT_LLM_PROVIDER)
 
     # Generate the text.
-    return provider.generate_text(prompt=prompt, llm_model=llm_model, **kwargs)
+    if stream:
+        if not provider.supports_streaming:
+            raise ValueError(f"{provider} does not support streaming.")
+
+        return provider.generate_stream_text(
+            prompt=prompt, llm_model=llm_model, **kwargs
+        )
+    else:
+        return provider.generate_text(prompt=prompt, llm_model=llm_model, **kwargs)
 
 
 def enable_logfire() -> None:

simplemind/models.py

@@ -116,17 +116,23 @@ class Conversation(SMBaseModel):
                 except NotImplementedError:
                     pass
 
-    def prepend_system_message(
-        self, role: MESSAGE_ROLE, text: str, meta: Dict[str, Any] | None = None
-    ):
+    def prepend_system_message(self, text: str, meta: Dict[str, Any] | None = None):
         """Prepend a system message to the conversation."""
-        self.messages = [Message(role=role, text=text, meta=meta or {})] + self.messages
+        self.messages = [
+            Message(role="system", text=text, meta=meta or {})
+        ] + self.messages
 
     def add_message(
-        self, role: MESSAGE_ROLE, text: str, meta: Optional[Dict[str, Any]] = None
+        self,
+        role: MESSAGE_ROLE = "user",
+        text: str | None = None,
+        *,
+        meta: Optional[Dict[str, Any]] = None,
     ):
         """Add a new message to the conversation."""
 
+        assert text is not None
+
         # Ensure meta is a dict.
         if meta is None:
             meta = {}
@@ -151,6 +157,8 @@ class Conversation(SMBaseModel):
     ) -> Message:
         """Send the conversation to the LLM."""
 
+        # TODO: llm_model and llm_provider should override the conversation's.
+
         # Execute all pre send hooks.
         for plugin in self.plugins:
             if hasattr(plugin, "pre_send_hook"):

simplemind/providers/__init__.py

@@ -7,5 +7,6 @@ from .groq import Groq
 from .ollama import Ollama
 from .openai import OpenAI
 from .xai import XAI
+from .amazon import Amazon
 
-providers: List[Type[BaseProvider]] = [Anthropic, Gemini, Groq, OpenAI, Ollama, XAI]
+providers: List[Type[BaseProvider]] = [Anthropic, Gemini, Groq, OpenAI, Ollama, XAI, Amazon]

simplemind/providers/_base.py

@@ -16,6 +16,8 @@ class BaseProvider(ABC):
 
     NAME: str
     DEFAULT_MODEL: str
+    supports_streaming: bool = False
+    supports_structured_responses: bool = True
 
     @cached_property
     @abstractmethod
@@ -40,6 +42,6 @@ class BaseProvider(ABC):
         raise NotImplementedError
 
     @abstractmethod
-    def generate_text(self, prompt: str, **kwargs) -> str:
+    def generate_text(self, prompt: str, *, stream: bool = False, **kwargs) -> str:
         """Generate text from a prompt."""
         raise NotImplementedError

simplemind/providers/amazon.py

@@ -0,0 +1,116 @@
+from typing import Type, TypeVar
+from functools import cached_property
+
+import instructor
+from pydantic import BaseModel
+
+from ._base import BaseProvider
+from ..settings import settings
+
+T = TypeVar("T", bound=BaseModel)
+
+PROVIDER_NAME = "amazon"
+DEFAULT_MODEL = "anthropic.claude-3-sonnet-20240229-v1:0"
+DEFAULT_MAX_TOKENS = 5_000
+
+
+class Amazon(BaseProvider):
+    NAME = PROVIDER_NAME
+    DEFAULT_MODEL = DEFAULT_MODEL
+    supports_streaming = True
+
+    def __init__(self, profile_name: str | None = None):
+        self.profile_name = profile_name or settings.AMAZON_PROFILE_NAME
+
+    @cached_property
+    def client(self):
+        """The AnthropicBedrock client."""
+        import anthropic
+
+        if not self.profile_name:
+            raise ValueError("Profile name is not provided")
+
+        return anthropic.AnthropicBedrock(aws_profile=self.profile_name)
+
+    @cached_property
+    def structured_client(self):
+        """A client patched with Instructor."""
+
+        return instructor.from_anthropic(self.client)
+
+    def send_conversation(self, conversation: "Conversation", **kwargs):
+        """Send a conversation to the OpenAI API."""
+
+        from ..models import Message
+
+        messages = [
+            {"role": msg.role, "content": msg.text} for msg in conversation.messages
+        ]
+
+        response = self.client.chat.completions.create(
+            model=conversation.llm_model or DEFAULT_MODEL, messages=messages, **kwargs
+        )
+
+        # Get the response content from the OpenAI response
+        assistant_message = response.choices[0].message
+
+        # Create and return a properly formatted Message instance
+        return Message(
+            role="assistant",
+            text=assistant_message.content or "",
+            raw=response,
+            llm_model=conversation.llm_model or DEFAULT_MODEL,
+            llm_provider=PROVIDER_NAME,
+        )
+
+    def structured_response(
+        self, prompt, response_model: Type[T], *, llm_model: str | None = None, **kwargs
+    ) -> T:
+        # Ensure messages are provided in kwargs
+        messages = [
+            {"role": "user", "content": prompt},
+        ]
+
+        response = self.structured_client.chat.completions.create(
+            messages=messages,
+            model=llm_model or self.DEFAULT_MODEL,
+            response_model=response_model,
+            max_tokens=DEFAULT_MAX_TOKENS,
+            **kwargs,
+        )
+        return response
+
+    def generate_text(self, prompt, *, llm_model, **kwargs):
+        messages = [
+            {"role": "user", "content": prompt},
+        ]
+
+        response = self.client.messages.create(
+            model=llm_model or self.DEFAULT_MODEL,
+            messages=messages,
+            max_tokens=DEFAULT_MAX_TOKENS,
+            **kwargs,
+        )
+
+        return response.content[0].text
+
+    def generate_stream_text(self, prompt, *, llm_model, **kwargs):
+        """Generate streaming text using the Amazon API."""
+
+        # Prepare the messages.
+        messages = [
+            {"role": "user", "content": prompt},
+        ]
+
+        # Send the request to the API.
+        response = self.client.messages.create(
+            model=llm_model or self.DEFAULT_MODEL,
+            messages=messages,
+            stream=True,
+            **kwargs,
+        )
+
+        # Yield the text chunks.
+        for chunk in response:
+            if chunk.text:
+                yield chunk.text

simplemind/providers/anthropic.py

@@ -24,6 +24,7 @@ class Anthropic(BaseProvider):
     NAME = PROVIDER_NAME
     DEFAULT_MODEL = DEFAULT_MODEL
     DEFAULT_KWARGS = DEFAULT_KWARGS
+    supports_streaming = True
 
     def __init__(self, api_key: str | None = None):
         self.api_key = api_key or settings.get_api_key(PROVIDER_NAME)
@@ -107,3 +108,20 @@ class Anthropic(BaseProvider):
         )
 
         return response.content[0].text
+
+    @logger
+    def generate_stream_text(self, prompt: str, *, llm_model: str, **kwargs):
+        # Prepare the messages.
+        messages = [
+            {"role": "user", "content": prompt},
+        ]
+
+        # Make the request.
+        with self.client.messages.stream(
+            model=llm_model or self.DEFAULT_MODEL,
+            messages=messages,
+            **{**self.DEFAULT_KWARGS, **kwargs},
+        ) as stream:
+            # Yield each chunk of text from the stream.
+            for chunk in stream.text_stream:
+                yield chunk

simplemind/providers/gemini.py

@@ -24,6 +24,7 @@ DEFAULT_MODEL = "models/gemini-1.5-flash-latest"
 class Gemini(BaseProvider):
     NAME = PROVIDER_NAME
     DEFAULT_MODEL = DEFAULT_MODEL
+    supports_streaming = True
 
     def __init__(self, api_key: str | None = None):
         self.api_key = api_key or settings.get_api_key(PROVIDER_NAME)
@@ -107,3 +108,17 @@ class Gemini(BaseProvider):
             # Handle the exception appropriately, e.g., log the error or raise a custom exception
             raise RuntimeError(f"Failed to generate text with Gemini API: {e}") from e
         return response.text
+
+    @logger
+    def generate_stream_text(self, prompt: str, **kwargs) -> str:
+        """Generate streaming text using the Gemini API."""
+        kwargs.pop("llm_model", None)
+        try:
+            response = self.client.generate_content(prompt, stream=True, **kwargs)
+            for chunk in response:
+                if chunk.text:
+                    yield chunk.text
+        except Exception as e:
+            raise RuntimeError(
+                f"Failed to generate streaming text with Gemini API: {e}"
+            ) from e

simplemind/providers/groq.py

@@ -24,6 +24,7 @@ class Groq(BaseProvider):
     NAME = PROVIDER_NAME
     DEFAULT_MODEL = DEFAULT_MODEL
     DEFAULT_KWARGS = DEFAULT_KWARGS
+    supports_streaming = True
 
     def __init__(self, api_key: str | None = None):
         self.api_key = api_key or settings.get_api_key(PROVIDER_NAME)
@@ -111,3 +112,32 @@ class Groq(BaseProvider):
         )
 
         return str(response.choices[0].message.content)
+
+    @logger
+    def generate_stream_text(
+        self,
+        prompt: str,
+        *,
+        llm_model: str | None = None,
+        **kwargs,
+    ) -> str:
+        """Generate streaming text using the Groq API."""
+        messages = [
+            {"role": "user", "content": prompt},
+        ]
+
+        response = self.client.chat.completions.create(
+            messages=messages,
+            model=llm_model or self.DEFAULT_MODEL,
+            stream=True,
+            **{**self.DEFAULT_KWARGS, **kwargs},
+        )
+
+        try:
+            for chunk in response:
+                if chunk.choices and chunk.choices[0].delta.content:
+                    yield chunk.choices[0].delta.content
+        except Exception as e:
+            raise RuntimeError(
+                f"Failed to generate streaming text with Groq API: {e}"
+            ) from e

simplemind/providers/ollama.py

@@ -26,6 +26,7 @@ class Ollama(BaseProvider):
     DEFAULT_MODEL = DEFAULT_MODEL
     DEFAULT_KWARGS = DEFAULT_KWARGS
     TIMEOUT = DEFAULT_TIMEOUT
+    supports_streaming = True
 
     def __init__(self, host_url: str | None = None):
         self.host_url = host_url or settings.OLLAMA_HOST_URL
@@ -116,3 +117,21 @@ class Ollama(BaseProvider):
         )
 
         return response.get("message", {}).get("content", "")
+
+    @logger
+    def generate_stream_text(self, prompt: str, *, llm_model: str, **kwargs) -> str:
+        # Prepare the messages.
+        messages = [
+            {"role": "user", "content": prompt},
+        ]
+
+        response = self.client.chat(
+            messages=messages,
+            model=llm_model or self.DEFAULT_MODEL,
+            stream=True,
+            **{**self.DEFAULT_KWARGS, **kwargs},
+        )
+
+        # Iterate over the response and yield the content.
+        for chunk in response:
+            yield chunk["message"]["content"]

simplemind/providers/openai.py

@@ -23,6 +23,8 @@ class OpenAI(BaseProvider):
     NAME = PROVIDER_NAME
     DEFAULT_MODEL = DEFAULT_MODEL
     DEFAULT_KWARGS = DEFAULT_KWARGS
+    supports_streaming = True
+    
 
     def __init__(self, api_key: str | None = None):
         self.api_key = api_key or settings.get_api_key(PROVIDER_NAME)
@@ -106,3 +108,25 @@ class OpenAI(BaseProvider):
             **{**self.DEFAULT_KWARGS, **kwargs},
         )
         return response.choices[0].message.content
+
+    @logger
+    def generate_stream_text(
+        self, prompt: str, *, llm_model: str | None = None, **kwargs
+    ):
+        """Generate streaming text using the OpenAI API.
+
+        Yields chunks of text as they are generated by the model.
+        """
+        messages = [
+            {"role": "user", "content": prompt},
+        ]
+        response = self.client.chat.completions.create(
+            messages=messages,
+            model=llm_model or self.DEFAULT_MODEL,
+            stream=True,  # Enable streaming
+            **{**self.DEFAULT_KWARGS, **kwargs},
+        )
+
+        for chunk in response:
+            if chunk.choices[0].delta.content is not None:
+                yield chunk.choices[0].delta.content

simplemind/providers/xai.py

@@ -25,6 +25,8 @@ class XAI(BaseProvider):
     NAME = PROVIDER_NAME
     DEFAULT_MODEL = DEFAULT_MODEL
     DEFAULT_KWARGS = DEFAULT_KWARGS
+    supports_streaming = True
+    supports_structured_responses = False
 
     def __init__(self, api_key: str | None = None):
         self.api_key = api_key or settings.get_api_key(PROVIDER_NAME)
@@ -85,14 +87,36 @@ class XAI(BaseProvider):
 
     @logger
     def generate_text(self, prompt: str, *, llm_model: str, **kwargs) -> str:
+        # Prepare the messages.
         messages = [
             {"role": "user", "content": prompt},
         ]
 
+        # Make the request.
         response = self.client.chat.completions.create(
             messages=messages,
             model=llm_model or self.DEFAULT_MODEL,
             **{**self.DEFAULT_KWARGS, **kwargs},
         )
 
+        # Return the response content.
         return str(response.choices[0].message.content)
+
+    @logger
+    def generate_stream_text(self, prompt: str, *, llm_model: str, **kwargs) -> str:
+        # Prepare the messages.
+        messages = [
+            {"role": "user", "content": prompt},
+        ]
+
+        # Make the request.
+        response = self.client.chat.completions.create(
+            messages=messages,
+            model=llm_model or self.DEFAULT_MODEL,
+            stream=True,
+            **{**self.DEFAULT_KWARGS, **kwargs},
+        )
+
+        # Iterate over the response and yield the content.
+        for chunk in response:
+            yield chunk.choices[0].delta.content

simplemind/settings.py

@@ -8,7 +8,6 @@ class LoggingConfig(BaseSettings):
     """The class that holds all the logging settings for the application."""
 
     is_enabled: bool = Field(False, description="Enable logging")
-
     model_config = SettingsConfigDict(extra="forbid")
 
     def enable_logfire(self, **kwargs) -> None:
@@ -41,6 +40,9 @@ class LoggingConfig(BaseSettings):
 class Settings(BaseSettings):
     """The class that holds all the API keys for the application."""
 
+    AMAZON_PROFILE_NAME: Optional[str] = Field(
+        "default", description="AWS Named Profile"
+    )
     ANTHROPIC_API_KEY: Optional[SecretStr] = Field(
         None, description="API key for Anthropic"
     )

tests/test_conversations.py

@@ -0,0 +1,28 @@
+import pytest
+
+from simplemind.providers import Anthropic, Gemini, OpenAI, Groq, Ollama, Amazon
+
+import simplemind as sm
+
+
+@pytest.mark.parametrize(
+    "provider_cls",
+    [
+        Anthropic,
+        Gemini,
+        OpenAI,
+        Groq,
+        Ollama,
+        # Amazon
+    ],
+)
+def test_generate_data(provider_cls):
+    conv = sm.create_conversation(
+        llm_model=provider_cls.DEFAULT_MODEL, llm_provider=provider_cls.NAME
+    )
+
+    conv.add_message(text="hey")
+    data = conv.send()
+
+    assert isinstance(data.text, str)
+    assert len(data.text) > 0

tests/test_generate_data.py

@@ -1,7 +1,8 @@
-from pydantic import BaseModel
-
 import pytest
-from simplemind.providers import Anthropic, Gemini, Groq, Ollama, OpenAI
+
+from simplemind.providers import Anthropic, Gemini, OpenAI, Groq, Ollama, Amazon
+
+from pydantic import BaseModel
 
 
 class ResponseModel(BaseModel):
@@ -16,6 +17,7 @@ class ResponseModel(BaseModel):
         OpenAI,
         Groq,
         Ollama,
+        # Amazon
     ],
 )
 def test_generate_data(provider_cls):

tests/test_generate_text.py

@@ -1,5 +1,6 @@
 import pytest
-from simplemind.providers import Anthropic, Gemini, Groq, Ollama, OpenAI
+
+from simplemind.providers import Anthropic, Gemini, OpenAI, Groq, Ollama, Amazon
 
 
 @pytest.mark.parametrize(
@@ -10,6 +11,7 @@ from simplemind.providers import Anthropic, Gemini, Groq, Ollama, OpenAI
         OpenAI,
         Groq,
         Ollama,
+        # Amazon,
     ],
 )
 def test_generate_text(provider_cls):