Add streaming support to generate_text method and refactor related functions

This commit refactors the `create_conversation` function in `simplemind/__init__.py` to use a more descriptive variable name (`conv`) for the conversation object. It also updates the `add_plugin` method to use the new variable name (`conv`) instead of `conversation`.

In `simplemind/models.py`, the `prepend_system_message` method now accepts an optional `meta` parameter. The method also adds a system message to the conversation by prepending it to the list of messages.

Additionally, the `add_message` method in `simplemind/models.py` has been modified to include type annotations and a default value for the `role` parameter. The method now requires the `text` parameter to be provided explicitly.

A new test file, `tests/test_conversations.py`, has been added to the repository. This file contains a test case for the `generate_data` function, which tests the functionality of different LLM providers.

Lastly, the test files `tests/test_generate_data.py` and `tests/test_generate_text.py` have been modified to remove the unused `Amazon` provider from the list of test cases.

CHANGELOG.md

@@ -1,6 +1,15 @@
 Release History
 ===============
 
+## 0.2.2 (2024-11-02)
+
+- `conv.prepend_system_message` now uses system role by default.
+- General improvements.
+
+## 0.2.1 (2024-11-01)
+
+- Add `cached_property` to Amazon provider.
+
 ## 0.2.0 (2024-11-01)
 
 - Add Amazon Bedrock provider.

Dockerfile

@@ -0,0 +1,21 @@
+FROM python:3.12-slim
+
+# Install system dependencies
+RUN apt-get update && apt-get install -y \
+    git \
+    && rm -rf /var/lib/apt/lists/*
+
+# Install uv
+RUN pip install uv
+
+# Create and set working directory
+WORKDIR /app
+
+# Copy requirements/project files
+ONBUILD COPY . .
+
+# Install dependencies using uv
+RUN uv pip install "simplemind[full]" --system
+
+# Set default command
+CMD ["python"]

README.md

@@ -10,31 +10,64 @@ Simplemind is AI library designed to simplify your experience with AI APIs in Py
 
 With Simplemind, tapping into AI is as easy as a friendly conversation.
 
-- **Easy-to-use AI tools**: SimpleMind provides simple interfaces to popular AI services.
+- **Easy-to-use AI tools**: Simplemind provides simple interfaces to most popular AI services.
 - **Human-centered design**: The library prioritizes readability and usability—no need to be an expert to start experimenting.
 - **Minimal configuration**: Get started quickly, without worrying about configuration headaches.
 
 ## Supported APIs
 
-To specify a specific provider or model, you can use the `llm_provider` and `llm_model` parameters when calling: `generate_text`, `generate_data`, or `create_conversation`. The APIs remain identital between all supported providers/models.
+The APIs remain identical between all supported providers / models:
 
-- [**Anthropic's Claude**](https://www.anthropic.com/claude)
-- [**Amazon Bedrock**](https://aws.amazon.com/bedrock/)
-- [**Google's Gemini**](https://gemini.google/)
-- [**Groq's Groq**](https://groq.com/)
-- [**Ollama**](https://ollama.com)
-- [**OpenAI's GPT**](https://openai.com/gpt)
-- [**xAI's Grok**](https://x.ai/)
+<table>
+  <thead>
+    <tr>
+      <th></th>
+      <th><code>llm_provider</code></th>
+      <th>Default <code>llm_model</code></th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td><a href="https://www.anthropic.com/claude">Anthropic's Claude</a></td>
+      <td><code>"anthropic"</code></td>
+      <td><code>"claude-3-5-sonnet-20241022"</code></td>
+    </tr>
+    <tr>
+      <td><a href="https://aws.amazon.com/bedrock/">Amazon's Bedrock</a></td>
+      <td><code>"amazon"</code></td>
+      <td><code>"anthropic.claude-3-sonnet-20240229-v1:0"</code></td>
+    </tr>
+    <tr>
+      <td><a href="https://gemini.google/">Google's Gemini</a></td>
+      <td><code>"gemini"</code></td>
+      <td><code>"models/gemini-1.5-pro"</code></td>
+    </tr>
+    <tr>
+      <td><a href="https://groq.com/">Groq's Groq</a></td>
+      <td><code>"groq"</code></td>
+      <td><code>"llama3-8b-8192"</code></td>
+    </tr>
+    <tr>
+      <td><a href="https://ollama.com">Ollama</a></td>
+      <td><code>"ollama"</code></td>
+      <td><code>"llama3.2"</code></td>
+    </tr>
+    <tr>
+      <td><a href="https://openai.com/gpt">OpenAI's GPT</a></td>
+      <td><code>"openai"</code></td>
+      <td><code>"gpt-4o-mini"</code></td>
+    </tr>
+    <tr>
+      <td><a href="https://x.ai/">xAI's Grok</a></td>
+      <td><code>"xai"</code></td>
+      <td><code>"grok-beta"</code></td>
+    </tr>
+  </tbody>
+</table>
 
-If you want to see Simplemind support, additional providers or models, please send a pull request!
+To specify a specific provider or model, you can use the `llm_provider` and `llm_model` parameters when calling: `generate_text`, `generate_data`, or `create_conversation`.
 
-## Why SimpleMind?
-
-- **Intuitive**: Built with Pythonic simplicity and readability in mind.
-- **For Humans**: Emphasizes a human-friendly interface, just like `requests` for HTTP.
-- **Open Source**: Simplemind is open source, and contributions are always welcome!
-
-Also, why not? :)
+If you want to see Simplemind support additional providers or models, please send a pull request!
 
 ## Quickstart
 
@@ -123,20 +156,20 @@ Special thanks to [@jxnl](https://github.com/jxnl) for building [Instructor](htt
 SimpleMind also allows for easy conversational flows:
 
 ```pycon
->>> conversation = sm.create_conversation(llm_model="gpt-4o-mini", llm_provider="openai")
+>>> conv = sm.create_conversation(llm_model="gpt-4o-mini", llm_provider="openai")
 
 >>> # Add a message to the conversation
->>> conversation.add_message("user", "Hi there, how are you?")
+>>> conv.add_message("user", "Hi there, how are you?")
 
->>> conversation.send()
+>>> conv.send()
 <Message role=assistant text="Hello! I'm just a computer program, so I don't have feelings, but I'm here and ready to help you. How can I assist you today?">
 ```
 
-To continue the conversation, you can call `conversation.send()` again, which returns the next message in the conversation:
+To continue the conversation, you can call `conv.send()` again, which returns the next message in the conversation:
 
 ```pycon
->>> conversation.add_message("user", "What is the meaning of life?")
->>> conversation.send()
+>>> conv.add_message("user", "What is the meaning of life?")
+>>> conv.send()
 <Message role=assistant text="The meaning of life is a profound philosophical question that has been explored by cultures, religions, and philosophers for centuries. Different people and belief systems offer varying interpretations:\n\n1. **Religious Perspectives:** Many religions propose that the meaning of life is to fulfill a divine purpose, serve God, or reach an afterlife. For example, Christianity often emphasizes love, faith, and service to God and others as central to life’s meaning.\n\n2. **Philosophical Views:** Philosophers offer diverse answers. Existentialists like Jean-Paul Sartre argue that life has no inherent meaning, and it is up to individuals to create their own purpose. Others, like Aristotle, suggest that achieving eudaimonia (flourishing or happiness) through virtuous living is the key to a meaningful life.\n\n3. **Scientific and Secular Approaches:** Some people find meaning through understanding the natural world, contributing to human knowledge, or through personal accomplishments and happiness. They may view life’s meaning as a product of connection, legacy, or the pursuit of knowledge and creativity.\n\n4. **Personal Perspective:** For many, the meaning of life is deeply personal, involving their relationships, passions, and goals. These individuals define life’s purpose through experiences, connections, and the impact they have on others and the world.\n\nUltimately, the meaning of life is a subjective question, with each person finding their own answers based on their beliefs, experiences, and reflections.">
 ```
 
@@ -229,7 +262,7 @@ Simple, yet effective.
 
 ### Logging
 
-Simplemind uses [logfire](https://logfire.ai) for logging. To enable logging, call `sm.enable_logfire()`.
+Simplemind uses [Logfire](https://pydantic.dev/logfire) for logging. To enable logging, call `sm.enable_logfire()`.
 
 ### More Examples
 

docs/conf.py

@@ -16,7 +16,7 @@ import simplemind
 project = "simplemind"
 copyright = "2024 Kenneth Reitz"
 author = "Kenneth Reitz"
-release = "v0.2.0"
+release = "v0.2.2"
 
 # -- General configuration ---------------------------------------------------
 # https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration

examples/_context.py

@@ -5,6 +5,7 @@ import sys
 sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), "..")))
 
 
+import simplemind
 import simplemind as sm
 
-__all__ = ["sm"]
+__all__ = ["simplemind", "sm"]

examples/bible_verses.py

@@ -0,0 +1,137 @@
+from _context import simplemind as sm
+from pydantic import BaseModel
+from rich.console import Console
+from rich.panel import Panel
+from rich.text import Text
+
+console = Console()
+
+gpt_4o_mini = sm.Session(llm_provider="openai")
+claude_sonnet = sm.Session(llm_provider="anthropic")
+
+
+class BibleVerse(BaseModel):
+    book: str
+    chapter: int
+    verse: int
+    text: str
+    translation: str
+
+
+class BiblePassage(BaseModel):
+    book: str
+    chapter: int
+    verses: list[BibleVerse]
+    translation: str
+
+
+class CrossReference(BaseModel):
+    passage: BiblePassage
+    notes: list[str]
+    origin_verse: BibleVerse
+    ai_perspective: str
+    anthropic_perspective: str
+
+
+def get_passage(book: str, chapter: int, translation: str = "ESV") -> BiblePassage:
+    passage = gpt_4o_mini.generate_data(
+        prompt=f"""Return {book} chapter {chapter} from the {translation} translation.
+Format each verse as plain text without any special characters or formatting.
+For example:
+- "Love is patient, love is kind."
+- "It does not envy, it does not boast"
+
+Return only the biblical text, formatted as a BiblePassage object.""",
+        response_model=BiblePassage,
+        max_tokens=8000,
+    )
+    return passage
+
+
+def get_cross_reference(passage: BiblePassage) -> CrossReference:
+    verses_text = "\n".join([f"Verse {v.verse}: {v.text}" for v in passage.verses])
+
+    # Get main cross-reference from OpenAI
+    ref = gpt_4o_mini.generate_data(
+        prompt=f"""Find a thematically related Bible passage that connects with this text:
+{verses_text}
+
+Return a CrossReference object with:
+1. A related passage (using plain text without special characters)
+2. A list of clear, specific notes explaining the thematic connections
+3. The original passage included
+4. An AI perspective that provides a thoughtful, modern interpretation of how these passages relate to contemporary life and universal human experiences""",
+        response_model=CrossReference,
+    )
+
+    # Get Anthropic's perspective separately
+    anthropic_insight = claude_sonnet.generate_text(
+        prompt=f"""Analyze these biblical passages from a philosophical and ethical perspective:
+
+Original passage:
+{verses_text}
+
+Cross-reference passage:
+{' '.join([f'Verse {v.verse}: {v.text}' for v in ref.passage.verses])}
+
+Provide a thoughtful analysis focusing on the philosophical and ethical implications of these passages, drawing from your training in ethics and philosophy.
+Return your response as a plain string.""",
+    )
+
+    # Add Anthropic's perspective to the reference object
+    ref.anthropic_perspective = anthropic_insight
+    return ref
+
+
+def pretty_print_reference(ref: CrossReference):
+    # Create origin passage panel
+    origin_text = Text()
+    origin_text.append(
+        f"{ref.origin_verse.book} {ref.origin_verse.chapter}\n",
+        style="bold blue",
+    )
+    origin_text.append(f"{ref.origin_verse.verse}. ", style="blue")
+    origin_text.append(f"{ref.origin_verse.text}\n", style="italic")
+    origin_text.append(f"\n({ref.origin_verse.translation})", style="dim")
+
+    origin_panel = Panel(origin_text, title="Original Passage", border_style="blue")
+
+    # Create cross reference panel
+    ref_text = Text()
+    ref_text.append(
+        f"{ref.passage.book} {ref.passage.chapter}\n",
+        style="bold green",
+    )
+    for verse in ref.passage.verses:
+        ref_text.append(f"{verse.verse}. ", style="green")
+        ref_text.append(f"{verse.text}\n", style="italic")
+    ref_text.append(f"\n({ref.passage.translation})", style="dim")
+
+    ref_panel = Panel(ref_text, title="Cross Reference", border_style="green")
+
+    # Create notes panel with bullet points
+    notes_text = Text()
+    for note in ref.notes:
+        notes_text.append("• ", style="yellow")
+        notes_text.append(f"{note}\n")
+
+    notes_panel = Panel(notes_text, title="Thematic Connections", border_style="yellow")
+
+    # Add new AI perspective panel
+    ai_text = Text()
+    ai_text.append(ref.ai_perspective)
+
+    ai_panel = Panel(ai_text, title="AI Perspective", border_style="magenta")
+
+    # Print all panels
+    console.print(origin_panel)
+    console.print(ref_panel)
+    console.print(notes_panel)
+    console.print(ai_panel)
+
+
+if __name__ == "__main__":
+    # Get 1 Corinthians 13 (The Love Chapter)
+    passage = get_passage("1 Corinthians", 13)
+    ref = get_cross_reference(passage)
+    pretty_print_reference(ref)

examples/cooking_recipe_example.py

@@ -0,0 +1,66 @@
+from pydantic import BaseModel
+import simplemind as sm
+
+
+class InstructionStep(BaseModel):
+    step_number: int
+    instruction: str
+
+class RecipeIngredient(BaseModel):
+    name: str
+    quantity: float
+    unit: str
+
+class Recipe(BaseModel):
+    name: str
+    ingredients: list[RecipeIngredient]
+    instructions: list[InstructionStep]
+    
+    def __str__(self) -> str:
+        output = f"\n=== {self.name.upper()} ===\n\n"
+        
+        output += "INGREDIENTS:\n"
+        for ing in self.ingredients:
+            output += f"• {ing.quantity} {ing.unit} {ing.name}\n"
+        
+        output += "\nINSTRUCTIONS:\n"
+        for step in self.instructions:
+            output += f"{step.step_number}. {step.instruction}\n"
+        
+        return output
+    
+
+recipe = sm.generate_data(
+    "Write a recipe for chocolate chip cookies",
+    llm_model="gpt-4o-mini",
+    llm_provider="openai",
+    response_model=Recipe,
+)
+
+print(recipe)
+# Expected output is something like this:
+#
+# === CHOCOLATE CHIP COOKIES ===
+#
+# INGREDIENTS:
+# • 2.25 cups all-purpose flour
+# • 1.0 teaspoon baking soda
+# • 0.5 teaspoon salt
+# • 1.0 cup unsalted butter
+# • 0.75 cup sugar
+# • 0.75 cup brown sugar
+# • 1.0 teaspoon vanilla extract
+# • 2.0 large eggs
+# • 2.0 cups semi-sweet chocolate chips
+#
+# INSTRUCTIONS:
+# 1. Preheat your oven to 350°F (175°C).
+# 2. In a small bowl, combine flour, baking soda, and salt; set aside.
+# 3. In a large bowl, cream together the butter, sugar, and brown sugar until smooth.
+# 4. Beat in the vanilla extract and eggs, one at a time.
+# 5. Gradually blend in the flour mixture until just combined.
+# 6. Stir in the chocolate chips.
+# 7. Drop by rounded tablespoon onto ungreased cookie sheets.
+# 8. Bake for 9 to 11 minutes, or until edges are golden.
+# 9. Let cool on the cookie sheet for a few minutes before transferring to wire racks to cool completely.
+

examples/discussion.py

@@ -0,0 +1,132 @@
+import time
+from typing import List, Tuple
+
+from rich.console import Console
+from rich.markdown import Markdown
+
+from _context import sm
+
+
+class MultiAIConversation:
+    """Orchestrates conversations between multiple AI models."""
+
+    MODEL_SESSIONS = {
+        "Llama3.2": sm.Session(
+            llm_provider="ollama",
+            llm_model="llama3.2",
+        ),
+        "Claude-3.5-Sonnet": sm.Session(
+            llm_provider="anthropic",
+            llm_model="claude-3-5-sonnet-20241022",
+        ),
+        "GPT-4o": sm.Session(
+            llm_provider="openai",
+            llm_model="gpt-4o",
+        ),
+        "Grok-Beta": sm.Session(
+            llm_provider="xai",
+            llm_model="grok-beta",
+        ),
+    }
+
+    def __init__(self, topic: str, turns_per_model: int = 1, max_rounds: int = 5):
+
+        self.topic = topic
+        self.turns_per_model = turns_per_model
+        self.max_rounds = max_rounds
+        self.conversation_history: List[Tuple[str, str]] = []
+        self.console = Console()
+
+    def _format_system_prompt(self, ai_name: str) -> str:
+        """Creates a system prompt for each AI model."""
+        return f"""You are {ai_name}. You are participating in a thoughtful discussion with other AI models about {self.topic}.
+
+Rules:
+1. Be concise but insightful (keep responses under 100 words)
+2. Build upon previous points made in the conversation
+3. Ask questions to deepen the discussion when appropriate
+4. Stay on topic while maintaining your unique perspective
+5. Be respectful of other viewpoints while maintaining your distinct voice
+
+Current discussion topic: {self.topic}"""
+
+    def _create_conversation(
+        self, session: sm.Session, ai_name: str
+    ) -> sm.Conversation:
+        """Creates a new conversation with appropriate context for an AI model."""
+        conv = session.create_conversation()
+
+        # Add system prompt
+        conv.add_message(role="user", text=self._format_system_prompt(ai_name))
+
+        # Add conversation history
+        for speaker, message in self.conversation_history[-3:]:  # Last 3 messages
+            conv.add_message(role="user", text=f"{speaker} said: {message}")
+
+        return conv
+
+    def _print_response(self, ai_name: str, response: str):
+        """Pretty prints an AI response using Rich."""
+        self.console.print(f"\n[bold blue]{ai_name}[/bold blue]:")
+        self.console.print(Markdown(response))
+        # Store in history
+        self.conversation_history.append((ai_name, response))
+
+    def run_conversation(self):
+        """Runs the multi-AI conversation."""
+
+        # Initialize the conversation
+        initial_prompt = (
+            f"Let's have a thoughtful discussion about {self.topic}. "
+            "Please share your initial thoughts in 2-3 sentences."
+        )
+
+        for round_num in range(self.max_rounds):
+            self.console.print(f"\n[bold green]Round {round_num + 1}[/bold green]")
+
+            for model_name, session in self.MODEL_SESSIONS.items():
+                for turn in range(self.turns_per_model):
+                    conversation = self._create_conversation(session, model_name)
+
+                    # Add the prompt
+                    prompt = (
+                        initial_prompt
+                        if round_num == 0 and turn == 0
+                        else (
+                            f"Continue the discussion about {self.topic}, "
+                            "responding to the previous points made."
+                        )
+                    )
+
+                    conversation.add_message(role="user", text=prompt)
+
+                    # Get and print response
+                    response = conversation.send()
+                    self._print_response(model_name, response.text)
+
+                    # Small delay to prevent rate limiting
+                    time.sleep(1)
+
+            # Optional: Add a separator between rounds
+            self.console.print("\n" + "-" * 50)
+
+
+def have_ai_discussion(topic: str, turns_per_model: int = 1, max_rounds: int = 3):
+    """Convenience function to start an AI discussion."""
+    debate = MultiAIConversation(
+        topic=topic, turns_per_model=turns_per_model, max_rounds=max_rounds
+    )
+
+    print(f"\nStarting AI discussion on: {topic}")
+    print("=" * 50)
+
+    debate.run_conversation()
+
+
+# Example usage
+if __name__ == "__main__":
+    # Example topics
+    topic = "The future of human-AI collaboration in creative fields",
+
+    # Run a discussion on the first topic
+    have_ai_discussion(topic=topic, turns_per_model=1, max_rounds=3)

examples/generate_stream_text.py

@@ -0,0 +1,9 @@
+from _context import sm
+
+# Defaults to the default provider (openai)
+r = sm.generate_text(
+    "Write a poem about the moon", llm_model="gpt-4o-mini", stream=True
+)
+
+for chunk in r:
+    print(chunk, end="", flush=True)

examples/math_plugin.py

@@ -2,7 +2,7 @@ from _context import sm
 
 
 class MathPlugin(sm.BasePlugin):
-    def send_hook(self, conversation: sm.Conversation):
+    def pre_send_hook(self, conversation: sm.Conversation):
         last_user_message = conversation.get_last_message(role="user")
         if last_user_message is None:
             return

examples/medicine_data.py

@@ -0,0 +1,94 @@
+from pydantic import BaseModel
+from _context import simplemind as sm
+from rich.console import Console
+from rich.panel import Panel
+from rich.table import Table
+
+
+class SideEffect(BaseModel):
+    effect: str
+    severity: str  # mild, moderate, severe
+    frequency: str  # common, uncommon, rare
+
+
+class Medication(BaseModel):
+    brand_name: str
+    generic_name: str
+    drug_class: str
+    half_life: str
+    common_uses: list[str]
+    side_effects: list[SideEffect]
+    typical_dosage: str
+    warnings: list[str]
+
+
+class MedicationList(BaseModel):
+    root: list[Medication]
+
+
+# Create a session with your preferred model
+session = sm.Session(llm_provider="openai", llm_model="gpt-4o-mini")
+
+
+# Update the prompt to use an f-string with a parameter
+def get_medication_prompt(medications: list[str]) -> str:
+    return f"""
+Provide detailed medical information about {', '.join(medications)}.
+Include their generic names, drug classes, half-lives, common uses, side effects (with severity and frequency),
+typical dosages, and important warnings.
+Return the information as separate medication entries.
+"""
+
+
+# Example usage
+medications_to_lookup = ["Abilify (aripiprazole)", "Trileptal (oxcarbazepine)"]
+prompt = get_medication_prompt(medications_to_lookup)
+
+# Generate structured data for medications
+medications = session.generate_data(prompt=prompt, response_model=MedicationList)
+
+# Create a Rich console
+console = Console()
+
+# Replace the print section with Rich formatting
+for med in medications.root:
+    # Create a table for the medication details
+    table = Table(show_header=False, box=None)
+    table.add_row("[bold cyan]Generic Name:[/]", med.generic_name)
+    table.add_row("[bold cyan]Drug Class:[/]", med.drug_class)
+    table.add_row("[bold cyan]Half Life:[/]", med.half_life)
+
+    # Create a nested table for common uses
+    uses_table = Table(show_header=False, box=None, padding=(0, 2))
+    for use in med.common_uses:
+        uses_table.add_row("•", use)
+
+    # Create a nested table for side effects
+    effects_table = Table(show_header=False, box=None, padding=(0, 2))
+    for effect in med.side_effects:
+        severity_color = {"mild": "green", "moderate": "yellow", "severe": "red"}.get(
+            effect.severity.lower(), "white"
+        )
+        effects_table.add_row(
+            "•",
+            effect.effect,
+            f"[{severity_color}]{effect.severity}[/]",
+            f"({effect.frequency})",
+        )
+
+    # Create a nested table for warnings
+    warnings_table = Table(show_header=False, box=None, padding=(0, 2))
+    for warning in med.warnings:
+        warnings_table.add_row("•", f"[red]{warning}[/]")
+
+    # Add the nested tables to the main table
+    table.add_row("[bold cyan]Common Uses:[/]", uses_table)
+    table.add_row("[bold cyan]Side Effects:[/]", effects_table)
+    table.add_row("[bold cyan]Typical Dosage:[/]", med.typical_dosage)
+    table.add_row("[bold cyan]Warnings:[/]", warnings_table)
+
+    # Create and print a panel for each medication
+    console.print(
+        Panel(table, title=f"[bold blue]{med.brand_name}[/]", border_style="blue")
+    )
+    console.print()  # Add a blank line between medications

examples/requirements.txt

@@ -2,3 +2,4 @@ numpy
 openai
 pydantic
 faiss-cpu
+rich

examples/sentiment_analysis.py

@@ -3,6 +3,7 @@ from typing import Literal
 from _context import sm
 from pydantic import BaseModel
 
+# Note: you should probably be using textblob for this.
 
 class SentimentAnalysis(BaseModel):
     sentiment: Literal["positive", "negative", "neutral"]

examples/simple_memory.py

@@ -13,7 +13,7 @@ class SimpleMemoryPlugin:
 
     def initialize_hook(self, conversation: sm.Conversation):
         for m in self.yield_memories():
-            conversation.prepend_system_message(role="system", text=m)
+            conversation.prepend_system_message(text=m)
 
 
 conversation = sm.create_conversation(llm_model="grok-beta", llm_provider="xai")

pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "simplemind"
-version = "0.2.0"
+version = "0.2.2"
 description = "An experimental client for AI providers that intends to replace LangChain and LangGraph for most common use cases."
 readme = "README.md"
 requires-python = ">=3.10"
@@ -16,6 +16,12 @@ full = [
     "botocore",
     "boto3"
 ]
+openai = ["openai"]
+anthropic = ["anthropic"]
+ollama = ["ollama", "openai"]
+groq = ["groq"]
+gemini = ["google-generativeai"]
+amazon = ["boto3", "botocore", "anthropic"]
 
 [build-system]
 requires = ["hatchling"]

simplemind/__init__.py

@@ -1,4 +1,4 @@
-from typing import List, Type
+from typing import Generator, List, Type
 
 from .models import BaseModel, BasePlugin, Conversation
 from .settings import settings
@@ -64,16 +64,16 @@ def create_conversation(
     """Create a new conversation."""
 
     # Create the conversation.
-    conversation = Conversation(
+    conv = Conversation(
         llm_model=llm_model,
         llm_provider=llm_provider or settings.DEFAULT_LLM_PROVIDER,
     )
 
     # Add plugins to the conversation.
     for plugin in plugins or []:
-        conversation.add_plugin(plugin)
+        conv.add_plugin(plugin)
 
-    return conversation
+    return conv
 
 
 def generate_data(
@@ -94,6 +94,7 @@ def generate_data(
         prompt=prompt,
         llm_model=llm_model,
         response_model=response_model,
+        **kwargs,
     )
 
 
@@ -102,6 +103,7 @@ def generate_text(
     *,
     llm_model: str | None = None,
     llm_provider: str | None = None,
+    stream: bool = False,
     **kwargs,
 ) -> str:
     """Generate text from a given prompt."""
@@ -110,7 +112,15 @@ def generate_text(
     provider = find_provider(llm_provider or settings.DEFAULT_LLM_PROVIDER)
 
     # Generate the text.
-    return provider.generate_text(prompt=prompt, llm_model=llm_model, **kwargs)
+    if stream:
+        if not provider.supports_streaming:
+            raise ValueError(f"{provider} does not support streaming.")
+
+        return provider.generate_stream_text(
+            prompt=prompt, llm_model=llm_model, **kwargs
+        )
+    else:
+        return provider.generate_text(prompt=prompt, llm_model=llm_model, **kwargs)
 
 
 def enable_logfire() -> None:

simplemind/models.py

@@ -116,17 +116,23 @@ class Conversation(SMBaseModel):
                 except NotImplementedError:
                     pass
 
-    def prepend_system_message(
-        self, role: MESSAGE_ROLE, text: str, meta: Dict[str, Any] | None = None
-    ):
+    def prepend_system_message(self, text: str, meta: Dict[str, Any] | None = None):
         """Prepend a system message to the conversation."""
-        self.messages = [Message(role=role, text=text, meta=meta or {})] + self.messages
+        self.messages = [
+            Message(role="system", text=text, meta=meta or {})
+        ] + self.messages
 
     def add_message(
-        self, role: MESSAGE_ROLE, text: str, meta: Optional[Dict[str, Any]] = None
+        self,
+        role: MESSAGE_ROLE = "user",
+        text: str | None = None,
+        *,
+        meta: Optional[Dict[str, Any]] = None,
     ):
         """Add a new message to the conversation."""
 
+        assert text is not None
+
         # Ensure meta is a dict.
         if meta is None:
             meta = {}
@@ -151,6 +157,8 @@ class Conversation(SMBaseModel):
     ) -> Message:
         """Send the conversation to the LLM."""
 
+        # TODO: llm_model and llm_provider should override the conversation's.
+
         # Execute all pre send hooks.
         for plugin in self.plugins:
             if hasattr(plugin, "pre_send_hook"):

simplemind/providers/_base.py

@@ -16,6 +16,7 @@ class BaseProvider(ABC):
 
     NAME: str
     DEFAULT_MODEL: str
+    supports_streaming: bool = False
 
     @cached_property
     @abstractmethod
@@ -40,6 +41,6 @@ class BaseProvider(ABC):
         raise NotImplementedError
 
     @abstractmethod
-    def generate_text(self, prompt: str, **kwargs) -> str:
+    def generate_text(self, prompt: str, *, stream: bool = False, **kwargs) -> str:
         """Generate text from a prompt."""
         raise NotImplementedError

simplemind/providers/amazon.py

@@ -1,7 +1,7 @@
 from typing import Type, TypeVar
+from functools import cached_property
 
 import instructor
-import anthropic
 from pydantic import BaseModel
 
 from ._base import BaseProvider
@@ -21,21 +21,25 @@ class Amazon(BaseProvider):
     def __init__(self, profile_name: str | None = None):
         self.profile_name = profile_name or settings.AMAZON_PROFILE_NAME
 
-    @property
+    @cached_property
     def client(self):
         """The AnthropicBedrock client."""
+        import anthropic
+
         if not self.profile_name:
             raise ValueError("Profile name is not provided")
 
         return anthropic.AnthropicBedrock(aws_profile=self.profile_name)
 
-    @property
+    @cached_property
     def structured_client(self):
         """A client patched with Instructor."""
+
         return instructor.from_anthropic(self.client)
 
     def send_conversation(self, conversation: "Conversation", **kwargs):
         """Send a conversation to the OpenAI API."""
+
         from ..models import Message
 
         messages = [

simplemind/providers/openai.py

@@ -23,6 +23,7 @@ class OpenAI(BaseProvider):
     NAME = PROVIDER_NAME
     DEFAULT_MODEL = DEFAULT_MODEL
     DEFAULT_KWARGS = DEFAULT_KWARGS
+    supports_streaming = True
 
     def __init__(self, api_key: str | None = None):
         self.api_key = api_key or settings.get_api_key(PROVIDER_NAME)
@@ -106,3 +107,25 @@ class OpenAI(BaseProvider):
             **{**self.DEFAULT_KWARGS, **kwargs},
         )
         return response.choices[0].message.content
+
+    @logger
+    def generate_stream_text(
+        self, prompt: str, *, llm_model: str | None = None, **kwargs
+    ):
+        """Generate streaming text using the OpenAI API.
+
+        Yields chunks of text as they are generated by the model.
+        """
+        messages = [
+            {"role": "user", "content": prompt},
+        ]
+        response = self.client.chat.completions.create(
+            messages=messages,
+            model=llm_model or self.DEFAULT_MODEL,
+            stream=True,  # Enable streaming
+            **{**self.DEFAULT_KWARGS, **kwargs},
+        )
+
+        for chunk in response:
+            if chunk.choices[0].delta.content is not None:
+                yield chunk.choices[0].delta.content

simplemind/settings.py

@@ -8,7 +8,6 @@ class LoggingConfig(BaseSettings):
     """The class that holds all the logging settings for the application."""
 
     is_enabled: bool = Field(False, description="Enable logging")
-
     model_config = SettingsConfigDict(extra="forbid")
 
     def enable_logfire(self, **kwargs) -> None:
@@ -41,7 +40,9 @@ class LoggingConfig(BaseSettings):
 class Settings(BaseSettings):
     """The class that holds all the API keys for the application."""
 
-    AMAZON_PROFILE_NAME: Optional[str] = Field("default", description="AWS Named Profile")
+    AMAZON_PROFILE_NAME: Optional[str] = Field(
+        "default", description="AWS Named Profile"
+    )
     ANTHROPIC_API_KEY: Optional[SecretStr] = Field(
         None, description="API key for Anthropic"
     )

tests/test_conversations.py

@@ -0,0 +1,28 @@
+import pytest
+
+from simplemind.providers import Anthropic, Gemini, OpenAI, Groq, Ollama, Amazon
+
+import simplemind as sm
+
+
+@pytest.mark.parametrize(
+    "provider_cls",
+    [
+        Anthropic,
+        Gemini,
+        OpenAI,
+        Groq,
+        Ollama,
+        # Amazon
+    ],
+)
+def test_generate_data(provider_cls):
+    conv = sm.create_conversation(
+        llm_model=provider_cls.DEFAULT_MODEL, llm_provider=provider_cls.NAME
+    )
+
+    conv.add_message(text="hey")
+    data = conv.send()
+
+    assert isinstance(data.text, str)
+    assert len(data.text) > 0

tests/test_generate_data.py

@@ -1,4 +1,3 @@
-
 import pytest
 
 from simplemind.providers import Anthropic, Gemini, OpenAI, Groq, Ollama, Amazon
@@ -18,7 +17,7 @@ class ResponseModel(BaseModel):
         OpenAI,
         Groq,
         Ollama,
-        Amazon
+        # Amazon
     ],
 )
 def test_generate_data(provider_cls):

tests/test_generate_text.py

@@ -11,7 +11,7 @@ from simplemind.providers import Anthropic, Gemini, OpenAI, Groq, Ollama, Amazon
         OpenAI,
         Groq,
         Ollama,
-        Amazon,
+        # Amazon,
     ],
 )
 def test_generate_text(provider_cls):