more docs

docs/examples/autodataframe.md

@@ -2,6 +2,9 @@
 
 In this example, we'll demonstrate how to convert a text into dataframes using OpenAI Function Call. We will define the necessary data structures using Pydantic and show how to convert the text into dataframes.
 
+!!! note "Motivation"
+    Often times when we parse data we have an opportunity to extract structured data, what if we could extract an arbitrary number of tables with arbitray schemas? By pulling out dataframes we could write tables or csv files and attach them to our retrived data.
+
 ## Defining the Data Structures
 
 Let's start by defining the data structures required for this task: `RowData`, `Dataframe`, and `Database`.

docs/examples/exact_citations.md

@@ -2,22 +2,19 @@
 
 In this example, we'll demonstrate how to use OpenAI Function Call to ask an AI a question and get back an answer with correct citations. We'll define the necessary data structures using Pydantic and show how to retrieve the citations for each answer.
 
+!!! note "Motivation"
+    Often times retrival augmented models hallucinate. Wouldn't it be great if each sentence came with citations that were no on the chunk level but at the substring level. Moreover, to ensure the quote exists we can search the string to find the exact quote!
+
 ## Defining the Data Structures
 
 Let's start by defining the data structures required for this task: `Fact` and `QuestionAnswer`.
 
 !!! tip "Prompting as documentation"
-    Make sure to include detailed and useful docstrings and fields for your class definitions. Naming becomes very important since they are semantically meaninful in the prompt 
+    Make sure to include detailed and useful docstrings and fields for your class definitions. Naming becomes very important since they are semantically meaninful in the prompt.
 
     * `substring_quote` performs better than `quote` since it suggests it should be a substring of the original content.
     * Notice that there are instructions on splitting facts in the docstring which will be used by OpenAI
 
-
-!!! tip "Embedding computation"
-    While its not thet best idea to get too crazy with adding 100 methods to your class
-    colocating some computation is oftentimes useful, here we implement the substring search directly with the `Fact` class.
-
-
 ```python
 import openai
 from pydantic import Field, BaseModel
@@ -74,6 +71,10 @@ The `Fact` class represents a single statement in the answer. It contains a `fac
 
 The `QuestionAnswer` class represents a question and its answer. It consists of a `question` attribute for the question asked and a list of `Fact` objects in the `answer` attribute.
 
+!!! tip "Embedding computation"
+    While its not thet best idea to get too crazy with adding 100 methods to your class
+    colocating some computation is oftentimes useful, here we implement the substring search directly with the `Fact` class.
+
 ## Asking AI a Question
 
 To ask the AI a question and get back an answer with citations, we can define a function `ask_ai` that takes a question and context as input and returns a `QuestionAnswer` object.
@@ -132,7 +133,7 @@ Let's evaluate the example by asking the AI a question and getting back an answe
 
 !!! usage "Highlight"
     This just adds some color and captures the citation in `<>`
-    
+
     ```python
     def highlight(text, span):
         return (

docs/examples/planning-tasks.md

@@ -1,9 +1,9 @@
 # Example: Planning and Executing a Query Plan
 
-In this example, we will demonstrate how to use the OpenAI Function Call `ChatCompletion` model to plan and execute a query plan using a question-answering system. We will define the necessary structures using Pydantic and show how to execute the query plan step-by-step.
+In this example, we will demonstrate how to use the OpenAI Function Call `ChatCompletion` model to plan and execute a query plan in a question-answering system. We will define the necessary structures using Pydantic and show how to execute the query plan step-by-step.
 
-!!! note "Graph Generation"
-    Notice that this example produces a flat list of items with dependencies that resemble a graph, while pydantic allows for recursive definitions, its much easier and less confusing for the model to generate flat schemas rather than recursive schemas. If y ou want to see a recursive example see [recursive schemas](recursive.md)
+!!! note "Motivation"
+    Multishop q/a is iterative and may never end, by trying to plan the queryin one step we know we'll have a finite number of steps. It also helps us break apart questions to figure what we can or cannot answer with available data.
 
 ## Defining the Structures
 
@@ -54,6 +54,9 @@ class QueryPlan(OpenAISchema):
         return [q for q in self.query_graph if q.id in ids]
 ```
 
+!!! warning "Graph Generation"
+    Notice that this example produces a flat list of items with dependencies that resemble a graph, while pydantic allows for recursive definitions, its much easier and less confusing for the model to generate flat schemas rather than recursive schemas. If y ou want to see a recursive example see [recursive schemas](recursive.md)
+
 ## Planning a Query Plan
 
 Now, let's demonstrate how to plan and execute a query plan using the defined models and the OpenAI API.
@@ -97,6 +100,9 @@ plan = query_planner(
 plan.dict()
 ```
 
+!!! warning "No RAG"
+    While we build the query plan in this example we do not propose a method to actually answer the question. You can implement your own answer function that perhaps makes a retrival and calls openai for retrival augmented generation. That step would also make use of function calls but goes beyond the scope of this example.
+
 ```python
 {'query_graph': [{'dependancies': [],
                     'id': 1,

docs/examples/recursive.md

@@ -1,6 +1,6 @@
 # Example: Parsing a Directory Tree
 
-In this example, we will demonstrate how to convert a string representing a directory tree into a filesystem structure using OpenAI's GPT-3 model. We will define the necessary structures using Pydantic, create a function to parse the tree, and provide an example of how to use it.
+In this example, we will demonstrate how define and use a recursive class definition to convert a string representing a directory tree into a filesystem structure using OpenAI's function call api. We will define the necessary structures using Pydantic, create a function to parse the tree, and provide an example of how to use it.
 
 ## Defining the Structures
 
@@ -80,7 +80,7 @@ The `DirectoryTree` class represents the entire directory tree. It has a single
 
 ## Parsing the Tree
 
-We define a function `parse_tree_to_filesystem` to convert a string representing a directory tree into a filesystem structure using OpenAI's GPT-3 model.
+We define a function `parse_tree_to_filesystem` to convert a string representing a directory tree into a filesystem structure using OpenAI.
 
 ```python
 import openai
@@ -143,11 +143,11 @@ root = parse_tree_to_filesystem(
 root.print_paths()
 ```
 
-In this example, we call `parse_tree_to_filesystem` with a string representing a directory tree. The directory tree has a root node named 'root' with two subfolders (folder1 and folder2). The 'folder1' subfolder contains two files (file1.txt and file2.txt), while the 'folder2' subfolder contains a file (file3.txt) and a subfolder (subfolder1) that, in turn, contains a file (file4.txt).
+In this example, we call `parse_tree_to_filesystem` with a string representing a directory tree.
 
 After parsing the string into a `DirectoryTree` object, we call `root.print_paths()` to print the paths of the root node and its children. The output of this example will be:
 
-```
+```python
 root                               NodeType.FOLDER
 root/folder1                       NodeType.FOLDER
 root/folder1/file1.txt             NodeType.FILE

docs/examples/search.md

@@ -1,6 +1,9 @@
 # Example: Segmenting Search Queries
 
-In this example, we will demonstrate how to leverage the `MultiTask` and `enum.Enum` features of OpenAI Function Call to segment search queries. We will define the necessary structures using Pydantic and demonstrate how to execute the segmented queries.
+In this example, we will demonstrate how to leverage the `MultiTask` and `enum.Enum` features of OpenAI Function Call to segment search queries. We will define the necessary structures using Pydantic and demonstrate how segment query into multiple sub queries and execute them in parallel with `asyncio`.
+
+!!! note "Motivation"
+    Extracting a list of tasks from text is one of the most common examples of using LLMs for 'intent' I can imagine a system like this powering Siri or Alexa fairly soon.
 
 ## Defining the Structures
 
@@ -42,9 +45,17 @@ class MultiSearch(OpenAISchema):
 
 The `MultiSearch` class has a single attribute, `tasks`, which is a list of `Search` objects.
 
-## Using the Prompt Pipeline
+This pattern is so common that we've added a helper function `MultiTask` to makes this simpler 
 
-To segment a search query, we will use the Prompt Pipeline in OpenAI Function Call. We can define a function that takes a string and returns segmented search queries using the `MultiSearch` class.
+```python
+from openai_function_call.dsl import MultiTask
+
+MultiSearch = MultiTask(Search)
+```
+
+## Calling Completions
+
+To segment a search query, we will use the base openai api. We can define a function that takes a string and returns segmented search queries using the `MultiSearch` class.
 
 ```python
 import openai
@@ -67,7 +78,7 @@ def segment(data: str) -> MultiSearch:
     return MultiSearch.from_response(completion)
 ```
 
-The `segment` function takes a string `data` and creates a completion using the Prompt Pipeline. It prompts the model to segment the data into multiple search queries and returns the result as a `MultiSearch` object.
+The `segment` function takes a string `data` and creates a completion. It prompts the model to segment the data into multiple search queries and returns the result as a `MultiSearch` object.
 
 ## Evaluating an Example
 

mkdocs.yml

@@ -40,9 +40,9 @@ nav:
       - "Introduction: Pipeline API": "pipeline-example.md"
       - "Message Templates": "chat-completion.md"
   - Examples:
-      - 'Table of contents': 'examples/index.md'
-      - 'Segmented Search': 'examples/search.md'
-      - 'One shot Query Planning': 'examples/planning-tasks.md'
-      - 'Recursive Schemas': 'examples/recursive.md'
-      - 'Exact Citations': 'examples/exact_citations.md'
-      - 'Automated Dataframe Extraction': "examples/autodataframe.md"
+      - 'Example applications': 'examples/index.md'
+      - 'Example: Segmented Search': 'examples/search.md'
+      - 'Example: One shot Query Planning': 'examples/planning-tasks.md'
+      - 'Example: Recursive Schemas': 'examples/recursive.md'
+      - 'Example: Exact Citations': 'examples/exact_citations.md'
+      - 'Example: Automated Dataframe Extraction': "examples/autodataframe.md"