instructor/tutorials/2.tips.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "8bb7d0d0-2b7f-4e9e-8565-467dc5c6fd22",
   "metadata": {},
   "source": [
    "# General Tips on Prompting\n",
    "\n",
    "Before we get into some big applications of schema engineering I want to equip you with the tools for success.\n",
    "This notebook is to share some general advice when using prompts to get the most of your models.\n",
    "\n",
    "Before you might think of prompt engineering as massaging this wall of text, almost like coding in a notepad. But with schema engineering you can get a lot more out of your prompts with a lot less work.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8a785c25-b08d-4ab4-bbd7-22e3b090c2ed",
   "metadata": {},
   "source": [
    "## Classification\n",
    "\n",
    "For classification we've found theres generally two methods of modeling.\n",
    "\n",
    "1. using Enums\n",
    "2. using Literals\n",
    "\n",
    "Use an enum in Python when you need a set of named constants that are related and you want to ensure type safety, readability, and prevent invalid values. Enums are helpful for grouping and iterating over these constants.\n",
    "\n",
    "Use literals when you have a small, unchanging set of values that you don't need to group or iterate over, and when type safety and preventing invalid values is less of a concern. Literals are simpler and more direct for basic, one-off values.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "fdf5e1d9-31ad-4e8a-a55e-e2e70fff598d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'age': 17, 'name': 'Harry Potter', 'house': <House.Gryffindor: 'gryffindor'>}"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import instructor\n",
    "from openai import OpenAI\n",
    "\n",
    "from enum import Enum\n",
    "from pydantic import BaseModel, Field\n",
    "from typing_extensions import Literal\n",
    "\n",
    "\n",
    "client = instructor.patch(OpenAI())\n",
    "\n",
    "\n",
    "# Tip: Do not use auto() as they cast to 1,2,3,4\n",
    "class House(Enum):\n",
    "    Gryffindor = \"gryffindor\"\n",
    "    Hufflepuff = \"hufflepuff\"\n",
    "    Ravenclaw = \"ravenclaw\"\n",
    "    Slytherin = \"slytherin\"\n",
    "\n",
    "\n",
    "class Character(BaseModel):\n",
    "    age: int\n",
    "    name: str\n",
    "    house: House\n",
    "\n",
    "    def say_hello(self):\n",
    "        print(\n",
    "            f\"Hello, I'm {self.name}, I'm {self.age} years old and I'm from {self.house.value.title()}\"\n",
    "        )\n",
    "\n",
    "\n",
    "resp = client.chat.completions.create(\n",
    "    model=\"gpt-4-1106-preview\",\n",
    "    messages=[{\"role\": \"user\", \"content\": \"Harry Potter\"}],\n",
    "    response_model=Character,\n",
    ")\n",
    "resp.model_dump()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "c609eb44",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Hello, I'm Harry Potter, I'm 17 years old and I'm from Gryffindor\n"
     ]
    }
   ],
   "source": [
    "resp.say_hello()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "03db160c-81e9-4373-bfec-7a107224b6dd",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'age': 17, 'name': 'Harry Potter', 'house': 'Gryffindor'}"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "class Character(BaseModel):\n",
    "    age: int\n",
    "    name: str\n",
    "    house: Literal[\"Gryffindor\", \"Hufflepuff\", \"Ravenclaw\", \"Slytherin\"]\n",
    "\n",
    "\n",
    "resp = client.chat.completions.create(\n",
    "    model=\"gpt-4-1106-preview\",\n",
    "    messages=[{\"role\": \"user\", \"content\": \"Harry Potter\"}],\n",
    "    response_model=Character,\n",
    ")\n",
    "resp.model_dump()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "803e0ce6-6e7e-4d86-a7a8-49ebaad0a40b",
   "metadata": {},
   "source": [
    "## Arbitrary properties\n",
    "\n",
    "Often times there are long properties that you might want to extract from data that we can not specify in advanced. We can get around this by defining an arbitrary key value store like so:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "0e7938b8-4666-4df4-bd80-f53e8baf7550",
   "metadata": {},
   "outputs": [],
   "source": [
    "from typing import List\n",
    "\n",
    "\n",
    "class Property(BaseModel):\n",
    "    key: str = Field(description=\"Must be snake case\")\n",
    "    value: str\n",
    "\n",
    "\n",
    "class Character(BaseModel):\n",
    "    age: int\n",
    "    name: str\n",
    "    house: Literal[\"Gryffindor\", \"Hufflepuff\", \"Ravenclaw\", \"Slytherin\"]\n",
    "    properties: List[Property]\n",
    "\n",
    "\n",
    "resp = client.chat.completions.create(\n",
    "    model=\"gpt-4-1106-preview\",\n",
    "    messages=[{\"role\": \"user\", \"content\": \"Snape from Harry Potter\"}],\n",
    "    response_model=Character,\n",
    ")\n",
    "resp.model_dump()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b3e62f68-a79f-4f65-9c1f-726e4e2d340a",
   "metadata": {},
   "source": [
    "## Limiting the length of lists\n",
    "\n",
    "In later chapters we'll talk about how to use validators to assert the length of lists but we can also use prompting tricks to enumerate values. Here we'll define a index to count the properties.\n",
    "\n",
    "In this following example instead of extraction we're going to work on generation instead.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "69a58d01-ab6f-41b6-bc0c-b0e55fdb6fe4",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'age': 38,\n",
       " 'name': 'Severus Snape',\n",
       " 'house': 'Slytherin',\n",
       " 'properties': [{'index': '1', 'key': 'patronus', 'value': 'Doe'},\n",
       "  {'index': '2',\n",
       "   'key': 'position',\n",
       "   'value': 'Potions Master, Defense Against the Dark Arts teacher, Headmaster'},\n",
       "  {'index': '3',\n",
       "   'key': 'loyalty',\n",
       "   'value': 'Hogwarts, Albus Dumbledore, Order of the Phoenix, Lily Evans'},\n",
       "  {'index': '4',\n",
       "   'key': 'skills',\n",
       "   'value': 'Potions expertise, Occlumency, Legilimency'},\n",
       "  {'index': '5',\n",
       "   'key': 'disguised_loyalty',\n",
       "   'value': 'Death Eater (formerly, as a double agent)'}]}"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "class Property(BaseModel):\n",
    "    index: str = Field(..., description=\"Monotonically increasing ID\")\n",
    "    key: str = Field(description=\"Must be snake case\")\n",
    "    value: str\n",
    "\n",
    "\n",
    "class Character(BaseModel):\n",
    "    age: int\n",
    "    name: str\n",
    "    house: Literal[\"Gryffindor\", \"Hufflepuff\", \"Ravenclaw\", \"Slytherin\"]\n",
    "    properties: List[Property] = Field(\n",
    "        ...,\n",
    "        description=\"Numbered list of arbitrary extracted properties, should be exactly 5\",\n",
    "    )\n",
    "\n",
    "\n",
    "resp = client.chat.completions.create(\n",
    "    model=\"gpt-4-1106-preview\",\n",
    "    messages=[{\"role\": \"user\", \"content\": \"Snape from Harry Potter\"}],\n",
    "    response_model=Character,\n",
    ")\n",
    "resp.model_dump()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bbc1d900-617a-4e4d-a401-6d10a5153cda",
   "metadata": {},
   "source": [
    "## Defining Multiple Entities\n",
    "\n",
    "Now that we see a single entity with many properties we can continue to nest them into many users\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1f2a2b14-a956-4f96-90c9-e11ca04ab7d1",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "age=38 name='Severus Snape' house='Slytherin'\n",
      "age=115 name='Albus Dumbledore' house='Gryffindor'\n"
     ]
    }
   ],
   "source": [
    "from typing import Iterable\n",
    "\n",
    "\n",
    "class Character(BaseModel):\n",
    "    age: int\n",
    "    name: str\n",
    "    house: Literal[\"Gryffindor\", \"Hufflepuff\", \"Ravenclaw\", \"Slytherin\"]\n",
    "\n",
    "\n",
    "resp = client.chat.completions.create(\n",
    "    model=\"gpt-4-1106-preview\",\n",
    "    messages=[{\"role\": \"user\", \"content\": \"Snape and Dumbledore from Harry Potter\"}],\n",
    "    response_model=Iterable[Character],\n",
    ")\n",
    "\n",
    "for character in resp:\n",
    "    print(character)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a3091aba",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "age=38 name='Severus Snape' house='Slytherin'\n",
      "age=115 name='Albus Dumbledore' house='Gryffindor'\n"
     ]
    }
   ],
   "source": [
    "from typing import Iterable\n",
    "\n",
    "\n",
    "class Character(BaseModel):\n",
    "    age: int\n",
    "    name: str\n",
    "    house: Literal[\"Gryffindor\", \"Hufflepuff\", \"Ravenclaw\", \"Slytherin\"]\n",
    "\n",
    "\n",
    "resp = client.chat.completions.create(\n",
    "    model=\"gpt-4-1106-preview\",\n",
    "    messages=[{\"role\": \"user\", \"content\": \"Snape and Dumbledore from Harry Potter\"}],\n",
    "    stream=True,\n",
    "    response_model=Iterable[Character],\n",
    ")\n",
    "\n",
    "for character in resp:\n",
    "    print(character)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f6ed3144-bde1-4033-9c94-a6926fa079d2",
   "metadata": {},
   "source": [
    "## Defining Relationships\n",
    "\n",
    "Now only can we define lists of users, with list of properties one of the more interesting things I've learned about prompting is that we can also easily define lists of references.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6de8768e-b36a-4a51-9cf9-940d178552f6",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "id=1 name='Harry Potter' friends=[2, 3, 4, 5]\n",
      "id=2 name='Hermione Granger' friends=[1, 3, 4, 5]\n",
      "id=3 name='Ron Weasley' friends=[1, 2, 4, 5]\n",
      "id=4 name='Draco Malfoy' friends=[5]\n",
      "id=5 name='Neville Longbottom' friends=[1, 2, 3, 4]\n"
     ]
    }
   ],
   "source": [
    "class Character(BaseModel):\n",
    "    id: int\n",
    "    name: str\n",
    "    friends: List[int]\n",
    "\n",
    "\n",
    "resp = client.chat.completions.create(\n",
    "    model=\"gpt-4-1106-preview\",\n",
    "    messages=[{\"role\": \"user\", \"content\": \"The 5 kids from Harry Potter\"}],\n",
    "    stream=True,\n",
    "    response_model=Iterable[Character],\n",
    ")\n",
    "\n",
    "for character in resp:\n",
    "    print(character)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b31e10d7-ebd2-49b4-b2c4-20dd67ca135d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       " \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Generated by graphviz version 9.0.0 (20230911.1827)\n",
       " -->\n",
       "<!-- Pages: 1 -->\n",
       "<svg width=\"338pt\" height=\"260pt\"\n",
       " viewBox=\"0.00 0.00 338.40 260.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 256)\">\n",
       "<polygon fill=\"white\" stroke=\"none\" points=\"-4,4 -4,-256 334.4,-256 334.4,4 -4,4\"/>\n",
       "<!-- 1 -->\n",
       "<g id=\"node1\" class=\"node\">\n",
       "<title>1</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"154.73\" cy=\"-234\" rx=\"56.98\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"154.73\" y=\"-228.95\" font-family=\"Times,serif\" font-size=\"14.00\">Harry Potter</text>\n",
       "</g>\n",
       "<!-- 2 -->\n",
       "<g id=\"node2\" class=\"node\">\n",
       "<title>2</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"100.73\" cy=\"-162\" rx=\"81.04\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"100.73\" y=\"-156.95\" font-family=\"Times,serif\" font-size=\"14.00\">Hermione Granger</text>\n",
       "</g>\n",
       "<!-- 1&#45;&gt;2 -->\n",
       "<g id=\"edge1\" class=\"edge\">\n",
       "<title>1&#45;&gt;2</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M141.66,-216.05C135.36,-207.89 127.67,-197.91 120.66,-188.82\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"123.49,-186.77 114.61,-180.99 117.95,-191.04 123.49,-186.77\"/>\n",
       "</g>\n",
       "<!-- 3 -->\n",
       "<g id=\"node3\" class=\"node\">\n",
       "<title>3</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"159.73\" cy=\"-90\" rx=\"60.05\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"159.73\" y=\"-84.95\" font-family=\"Times,serif\" font-size=\"14.00\">Ron Weasley</text>\n",
       "</g>\n",
       "<!-- 1&#45;&gt;3 -->\n",
       "<g id=\"edge2\" class=\"edge\">\n",
       "<title>1&#45;&gt;3</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M169.97,-216.24C177.83,-206.44 186.61,-193.4 190.73,-180 195.44,-164.71 194.92,-159.44 190.73,-144 188.22,-134.74 183.65,-125.44 178.77,-117.27\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"181.73,-115.41 173.38,-108.9 175.84,-119.2 181.73,-115.41\"/>\n",
       "</g>\n",
       "<!-- 4 -->\n",
       "<g id=\"node4\" class=\"node\">\n",
       "<title>4</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"71.73\" cy=\"-18\" rx=\"63.63\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"71.73\" y=\"-12.95\" font-family=\"Times,serif\" font-size=\"14.00\">Draco Malfoy</text>\n",
       "</g>\n",
       "<!-- 1&#45;&gt;4 -->\n",
       "<g id=\"edge3\" class=\"edge\">\n",
       "<title>1&#45;&gt;4</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M100.63,-228.11C68.79,-222.13 30.93,-208.95 10.73,-180 -19.12,-137.2 20.11,-77.9 48.09,-44.51\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"50.73,-46.81 54.63,-36.96 45.43,-42.23 50.73,-46.81\"/>\n",
       "</g>\n",
       "<!-- 5 -->\n",
       "<g id=\"node5\" class=\"node\">\n",
       "<title>5</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"243.73\" cy=\"-18\" rx=\"86.67\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"243.73\" y=\"-12.95\" font-family=\"Times,serif\" font-size=\"14.00\">Neville Longbottom</text>\n",
       "</g>\n",
       "<!-- 1&#45;&gt;5 -->\n",
       "<g id=\"edge4\" class=\"edge\">\n",
       "<title>1&#45;&gt;5</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M174.21,-216.8C184.81,-207.08 197.43,-193.93 205.73,-180 230.62,-138.26 239.21,-81.35 242.17,-47.57\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"245.65,-47.99 242.92,-37.75 238.67,-47.46 245.65,-47.99\"/>\n",
       "</g>\n",
       "<!-- 2&#45;&gt;3 -->\n",
       "<g id=\"edge5\" class=\"edge\">\n",
       "<title>2&#45;&gt;3</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M115.02,-144.05C121.97,-135.8 130.49,-125.7 138.21,-116.54\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"140.84,-118.85 144.61,-108.95 135.49,-114.34 140.84,-118.85\"/>\n",
       "</g>\n",
       "<!-- 2&#45;&gt;4 -->\n",
       "<g id=\"edge6\" class=\"edge\">\n",
       "<title>2&#45;&gt;4</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M97.18,-143.59C92.26,-119.5 83.32,-75.75 77.46,-47.03\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"80.96,-46.69 75.53,-37.59 74.1,-48.09 80.96,-46.69\"/>\n",
       "</g>\n",
       "<!-- 3&#45;&gt;4 -->\n",
       "<g id=\"edge7\" class=\"edge\">\n",
       "<title>3&#45;&gt;4</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M139.32,-72.76C127.96,-63.73 113.63,-52.33 101.09,-42.35\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"103.53,-39.82 93.52,-36.33 99.17,-45.3 103.53,-39.82\"/>\n",
       "</g>\n",
       "<!-- 3&#45;&gt;5 -->\n",
       "<g id=\"edge8\" class=\"edge\">\n",
       "<title>3&#45;&gt;5</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M179.22,-72.76C189.8,-63.95 203.09,-52.87 214.85,-43.07\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"216.88,-45.94 222.32,-36.85 212.39,-40.56 216.88,-45.94\"/>\n",
       "</g>\n",
       "</g>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<graphviz.graphs.Digraph at 0x119894f90>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from graphviz import Digraph\n",
    "from IPython.display import display\n",
    "\n",
    "dot = Digraph()\n",
    "\n",
    "resp = client.chat.completions.create(\n",
    "    model=\"gpt-4-1106-preview\",\n",
    "    messages=[{\"role\": \"user\", \"content\": \"The 5 kids from Harry Potter\"}],\n",
    "    response_model=Iterable[Character],\n",
    ")\n",
    "\n",
    "# Create nodes for each user\n",
    "for user in resp:\n",
    "    dot.node(str(user.id), user.name)\n",
    "\n",
    "# Create edges for friends\n",
    "for user in resp:\n",
    "    for friend_id in user.friends:\n",
    "        # To avoid duplicating edges, only add an edge if the friend ID is greater than the user ID\n",
    "        if friend_id > user.id:\n",
    "            dot.edge(str(user.id), str(friend_id))\n",
    "\n",
    "\n",
    "# Render the graph to a file\n",
    "display(dot)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "523b5797-71a5-4a96-a4b7-21280fb73015",
   "metadata": {},
   "source": [
    "With the tools we've discussed, we can find numerous real-world applications in production settings. These include extracting action items from transcripts, generating fake data, filling out forms, and creating objects that correspond to generative UI. These simple tricks will be highly useful.\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}