Add refactoring example to gpt-engineer (#61)

* Add refactoring capabilities to gpt-engineer

* Improving results of refactoring

* Refactor script saves to file

* Add documentation about the new example in gpt-engineer

.gitignore

@@ -127,6 +127,7 @@ venv/
 ENV/
 env.bak/
 venv.bak/
+.envrc
 
 # Spyder project settings
 .spyderproject

docs/examples/gpt-engineer.md

@@ -54,7 +54,7 @@ We can define a function that takes in a string and returns a `Program` object.
 ```python
 import openai
 
-def segment(data: str) -> Program:
+def develop(data: str) -> Program:
     completion = openai.ChatCompletion.create(
         model="gpt-3.5-turbo-0613",
         temperature=0.1,
@@ -80,7 +80,7 @@ def segment(data: str) -> Program:
 Let's evaluate the example by specifying the program to create and print the resulting files.
 
 ```python
-queries = segment(
+program = develop(
         """
         Create a fastapi app with a readme.md file and a main.py file with 
         some basic math functions. the datamodels should use pydantic and 
@@ -89,7 +89,7 @@ queries = segment(
         and a curl example"""
     )
 
-for file in queries.files:
+for file in program.files:
     print(file.file_name)
     print("-")
     print(file.body)
@@ -161,4 +161,214 @@ The output will be:
     fastapi
     uvicorn
     pydantic
-```
+```
+
+## Add Refactoring Capabilities
+
+This second part of the example shows how OpenAI API can be used to update the multiples files previously created, based on new specifications.
+
+In order to do that, we'll rely on the standard [unidiff](https://en.wikipedia.org/wiki/Diff#Unified_format) format.
+
+This will be our definition for a change in our code base:
+
+```python
+from pydantic import Field
+from openai_function_call import OpenAISchema
+
+class Diff(OpenAISchema):
+    """
+    Changes that must be correctly made in a program's code repository defined as a
+    complete diff (Unified Format) file which will be used to `patch` the repository.
+
+    Example:
+      --- /path/to/original	timestamp
+      +++ /path/to/new	timestamp
+      @@ -1,3 +1,9 @@
+      +This is an important
+      +notice! It should
+      +therefore be located at
+      +the beginning of this
+      +document!
+      +
+       This part of the
+       document has stayed the
+       same from version to
+      @@ -8,13 +14,8 @@
+       compress the size of the
+       changes.
+      -This paragraph contains
+      -text that is outdated.
+      -It will be deleted in the
+      -near future.
+      -
+       It is important to spell
+      -check this dokument. On
+      +check this document. On
+       the other hand, a
+       misspelled word isn't
+       the end of the world.
+      @@ -22,3 +23,7 @@
+       this paragraph needs to
+       be changed. Things can
+       be added after it.
+      +
+      +This paragraph contains
+      +important new additions
+      +to this document.
+    """
+
+    diff: str = Field(
+        ...,
+        description=(
+            "Changes in a code repository correctly represented in 'diff' format, "
+            "correctly escaped so it could be used in a JSON"
+        ),
+    )
+```
+
+The `diff` class represents a *diff* file, with a set of changes that can be applied to our program using a tool like patch or Git.
+
+## Calling Refactor Completions
+
+We'll define a function that will pass the program and the new specifications to the OpenAI API:
+
+```python
+import openai
+from generate import Program
+
+def refactor(new_requirements: str, program: Program) -> Diff:
+    program_description = "\n".join(
+        [f"{code.file_name}\n[[[\n{code.body}\n]]]\n" for code in program.files]
+    )
+    completion = openai.ChatCompletion.create(
+        # model="gpt-3.5-turbo-0613",
+        model="gpt-4",
+        temperature=0,
+        functions=[Diff.openai_schema],
+        function_call={"name": Diff.openai_schema["name"]},
+        messages=[
+            {
+                "role": "system",
+                "content": "You are a world class programming AI capable of refactor "
+                "existing python repositories. You will name files correct, include "
+                "__init__.py files and write correct python code, with correct imports. "
+                "You'll deliver your changes in valid 'diff' format so that they could "
+                "be applied using the 'patch' command. "
+                "Make sure you put the correct line numbers, "
+                "and that all lines that must be changed are correctly marked.",
+            },
+            {
+                "role": "user",
+                "content": new_requirements,
+            },
+            {
+                "role": "user",
+                "content": program_description,
+            },
+        ],
+        max_tokens=1000,
+    )
+    return Diff.from_response(completion)
+```
+
+Notice we're using here the version `gpt-4` of the model, which is more powerful but, also, more expensive.
+
+## Creating an Example Refactoring
+
+To tests these refactoring, we'll use the `program` object, generated in the first part of this example.
+
+```python
+changes = refactor(
+    new_requirements="Refactor this code to use flask instead.",
+    program=program,
+)
+print(changes.diff)
+```
+
+The output will be this:
+
+```diff
+--- readme.md
++++ readme.md
+@@ -1,9 +1,9 @@
+ # FastAPI App
+
+-This is a FastAPI app that provides some basic math functions.
++This is a Flask app that provides some basic math functions.
+
+ ## Usage
+
+ To use this app, follow the instructions below:
+
+ 1. Install the required dependencies by running `pip install -r requirements.txt`.
+-2. Start the app by running `uvicorn main:app --reload`.
++2. Start the app by running `flask run`.
+ 3. Open your browser and navigate to `http://localhost:5000/docs` to access the Swagger UI documentation.
+
+ ## Example
+
+ To perform a basic math operation, you can use the following curl command:
+
+ ```bash
+-curl -X POST -H "Content-Type: application/json" -d '{"operation": "add", "operands": [2, 3]}' http://localhost:8000/calculate
++curl -X POST -H "Content-Type: application/json" -d '{"operation": "add", "operands": [2, 3]}' http://localhost:5000/calculate
+ ```
+
+--- main.py
++++ main.py
+@@ -1,29 +1,29 @@
+-from fastapi import FastAPI
+-from pydantic import BaseModel
++from flask import Flask, request, jsonify
+
+-app = FastAPI()
++app = Flask(__name__)
+
+
+-class Operation(BaseModel):
+-    operation: str
+-    operands: list
++@app.route('/calculate', methods=['POST'])
++def calculate():
++    data = request.get_json()
++    operation = data.get('operation')
++    operands = data.get('operands')
+
+
+-@app.post('/calculate')
+-async def calculate(operation: Operation):
+-    if operation.operation == 'add':
+-        result = sum(operation.operands)
+-    elif operation.operation == 'subtract':
+-        result = operation.operands[0] - sum(operation.operands[1:])
+-    elif operation.operation == 'multiply':
++    if operation == 'add':
++        result = sum(operands)
++    elif operation == 'subtract':
++        result = operands[0] - sum(operands[1:])
++    elif operation == 'multiply':
+         result = 1
+-        for operand in operation.operands:
++        for operand in operands:
+             result *= operand
+-    elif operation.operation == 'divide':
+-        result = operation.operands[0]
+-        for operand in operation.operands[1:]:
++    elif operation == 'divide':
++        result = operands[0]
++        for operand in operands[1:]:
+             result /= operand
+     else:
+         result = None
+-    return {'result': result}
++    return jsonify({'result': result})
+
+--- requirements.txt
++++ requirements.txt
+@@ -1,3 +1,2 @@
+-fastapi
+-uvicorn
+-pydantic
++flask
++flask-cors
+```

examples/gpt-engineer/changes.diff

@@ -0,0 +1,83 @@
+--- readme.md
++++ readme.md
+@@ -1,9 +1,9 @@
+ # FastAPI App
+ 
+-This is a FastAPI app that provides some basic math functions.
++This is a Flask app that provides some basic math functions.
+ 
+ ## Usage
+ 
+ To use this app, follow the instructions below:
+ 
+ 1. Install the required dependencies by running `pip install -r requirements.txt`.
+-2. Start the app by running `uvicorn main:app --reload`.
++2. Start the app by running `flask run`.
+ 3. Open your browser and navigate to `http://localhost:5000/docs` to access the Swagger UI documentation.
+ 
+ ## Example
+ 
+ To perform a basic math operation, you can use the following curl command:
+ 
+ ```bash
+-curl -X POST -H "Content-Type: application/json" -d '{"operation": "add", "operands": [2, 3]}' http://localhost:8000/calculate
++curl -X POST -H "Content-Type: application/json" -d '{"operation": "add", "operands": [2, 3]}' http://localhost:5000/calculate
+ ```
+
+--- main.py
++++ main.py
+@@ -1,29 +1,29 @@
+-from fastapi import FastAPI
+-from pydantic import BaseModel
++from flask import Flask, request, jsonify
+ 
+-app = FastAPI()
++app = Flask(__name__)
+ 
+ 
+-class Operation(BaseModel):
+-    operation: str
+-    operands: list
++@app.route('/calculate', methods=['POST'])
++def calculate():
++    data = request.get_json()
++    operation = data.get('operation')
++    operands = data.get('operands')
+ 
+ 
+-@app.post('/calculate')
+-async def calculate(operation: Operation):
+-    if operation.operation == 'add':
+-        result = sum(operation.operands)
+-    elif operation.operation == 'subtract':
+-        result = operation.operands[0] - sum(operation.operands[1:])
+-    elif operation.operation == 'multiply':
++    if operation == 'add':
++        result = sum(operands)
++    elif operation == 'subtract':
++        result = operands[0] - sum(operands[1:])
++    elif operation == 'multiply':
+         result = 1
+-        for operand in operation.operands:
++        for operand in operands:
+             result *= operand
+-    elif operation.operation == 'divide':
+-        result = operation.operands[0]
+-        for operand in operation.operands[1:]:
++    elif operation == 'divide':
++        result = operands[0]
++        for operand in operands[1:]:
+             result /= operand
+     else:
+         result = None
+-    return {'result': result}
++    return jsonify({'result': result})
+
+--- requirements.txt
++++ requirements.txt
+@@ -1,3 +1,2 @@
+-fastapi
+-uvicorn
+-pydantic
++flask
++flask-cors

examples/gpt-engineer/generate.py

@@ -28,7 +28,7 @@ class Program(OpenAISchema):
     files: List[File] = Field(..., description="List of files")
 
 
-def segment(data: str) -> Program:
+def develop(data: str) -> Program:
     completion = openai.ChatCompletion.create(
         model="gpt-3.5-turbo-0613",
         temperature=0.1,
@@ -50,16 +50,16 @@ def segment(data: str) -> Program:
 
 
 if __name__ == "__main__":
-    queries = segment(
+    program = develop(
         """
-        Create a fastapi app with a readme.md file and a main.py file with 
-        some basic math functions. the datamodels should use pydantic and 
-        the main.py should use fastapi. the readme.md should have a title 
-        and a description. The readme should contain some helpful infromation 
+        Create a fastapi app with a readme.md file and a main.py file with
+        some basic math functions. the datamodels should use pydantic and
+        the main.py should use fastapi. the readme.md should have a title
+        and a description. The readme should contain some helpful infromation
         and a curl example"""
     )
 
-    for file in queries.files:
+    for file in program.files:
         print(file.file_name)
         print("-")
         print(file.body)
@@ -81,12 +81,16 @@ if __name__ == "__main__":
 
     ## Example
 
-    You can use the following curl command to test the `/add` endpoint:
+    To perform a basic math operation, you can use the following curl command:
 
     ```bash
-    $ curl -X POST -H "Content-Type: application/json" -d '{"a": 2, "b": 3}' http://localhost:8000/add
+    curl -X POST -H "Content-Type: application/json" -d '{"operation": "add", "operands": [2, 3]}' http://localhost:8000/calculate
     ```
 
+
+
+
+
     main.py
     -
     from fastapi import FastAPI
@@ -95,32 +99,32 @@ if __name__ == "__main__":
     app = FastAPI()
 
 
-    class Numbers(BaseModel):
-        a: int
-        b: int
+    class Operation(BaseModel):
+        operation: str
+        operands: list
 
 
-    @app.post('/add')
-    def add_numbers(numbers: Numbers):
-        return {'result': numbers.a + numbers.b}
+    @app.post('/calculate')
+    async def calculate(operation: Operation):
+        if operation.operation == 'add':
+            result = sum(operation.operands)
+        elif operation.operation == 'subtract':
+            result = operation.operands[0] - sum(operation.operands[1:])
+        elif operation.operation == 'multiply':
+            result = 1
+            for operand in operation.operands:
+                result *= operand
+        elif operation.operation == 'divide':
+            result = operation.operands[0]
+            for operand in operation.operands[1:]:
+                result /= operand
+        else:
+            result = None
+        return {'result': result}
 
 
-    @app.post('/subtract')
-    def subtract_numbers(numbers: Numbers):
-        return {'result': numbers.a - numbers.b}
 
 
-    @app.post('/multiply')
-    def multiply_numbers(numbers: Numbers):
-        return {'result': numbers.a * numbers.b}
-
-
-    @app.post('/divide')
-    def divide_numbers(numbers: Numbers):
-        if numbers.b == 0:
-            return {'error': 'Cannot divide by zero'}
-        return {'result': numbers.a / numbers.b}
-
 
     requirements.txt
     -
@@ -128,3 +132,6 @@ if __name__ == "__main__":
     uvicorn
     pydantic
     """
+
+    with open("program.json", "w") as f:
+        f.write(Program.parse_obj(program).json())

examples/gpt-engineer/program.json

@@ -0,0 +1 @@
+{"files": [{"file_name": "readme.md", "body": "# FastAPI App\n\nThis is a FastAPI app that provides some basic math functions.\n\n## Usage\n\nTo use this app, follow the instructions below:\n\n1. Install the required dependencies by running `pip install -r requirements.txt`.\n2. Start the app by running `uvicorn main:app --reload`.\n3. Open your browser and navigate to `http://localhost:8000/docs` to access the Swagger UI documentation.\n\n## Example\n\nTo perform a basic math operation, you can use the following curl command:\n\n```bash\ncurl -X POST -H \"Content-Type: application/json\" -d '{\"operation\": \"add\", \"operands\": [2, 3]}' http://localhost:8000/calculate\n```\n"}, {"file_name": "main.py", "body": "from fastapi import FastAPI\nfrom pydantic import BaseModel\n\napp = FastAPI()\n\n\nclass Operation(BaseModel):\n    operation: str\n    operands: list\n\n\n@app.post('/calculate')\nasync def calculate(operation: Operation):\n    if operation.operation == 'add':\n        result = sum(operation.operands)\n    elif operation.operation == 'subtract':\n        result = operation.operands[0] - sum(operation.operands[1:])\n    elif operation.operation == 'multiply':\n        result = 1\n        for operand in operation.operands:\n            result *= operand\n    elif operation.operation == 'divide':\n        result = operation.operands[0]\n        for operand in operation.operands[1:]:\n            result /= operand\n    else:\n        result = None\n    return {'result': result}\n"}, {"file_name": "requirements.txt", "body": "fastapi\nuvicorn\npydantic"}]}

examples/gpt-engineer/refactor.py

@@ -0,0 +1,190 @@
+import openai
+
+from pydantic import Field, parse_file_as
+from openai_function_call import OpenAISchema
+
+from generate import Program
+
+
+class Diff(OpenAISchema):
+    """
+    Changes that must be correctly made in a program's code repository defined as a
+    complete diff (Unified Format) file which will be used to `patch` the repository.
+
+    Example:
+      --- /path/to/original	timestamp
+      +++ /path/to/new	timestamp
+      @@ -1,3 +1,9 @@
+      +This is an important
+      +notice! It should
+      +therefore be located at
+      +the beginning of this
+      +document!
+      +
+       This part of the
+       document has stayed the
+       same from version to
+      @@ -8,13 +14,8 @@
+       compress the size of the
+       changes.
+      -This paragraph contains
+      -text that is outdated.
+      -It will be deleted in the
+      -near future.
+      -
+       It is important to spell
+      -check this dokument. On
+      +check this document. On
+       the other hand, a
+       misspelled word isn't
+       the end of the world.
+      @@ -22,3 +23,7 @@
+       this paragraph needs to
+       be changed. Things can
+       be added after it.
+      +
+      +This paragraph contains
+      +important new additions
+      +to this document.
+    """
+
+    diff: str = Field(
+        ...,
+        description=(
+            "Changes in a code repository correctly represented in 'diff' format, "
+            "correctly escaped so it could be used in a JSON"
+        ),
+    )
+
+
+def refactor(new_requirements: str, program: Program) -> Diff:
+    program_description = "\n".join(
+        [f"{code.file_name}\n[[[\n{code.body}\n]]]\n" for code in program.files]
+    )
+    completion = openai.ChatCompletion.create(
+        # model="gpt-3.5-turbo-0613",
+        model="gpt-4",
+        temperature=0,
+        functions=[Diff.openai_schema],
+        function_call={"name": Diff.openai_schema["name"]},
+        messages=[
+            {
+                "role": "system",
+                "content": "You are a world class programming AI capable of refactor "
+                "existing python repositories. You will name files correct, include "
+                "__init__.py files and write correct python code, with correct imports. "
+                "You'll deliver your changes in valid 'diff' format so that they could "
+                "be applied using the 'patch' command. "
+                "Make sure you put the correct line numbers, "
+                "and that all lines that must be changed are correctly marked.",
+            },
+            {
+                "role": "user",
+                "content": new_requirements,
+            },
+            {
+                "role": "user",
+                "content": program_description,
+            },
+        ],
+        max_tokens=1000,
+    )
+    return Diff.from_response(completion)
+
+
+if __name__ == "__main__":
+    program = parse_file_as(path="program.json", type_=Program)
+
+    changes = refactor(
+        new_requirements="Refactor this code to use flask instead.",
+        program=program,
+    )
+    print(changes.diff)
+    """
+    --- readme.md
+    +++ readme.md
+    @@ -1,9 +1,9 @@
+     # FastAPI App
+
+    -This is a FastAPI app that provides some basic math functions.
+    +This is a Flask app that provides some basic math functions.
+
+     ## Usage
+
+     To use this app, follow the instructions below:
+
+     1. Install the required dependencies by running `pip install -r requirements.txt`.
+    -2. Start the app by running `uvicorn main:app --reload`.
+    +2. Start the app by running `flask run`.
+     3. Open your browser and navigate to `http://localhost:5000/docs` to access the Swagger UI documentation.
+
+     ## Example
+
+     To perform a basic math operation, you can use the following curl command:
+
+     ```bash
+    -curl -X POST -H "Content-Type: application/json" -d '{"operation": "add", "operands": [2, 3]}' http://localhost:8000/calculate
+    +curl -X POST -H "Content-Type: application/json" -d '{"operation": "add", "operands": [2, 3]}' http://localhost:5000/calculate
+     ```
+
+    --- main.py
+    +++ main.py
+    @@ -1,29 +1,29 @@
+    -from fastapi import FastAPI
+    -from pydantic import BaseModel
+    +from flask import Flask, request, jsonify
+
+    -app = FastAPI()
+    +app = Flask(__name__)
+
+
+    -class Operation(BaseModel):
+    -    operation: str
+    -    operands: list
+    +@app.route('/calculate', methods=['POST'])
+    +def calculate():
+    +    data = request.get_json()
+    +    operation = data.get('operation')
+    +    operands = data.get('operands')
+
+
+    -@app.post('/calculate')
+    -async def calculate(operation: Operation):
+    -    if operation.operation == 'add':
+    -        result = sum(operation.operands)
+    -    elif operation.operation == 'subtract':
+    -        result = operation.operands[0] - sum(operation.operands[1:])
+    -    elif operation.operation == 'multiply':
+    +    if operation == 'add':
+    +        result = sum(operands)
+    +    elif operation == 'subtract':
+    +        result = operands[0] - sum(operands[1:])
+    +    elif operation == 'multiply':
+             result = 1
+    -        for operand in operation.operands:
+    +        for operand in operands:
+                 result *= operand
+    -    elif operation.operation == 'divide':
+    -        result = operation.operands[0]
+    -        for operand in operation.operands[1:]:
+    +    elif operation == 'divide':
+    +        result = operands[0]
+    +        for operand in operands[1:]:
+                 result /= operand
+         else:
+             result = None
+    -    return {'result': result}
+    +    return jsonify({'result': result})
+
+    --- requirements.txt
+    +++ requirements.txt
+    @@ -1,3 +1,2 @@
+    -fastapi
+    -uvicorn
+    -pydantic
+    +flask
+    +flask-cors
+    """
+
+    with open("changes.diff", "w") as f:
+        f.write(changes.diff)