mirror of https://github.com/kennethreitz/instructor.git synced 2026-06-05 22:50:18 +00:00

Files

T

Jason Liu 64e7f51601 Implement CLI for finetuning (#85 )

* add cli

* remove hint

* fix

* updat docs

* lock

2023-08-23 23:53:22 -07:00

5.9 KiB

Raw Blame History

Using the Command Line Interface

The instructor CLI provides functionalities for managing fine-tuning jobs on OpenAI.

Creating a Fine-Tuning Job

View Jobs Options

$ instructor jobs --help          

Usage: instructor jobs [OPTIONS] COMMAND [ARGS]...

  Monitor and create fine tuning jobs

Options:
  --help  Show this message and exit.

Commands:
  cancel            Cancel a fine-tuning job.
  create-from-file  Create a fine-tuning job from a file.
  create-from-id    Create a fine-tuning job from an existing ID.
  list              Monitor the status of the most recent fine-tuning jobs.

Create from File

The create-from-file command uploads and trains a model in a single step:

$ instructor jobs create-from-file transformed_data.jsonl

Create from ID

The create-from-id command uses an uploaded file and trains a model

$ instructor files upload transformed_data.jsonl 
$ instructor files list
...
$ instructor jobs create-from-file <file_id>

Viewing Files and Jobs

Viewing Jobs

$ instructor jobs list 

OpenAI Fine Tuning Job Monitoring                                                
┏━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━┓
┃                ┃              ┃                ┃     Completion ┃                 ┃                ┃        ┃                 ┃
┃ Job ID         ┃ Status       ┃  Creation Time ┃           Time ┃ Model Name      ┃ File ID        ┃ Epochs ┃ Base Model      ┃
┡━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━┩
│ ftjob-PWo6uwk… │ 🚫 cancelled │     2023-08-23 │            N/A │                 │ file-F7lJg6Z4… │ 3      │ gpt-3.5-turbo-… │
│                │              │       23:10:54 │                │                 │                │        │                 │
│ ftjob-1whjva8… │ 🚫 cancelled │     2023-08-23 │            N/A │                 │ file-F7lJg6Z4… │ 3      │ gpt-3.5-turbo-… │
│                │              │       22:47:05 │                │                 │                │        │                 │
│ ftjob-wGoBDld… │ 🚫 cancelled │     2023-08-23 │            N/A │                 │ file-F7lJg6Z4… │ 3      │ gpt-3.5-turbo-… │
│                │              │       22:44:12 │                │                 │                │        │                 │
│ ftjob-yd5aRTc… │ ✅ succeeded │     2023-08-23 │     2023-08-23 │ ft:gpt-3.5-tur… │ file-IQxAUDqX… │ 3      │ gpt-3.5-turbo-… │
│                │              │       14:26:03 │       15:02:29 │                 │                │        │                 │
└────────────────┴──────────────┴────────────────┴────────────────┴─────────────────┴────────────────┴────────┴─────────────────┘
                                    Automatically refreshes every 5 seconds, press Ctrl+C to exit

Viewing Files

$ instructor files list 

OpenAI Files                                                      
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━┓                         
┃ File ID                       ┃ Size (bytes) ┃ Creation Time       ┃ Filename ┃ Purpose   ┃                         
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━┩                         
│ file-0lw2BSNRUlXZXRRu2beCCWjl │       369523 │ 2023-08-23 23:31:57 │ file     │ fine-tune │                         
│ file-IHaUXcMEykmFUp1kt2puCDEq │       369523 │ 2023-08-23 23:09:35 │ file     │ fine-tune │                         
│ file-ja9vRBf0FydEOTolaa3BMqES │       369523 │ 2023-08-23 22:42:29 │ file     │ fine-tune │                         
│ file-F7lJg6Z47CREvmx4kyvyZ6Sn │       369523 │ 2023-08-23 22:42:03 │ file     │ fine-tune │                         
│ file-YUxqZPyJRl5GJCUTw3cNmA46 │       369523 │ 2023-08-23 22:29:10 │ file     │ fine-tune │                         
└───────────────────────────────┴──────────────┴─────────────────────┴──────────┴───────────┘

Conclusion

The instructor CLI offers an intuitive interface for managing OpenAI's fine-tuning jobs and related files. By utilizing simple commands, you can create, monitor, and manage your fine-tuning tasks with ease. Feel free to explore further options and parameters by using the --help flag with any command.

5.9 KiB Raw Blame History