-
Notifications
You must be signed in to change notification settings - Fork 9.4k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add cookbook for using chained calls for o1 structured outputs (#1420)
- Loading branch information
1 parent
339300b
commit 4f950d6
Showing
3 changed files
with
270 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
256 changes: 256 additions & 0 deletions
256
examples/o1/Using_chained_calls_for_o1_structured_outputs.ipynb
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,256 @@ | ||
| { | ||
| "cells": [ | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "# Using chained calls for reasoning structured outputs" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "The initially released versions (September 2024) of [o1](https://openai.com/index/introducing-openai-o1-preview/) reasoning models have advanced capabilities but do not have [structured outputs](https://platform.openai.com/docs/guides/structured-outputs/examples) support. \n", | ||
| "\n", | ||
| "This means that requests with o1 don't have reliable type-safety and rely on the prompt itself to return a useful JSON. \n", | ||
| "\n", | ||
| "In this guide, we'll explore two methods to prompt o1 models, specifically `o1-preview`, to return a valid JSON format when using the OpenAI API." | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "# Prompting\n", | ||
| "\n", | ||
| "The simplest way to return a JSON response using `o1-preview` is to explicitly prompt it. \n", | ||
| "\n", | ||
| "Let's run through an example of:\n", | ||
| "- Fetching a wikipedia page of companies\n", | ||
| "- Determining which could benefit the most from AI capabilities\n", | ||
| "- Returning them in a JSON format, which could then be ingested by other systems" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": 11, | ||
| "metadata": {}, | ||
| "outputs": [ | ||
| { | ||
| "name": "stdout", | ||
| "output_type": "stream", | ||
| "text": [ | ||
| "{\n", | ||
| " \"companies\": [\n", | ||
| " {\n", | ||
| " \"company_name\": \"Walmart\",\n", | ||
| " \"page_link\": \"https://en.wikipedia.org/wiki/Walmart\",\n", | ||
| " \"reason\": \"Walmart could benefit from AI technology by enhancing their supply chain management, optimizing inventory levels, improving customer service through AI-powered chatbots, and providing personalized shopping experiences. AI can help Walmart forecast demand more accurately, reduce operational costs, and increase overall efficiency.\"\n", | ||
| " },\n", | ||
| " {\n", | ||
| " \"company_name\": \"UnitedHealth Group\",\n", | ||
| " \"page_link\": \"https://en.wikipedia.org/wiki/UnitedHealth_Group\",\n", | ||
| " \"reason\": \"UnitedHealth Group could leverage AI technology to improve patient care through predictive analytics, personalize treatment plans, detect fraudulent claims, and streamline administrative processes. AI can assist in early disease detection, improve diagnostic accuracy, and enhance data analysis for better health outcomes.\"\n", | ||
| " },\n", | ||
| " {\n", | ||
| " \"company_name\": \"Ford Motor Company\",\n", | ||
| " \"page_link\": \"https://en.wikipedia.org/wiki/Ford_Motor_Company\",\n", | ||
| " \"reason\": \"Ford Motor Company could benefit from AI technology by advancing autonomous vehicle development, optimizing manufacturing processes with automation and robotics, implementing predictive maintenance, and enhancing the in-car experience with AI-driven features. AI can help Ford improve safety, reduce production costs, and innovate new transportation solutions.\"\n", | ||
| " }\n", | ||
| " ]\n", | ||
| "}\n" | ||
| ] | ||
| } | ||
| ], | ||
| "source": [ | ||
| "import requests\n", | ||
| "from openai import OpenAI\n", | ||
| "\n", | ||
| "client = OpenAI()\n", | ||
| "\n", | ||
| "def fetch_html(url):\n", | ||
| " response = requests.get(url)\n", | ||
| " if response.status_code == 200:\n", | ||
| " return response.text\n", | ||
| " else:\n", | ||
| " return None\n", | ||
| "\n", | ||
| "url = \"https://en.wikipedia.org/wiki/List_of_largest_companies_in_the_United_States_by_revenue\"\n", | ||
| "html_content = fetch_html(url)\n", | ||
| "\n", | ||
| "json_format = \"\"\"\n", | ||
| "{\n", | ||
| " companies: [\n", | ||
| " {\n", | ||
| " \\\"company_name\\\": \\\"OpenAI\\\",\n", | ||
| " \\\"page_link\\\": \\\"https://en.wikipedia.org/wiki/OpenAI\\\",\n", | ||
| " \\\"reason\\\": \\\"OpenAI would benefit because they are an AI company...\\\"\n", | ||
| " }\n", | ||
| " ]\n", | ||
| "}\n", | ||
| "\"\"\"\n", | ||
| "\n", | ||
| "o1_response = client.chat.completions.create(\n", | ||
| " model=\"o1-preview\",\n", | ||
| " messages=[\n", | ||
| " {\n", | ||
| " \"role\": \"user\", \n", | ||
| " \"content\": f\"\"\"\n", | ||
| "You are a business analyst designed to understand how AI technology could be used across large corporations.\n", | ||
| "\n", | ||
| "- Read the following html and return which companies would benefit from using AI technology: {html_content}.\n", | ||
| "- Rank these propects by opportunity by comparing them and show me the top 3. Return only as a JSON with the following format: {json_format}\"\n", | ||
| "\"\"\"\n", | ||
| " }\n", | ||
| " ]\n", | ||
| ")\n", | ||
| "\n", | ||
| "print(o1_response.choices[0].message.content)" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "Note that the response is already quite good - it returns the JSON with the appropriate responses. However, it runs into the same pitfalls as existing use-cases of prompt-only JSON inference: \n", | ||
| "- You must manually process this JSON into your type-safe structure\n", | ||
| "- Model refusals are not [explicitly returned from the API as a separate structure](https://platform.openai.com/docs/guides/structured-outputs/refusals)" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "# Structured Outputs" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "Let's now do this with [structured outputs.](https://platform.openai.com/docs/guides/structured-outputs). To enable this functionality, we’ll link the `o1-preview` response with a follow-up request to `gpt-4o-mini`, which can effectively process the data returned from the initial o1-preview response." | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "code", | ||
| "execution_count": 12, | ||
| "metadata": {}, | ||
| "outputs": [ | ||
| { | ||
| "name": "stdout", | ||
| "output_type": "stream", | ||
| "text": [ | ||
| "CompaniesData(\n", | ||
| " companies=[\n", | ||
| " CompanyData(\n", | ||
| " company_name='Walmart',\n", | ||
| " page_link='https://en.wikipedia.org/wiki/Walmart',\n", | ||
| " reason=(\n", | ||
| " 'As the largest retailer, Walmart can significantly benefit from AI by optimizing supply chain and inv'\n", | ||
| " 'entory management, improving demand forecasting, personalizing customer experiences, and enhancing in'\n", | ||
| " '-store operations through AI-driven analytics.'\n", | ||
| " ),\n", | ||
| " ),\n", | ||
| " CompanyData(\n", | ||
| " company_name='JPMorgan Chase',\n", | ||
| " page_link='https://en.wikipedia.org/wiki/JPMorgan_Chase',\n", | ||
| " reason=(\n", | ||
| " 'As a leading financial institution, JPMorgan Chase can leverage AI for fraud detection, risk manageme'\n", | ||
| " 'nt, personalized banking services, algorithmic trading, and enhancing customer service with AI-powere'\n", | ||
| " 'd chatbots and virtual assistants.'\n", | ||
| " ),\n", | ||
| " ),\n", | ||
| " CompanyData(\n", | ||
| " company_name='UnitedHealth Group',\n", | ||
| " page_link='https://en.wikipedia.org/wiki/UnitedHealth_Group',\n", | ||
| " reason=(\n", | ||
| " 'Being a major player in healthcare, UnitedHealth Group can utilize AI to improve patient care through'\n", | ||
| " ' predictive analytics, enhance diagnostics, streamline administrative processes, and reduce costs by '\n", | ||
| " 'optimizing operations with AI-driven solutions.'\n", | ||
| " ),\n", | ||
| " ),\n", | ||
| " ],\n", | ||
| ")\n" | ||
| ] | ||
| } | ||
| ], | ||
| "source": [ | ||
| "from pydantic import BaseModel\n", | ||
| "from devtools import pprint\n", | ||
| "\n", | ||
| "class CompanyData(BaseModel):\n", | ||
| " company_name: str\n", | ||
| " page_link: str\n", | ||
| " reason: str\n", | ||
| "\n", | ||
| "class CompaniesData(BaseModel):\n", | ||
| " companies: list[CompanyData]\n", | ||
| "\n", | ||
| "o1_response = client.chat.completions.create(\n", | ||
| " model=\"o1-preview\",\n", | ||
| " messages=[\n", | ||
| " {\n", | ||
| " \"role\": \"user\", \n", | ||
| " \"content\": f\"\"\"\n", | ||
| "You are a business analyst designed to understand how AI technology could be used across large corporations.\n", | ||
| "\n", | ||
| "- Read the following html and return which companies would benefit from using AI technology: {html_content}.\n", | ||
| "- Rank these propects by opportunity by comparing them and show me the top 3. Return each with {CompanyData.__fields__.keys()}\n", | ||
| "\"\"\"\n", | ||
| " }\n", | ||
| " ]\n", | ||
| ")\n", | ||
| "\n", | ||
| "o1_response_content = o1_response.choices[0].message.content\n", | ||
| "\n", | ||
| "response = client.beta.chat.completions.parse(\n", | ||
| " model=\"gpt-4o-mini\",\n", | ||
| " messages=[\n", | ||
| " {\n", | ||
| " \"role\": \"user\", \n", | ||
| " \"content\": f\"\"\"\n", | ||
| "Given the following data, format it with the given response format: {o1_response_content}\n", | ||
| "\"\"\"\n", | ||
| " }\n", | ||
| " ],\n", | ||
| " response_format=CompaniesData,\n", | ||
| ")\n", | ||
| "\n", | ||
| "pprint(response.choices[0].message.parsed)" | ||
| ] | ||
| }, | ||
| { | ||
| "cell_type": "markdown", | ||
| "metadata": {}, | ||
| "source": [ | ||
| "# Conclusion\n", | ||
| "\n", | ||
| "Structured outputs allow your code to have reliable type-safety and simpler prompting. In addition, it allows you to re-use your object schemas for easier integration into your existing workflows.\n", | ||
| "\n", | ||
| "The o1 class of models currently doesn't have structured outputs support, but we can re-use existing structured outputs functionality from `gpt-4o-mini` by chaining two requests together. This flow currently requires two calls, but the second `gpt-4o-mini` call cost should be minimal compared to the `o1-preview`/`o1-mini` calls." | ||
| ] | ||
| } | ||
| ], | ||
| "metadata": { | ||
| "kernelspec": { | ||
| "display_name": "Python 3", | ||
| "language": "python", | ||
| "name": "python3" | ||
| }, | ||
| "language_info": { | ||
| "codemirror_mode": { | ||
| "name": "ipython", | ||
| "version": 3 | ||
| }, | ||
| "file_extension": ".py", | ||
| "mimetype": "text/x-python", | ||
| "name": "python", | ||
| "nbconvert_exporter": "python", | ||
| "pygments_lexer": "ipython3", | ||
| "version": "3.12.6" | ||
| } | ||
| }, | ||
| "nbformat": 4, | ||
| "nbformat_minor": 2 | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters