Skip to main content

ChatOpenAI

You can use OpenAI's chat models as follows:

tip

See this section for general instructions on installing integration packages.

npm install @langchain/openai
tip

We're unifying model params across all packages. We now suggest using model instead of modelName, and apiKey for API keys.

import { ChatOpenAI } from "@langchain/openai";
import { HumanMessage } from "@langchain/core/messages";

const model = new ChatOpenAI({
  temperature: 0.9,
  apiKey: "YOUR-API-KEY", // In Node.js defaults to process.env.OPENAI_API_KEY
});

// You can also pass tools or functions to the model, learn more here
// https://platform.openai.com/docs/guides/gpt/function-calling

const modelForFunctionCalling = new ChatOpenAI({
  model: "gpt-4",
  temperature: 0,
});

await modelForFunctionCalling.invoke(
  [new HumanMessage("What is the weather in New York?")],
  {
    functions: [
      {
        name: "get_current_weather",
        description: "Get the current weather in a given location",
        parameters: {
          type: "object",
          properties: {
            location: {
              type: "string",
              description: "The city and state, e.g. San Francisco, CA",
            },
            unit: { type: "string", enum: ["celsius", "fahrenheit"] },
          },
          required: ["location"],
        },
      },
    ],
    // You can set the `function_call` arg to force the model to use a function
    function_call: {
      name: "get_current_weather",
    },
  }
);
/*
AIMessage {
  text: '',
  name: undefined,
  additional_kwargs: {
    function_call: {
      name: 'get_current_weather',
      arguments: '{\n  "location": "New York"\n}'
    }
  }
}
*/

// Coerce response type with JSON mode.
// Requires "gpt-4-1106-preview" or later
const jsonModeModel = new ChatOpenAI({
  model: "gpt-4-1106-preview",
  maxTokens: 128,
}).bind({
  response_format: {
    type: "json_object",
  },
});

// Must be invoked with a system message containing the string "JSON":
// https://platform.openai.com/docs/guides/text-generation/json-mode
const res = await jsonModeModel.invoke([
  ["system", "Only return JSON"],
  ["human", "Hi there!"],
]);
console.log(res);

/*
  AIMessage {
    content: '{\n  "response": "How can I assist you today?"\n}',
    name: undefined,
    additional_kwargs: { function_call: undefined, tool_calls: undefined }
  }
*/

API Reference:

If you're part of an organization, you can set process.env.OPENAI_ORGANIZATION with your OpenAI organization id, or pass it in as organization when initializing the model.

Multimodal messages​

info

This feature is currently in preview. The message schema may change in future releases.

OpenAI supports interleaving images with text in input messages with their gpt-4-vision-preview. Here's an example of how this looks:

import * as fs from "node:fs/promises";

import { ChatOpenAI } from "@langchain/openai";
import { HumanMessage } from "@langchain/core/messages";

const imageData = await fs.readFile("./hotdog.jpg");
const chat = new ChatOpenAI({
  model: "gpt-4-vision-preview",
  maxTokens: 1024,
});
const message = new HumanMessage({
  content: [
    {
      type: "text",
      text: "What's in this image?",
    },
    {
      type: "image_url",
      image_url: {
        url: `data:image/jpeg;base64,${imageData.toString("base64")}`,
      },
    },
  ],
});

const res = await chat.invoke([message]);
console.log({ res });

/*
  {
    res: AIMessage {
      content: 'The image shows a hot dog, which consists of a grilled or steamed sausage served in the slit of a partially sliced bun. This particular hot dog appears to be plain, without any visible toppings or condiments.',
      additional_kwargs: { function_call: undefined }
    }
  }
*/

const hostedImageMessage = new HumanMessage({
  content: [
    {
      type: "text",
      text: "What does this image say?",
    },
    {
      type: "image_url",
      image_url:
        "https://www.freecodecamp.org/news/content/images/2023/05/Screenshot-2023-05-29-at-5.40.38-PM.png",
    },
  ],
});
const res2 = await chat.invoke([hostedImageMessage]);
console.log({ res2 });

/*
  {
    res2: AIMessage {
      content: 'The image contains the text "LangChain" with a graphical depiction of a parrot on the left and two interlocked rings on the left side of the text.',
      additional_kwargs: { function_call: undefined }
    }
  }
*/

const lowDetailImage = new HumanMessage({
  content: [
    {
      type: "text",
      text: "Summarize the contents of this image.",
    },
    {
      type: "image_url",
      image_url: {
        url: "https://blog.langchain.dev/content/images/size/w1248/format/webp/2023/10/Screenshot-2023-10-03-at-4.55.29-PM.png",
        detail: "low",
      },
    },
  ],
});
const res3 = await chat.invoke([lowDetailImage]);
console.log({ res3 });

/*
  {
    res3: AIMessage {
      content: 'The image shows a user interface for a service named "WebLangChain," which appears to be powered by "Twalv." It includes a text box with the prompt "Ask me anything about anything!" suggesting that users can enter questions on various topics. Below the text box, there are example questions that users might ask, such as "what is langchain?", "history of mesopotamia," "how to build a discord bot," "leonardo dicaprio girlfriend," "fun gift ideas for software engineers," "how does a prism separate light," and "what beer is best." The interface also includes a round blue button with a paper plane icon, presumably to submit the question. The overall theme of the image is dark with blue accents.',
      additional_kwargs: { function_call: undefined }
    }
  }
*/

API Reference:

Tool calling​

info

This feature is currently only available for gpt-3.5-turbo-1106 and gpt-4-1106-preview models.

More recent OpenAI chat models support calling multiple functions to get all required data to answer a question. Here's an example how a conversation turn with this functionality might look:

import { ChatOpenAI } from "@langchain/openai";
import { ToolMessage } from "@langchain/core/messages";

// Mocked out function, could be a database/API call in production
function getCurrentWeather(location: string, _unit?: string) {
  if (location.toLowerCase().includes("tokyo")) {
    return JSON.stringify({ location, temperature: "10", unit: "celsius" });
  } else if (location.toLowerCase().includes("san francisco")) {
    return JSON.stringify({
      location,
      temperature: "72",
      unit: "fahrenheit",
    });
  } else {
    return JSON.stringify({ location, temperature: "22", unit: "celsius" });
  }
}

// Bind function to the model as a tool
const chat = new ChatOpenAI({
  model: "gpt-3.5-turbo-1106",
  maxTokens: 128,
}).bind({
  tools: [
    {
      type: "function",
      function: {
        name: "get_current_weather",
        description: "Get the current weather in a given location",
        parameters: {
          type: "object",
          properties: {
            location: {
              type: "string",
              description: "The city and state, e.g. San Francisco, CA",
            },
            unit: { type: "string", enum: ["celsius", "fahrenheit"] },
          },
          required: ["location"],
        },
      },
    },
  ],
  tool_choice: "auto",
});

// Ask initial question that requires multiple tool calls
const res = await chat.invoke([
  ["human", "What's the weather like in San Francisco, Tokyo, and Paris?"],
]);
console.log(res.additional_kwargs.tool_calls);
/*
  [
    {
      id: 'call_IiOsjIZLWvnzSh8iI63GieUB',
      type: 'function',
      function: {
        name: 'get_current_weather',
        arguments: '{"location": "San Francisco", "unit": "celsius"}'
      }
    },
    {
      id: 'call_blQ3Oz28zSfvS6Bj6FPEUGA1',
      type: 'function',
      function: {
        name: 'get_current_weather',
        arguments: '{"location": "Tokyo", "unit": "celsius"}'
      }
    },
    {
      id: 'call_Kpa7FaGr3F1xziG8C6cDffsg',
      type: 'function',
      function: {
        name: 'get_current_weather',
        arguments: '{"location": "Paris", "unit": "celsius"}'
      }
    }
  ]
*/

// Format the results from calling the tool calls back to OpenAI as ToolMessages
const toolMessages = res.additional_kwargs.tool_calls?.map((toolCall) => {
  const toolCallResult = getCurrentWeather(
    JSON.parse(toolCall.function.arguments).location
  );
  return new ToolMessage({
    tool_call_id: toolCall.id,
    name: toolCall.function.name,
    content: toolCallResult,
  });
});

// Send the results back as the next step in the conversation
const finalResponse = await chat.invoke([
  ["human", "What's the weather like in San Francisco, Tokyo, and Paris?"],
  res,
  ...(toolMessages ?? []),
]);

console.log(finalResponse);
/*
  AIMessage {
    content: 'The current weather in:\n' +
      '- San Francisco is 72°F\n' +
      '- Tokyo is 10°C\n' +
      '- Paris is 22°C',
    additional_kwargs: { function_call: undefined, tool_calls: undefined }
  }
*/

API Reference:

.withStructuredOutput({ ... })​

info

The .withStructuredOutput method is in beta. It is actively being worked on, so the API may change.

You can also use the .withStructuredOutput({ ... }) method to coerce ChatOpenAI into returning a structured output.

The method allows for passing in either a Zod object, or a valid JSON schema (like what is returned from zodToJsonSchema).

Using the method is simple. Just define your LLM and call .withStructuredOutput({ ... }) on it, passing the desired schema.

Here is an example using a Zod schema and the functionCalling mode (default mode):

import { ChatPromptTemplate } from "@langchain/core/prompts";
import { ChatOpenAI } from "@langchain/openai";
import { z } from "zod";

const model = new ChatOpenAI({
  temperature: 0,
  model: "gpt-4-turbo-preview",
});

const calculatorSchema = z.object({
  operation: z.enum(["add", "subtract", "multiply", "divide"]),
  number1: z.number(),
  number2: z.number(),
});

const modelWithStructuredOutput = model.withStructuredOutput(calculatorSchema);

const prompt = ChatPromptTemplate.fromMessages([
  ["system", "You are VERY bad at math and must always use a calculator."],
  ["human", "Please help me!! What is 2 + 2?"],
]);
const chain = prompt.pipe(modelWithStructuredOutput);
const result = await chain.invoke({});
console.log(result);
/*
{ operation: 'add', number1: 2, number2: 2 }
 */

/**
 * You can also specify 'includeRaw' to return the parsed
 * and raw output in the result.
 */
const includeRawModel = model.withStructuredOutput(calculatorSchema, {
  name: "calculator",
  includeRaw: true,
});

const includeRawChain = prompt.pipe(includeRawModel);
const includeRawResult = await includeRawChain.invoke({});
console.log(JSON.stringify(includeRawResult, null, 2));
/*
{
  "raw": {
    "kwargs": {
      "content": "",
      "additional_kwargs": {
        "tool_calls": [
          {
            "id": "call_A8yzNBDMiRrCB8dFYqJLhYW7",
            "type": "function",
            "function": {
              "name": "calculator",
              "arguments": "{\"operation\":\"add\",\"number1\":2,\"number2\":2}"
            }
          }
        ]
      }
    }
  },
  "parsed": {
    "operation": "add",
    "number1": 2,
    "number2": 2
  }
}
 */

API Reference:

Additionally, you can pass in an OpenAI function definition or JSON schema directly:

info

If using jsonMode as the method you must include context in your prompt about the structured output you want. This must include the keyword: JSON.

import { ChatPromptTemplate } from "@langchain/core/prompts";
import { ChatOpenAI } from "@langchain/openai";

const model = new ChatOpenAI({
  temperature: 0,
  model: "gpt-4-turbo-preview",
});

const calculatorSchema = {
  type: "object",
  properties: {
    operation: {
      type: "string",
      enum: ["add", "subtract", "multiply", "divide"],
    },
    number1: { type: "number" },
    number2: { type: "number" },
  },
  required: ["operation", "number1", "number2"],
};

// Default mode is "functionCalling"
const modelWithStructuredOutput = model.withStructuredOutput(calculatorSchema);

const prompt = ChatPromptTemplate.fromMessages([
  [
    "system",
    `You are VERY bad at math and must always use a calculator.
Respond with a JSON object containing three keys:
'operation': the type of operation to execute, either 'add', 'subtract', 'multiply' or 'divide',
'number1': the first number to operate on,
'number2': the second number to operate on.
`,
  ],
  ["human", "Please help me!! What is 2 + 2?"],
]);
const chain = prompt.pipe(modelWithStructuredOutput);
const result = await chain.invoke({});
console.log(result);
/*
{ operation: 'add', number1: 2, number2: 2 }
 */

/**
 * You can also specify 'includeRaw' to return the parsed
 * and raw output in the result, as well as a "name" field
 * to give the LLM additional context as to what you are generating.
 */
const includeRawModel = model.withStructuredOutput(calculatorSchema, {
  name: "calculator",
  includeRaw: true,
  method: "jsonMode",
});

const includeRawChain = prompt.pipe(includeRawModel);
const includeRawResult = await includeRawChain.invoke({});
console.log(JSON.stringify(includeRawResult, null, 2));
/*
{
  "raw": {
    "kwargs": {
      "content": "{\n  \"operation\": \"add\",\n  \"number1\": 2,\n  \"number2\": 2\n}",
      "additional_kwargs": {}
    }
  },
  "parsed": {
    "operation": "add",
    "number1": 2,
    "number2": 2
  }
}
 */

API Reference:

Custom URLs​

You can customize the base URL the SDK sends requests to by passing a configuration parameter like this:

import { ChatOpenAI } from "@langchain/openai";

const model = new ChatOpenAI({
  temperature: 0.9,
  configuration: {
    baseURL: "https://your_custom_url.com",
  },
});

const message = await model.invoke("Hi there!");

console.log(message);

/*
  AIMessage {
    content: 'Hello! How can I assist you today?',
    additional_kwargs: { function_call: undefined }
  }
*/

API Reference:

You can also pass other ClientOptions parameters accepted by the official SDK.

If you are hosting on Azure OpenAI, see the dedicated page instead.

Calling fine-tuned models​

You can call fine-tuned OpenAI models by passing in your corresponding modelName parameter.

This generally takes the form of ft:{OPENAI_MODEL_NAME}:{ORG_NAME}::{MODEL_ID}. For example:

import { ChatOpenAI } from "@langchain/openai";

const model = new ChatOpenAI({
  temperature: 0.9,
  model: "ft:gpt-3.5-turbo-0613:{ORG_NAME}::{MODEL_ID}",
});

const message = await model.invoke("Hi there!");

console.log(message);

/*
  AIMessage {
    content: 'Hello! How can I assist you today?',
    additional_kwargs: { function_call: undefined }
  }
*/

API Reference:

Generation metadata​

If you need additional information like logprobs or token usage, these will be returned directly in the .invoke response.

tip

Requires @langchain/core version >=0.1.48.

import { ChatOpenAI } from "@langchain/openai";

// See https://cookbook.openai.com/examples/using_logprobs for details
const model = new ChatOpenAI({
  logprobs: true,
  // topLogprobs: 5,
});

const responseMessage = await model.invoke("Hi there!");

console.log(JSON.stringify(responseMessage, null, 2));

/*
  {
    "lc": 1,
    "type": "constructor",
    "id": [
      "langchain_core",
      "messages",
      "AIMessage"
    ],
    "kwargs": {
      "content": "Hello! How can I assist you today?",
      "additional_kwargs": {},
      "response_metadata": {
        "tokenUsage": {
          "completionTokens": 9,
          "promptTokens": 10,
          "totalTokens": 19
        },
        "finish_reason": "stop",
        "logprobs": {
          "content": [
            {
              "token": "Hello",
              "logprob": -0.0006793116,
              "bytes": [
                72,
                101,
                108,
                108,
                111
              ],
              "top_logprobs": []
            },
            {
              "token": "!",
              "logprob": -0.00011725161,
              "bytes": [
                33
              ],
              "top_logprobs": []
            },
            {
              "token": " How",
              "logprob": -0.000038457987,
              "bytes": [
                32,
                72,
                111,
                119
              ],
              "top_logprobs": []
            },
            {
              "token": " can",
              "logprob": -0.00094290765,
              "bytes": [
                32,
                99,
                97,
                110
              ],
              "top_logprobs": []
            },
            {
              "token": " I",
              "logprob": -0.0000013856493,
              "bytes": [
                32,
                73
              ],
              "top_logprobs": []
            },
            {
              "token": " assist",
              "logprob": -0.14702488,
              "bytes": [
                32,
                97,
                115,
                115,
                105,
                115,
                116
              ],
              "top_logprobs": []
            },
            {
              "token": " you",
              "logprob": -0.000001147242,
              "bytes": [
                32,
                121,
                111,
                117
              ],
              "top_logprobs": []
            },
            {
              "token": " today",
              "logprob": -0.000067901296,
              "bytes": [
                32,
                116,
                111,
                100,
                97,
                121
              ],
              "top_logprobs": []
            },
            {
              "token": "?",
              "logprob": -0.000014974867,
              "bytes": [
                63
              ],
              "top_logprobs": []
            }
          ]
        }
      }
    }
  }
*/

API Reference:

With callbacks​

You can also use the callbacks system:

import { ChatOpenAI } from "@langchain/openai";

// See https://cookbook.openai.com/examples/using_logprobs for details
const model = new ChatOpenAI({
  logprobs: true,
  // topLogprobs: 5,
});

const result = await model.invoke("Hi there!", {
  callbacks: [
    {
      handleLLMEnd(output) {
        console.log("GENERATION OUTPUT:", JSON.stringify(output, null, 2));
      },
    },
  ],
});

console.log("FINAL OUTPUT", result);

/*
  GENERATION OUTPUT: {
    "generations": [
      [
        {
          "text": "Hello! How can I assist you today?",
          "message": {
            "lc": 1,
            "type": "constructor",
            "id": [
              "langchain_core",
              "messages",
              "AIMessage"
            ],
            "kwargs": {
              "content": "Hello! How can I assist you today?",
              "additional_kwargs": {}
            }
          },
          "generationInfo": {
            "finish_reason": "stop",
            "logprobs": {
              "content": [
                {
                  "token": "Hello",
                  "logprob": -0.0010195904,
                  "bytes": [
                    72,
                    101,
                    108,
                    108,
                    111
                  ],
                  "top_logprobs": []
                },
                {
                  "token": "!",
                  "logprob": -0.0004447316,
                  "bytes": [
                    33
                  ],
                  "top_logprobs": []
                },
                {
                  "token": " How",
                  "logprob": -0.00006682846,
                  "bytes": [
                    32,
                    72,
                    111,
                    119
                  ],
                  "top_logprobs": []
                },
                ...
              ]
            }
          }
        }
      ]
    ],
    "llmOutput": {
      "tokenUsage": {
        "completionTokens": 9,
        "promptTokens": 10,
        "totalTokens": 19
      }
    }
  }
  FINAL OUTPUT AIMessage {
    content: 'Hello! How can I assist you today?',
    additional_kwargs: { function_call: undefined, tool_calls: undefined }
  }
*/

API Reference:

With .generate()​

import { ChatOpenAI } from "@langchain/openai";
import { HumanMessage } from "@langchain/core/messages";

// See https://cookbook.openai.com/examples/using_logprobs for details
const model = new ChatOpenAI({
  logprobs: true,
  // topLogprobs: 5,
});

const generations = await model.invoke([new HumanMessage("Hi there!")]);

console.log(JSON.stringify(generations, null, 2));

/*
  {
    "generations": [
      [
        {
          "text": "Hello! How can I assist you today?",
          "message": {
            "lc": 1,
            "type": "constructor",
            "id": [
              "langchain_core",
              "messages",
              "AIMessage"
            ],
            "kwargs": {
              "content": "Hello! How can I assist you today?",
              "additional_kwargs": {}
            }
          },
          "generationInfo": {
            "finish_reason": "stop",
            "logprobs": {
              "content": [
                {
                  "token": "Hello",
                  "logprob": -0.0011337858,
                  "bytes": [
                    72,
                    101,
                    108,
                    108,
                    111
                  ],
                  "top_logprobs": []
                },
                {
                  "token": "!",
                  "logprob": -0.00044127836,
                  "bytes": [
                    33
                  ],
                  "top_logprobs": []
                },
                {
                  "token": " How",
                  "logprob": -0.000065994034,
                  "bytes": [
                    32,
                    72,
                    111,
                    119
                  ],
                  "top_logprobs": []
                },
                ...
              ]
            }
          }
        }
      ]
    ],
    "llmOutput": {
      "tokenUsage": {
        "completionTokens": 9,
        "promptTokens": 10,
        "totalTokens": 19
      }
    }
  }
*/

API Reference:

Streaming tokens​

OpenAI supports streaming token counts via an opt-in call option. This can be set by passing { stream_options: { include_usage: true } }. Setting this call option will cause the model to return an additional chunk at the end of the stream, containing the token usage.

import type { AIMessageChunk } from "@langchain/core/messages";
import { ChatOpenAI } from "@langchain/openai";
import { concat } from "@langchain/core/utils/stream";

// Instantiate the model
const model = new ChatOpenAI();

const response = await model.stream("Hello, how are you?", {
  // Pass the stream options
  stream_options: {
    include_usage: true,
  },
});

// Iterate over the response, only saving the last chunk
let finalResult: AIMessageChunk | undefined;
for await (const chunk of response) {
  if (finalResult) {
    finalResult = concat(finalResult, chunk);
  } else {
    finalResult = chunk;
  }
}

console.log(finalResult?.usage_metadata);

/*
  { input_tokens: 13, output_tokens: 30, total_tokens: 43 }
*/

API Reference:

tip

See the LangSmith trace here

Disabling parallel tool calls​

If you have multiple tools bound to the model, but you'd only like for a single tool to be called at a time, you can pass the parallel_tool_calls call option to enable/disable this behavior. By default, parallel_tool_calls is set to true.

import { ChatOpenAI } from "@langchain/openai";
import { z } from "zod";
import { zodToJsonSchema } from "zod-to-json-schema";

const model = new ChatOpenAI({
  temperature: 0,
  model: "gpt-4o",
});

// Define your tools
const calculatorSchema = z
  .object({
    operation: z.enum(["add", "subtract", "multiply", "divide"]),
    number1: z.number(),
    number2: z.number(),
  })
  .describe("A tool to perform basic arithmetic operations");
const weatherSchema = z
  .object({
    city: z.enum(["add", "subtract", "multiply", "divide"]),
  })
  .describe("A tool to get the weather in a city");

// Bind tools to the model
const modelWithTools = model.bindTools([
  {
    type: "function",
    function: {
      name: "calculator",
      description: calculatorSchema.description,
      parameters: zodToJsonSchema(calculatorSchema),
    },
  },
  {
    type: "function",
    function: {
      name: "weather",
      description: weatherSchema.description,
      parameters: zodToJsonSchema(weatherSchema),
    },
  },
]);

// Invoke the model with `parallel_tool_calls` set to `true`
const response = await modelWithTools.invoke(
  ["What is the weather in san francisco and what is 23716 times 27342?"],
  {
    parallel_tool_calls: true,
  }
);
console.log(response.tool_calls);
// We can see it called two tools
/*
[
  {
    name: 'weather',
    args: { city: 'san francisco' },
    id: 'call_c1KymEIix7mdlFtgLSnTXmDc'
  },
  {
    name: 'calculator',
    args: { operation: 'multiply', number1: 23716, number2: 27342 },
    id: 'call_ANLYclAmXQ4TwUCLXakbPr3Z'
  }
]
*/

// Invoke the model with `parallel_tool_calls` set to `false`
const response2 = await modelWithTools.invoke(
  ["What is the weather in san francisco and what is 23716 times 27342?"],
  {
    parallel_tool_calls: false,
  }
);
console.log(response2.tool_calls);
// We can see it called one tool
/*
[
  {
    name: 'weather',
    args: { city: 'san francisco' },
    id: 'call_Rk34XffawJjgZ2BCK9E4CwlT'
  }
]
*/

API Reference:

tip

See the LangSmith trace for the first invocation here and the second invocation here

Was this page helpful?


You can leave detailed feedback on GitHub.