Skip to main content

Quickstart

In this guide, we will go over the basic ways to create Chains and Agents that call Tools. Tools can be just about anything — APIs, functions, databases, etc. Tools allow us to extend the capabilities of a model beyond just outputting text/messages. The key to using models with tools is correctly prompting a model and parsing its response so that it chooses the right tools and provides the right inputs for them.

Setup​

We’ll use OpenAI for this guide, and will need to install its partner package:

tip

See this section for general instructions on installing integration packages.

npm install @langchain/openai

You'll need to sign up for an OpenAI key and set it as an environment variable named OPENAI_API_KEY.

We'll also use the popular validation library Zod to define our tool schemas. It's already a dependency of langchain, but you can install it explicitly like this too:

npm install zod

Create a tool​

First, we need to create a tool to call. For this example, we will create a custom tool from a function. For more information on creating custom tools, please see this guide.

import { z } from "zod";
import { DynamicStructuredTool } from "@langchain/core/tools";

const multiplyTool = new DynamicStructuredTool({
  name: "multiply",
  description: "Multiply two integers together.",
  schema: z.object({
    firstInt: z.number(),
    secondInt: z.number(),
  }),
  func: async ({ firstInt, secondInt }) => {
    return (firstInt * secondInt).toString();
  },
});

await multiplyTool.invoke({ firstInt: 4, secondInt: 5 });
20

Chains​

If we know that we only need to use a tool a fixed number of times, we can create a chain for doing so. Let’s create a simple chain that just multiplies user-specified numbers.

Function calling​

One of the most reliable ways to use tools with LLMs is with function calling APIs (also sometimes called tool calling or parallel function calling). This only works with models that explicitly support function calling, like OpenAI models.

First we’ll define our model:

import { ChatOpenAI } from "@langchain/openai";

const model = new ChatOpenAI({
  model: "gpt-3.5-turbo-1106",
});

Next we’ll convert our LangChain Tool to the format an OpenAI function accepts, JSONSchema, and bind this as the tools argument to be passed to all ChatOpenAI calls. Since we only have a single Tool and in this initial chain we want to make sure it’s always used, we’ll also specify tool_choice. See the OpenAI chat API reference for more on these parameters.

import { convertToOpenAITool } from "@langchain/core/utils/function_calling";

const formattedTools = [convertToOpenAITool(multiplyTool)];
[
  {
    "type": "function",
    "function": {
      "name": "multiply",
      "description": "Multiply two integers together.",
      "parameters": {
        "type": "object",
        "properties": {
          "firstInt": {
            "type": "number"
          },
          "secondInt": {
            "type": "number"
          }
        },
        "required": [
          "firstInt",
          "secondInt"
        ],
        "additionalProperties": false,
        "$schema": "http://json-schema.org/draft-07/schema#"
      }
    }
  }
]
const modelWithTools = model.bind({
  tools: formattedTools,
  // We specify tool_choice to enforce that the 'multiply' function is called by the model.
  tool_choice: {
    type: "function",
    function: { name: "multiply" },
  },
});

Now we’ll compose out tool-calling model with a JsonOutputToolsParser, a built-in LangChain output parser that converts an OpenAI function-calling response to a list of {"type": "TOOL_NAME", "args": {...}} objects with the tools to invoke and arguments to invoke them with.

import { JsonOutputToolsParser } from "langchain/output_parsers";

const chain = modelWithTools.pipe(new JsonOutputToolsParser());

await chain.invoke("What's 4 times 23?");
[ { type: 'multiply', args: { firstInt: 4, secondInt: 23 } } ]

Since we know we’re always invoking the multiply tool, we can simplify our output a bit to return only the args for the multiply tool using the JsonOutputKeyToolsParser. To further simplify we’ll specify returnSingle: true, so that instead of a list of tool invocations our output parser returns only the first tool invocation.

import { JsonOutputKeyToolsParser } from "langchain/output_parsers";

const chain2 = modelWithTools.pipe(
  new JsonOutputKeyToolsParser({ keyName: "multiply", returnSingle: true })
);

await chain2.invoke("What's 4 times 23?");
{ firstInt: 4, secondInt: 23 }

Invoking the tool​

Great! We’re able to generate tool invocations. But what if we want to actually call the tool with the LLM-generated args? To do that we just need to pass them to the tool:

info

You can see a LangSmith trace of this example here

import { RunnableSequence } from "@langchain/core/runnables";

const chain3 = RunnableSequence.from([
  modelWithTools,
  new JsonOutputKeyToolsParser({ keyName: "multiply", returnSingle: true }),
  multiplyTool,
]);

await chain3.invoke("What's 4 times 23?");
92

And there we have our answer!

Agents​

Chains are great when we know the specific sequence of tool usage needed for any user input. But for certain use cases, how many times we use tools depends on the input. In these cases, we want to let the model itself decide how many times to use tools and in what order. That's where Agents come in!

LangChain comes with a number of built-in agents that are optimized for different use cases. Read about all the available agent types here.

For this example, let’s try out the OpenAI tools agent, which makes use of the new OpenAI tool-calling API (this is only available in the latest OpenAI models, and differs from function-calling in that the model can return multiple function invocations at once).

Keep in mind that some agents only support single-argument tools - for these agents, you will need to use a DynamicTool instead and parse the input string yourself.

import { pull } from "langchain/hub";
import { AgentExecutor, createOpenAIToolsAgent } from "langchain/agents";
import { ChatOpenAI } from "@langchain/openai";
import type { ChatPromptTemplate } from "@langchain/core/prompts";

// Get the prompt to use - you can modify this!
// You can also see the full prompt at:
// https://smith.langchain.com/hub/hwchase17/openai-tools-agent
const prompt = await pull<ChatPromptTemplate>("hwchase17/openai-tools-agent");

Agents can also choose between multiple tools to solve a problem. To learn how to build Chains that use multiple tools, check out the Chains with multiple tools guide.

import { z } from "zod";
import { DynamicStructuredTool } from "@langchain/core/tools";

const addTool = new DynamicStructuredTool({
  name: "add",
  description: "Add two integers together.",
  schema: z.object({
    firstInt: z.number(),
    secondInt: z.number(),
  }),
  func: async ({ firstInt, secondInt }) => {
    return (firstInt + secondInt).toString();
  },
});

const multiplyTool = new DynamicStructuredTool({
  name: "multiply",
  description: "Multiply two integers together.",
  schema: z.object({
    firstInt: z.number(),
    secondInt: z.number(),
  }),
  func: async ({ firstInt, secondInt }) => {
    return (firstInt * secondInt).toString();
  },
});

const exponentiateTool = new DynamicStructuredTool({
  name: "exponentiate",
  description: "Exponentiate the base to the exponent power.",
  schema: z.object({
    base: z.number(),
    exponent: z.number(),
  }),
  func: async ({ base, exponent }) => {
    return (base ** exponent).toString();
  },
});

const model = new ChatOpenAI({
  model: "gpt-3.5-turbo-1106",
  temperature: 0,
});

const tools = [addTool, multiplyTool, exponentiateTool];

const agent = await createOpenAIToolsAgent({
  llm: model,
  tools,
  prompt,
});

const agentExecutor = new AgentExecutor({
  agent,
  tools,
  verbose: true,
});

With an agent, we can ask questions that require arbitrarily-many uses of our tools:

info

You can see a LangSmith trace of this example here

await agentExecutor.invoke({
  input:
    "Take 3 to the fifth power and multiply that by the sum of twelve and three, then square the whole result",
});


> Entering new AgentExecutor chain...

Invoking: `exponentiate` with `{'base': 3, 'exponent': 5}`


243
Invoking: `add` with `{'first_int': 12, 'second_int': 3}`


15
Invoking: `multiply` with `{'first_int': 243, 'second_int': 15}`


3645
Invoking: `exponentiate` with `{'base': 3645, 'exponent': 2}`


13286025

The result of taking 3 to the fifth power and multiplying that by the sum of twelve and three, then squaring the whole result is 13,286,025.

> Finished chain.
{
  input: 'Take 3 to the fifth power and multiply that by the sum of twelve and three, then square the whole result',
  output: 'The result of taking 3 to the fifth power and multiplying that by the sum of twelve and three, then squaring the whole result is 13,286,025.'
}

Next steps​

Here we’ve gone over the basic ways to use Tools with Chains and Agents. We recommend the following sections to explore next:

Help us out by providing feedback on this documentation page: