Skip to main content

Tool usage

This section will cover how to create conversational agents: chatbots that can interact with other systems and APIs using tools.

Before reading this guide, we recommend you read both the chatbot quickstart in this section and be familiar with the documentation on agents.

Setup​

For this guide, we’ll be using an OpenAI tools agent with a single tool for searching the web. The default will be powered by Tavily, but you can switch it out for any similar tool. The rest of this section will assume you’re using Tavily.

You’ll need to sign up for an account on the Tavily website, and install the following packages:

npm install @langchain/openai @langchain/community

You will also need your OpenAI key set as OPENAI_API_KEY and your Tavily API key set as TAVILY_API_KEY.

Creating an agent​

Our end goal is to create an agent that can respond conversationally to user questions while looking up information as needed.

First, let’s initialize Tavily and an OpenAI chat model capable of tool calling:

import { TavilySearchResults } from "@langchain/community/tools/tavily_search";
import { ChatOpenAI } from "@langchain/openai";

const tools = [
  new TavilySearchResults({
    maxResults: 1,
  }),
];

const chat = new ChatOpenAI({
  model: "gpt-3.5-turbo-1106",
  temperature: 0,
});

To make our agent conversational, we must also choose a prompt with a placeholder for our chat history. Here’s an example:

import {
  ChatPromptTemplate,
  MessagesPlaceholder,
} from "@langchain/core/prompts";

// Adapted from https://smith.langchain.com/hub/hwchase17/openai-tools-agent
const prompt = ChatPromptTemplate.fromMessages([
  [
    "system",
    "You are a helpful assistant. You may not need to use tools for every query - the user may just want to chat!",
  ],
  new MessagesPlaceholder("messages"),
  new MessagesPlaceholder("agent_scratchpad"),
]);

Great! Now let’s assemble our agent:

import { AgentExecutor, createOpenAIToolsAgent } from "langchain/agents";

const agent = await createOpenAIToolsAgent({
  llm: chat,
  tools,
  prompt,
});

const agentExecutor = new AgentExecutor({ agent, tools });

Running the agent​

Now that we’ve set up our agent, let’s try interacting with it! It can handle both trivial queries that require no lookup:

import { HumanMessage } from "@langchain/core/messages";

await agentExecutor.invoke({
  messages: [new HumanMessage("I'm Nemo!")],
});


> Entering new AgentExecutor chain...
Hi Nemo! It's great to meet you. How can I assist you today?

> Finished chain.
{
  messages: [
    HumanMessage {
      content: "I'm Nemo!"
    }
  ],
  output: "Hi Nemo! It's great to meet you. How can I assist you today?"
}

Or, it can use of the passed search tool to get up to date information if needed:

await agentExecutor.invoke({
  messages: [
    new HumanMessage(
      "What is the current conservation status of the Great Barrier Reef?"
    ),
  ],
});


> Entering new AgentExecutor chain...

Invoking: `tavily_search_results_json` with `{'query': 'current conservation status of the Great Barrier Reef'}`


[{"title":"The Great Barrier Reef has avoided an 'in danger' listing, but still ...","url":"https://www.abc.net.au/news/2023-09-14/great-barrier-reef-off-in-danger-list-analysis/102854968","content":"Posted Wed 13 Sep 2023 at 10:32pm Coral reefs, including the Great Barrier Reef, are facing a bleak future. (Supplied: The Ocean Agency / XL Catlin Seaview Survey) abc.net.au/news/great-barrier-reef-off-in-danger-list-analysis/102854968 It's official.","score":0.96777,"raw_content":null}]

> Finished chain.
{
  messages: [
    HumanMessage {
      content: 'What is the current conservation status of the Great Barrier Reef?'
    }
  ],
  output: 'The Great Barrier Reef has avoided an "in danger" listing, but it is still facing a bleak future in terms of conservation. You can find more information about this on the ABC News website: [Great Barrier Reef Conservation Status](https://www.abc.net.au/news/2023-09-14/great-barrier-reef-off-in-danger-list-analysis/102854968)'
}

Conversational responses​

Because our prompt contains a placeholder for chat history messages, our agent can also take previous interactions into account and respond conversationally like a standard chatbot:

import { AIMessage } from "@langchain/core/messages";

await agentExecutor.invoke({
  messages: [
    new HumanMessage("I'm Nemo!"),
    new AIMessage("Hello Nemo! How can I assist you today?"),
    new HumanMessage("What is my name?"),
  ],
});


> Entering new AgentExecutor chain...
Your name is Nemo!

> Finished chain.
{
  messages: [
    HumanMessage {
      content: "I'm Nemo!"
    },
    AIMessage {
      content: 'Hello Nemo! How can I assist you today?'
    },
    HumanMessage {
      content: 'What is my name?'
    }
  ],
  output: 'Your name is Nemo!'
}

If preferred, you can also wrap the agent executor in a RunnableWithMessageHistory class to internally manage history messages. First, we need to slightly modify the prompt to take a separate input variable so that the wrapper can parse which input value to store as history:

// Adapted from https://smith.langchain.com/hub/hwchase17/openai-tools-agent
const prompt2 = ChatPromptTemplate.fromMessages([
  [
    "system",
    "You are a helpful assistant. You may not need to use tools for every query - the user may just want to chat!",
  ],
  new MessagesPlaceholder("chat_history"),
  ["human", "{input}"],
  new MessagesPlaceholder("agent_scratchpad"),
]);

const agent2 = await createOpenAIToolsAgent({
  llm: chat,
  tools,
  prompt: prompt2,
});

const agentExecutor2 = new AgentExecutor({ agent: agent2, tools });

Then, because our agent executor has multiple outputs, we also have to set the outputMessagesKey property when initializing the wrapper:

import { ChatMessageHistory } from "langchain/stores/message/in_memory";
import { RunnableWithMessageHistory } from "@langchain/core/runnables";

const demoEphemeralChatMessageHistory = new ChatMessageHistory();

const conversationalAgentExecutor = new RunnableWithMessageHistory({
  runnable: agentExecutor2,
  getMessageHistory: (_sessionId) => demoEphemeralChatMessageHistory,
  inputMessagesKey: "input",
  outputMessagesKey: "output",
  historyMessagesKey: "chat_history",
});
await conversationalAgentExecutor.invoke(
  { input: "I'm Nemo!" },
  { configurable: { sessionId: "unused" } }
);


> Entering new AgentExecutor chain...
Hello Nemo! It's great to meet you. How can I assist you today?

> Finished chain.
{
  input: "I'm Nemo!",
  chat_history: [
    HumanMessage {
      content: "I'm Nemo!"
    },
    AIMessage {
      content: "Hello Nemo! It's great to meet you. How can I assist you today?"
    }
  ],
  output: "Hello Nemo! It's great to meet you. How can I assist you today?"
}
await conversationalAgentExecutor.invoke(
  { input: "What is my name?" },
  { configurable: { sessionId: "unused" } }
);


> Entering new AgentExecutor chain...
Your name is Nemo!

> Finished chain.
{
  input: 'What is my name?',
  chat_history: [
    HumanMessage {
      content: "I'm Nemo!"
    },
    AIMessage {
      content: "Hello Nemo! It's great to meet you. How can I assist you today?"
    },
    HumanMessage {
      content: 'What is my name?'
    },
    AIMessage {
      content: 'Your name is Nemo!'
    }
  ],
  output: 'Your name is Nemo!'
}

Help us out by providing feedback on this documentation page: