Skip to main content

How to use output parsers to parse an LLM response into structured format

Prerequisites

This guide assumes familiarity with the following concepts:

Language models output text. But there are times where you want to get more structured information than just text back. While some model providers support built-in ways to return structured output, not all do. For these providers, you must use prompting to encourage the model to return structured data in the desired format.

LangChain has output parsers which can help parse model outputs into usable objects. We’ll go over a few examples below.

Get started​

The primary type of output parser for working with structured data in model responses is the StructuredOutputParser. In the below example, we define a schema for the type of output we expect from the model using zod.

First, let’s see the default formatting instructions we’ll plug into the prompt:

Pick your chat model:

Install dependencies

tip

See this section for general instructions on installing integration packages.

yarn add @langchain/openai

Add environment variables

OPENAI_API_KEY=your-api-key

Instantiate the model

import { ChatOpenAI } from "@langchain/openai";

const model = new ChatOpenAI({
  model: "gpt-3.5-turbo",
  temperature: 0
});
import { z } from "zod";
import { RunnableSequence } from "@langchain/core/runnables";
import { StructuredOutputParser } from "@langchain/core/output_parsers";
import { ChatPromptTemplate } from "@langchain/core/prompts";

const zodSchema = z.object({
  answer: z.string().describe("answer to the user's question"),
  source: z
    .string()
    .describe(
      "source used to answer the user's question, should be a website."
    ),
});

const parser = StructuredOutputParser.fromZodSchema(zodSchema);

const chain = RunnableSequence.from([
  ChatPromptTemplate.fromTemplate(
    "Answer the users question as best as possible.\n{format_instructions}\n{question}"
  ),
  model,
  parser,
]);

console.log(parser.getFormatInstructions());
You must format your output as a JSON value that adheres to a given "JSON Schema" instance.

"JSON Schema" is a declarative language that allows you to annotate and validate JSON documents.

For example, the example "JSON Schema" instance {{"properties": {{"foo": {{"description": "a list of test words", "type": "array", "items": {{"type": "string"}}}}}}, "required": ["foo"]}}}}
would match an object with one required property, "foo". The "type" property specifies "foo" must be an "array", and the "description" property semantically describes it as "a list of test words". The items within "foo" must be strings.
Thus, the object {{"foo": ["bar", "baz"]}} is a well-formatted instance of this example "JSON Schema". The object {{"properties": {{"foo": ["bar", "baz"]}}}} is not well-formatted.

Your output will be parsed and type-checked according to the provided schema instance, so make sure all fields in your output match the schema exactly and there are no trailing commas!

Here is the JSON Schema instance your output must adhere to. Include the enclosing markdown codeblock:
```json
{"type":"object","properties":{"answer":{"type":"string","description":"answer to the user's question"},"source":{"type":"string","description":"source used to answer the user's question, should be a website."}},"required":["answer","source"],"additionalProperties":false,"$schema":"http://json-schema.org/draft-07/schema#"}
```

Next, let’s invoke the chain:

const response = await chain.invoke({
  question: "What is the capital of France?",
  format_instructions: parser.getFormatInstructions(),
});

console.log(response);
{
  answer: "The capital of France is Paris.",
  source: "https://en.wikipedia.org/wiki/Paris"
}

Output parsers implement the Runnable interface, the basic building block of the LangChain Expression Language (LCEL). This means they support invoke, stream, batch, streamLog calls.

Validation​

One feature of the StructuredOutputParser is that it supports stricter Zod validations. For example, if you pass a simulated model output that does not conform to the schema, we get a detailed type error:

import { AIMessage } from "@langchain/core/messages";

await parser.invoke(new AIMessage(`{"badfield": "foo"}`));
Error: Failed to parse. Text: "{"badfield": "foo"}". Error: [
  {
    "code": "invalid_type",
    "expected": "string",
    "received": "undefined",
    "path": [
      "answer"
    ],
    "message": "Required"
  },
  {
    "code": "invalid_type",
    "expected": "string",
    "received": "undefined",
    "path": [
      "source"
    ],
    "message": "Required"
  }
]

Compared to:

await parser.invoke(
  new AIMessage(`{"answer": "Paris", "source": "I made it up"}`)
);
{ answer: "Paris", source: "I made it up" }

More advanced Zod validations are supported as well. To learn more, check out the Zod documentation.

Streaming​

While all parsers are runnables and support the streaming interface, only certain parsers can stream through partially parsed objects, since this is highly dependent on the output type. The StructuredOutputParser does not support partial streaming because it validates the output at each step. If you try to stream using a chain with this output parser, the chain will simply yield the fully parsed output:

const stream = await chain.stream({
  question: "What is the capital of France?",
  format_instructions: parser.getFormatInstructions(),
});

for await (const s of stream) {
  console.log(s);
}
{
  answer: "The capital of France is Paris.",
  source: "https://en.wikipedia.org/wiki/Paris"
}

The simpler JsonOutputParser, however, supports streaming through partial outputs:

import { JsonOutputParser } from "@langchain/core/output_parsers";

const template = `Return a JSON object with a single key named "answer" that answers the following question: {question}.
Do not wrap the JSON output in markdown blocks.`;

const jsonPrompt = ChatPromptTemplate.fromTemplate(template);
const jsonParser = new JsonOutputParser();
const jsonChain = jsonPrompt.pipe(model).pipe(jsonParser);

const stream = await jsonChain.stream({
  question: "Who invented the microscope?",
});

for await (const s of stream) {
  console.log(s);
}
{}
{ answer: "" }
{ answer: "The" }
{ answer: "The invention" }
{ answer: "The invention of" }
{ answer: "The invention of the" }
{ answer: "The invention of the microscope" }
{ answer: "The invention of the microscope is" }
{ answer: "The invention of the microscope is attributed" }
{ answer: "The invention of the microscope is attributed to" }
{ answer: "The invention of the microscope is attributed to Hans" }
{ answer: "The invention of the microscope is attributed to Hans L" }
{
  answer: "The invention of the microscope is attributed to Hans Lippers"
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey"
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey,"
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zach"
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias"
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Jans"
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen"
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen,"
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and"
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Anton"
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie"
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 4 more characters
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 8 more characters
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 12 more characters
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 13 more characters
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 18 more characters
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 20 more characters
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 26 more characters
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 29 more characters
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 33 more characters
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 38 more characters
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 43 more characters
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 48 more characters
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 51 more characters
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 52 more characters
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 57 more characters
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 63 more characters
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 73 more characters
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 80 more characters
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 81 more characters
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 85 more characters
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 94 more characters
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 99 more characters
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 108 more characters
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 112 more characters
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 118 more characters
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 127 more characters
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 138 more characters
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 145 more characters
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 149 more characters
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 150 more characters
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 151 more characters
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 157 more characters
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 159 more characters
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 163 more characters
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 167 more characters
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 171 more characters
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 175 more characters
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 176 more characters
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 181 more characters
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 186 more characters
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 190 more characters
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 202 more characters
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 203 more characters
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 209 more characters
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 214 more characters
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 226 more characters
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 239 more characters
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 242 more characters
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 246 more characters
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 253 more characters
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 257 more characters
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 262 more characters
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 265 more characters
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 268 more characters
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 273 more characters
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 288 more characters
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 300 more characters
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 303 more characters
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 311 more characters
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 316 more characters
}
{
  answer: "The invention of the microscope is attributed to Hans Lippershey, Zacharias Janssen, and Antonie van"... 317 more characters
}

Next steps​

You’ve learned about using output parsers to parse structured outputs from prompted model outputs.

Next, check out the guide on tool calling, a more built-in way of obtaining structured output that some model providers support, or read more about output parsers for other types of structured data like XML.

Was this page helpful?


You can leave detailed feedback on GitHub.