Skip to main content

Azure OpenAI

caution

You are currently on a page documenting the use of Azure OpenAI text completion models. The latest and most popular Azure OpenAI models are chat completion models.

Unless you are specifically using gpt-3.5-turbo-instruct, you are probably looking for this page instead.

Azure OpenAI is a cloud service to help you quickly develop generative AI experiences with a diverse set of prebuilt and curated models from OpenAI, Meta and beyond.

LangChain.js supports integration with Azure OpenAI using the new Azure integration in the OpenAI SDK.

You can learn more about Azure OpenAI and its difference with the OpenAI API on this page. If you don't have an Azure account, you can create a free account to get started.

info

Previously, LangChain.js supported integration with Azure OpenAI using the dedicated Azure OpenAI SDK. This SDK is now deprecated in favor of the new Azure integration in the OpenAI SDK, which allows to access the latest OpenAI models and features the same day they are released, and allows seemless transition between the OpenAI API and Azure OpenAI.

If you are using Azure OpenAI with the deprecated SDK, see the migration guide to update to the new API.

Setup​

You'll first need to install the @langchain/openai package:

tip

See this section for general instructions on installing integration packages.

npm install -S @langchain/openai

You'll also need to have an Azure OpenAI instance deployed. You can deploy a version on Azure Portal following this guide.

Once you have your instance running, make sure you have the name of your instance and key. You can find the key in the Azure Portal, under the "Keys and Endpoint" section of your instance.

If you're using Node.js, you can define the following environment variables to use the service:

AZURE_OPENAI_API_INSTANCE_NAME=<YOUR_INSTANCE_NAME>
AZURE_OPENAI_API_DEPLOYMENT_NAME=<YOUR_DEPLOYMENT_NAME>
AZURE_OPENAI_API_KEY=<YOUR_KEY>
AZURE_OPENAI_API_VERSION="2024-02-01"

Alternatively, you can pass the values directly to the AzureOpenAI constructor:

tip

We're unifying model params across all packages. We now suggest using model instead of modelName, and apiKey for API keys.

import { AzureOpenAI } from "@langchain/openai";

const model = new AzureOpenAI({
  azureOpenAIApiKey: "<your_key>", // In Node.js defaults to process.env.AZURE_OPENAI_API_KEY
  azureOpenAIApiInstanceName: "<your_instance_name>", // In Node.js defaults to process.env.AZURE_OPENAI_API_INSTANCE_NAME
  azureOpenAIApiDeploymentName: "<your_deployment_name>", // In Node.js defaults to process.env.AZURE_OPENAI_API_DEPLOYMENT_NAME
  azureOpenAIApiVersion: "<api_version>", // In Node.js defaults to process.env.AZURE_OPENAI_API_VERSION
});

API Reference:

info

You can find the list of supported API versions in the Azure OpenAI documentation.

Using Azure Managed Identity​

If you're using Azure Managed Identity, you can configure the credentials like this:

import {
  DefaultAzureCredential,
  getBearerTokenProvider,
} from "@azure/identity";
import { AzureOpenAI } from "@langchain/openai";

const credentials = new DefaultAzureCredential();
const azureADTokenProvider = getBearerTokenProvider(
  credentials,
  "https://cognitiveservices.azure.com/.default"
);

const model = new AzureOpenAI({
  azureADTokenProvider,
  azureOpenAIApiInstanceName: "<your_instance_name>",
  azureOpenAIApiDeploymentName: "<your_deployment_name>",
  azureOpenAIApiVersion: "<api_version>",
});

API Reference:

Using a different domain​

If your instance is hosted under a domain other than the default openai.azure.com, you'll need to use the alternate AZURE_OPENAI_BASE_PATH environment variable. For example, here's how you would connect to the domain https://westeurope.api.microsoft.com/openai/deployments/{DEPLOYMENT_NAME}:

import { AzureOpenAI } from "@langchain/openai";

const model = new AzureOpenAI({
  azureOpenAIApiKey: "<your_key>", // In Node.js defaults to process.env.AZURE_OPENAI_API_KEY
  azureOpenAIApiDeploymentName: "<your_deployment_name>", // In Node.js defaults to process.env.AZURE_OPENAI_API_DEPLOYMENT_NAME
  azureOpenAIApiVersion: "<api_version>", // In Node.js defaults to process.env.AZURE_OPENAI_API_VERSION
  azureOpenAIBasePath:
    "https://westeurope.api.microsoft.com/openai/deployments", // In Node.js defaults to process.env.AZURE_OPENAI_BASE_PATH
});

API Reference:

LLM usage example​

import { AzureOpenAI } from "@langchain/openai";

export const run = async () => {
  const model = new AzureOpenAI({
    temperature: 0.7,
    maxTokens: 1000,
    maxRetries: 5,
  });
  const res = await model.invoke(
    "Question: What would be a good company name for a company that makes colorful socks?\nAnswer:"
  );
  console.log({ res });
};

API Reference:

Chat usage example​

import { AzureChatOpenAI } from "@langchain/openai";

export const run = async () => {
  const model = new AzureChatOpenAI({
    prefixMessages: [
      {
        role: "system",
        content: "You are a helpful assistant that answers in pirate language",
      },
    ],
    maxTokens: 50,
  });
  const res = await model.invoke(
    "What would be a good company name for a company that makes colorful socks?"
  );
  console.log({ res });
};

API Reference:

Migration from Azure OpenAI SDK​

If you are using the deprecated Azure OpenAI SDK with the @langchain/azure-openai package, you can update your code to use the new Azure integration following these steps:

  1. Install the new @langchain/openai package and remove the previous @langchain/azure-openai package:

    npm install @langchain/openai
    npm uninstall @langchain/azure-openai

  2. Update your imports to use the new AzureOpenAI and AzureChatOpenAI classes from the @langchain/openai package:

    import { AzureOpenAI } from "@langchain/openai";

  3. Update your code to use the new AzureOpenAI and AzureChatOpenAI classes and pass the required parameters:

    const model = new AzureOpenAI({
      azureOpenAIApiKey: "<your_key>",
      azureOpenAIApiInstanceName: "<your_instance_name>",
      azureOpenAIApiDeploymentName: "<your_deployment_name>",
      azureOpenAIApiVersion: "<api_version>",
    });

    Notice that the constructor now requires the azureOpenAIApiInstanceName parameter instead of the azureOpenAIEndpoint parameter, and adds the azureOpenAIApiVersion parameter to specify the API version.

    • If you were using Azure Managed Identity, you now need to use the azureADTokenProvider parameter to the constructor instead of credentials, see the Azure Managed Identity section for more details.

    • If you were using environment variables, you now have to set the AZURE_OPENAI_API_INSTANCE_NAME environment variable instead of AZURE_OPENAI_API_ENDPOINT, and add the AZURE_OPENAI_API_VERSION environment variable to specify the API version.

Was this page helpful?


You can leave detailed feedback on GitHub.