import Admonition from "@theme/Admonition";

# Models

<Admonition type="warning" title="warning">
This page may contain outdated information. It will be updated as soon as possible.
</Admonition>

## Amazon Bedrock

This component facilitates the generation of text using the LLM (Large Language Model) model from Amazon Bedrock.

**Params**

- **Input Value:** Specifies the input text for text generation.

- **System Message (Optional):** A system message to pass to the model.

- **Model ID (Optional):** Specifies the model ID to be used for text generation. Defaults to _`"anthropic.claude-instant-v1"`_. Available options include:

  - _`"ai21.j2-grande-instruct"`_
  - _`"ai21.j2-jumbo-instruct"`_
  - _`"ai21.j2-mid"`_
  - _`"ai21.j2-mid-v1"`_
  - _`"ai21.j2-ultra"`_
  - _`"ai21.j2-ultra-v1"`_
  - _`"anthropic.claude-instant-v1"`_
  - _`"anthropic.claude-v1"`_
  - _`"anthropic.claude-v2"`_
  - _`"cohere.command-text-v14"`_

- **Credentials Profile Name (Optional):** Specifies the name of the credentials profile.

- **Region Name (Optional):** Specifies the region name.

- **Model Kwargs (Optional):** Additional keyword arguments for the model.

- **Endpoint URL (Optional):** Specifies the endpoint URL.

- **Streaming (Optional):** Specifies whether to stream the response from the model. Defaults to _`False`_.

- **Cache (Optional):** Specifies whether to cache the response.

- **Stream (Optional):** Specifies whether to stream the response from the model. Defaults to _`False`_.

<Admonition type="note" title="Note">
  <p>
    Ensure that necessary credentials are provided to connect to the Amazon
    Bedrock API. If connection fails, a ValueError will be raised.
  </p>
</Admonition>

---

## Anthropic

This component allows the generation of text using Anthropic Chat&Completion large language models.

**Params**

- **Model Name:** Specifies the name of the Anthropic model to be used for text generation. Available options include:

  - _`"claude-2.1"`_
  - _`"claude-2.0"`_
  - _`"claude-instant-1.2"`_
  - _`"claude-instant-1"`_

- **Anthropic API Key:** Your Anthropic API key.

- **Max Tokens (Optional):** Specifies the maximum number of tokens to generate. Defaults to _`256`_.

- **Temperature (Optional):** Specifies the sampling temperature. Defaults to _`0.7`_.

- **API Endpoint (Optional):** Specifies the endpoint of the Anthropic API. Defaults to _`"https://api.anthropic.com"`_ if not specified.

- **Input Value:** Specifies the input text for text generation.

- **Stream (Optional):** Specifies whether to stream the response from the model. Defaults to _`False`_.

- **System Message (Optional):** A system message to pass to the model.

For detailed documentation and integration guides, please refer to the [Anthropic Component Documentation](https://python.langchain.com/docs/integrations/chat/anthropic).

---

## Azure OpenAI

This component allows the generation of text using the LLM (Large Language Model) model from Azure OpenAI.

**Params**

- **Model Name:** Specifies the name of the Azure OpenAI model to be used for text generation. Available options include:

  - _`"gpt-35-turbo"`_
  - _`"gpt-35-turbo-16k"`_
  - _`"gpt-35-turbo-instruct"`_
  - _`"gpt-4"`_
  - _`"gpt-4-32k"`_
  - _`"gpt-4-vision"`_

- **Azure Endpoint:** Your Azure endpoint, including the resource. Example: `https://example-resource.azure.openai.com/`.

- **Deployment Name:** Specifies the name of the deployment.

- **API Version:** Specifies the version of the Azure OpenAI API to be used. Available options include:

  - _`"2023-03-15-preview"`_
  - _`"2023-05-15"`_
  - _`"2023-06-01-preview"`_
  - _`"2023-07-01-preview"`_
  - _`"2023-08-01-preview"`_
  - _`"2023-09-01-preview"`_
  - _`"2023-12-01-preview"`_

- **API Key:** Your Azure OpenAI API key.

- **Temperature (Optional):** Specifies the sampling temperature. Defaults to _`0.7`_.

- **Max Tokens (Optional):** Specifies the maximum number of tokens to generate. Defaults to _`1000`_.

- **Input Value:** Specifies the input text for text generation.

- **Stream (Optional):** Specifies whether to stream the response from the model. Defaults to _`False`_.

- **System Message (Optional):** A system message to pass to the model.

For detailed documentation and integration guides, please refer to the [Azure OpenAI Component Documentation](https://python.langchain.com/docs/integrations/llms/azure_openai).

---

## Cohere

This component enables text generation using Cohere large language models.

**Params**

- **Cohere API Key:** Your Cohere API key.

- **Max Tokens (Optional):** Specifies the maximum number of tokens to generate. Defaults to _`256`_.

- **Temperature (Optional):** Specifies the sampling temperature. Defaults to _`0.75`_.

- **Input Value:** Specifies the input text for text generation.

- **Stream (Optional):** Specifies whether to stream the response from the model. Defaults to _`False`_.

- **System Message (Optional):** A system message to pass to the model.

---

## Google Generative AI

This component enables text generation using Google Generative AI.

**Params**

- **Google API Key:** Your Google API key to use for the Google Generative AI.

- **Model:** The name of the model to use. Supported examples are _`"gemini-pro"`_ and _`"gemini-pro-vision"`_.

- **Max Output Tokens (Optional):** The maximum number of tokens to generate.

- **Temperature:** Run inference with this temperature. Must be in the closed interval [0.0, 1.0].

- **Top K (Optional):** Decode using top-k sampling: consider the set of top_k most probable tokens. Must be positive.

- **Top P (Optional):** The maximum cumulative probability of tokens to consider when sampling.

- **N (Optional):** Number of chat completions to generate for each prompt. Note that the API may not return the full n completions if duplicates are generated.

- **Input Value:** The input to the model.

- **Stream (Optional):** Specifies whether to stream the response from the model. Defaults to _`False`_.

- **System Message (Optional):** A system message to pass to the model.

---

## Hugging Face API

This component facilitates text generation using LLM models from the Hugging Face Inference API.

**Params**

- **Endpoint URL:** The URL of the Hugging Face Inference API endpoint. Should be provided along with necessary authentication credentials.

- **Task:** Specifies the task for text generation. Options include _`"text2text-generation"`_, _`"text-generation"`_, and _`"summarization"`_.

- **API Token:** The API token required for authentication with the Hugging Face Hub.

- **Model Keyword Arguments (Optional):** Additional keyword arguments for the model. Should be provided as a Python dictionary.

- **Input Value:** The input text for text generation.

- **Stream (Optional):** Specifies whether to stream the response from the model. Defaults to _`False`_.

- **System Message (Optional):** A system message to pass to the model.

---

## LiteLLM Model

Generates text using the `LiteLLM` collection of large language models.

**Parameters**

- **Model name:** The name of the model to use. For example, `gpt-3.5-turbo`. (Type: str)
- **API key:** The API key to use for accessing the provider's API. (Type: str, Optional)
- **Provider:** The provider of the API key. (Type: str, Choices: "OpenAI", "Azure", "Anthropic", "Replicate", "Cohere", "OpenRouter")
- **Temperature:** Controls the randomness of the text generation. (Type: float, Default: 0.7)
- **Model kwargs:** Additional keyword arguments for the model. (Type: Dict, Optional)
- **Top p:** Filter responses to keep the cumulative probability within the top p tokens. (Type: float, Optional)
- **Top k:** Filter responses to only include the top k tokens. (Type: int, Optional)
- **N:** Number of chat completions to generate for each prompt. (Type: int, Default: 1)
- **Max tokens:** The maximum number of tokens to generate for each chat completion. (Type: int, Default: 256)
- **Max retries:** Maximum number of retries for failed requests. (Type: int, Default: 6)
- **Verbose:** Whether to print verbose output. (Type: bool, Default: False)
- **Input:** The input prompt for text generation. (Type: str)
- **Stream:** Whether to stream the output. (Type: bool, Default: False)
- **System message:** System message to pass to the model. (Type: str, Optional)

---

## Ollama

Generate text using Ollama Local LLMs.

**Parameters**

- **Base URL:** Endpoint of the Ollama API. Defaults to 'http://localhost:11434' if not specified.
- **Model Name:** The model name to use. Refer to [Ollama Library](https://ollama.ai/library) for more models.
- **Temperature:** Controls the creativity of model responses. (Default: 0.8)
- **Cache:** Enable or disable caching. (Default: False)
- **Format:** Specify the format of the output (e.g., json). (Advanced)
- **Metadata:** Metadata to add to the run trace. (Advanced)
- **Mirostat:** Enable/disable Mirostat sampling for controlling perplexity. (Default: Disabled)
- **Mirostat Eta:** Learning rate for Mirostat algorithm. (Default: None) (Advanced)
- **Mirostat Tau:** Controls the balance between coherence and diversity of the output. (Default: None) (Advanced)
- **Context Window Size:** Size of the context window for generating tokens. (Default: None) (Advanced)
- **Number of GPUs:** Number of GPUs to use for computation. (Default: None) (Advanced)
- **Number of Threads:** Number of threads to use during computation. (Default: None) (Advanced)
- **Repeat Last N:** How far back the model looks to prevent repetition. (Default: None) (Advanced)
- **Repeat Penalty:** Penalty for repetitions in generated text. (Default: None) (Advanced)
- **TFS Z:** Tail free sampling value. (Default: None) (Advanced)
- **Timeout:** Timeout for the request stream. (Default: None) (Advanced)
- **Top K:** Limits token selection to top K. (Default: None) (Advanced)
- **Top P:** Works together with top-k. (Default: None) (Advanced)
- **Verbose:** Whether to print out response text.
- **Tags:** Tags to add to the run trace. (Advanced)
- **Stop Tokens:** List of tokens to signal the model to stop generating text. (Advanced)
- **System:** System to use for generating text. (Advanced)
- **Template:** Template to use for generating text. (Advanced)
- **Input:** The input text.
- **Stream:** Whether to stream the response.
- **System Message:** System message to pass to the model. (Advanced)

---

## OpenAI

This component facilitates text generation using OpenAI's models.

**Params**

- **Input Value:** The input text for text generation.

- **Max Tokens (Optional):** The maximum number of tokens to generate. Defaults to _`256`_.

- **Model Kwargs (Optional):** Additional keyword arguments for the model. Should be provided as a nested dictionary.

- **Model Name (Optional):** The name of the model to use. Defaults to _`gpt-4-1106-preview`_. Supported options include: _`gpt-4-turbo-preview`_, _`gpt-4-0125-preview`_, _`gpt-4-1106-preview`_, _`gpt-4-vision-preview`_, _`gpt-3.5-turbo-0125`_, _`gpt-3.5-turbo-1106`_.

- **OpenAI API Base (Optional):** The base URL of the OpenAI API. Defaults to _`https://api.openai.com/v1`_.

- **OpenAI API Key (Optional):** The API key for accessing the OpenAI API.

- **Temperature:** Controls the creativity of model responses. Defaults to _`0.7`_.

- **Stream (Optional):** Specifies whether to stream the response from the model. Defaults to _`False`_.

- **System Message (Optional):** System message to pass to the model.

---

## Qianfan

This component facilitates the generation of text using Baidu Qianfan chat models.

**Params**

- **Model Name:** Specifies the name of the Qianfan chat model to be used for text generation. Available options include:

  - _`"ERNIE-Bot"`_
  - _`"ERNIE-Bot-turbo"`_
  - _`"BLOOMZ-7B"`_
  - _`"Llama-2-7b-chat"`_
  - _`"Llama-2-13b-chat"`_
  - _`"Llama-2-70b-chat"`_
  - _`"Qianfan-BLOOMZ-7B-compressed"`_
  - _`"Qianfan-Chinese-Llama-2-7B"`_
  - _`"ChatGLM2-6B-32K"`_
  - _`"AquilaChat-7B"`_

- **Qianfan Ak:** Your Baidu Qianfan access key, obtainable from [here](https://cloud.baidu.com/product/wenxinworkshop).

- **Qianfan Sk:** Your Baidu Qianfan secret key, obtainable from [here](https://cloud.baidu.com/product/wenxinworkshop).

- **Top p (Optional):** Model parameter. Specifies the top-p value. Only supported in ERNIE-Bot and ERNIE-Bot-turbo models. Defaults to _`0.8`_.

- **Temperature (Optional):** Model parameter. Specifies the sampling temperature. Only supported in ERNIE-Bot and ERNIE-Bot-turbo models. Defaults to _`0.95`_.

- **Penalty Score (Optional):** Model parameter. Specifies the penalty score. Only supported in ERNIE-Bot and ERNIE-Bot-turbo models. Defaults to _`1.0`_.

- **Endpoint (Optional):** Endpoint of the Qianfan LLM, required if custom model is used.

- **Input Value:** Specifies the input text for text generation.

- **Stream (Optional):** Specifies whether to stream the response from the model. Defaults to _`False`_.

- **System Message (Optional):** A system message to pass to the model.

---

## Vertex AI

The `ChatVertexAI` is a component for generating text using Vertex AI Chat large language models API.

**Params**

- **Credentials:** The JSON file containing the credentials for accessing the Vertex AI Chat API.

- **Project:** The name of the project associated with the Vertex AI Chat API.

- **Examples (Optional):** List of examples to provide context for text generation.

- **Location:** The location of the Vertex AI Chat API service. Defaults to _`us-central1`_.

- **Max Output Tokens:** The maximum number of tokens to generate. Defaults to _`128`_.

- **Model Name:** The name of the model to use. Defaults to _`chat-bison`_.

- **Temperature:** Controls the creativity of model responses. Defaults to _`0.0`_.

- **Input Value:** The input text for text generation.

- **Top K:** Limits token selection to top K. Defaults to _`40`_.

- **Top P:** Works together with top-k. Defaults to _`0.95`_.

- **Verbose:** Whether to print out response text. Defaults to _`False`_.

- **Stream (Optional):** Specifies whether to stream the response from the model. Defaults to _`False`_.

- **System Message (Optional):** System message to pass to the model.