575cde3e8d* add some related links * admonitions audit * initial prereq audit * standardize install LF prereqs * some coderabbit
90 lines
No EOL
5.9 KiB
Text
90 lines
No EOL
5.9 KiB
Text
---
|
||
title: Vector store RAG
|
||
slug: /vector-store-rag
|
||
---
|
||
|
||
import Icon from "@site/src/components/icon";
|
||
|
||
Retrieval Augmented Generation, or RAG, is a pattern for training LLMs on your data and querying it.
|
||
|
||
RAG is backed by a **vector store**, a vector database which stores embeddings of the ingested data.
|
||
|
||
This enables **vector search**, a more powerful and context-aware search.
|
||
|
||
We've chosen [Astra DB](https://astra.datastax.com/signup?utm_source=langflow-pre-release&utm_medium=referral&utm_campaign=langflow-announcement&utm_content=create-a-free-astra-db-account) as the vector database for this starter flow, but you can follow along with any of Langflow's vector database options.
|
||
|
||
## Prerequisites
|
||
|
||
* [Install and start Langflow](/get-started-installation)
|
||
* Create an [OpenAI API key](https://platform.openai.com/api-keys)
|
||
* Create an [Astra DB vector database](https://docs.datastax.com/en/astra-db-serverless/get-started/quickstart.html)
|
||
* Create an [Astra application token scoped to your database](https://docs.datastax.com/en/astra-db-serverless/administration/manage-application-tokens.html#database-token)
|
||
|
||
You can use an existing [collection](https://docs.datastax.com/en/astra-db-serverless/databases/manage-collections.html#create-collection) or create a new collection through the **Astra DB** component.
|
||
|
||
## Open Langflow and start a new project
|
||
|
||
1. From the Langflow dashboard, click **New Flow**.
|
||
2. Select **Vector Store RAG**.
|
||
3. The **Vector Store RAG** flow is created.
|
||
|
||
## Build the vector RAG flow
|
||
|
||
The vector store RAG flow is built of two separate flows for ingestion and query.
|
||
|
||
|
||
|
||
The **Load Data Flow** (bottom of the screen) creates a searchable index to be queried for contextual similarity.
|
||
This flow populates the vector store with data from a local file.
|
||
It ingests data from a local file, splits it into chunks, indexes it in Astra DB, and computes embeddings for the chunks using the **Embeddings model** component.
|
||
|
||
The **Retriever Flow** (top of the screen) embeds the user's queries into vectors, which are compared to the vector store data from the **Load Data Flow** for contextual similarity.
|
||
|
||
- **Chat Input** receives user input from the **Playground**.
|
||
- **Embeddings model** converts the user query into vector form.
|
||
- **Astra DB** performs similarity search using the query vector.
|
||
- **Parser** processes the retrieved chunks.
|
||
- **Prompt** combines the user query with relevant context.
|
||
- **Language model** generates the response using the prompt and the OpenAI LLM.
|
||
- **Chat Output** returns the response to the **Playground**.
|
||
|
||
1. Add your **OpenAI API key** to the **Language model** component.
|
||
|
||
Optionally, create a [global variable](/configuration-global-variables) for the **OpenAI API key**.
|
||
|
||
1. In the **OpenAI API Key** field, click <Icon name="Globe" aria-hidden="True" /> **Globe**, and then click **Add New Variable**.
|
||
2. In the **Variable Name** field, enter `openai_api_key`.
|
||
3. In the **Value** field, paste your OpenAI API Key (`sk-...`).
|
||
4. Click **Save Variable**.
|
||
2. Configure the **Astra DB** component.
|
||
1. In the **Astra DB Application Token** field, add your **Astra DB** application token.
|
||
The component connects to your database and populates the menus with existing databases and collections.
|
||
2. Select your **Database**.
|
||
If you don't have a collection, select **New database**.
|
||
Complete the **Name**, **Cloud provider**, and **Region** fields, and then click **Create**. **Database creation takes a few minutes**.
|
||
3. Select your **Collection**. Collections are created in your [Astra DB deployment](https://astra.datastax.com) for storing vector data.
|
||
:::info
|
||
If your collection use the Astra vectorize NVIDIA integration, the **Embedding Model** port is removed, because the collection already generates embeddings with the integrated NVIDIA model. The component fetches the data from the collection, and uses the same model to generate embeddings for queries.
|
||
:::
|
||
|
||
3. If you don't have a collection, create a new one within the component.
|
||
1. Select **New collection**.
|
||
2. Complete the **Name**, **Embedding generation method**, **Embedding model**, and **Dimensions** fields, and then click **Create**.
|
||
|
||
Your choice for the **Embedding generation method** and **Embedding model** depends on whether you want to use embeddings generated by a provider through Astra's vectorize service, or generated by a component in Langflow.
|
||
|
||
* To use embeddings generated by a provider through Astra's vectorize service, select the model from the **Embedding generation method** dropdown menu, and then select the model from the **Embedding model** dropdown menu.
|
||
* To use embeddings generated by a component in Langflow, select **Bring your own** for both the **Embedding generation method** and **Embedding model** fields. In this starter project, the option for the embeddings method and model is the **OpenAI Embeddings** component connected to the **Astra DB** component.
|
||
* The **Dimensions** value must match the dimensions of your collection. This field is **not required** if you use embeddings generated through Astra's vectorize service. You can find this value in the **Collection** in your [Astra DB deployment](https://astra.datastax.com).
|
||
|
||
For more information, see the [DataStax Astra DB Serverless documentation](https://docs.datastax.com/en/astra-db-serverless/databases/embedding-generation.html).
|
||
|
||
|
||
If you used Langflow's **Global Variables** feature, the RAG application flow components are already configured with the necessary credentials.
|
||
|
||
## Run the Vector Store RAG flow
|
||
|
||
1. To run the flow, click <Icon name="Play" aria-hidden="true"/> **Playground**.
|
||
In the **Playground**, you can chat with the LLM that uses context from the database you created.
|
||
2. Enter a message and press <kbd>Enter</kbd>. Try something like, "What topics do you know about?"
|
||
3. The bot will respond with a summary of the data you've embedded. |