jyapayne/langflow
1
0
Fork 0
Code Issues Pull requests Projects Releases 1 Packages Wiki Activity Actions

docs: integrate nvidia NIM on WSL2 (#7192)

Browse source 
* initial-content

* more-on-wsl2

* update flow docs

* cleanup

* title-sidebar

---------

Co-authored-by: Jordan Frazier <jordan.frazier@datastax.com>
This commit is contained in:
Mendon Kissling 2025-03-20 21:18:55 -04:00 • committed by GitHub
commit ddda5970aa
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
3 changed files with 35 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

2
docs/docs/Integrations/Nvidia/integrations-nvidia-ingest.md
View file

@ -1,5 +1,5 @@
---
title: Integrate Nvidia Ingest with Langflow
title: Integrate NVIDIA Ingest with Langflow
slug: /integrations-nvidia-ingest
---

33
docs/docs/Integrations/Nvidia/integrations-nvidia-nim-wsl2.md Normal file
View file

@ -0,0 +1,33 @@
---
title: Integrate NVIDIA NIMs with Langflow
slug: /integrations-nvidia-ingest-wsl2
---
Connect **Langflow** with **NVIDIA NIM** on an RTX Windows system with [Windows Subsystem for Linux 2 (WSL2)](https://learn.microsoft.com/en-us/windows/wsl/install) installed.
[NVIDIA NIM](https://docs.nvidia.com/nim/index.html) provides containers to self-host GPU-accelerated inferencing microservices.
This example deploys the `mistral-nemo-12b-instruct` NIM on an **RTX Windows system** with **WSL2** and uses it as a model component in **Langflow**.
For more information on NVIDIA NIM, see the [NVIDIA documentation](https://docs.nvidia.com/nim/index.html).
## Prerequisites
* [NVIDIA NIM WSL2 installed](https://docs.nvidia.com/nim/wsl2/latest/getting-started.html)
* A NIM container deployed. The prerequisites vary between models.
For example, to deploy the `mistral-nemo-12b-instruct` NIM, follow the instructions for **Windows on RTX AI PCs (Beta)** on your [model's deployment overview](https://build.nvidia.com/nv-mistralai/mistral-nemo-12b-instruct/deploy?environment=wsl2.md)
* [WSL2 installed](https://learn.microsoft.com/en-us/windows/wsl/install)
* Windows 11 build 23H2 (and later)
* At least 12 GB of RAM
## Use the NVIDIA NIM in a flow
To connect the NIM you've deployed with Langflow, add the **NVIDIA** model component to a flow.
1. Create a [basic prompting flow](/get-started-quickstart).
2. Replace the **OpenAI** model component with the **NVIDIA** component.
3. In the **NVIDIA** component's **Base URL** field, add the URL your NIM is accessible at. If you followed your model's [deployment instructions](https://build.nvidia.com/nv-mistralai/mistral-nemo-12b-instruct/deploy?environment=wsl2.md), the value is `http://0.0.0.0:8000/v1`.
4. In the **NVIDIA** component's **NVIDIA API Key** field, add your NVIDIA API Key.
5. Select your model from the **Model Name** dropdown.
6. Open the **Playground** and chat with your **NIM** model.

1
docs/sidebars.js
View file

@ -174,6 +174,7 @@ module.exports = {
label: "NVIDIA",
items: [
"Integrations/Nvidia/integrations-nvidia-ingest",
"Integrations/Nvidia/integrations-nvidia-nim-wsl2",
],
},
],