jyapayne/langflow
1
0
Fork 0
Code Issues Pull requests Projects Releases 1 Packages Wiki Activity Actions

Merge branch 'logspace-ai:dev' into dev

Browse source
This commit is contained in:
Dave Morris 2023-08-06 16:02:03 -05:00 • committed by GitHub
commit 4984a1282d
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
59 changed files with 485 additions and 298 deletions
Download patch file Download diff file Expand all files Collapse all files

4
.github/workflows/release.yml vendored
View file

@ -14,9 +14,7 @@ env:
jobs:
if_release:
if: |
${{ github.event.pull_request.merged == true }}
&& ${{ contains(github.event.pull_request.labels.*.name, 'Release') }}
if: ${{ (github.event.pull_request.merged == true) && contains(github.event.pull_request.labels.*.name, 'Release') }}
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3

3
README.md
View file

@ -33,7 +33,7 @@
- [HuggingFace Spaces](#huggingface-spaces)
- [🖥️ Command Line Interface (CLI)](#️-command-line-interface-cli)
- [Usage](#usage)
- [Environment Variables](#environment-variables)
- [Environment Variables](#environment-variables)
- [Deployment](#deployment)
- [Deploy Langflow on Google Cloud Platform](#deploy-langflow-on-google-cloud-platform)
- [Deploy Langflow on Jina AI Cloud](#deploy-langflow-on-jina-ai-cloud)
@ -112,7 +112,6 @@ Each option is detailed below:
- `--cache`: Selects the type of cache to use. Options are `InMemoryCache` and `SQLiteCache`. Can be set using the `LANGFLOW_LANGCHAIN_CACHE` environment variable. The default is `SQLiteCache`.
- `--jcloud/--no-jcloud`: Toggles the option to deploy on Jina AI Cloud. The default is `no-jcloud`.
- `--dev/--no-dev`: Toggles the development mode. The default is `no-dev`.
- `--database-url`: Sets the database URL to connect to. If not provided, a local SQLite database will be used. Can be set using the `LANGFLOW_DATABASE_URL` environment variable.
- `--path`: Specifies the path to the frontend directory containing build files. This option is for development purposes only. Can be set using the `LANGFLOW_FRONTEND_PATH` environment variable.
- `--open-browser/--no-open-browser`: Toggles the option to open the browser after starting the server. Can be set using the `LANGFLOW_OPEN_BROWSER` environment variable. The default is `open-browser`.
- `--remove-api-keys/--no-remove-api-keys`: Toggles the option to remove API keys from the projects saved in the database. Can be set using the `LANGFLOW_REMOVE_API_KEYS` environment variable. The default is `no-remove-api-keys`.

2
dev.Dockerfile
View file

@ -15,4 +15,4 @@ COPY ./ ./
# Install dependencies
RUN poetry config virtualenvs.create false && poetry install --no-interaction --no-ansi
CMD ["uvicorn", "langflow.main:app", "--host", "0.0.0.0", "--port", "5003", "--reload", "log-level", "debug"]
CMD ["uvicorn","--factory", "langflow.main:create_app", "--host", "0.0.0.0", "--port", "5003", "--reload", "log-level", "debug"]

17
docker-compose.debug.yml
View file

@ -1,4 +1,4 @@
version: '3.4'
version: "3.4"
services:
backend:
@ -7,7 +7,12 @@ services:
build:
context: ./
dockerfile: ./dev.Dockerfile
command: ["sh", "-c", "pip install debugpy -t /tmp && python /tmp/debugpy --wait-for-client --listen 0.0.0.0:5678 -m uvicorn langflow.main:app --host 0.0.0.0 --port 7860 --reload"]
command:
[
"sh",
"-c",
"pip install debugpy -t /tmp && python /tmp/debugpy --wait-for-client --listen 0.0.0.0:5678 -m uvicorn --factory langflow.main:create_app --host 0.0.0.0 --port 7860 --reload",
]
ports:
- 7860:7860
- 5678:5678
@ -22,7 +27,7 @@ services:
ports:
- "3000:3000"
volumes:
- ./src/frontend/public:/home/node/app/public
- ./src/frontend/src:/home/node/app/src
- ./src/frontend/package.json:/home/node/app/package.json
restart: on-failure
- ./src/frontend/public:/home/node/app/public
- ./src/frontend/src:/home/node/app/src
- ./src/frontend/package.json:/home/node/app/package.json
restart: on-failure

12
docker-compose.yml
View file

@ -1,4 +1,4 @@
version: '3'
version: "3"
services:
backend:
@ -9,7 +9,7 @@ services:
- "7860:7860"
volumes:
- ./:/app
command: bash -c "uvicorn langflow.main:app --host 0.0.0.0 --port 7860 --reload"
command: bash -c "uvicorn --factory langflow.main:create_app --host 0.0.0.0 --port 7860 --reload"
frontend:
build:
@ -22,7 +22,7 @@ services:
ports:
- "3000:3000"
volumes:
- ./src/frontend/public:/home/node/app/public
- ./src/frontend/src:/home/node/app/src
- ./src/frontend/package.json:/home/node/app/package.json
restart: on-failure
- ./src/frontend/public:/home/node/app/public
- ./src/frontend/src:/home/node/app/src
- ./src/frontend/package.json:/home/node/app/package.json
restart: on-failure

14
docs/docs/guidelines/custom-component.mdx
View file

@ -385,19 +385,19 @@ Your structure should look something like this:
### Loading Custom Components
You can specify the path to your custom components using the _`--components-path`_ argument when running the Langflow CLI, as shown below:
```bash
langflow --components-path /path/to/components
```
Alternatively, you can set the `LANGFLOW_COMPONENTS_PATH` environment variable:
The recommended way to load custom components is to set the _`LANGFLOW_COMPONENTS_PATH`_ environment variable to the path of your custom components directory. Then, run the Langflow CLI as usual.
```bash
export LANGFLOW_COMPONENTS_PATH=/path/to/components
langflow
```
Alternatively, you can specify the path to your custom components using the _`--components-path`_ argument when running the Langflow CLI, as shown below:
```bash
langflow --components-path /path/to/components
```
Langflow will attempt to load all of the components found in the specified directory. If a component fails to load due to errors in the component's code, Langflow will print an error message to the console but will continue loading the rest of the components.
### Interacting with Custom Components

76
poetry.lock generated
View file

@ -146,13 +146,13 @@ files = [
[[package]]
name = "anthropic"
version = "0.3.7"
version = "0.3.8"
description = "Client library for the anthropic API"
optional = false
python-versions = ">=3.7,<4.0"
files = [
{file = "anthropic-0.3.7-py3-none-any.whl", hash = "sha256:ca57635d7f13d609aa8a5b93a834e067760d96b9657bdf81e0c7444ddf41fc64"},
{file = "anthropic-0.3.7.tar.gz", hash = "sha256:0453f80ba8224364c8b0dae0b5088becd67277de57708d7b887ebb6c2ceb3c49"},
{file = "anthropic-0.3.8-py3-none-any.whl", hash = "sha256:97ffe1bacc4214dc89b19f496cf2769746971e86f7c835a05aa21b76f260d279"},
{file = "anthropic-0.3.8.tar.gz", hash = "sha256:6651099807456c3b95b3879f5ad7d00f7e7e4f7649a2394d18032ab8be54ef16"},
]
[package.dependencies]
@ -739,18 +739,19 @@ sqlalchemy = ["sqlalchemy (>1.3.21,<2.0)"]
[[package]]
name = "cohere"
version = "4.17.0"
version = "4.19.2"
description = ""
optional = false
python-versions = ">=3.7,<4.0"
files = [
{file = "cohere-4.17.0-py3-none-any.whl", hash = "sha256:44e0bdb0a2d9467506d27b285f542177b98f92647f27e17ea921a01006fe2f33"},
{file = "cohere-4.17.0.tar.gz", hash = "sha256:9f479543b50490b4cb6385468d7571ad891a09cde7bd6b028171596bac6ce6ff"},
{file = "cohere-4.19.2-py3-none-any.whl", hash = "sha256:0b6a4fe04380a481a8e975ebcc9bb6433febe4d3eb583b6d6e04342a5e998345"},
{file = "cohere-4.19.2.tar.gz", hash = "sha256:a0b0fa698b3d3983fb328bb90d68fcf08faaa2268f3772ebc6bfea6ba55acf27"},
]
[package.dependencies]
aiohttp = ">=3.0,<4.0"
backoff = ">=2.0,<3.0"
fastavro = {version = "1.8.2", markers = "python_version >= \"3.8\""}
importlib_metadata = ">=6.0,<7.0"
requests = ">=2.25.0,<3.0.0"
urllib3 = ">=1.26,<3"
@ -924,13 +925,13 @@ test-randomorder = ["pytest-randomly"]
[[package]]
name = "ctransformers"
version = "0.2.17"
version = "0.2.18"
description = "Python bindings for the Transformer models implemented in C/C++ using GGML library."
optional = true
python-versions = "*"
files = [
{file = "ctransformers-0.2.17-py3-none-any.whl", hash = "sha256:903c16b38f5b2750ee34b90107c3a72351d7a9a201a6987a6560bd50874e9698"},
{file = "ctransformers-0.2.17.tar.gz", hash = "sha256:0c9de34cc8295ba6cb940e413130e6658fac54a99cecfa6098ac04638fd9301e"},
{file = "ctransformers-0.2.18-py3-none-any.whl", hash = "sha256:6d693e269d797f2485c86c9e8a322acb1464e782bf22547ec57e09d4ccbb4523"},
{file = "ctransformers-0.2.18.tar.gz", hash = "sha256:fc09c399baf16d99dd44aa6671d49aa4060d8904324c2527aa58e8831476c370"},
]
[package.dependencies]
@ -938,6 +939,7 @@ huggingface-hub = "*"
py-cpuinfo = ">=9.0.0,<10.0.0"
[package.extras]
gptq = ["exllama (==0.1.0)"]
tests = ["pytest"]
[[package]]
@ -1641,13 +1643,13 @@ six = "*"
[[package]]
name = "google-cloud-aiplatform"
version = "1.28.1"
version = "1.29.0"
description = "Vertex AI API client library"
optional = false
python-versions = ">=3.7"
files = [
{file = "google-cloud-aiplatform-1.28.1.tar.gz", hash = "sha256:b6468db7dc50295c988edf6505f0bf4d4bb2321de28873b2a4a87fd384be6308"},
{file = "google_cloud_aiplatform-1.28.1-py2.py3-none-any.whl", hash = "sha256:5587b8d4599047117b0c787635a00e8e3893b75944993e2faf784176442e9de4"},
{file = "google-cloud-aiplatform-1.29.0.tar.gz", hash = "sha256:fceabb924d2d26057e3c8c5c2e251929389aa6d553361377bc402781150c0db3"},
{file = "google_cloud_aiplatform-1.29.0-py2.py3-none-any.whl", hash = "sha256:cf81c1d93c61ccf3df60a65e3a5a1e465e044059d36b6fc1202b940c46c4c1e1"},
]
[package.dependencies]
@ -1728,13 +1730,13 @@ grpc = ["grpcio (>=1.38.0,<2.0dev)"]
[[package]]
name = "google-cloud-resource-manager"
version = "1.10.2"
version = "1.10.3"
description = "Google Cloud Resource Manager API client library"
optional = false
python-versions = ">=3.7"
files = [
{file = "google-cloud-resource-manager-1.10.2.tar.gz", hash = "sha256:9a7bdd0347ad553376cc66ad317c5223d1ae04bdcf74edcbfcd12605cff7b510"},
{file = "google_cloud_resource_manager-1.10.2-py2.py3-none-any.whl", hash = "sha256:9e074c28326bd1632f1a270c20cfea1ffe98f49cf821033e65bdac55661ffbd5"},
{file = "google-cloud-resource-manager-1.10.3.tar.gz", hash = "sha256:f80efcea36f10c5a81889afe93910926e3978b4b1ceeb82f563a2fc863072d14"},
{file = "google_cloud_resource_manager-1.10.3-py2.py3-none-any.whl", hash = "sha256:1381a4b0f522248ebe0ebd1289d8822b99c54f4e1fe03924a6e723b2ed93dd7f"},
]
[package.dependencies]
@ -2561,12 +2563,12 @@ testing = ["pytest (>=6)", "pytest-black (>=0.3.7)", "pytest-checkdocs (>=2.4)",
[[package]]
name = "jcloud"
version = "0.2.14"
version = "0.2.16"
description = "Simplify deploying and managing Jina projects on Jina Cloud"
optional = false
python-versions = "*"
files = [
{file = "jcloud-0.2.14.tar.gz", hash = "sha256:60798a41be370cacf380bb024f04086df8b0e65f20e51b4fbe1dd11e04d95be7"},
{file = "jcloud-0.2.16.tar.gz", hash = "sha256:abf39a70fc5852574a05e03dac5e5cc364df87ae1f8476cbd441178f14adb578"},
]
[package.dependencies]
@ -2890,6 +2892,20 @@ openai = ["openai (>=0,<1)", "tiktoken (>=0.3.2,<0.4.0)"]
qdrant = ["qdrant-client (>=1.3.1,<2.0.0)"]
text-helpers = ["chardet (>=5.1.0,<6.0.0)"]
[[package]]
name = "langchain-experimental"
version = "0.0.8"
description = "Building applications with LLMs through composability"
optional = false
python-versions = ">=3.8.1,<4.0"
files = [
{file = "langchain_experimental-0.0.8-py3-none-any.whl", hash = "sha256:34cf202ba29fdef178c5d68772cf2fa08dd2a0fad588ca3ef324ae71e596cc21"},
{file = "langchain_experimental-0.0.8.tar.gz", hash = "sha256:35d198f8e70a053ccd84273198ff08b4b700ac03ec5f43ba9b7ef797dd42ad14"},
]
[package.dependencies]
langchain = ">=0.0.239"
[[package]]
name = "langchain-serve"
version = "0.0.58"
@ -2917,13 +2933,13 @@ test = ["psutil", "pytest", "pytest-asyncio"]
[[package]]
name = "langsmith"
version = "0.0.16"
version = "0.0.18"
description = "Client library to connect to the LangSmith LLM Tracing and Evaluation Platform."
optional = false
python-versions = ">=3.8.1,<4.0"
files = [
{file = "langsmith-0.0.16-py3-none-any.whl", hash = "sha256:a9a5b30dc1c0ee0c9f8bbd2076856f1db3856a9521464f130be1db519c7077e5"},
{file = "langsmith-0.0.16.tar.gz", hash = "sha256:0729df30dd5b1c9d0c83614b5d348c7358ac182d7e3003dafbfcbedb88cbc8b8"},
{file = "langsmith-0.0.18-py3-none-any.whl", hash = "sha256:58f3eb029495e3696c3c94c95adfb2dd15043d82928ef932a7829fe020ad5522"},
{file = "langsmith-0.0.18.tar.gz", hash = "sha256:d8d97542f99352c5470bff74a4fbc9bd322dea850217a8ab71badcefa041566b"},
]
[package.dependencies]
@ -3311,13 +3327,13 @@ files = [
[[package]]
name = "more-itertools"
version = "10.0.0"
version = "10.1.0"
description = "More routines for operating on iterables, beyond itertools"
optional = false
python-versions = ">=3.8"
files = [
{file = "more-itertools-10.0.0.tar.gz", hash = "sha256:cd65437d7c4b615ab81c0640c0480bc29a550ea032891977681efd28344d51e1"},
{file = "more_itertools-10.0.0-py3-none-any.whl", hash = "sha256:928d514ffd22b5b0a8fce326d57f423a55d2ff783b093bab217eda71e732330f"},
{file = "more-itertools-10.1.0.tar.gz", hash = "sha256:626c369fa0eb37bac0291bce8259b332fd59ac792fa5497b59837309cd5b114a"},
{file = "more_itertools-10.1.0-py3-none-any.whl", hash = "sha256:64e0735fcfdc6f3464ea133afe8ea4483b1c5fe3a3d69852e6503b43a0b222e6"},
]
[[package]]
@ -5477,13 +5493,13 @@ cffi = {version = "*", markers = "implementation_name == \"pypy\""}
[[package]]
name = "qdrant-client"
version = "1.3.2"
version = "1.4.0"
description = "Client library for the Qdrant vector search engine"
optional = false
python-versions = ">=3.7,<3.12"
files = [
{file = "qdrant_client-1.3.2-py3-none-any.whl", hash = "sha256:66a076016fb9d33bec8170e96516d7e4a0ee5c611824cc9be18590ffeb3cf9aa"},
{file = "qdrant_client-1.3.2.tar.gz", hash = "sha256:6638c9eac027f2c0fdb1f63c3bd7b403fe8c3f73cb1f21fd15fd60f71012d537"},
{file = "qdrant_client-1.4.0-py3-none-any.whl", hash = "sha256:2f9e563955b5163da98016f2ed38d9aea5058576c7c5844e9aa205d28155f56d"},
{file = "qdrant_client-1.4.0.tar.gz", hash = "sha256:2e54f5a80eb1e7e67f4603b76365af4817af15fb3d0c0f44de4fd93afbbe5537"},
]
[package.dependencies]
@ -6380,13 +6396,13 @@ doc = ["reno", "sphinx", "tornado (>=4.5)"]
[[package]]
name = "textual"
version = "0.31.0"
version = "0.32.0"
description = "Modern Text User Interface framework"
optional = true
python-versions = ">=3.7,<4.0"
files = [
{file = "textual-0.31.0-py3-none-any.whl", hash = "sha256:1243bccadb28e1ff46bdfe676ee25a6ce52756842bc9dca4d824e0bc4d7d9a42"},
{file = "textual-0.31.0.tar.gz", hash = "sha256:e2b43f1c26b21731ee83f558f8d6cb4f7163e3a713854c36cd7785139a0e4e51"},
{file = "textual-0.32.0-py3-none-any.whl", hash = "sha256:81fc68406c8806bc864e2f035874a868b4ff0cf466289dce5f7b31869949383b"},
{file = "textual-0.32.0.tar.gz", hash = "sha256:f7b6683bc18faee6fd3c47cfbad43fbf8273c5fecc12230d52ce5ee089021327"},
]
[package.dependencies]
@ -7564,4 +7580,4 @@ local = ["ctransformers", "llama-cpp-python", "sentence-transformers"]
[metadata]
lock-version = "2.0"
python-versions = ">=3.9,<3.11"
content-hash = "154238df4e8b249196011f592d63e2fad0e18316960e400037f54453c813d47a"
content-hash = "57abce2ebcdc3cd7e359c36805822b9398d3bfb500c175b173a6d784d1276df6"

3
pyproject.toml
View file

@ -1,6 +1,6 @@
[tool.poetry]
name = "langflow"
version = "0.4.0"
version = "0.4.1"
description = "A Python package with a built-in web application"
authors = ["Logspace <contact@logspace.ai>"]
maintainers = [
@ -76,6 +76,7 @@ google-cloud-aiplatform = "^1.26.1"
psycopg = "^3.1.9"
psycopg-binary = "^3.1.9"
fastavro = "^1.8.0"
langchain-experimental = "^0.0.8"
[tool.poetry.group.dev.dependencies]
black = "^23.1.0"

47
src/backend/langflow/__main__.py
View file

@ -1,4 +1,3 @@
import os
import sys
import time
import httpx
@ -25,49 +24,25 @@ def update_settings(
config: str,
cache: str,
dev: bool = False,
database_url: Optional[str] = None,
remove_api_keys: bool = False,
components_path: Optional[Path] = None,
):
"""Update the settings from a config file."""
# Check for database_url in the environment variables
database_url = database_url or os.getenv("langflow_database_url")
if config:
logger.debug(f"Loading settings from {config}")
settings.update_from_yaml(config, dev=dev)
if database_url:
settings.update_settings(database_url=database_url)
if remove_api_keys:
logger.debug(f"Setting remove_api_keys to {remove_api_keys}")
settings.update_settings(remove_api_keys=remove_api_keys)
settings.update_settings(REMOVE_API_KEYS=remove_api_keys)
if cache:
logger.debug(f"Setting cache to {cache}")
settings.update_settings(cache=cache)
settings.update_settings(CACHE=cache)
if components_path:
logger.debug(f"Adding component path {components_path}")
settings.update_settings(components_path=components_path)
def load_params():
"""
Load the parameters from the environment variables.
"""
global_vars = globals()
for key, value in global_vars.items():
env_key = f"LANGFLOW_{key.upper()}"
if env_key in os.environ:
if isinstance(value, bool):
# Handle booleans
global_vars[key] = os.getenv(env_key, str(value)).lower() == "true"
elif isinstance(value, int):
# Handle integers
global_vars[key] = int(os.getenv(env_key, str(value)))
elif isinstance(value, str) or value is None:
# Handle strings and None values
global_vars[key] = os.getenv(env_key, str(value))
settings.update_settings(COMPONENTS_PATH=components_path)
def serve_on_jcloud():
@ -134,7 +109,7 @@ def serve(
config: str = typer.Option("config.yaml", help="Path to the configuration file."),
# .env file param
env_file: Path = typer.Option(
".env", help="Path to the .env file containing environment variables."
None, help="Path to the .env file containing environment variables."
),
log_level: str = typer.Option(
"critical", help="Logging level.", envvar="LANGFLOW_LOG_LEVEL"
@ -149,11 +124,13 @@ def serve(
),
jcloud: bool = typer.Option(False, help="Deploy on Jina AI Cloud"),
dev: bool = typer.Option(False, help="Run in development mode (may contain bugs)"),
database_url: str = typer.Option(
None,
help="Database URL to connect to. If not provided, a local SQLite database will be used.",
envvar="LANGFLOW_DATABASE_URL",
),
# This variable does not work but is set by the .env file
# and works with Pydantic
# database_url: str = typer.Option(
# None,
# help="Database URL to connect to. If not provided, a local SQLite database will be used.",
# envvar="LANGFLOW_DATABASE_URL",
# ),
path: str = typer.Option(
None,
help="Path to the frontend directory containing build files. This is for development purposes only.",
@ -176,7 +153,6 @@ def serve(
# override env variables with .env file
if env_file:
load_dotenv(env_file, override=True)
load_params()
if jcloud:
return serve_on_jcloud()
@ -185,7 +161,6 @@ def serve(
update_settings(
config,
dev=dev,
database_url=database_url,
remove_api_keys=remove_api_keys,
cache=cache,
components_path=components_path,

5
src/backend/langflow/api/v1/chat.py
View file

@ -125,9 +125,8 @@ async def stream_build(flow_id: str):
vertex.build()
params = vertex._built_object_repr()
valid = True
logger.debug(
f"Building node {str(params)[:50]}{'...' if len(str(params)) > 50 else ''}"
)
logger.debug(f"Building node {str(vertex.vertex_type)}")
logger.debug(f"Output: {params}")
if vertex.artifacts:
# The artifacts will be prompt variables
# passed to build_input_keys_response

39
src/backend/langflow/api/v1/endpoints.py
View file

@ -11,9 +11,6 @@ from fastapi import APIRouter, Depends, HTTPException, UploadFile, Body
from langflow.interface.custom.custom_component import CustomComponent
from langflow.interface.custom.directory_reader import (
CustomComponentPathValueError,
)
from langflow.api.v1.schemas import (
ProcessResponse,
@ -38,49 +35,27 @@ router = APIRouter(tags=["Base"])
@router.get("/all")
def get_all():
logger.debug("Building langchain types dict")
native_components = build_langchain_types_dict()
# custom_components is a list of dicts
# need to merge all the keys into one dict
custom_components_from_file = {}
if settings.components_path:
if settings.COMPONENTS_PATH:
logger.info(f"Building custom components from {settings.COMPONENTS_PATH}")
custom_component_dicts = [
build_langchain_custom_component_list_from_path(str(path))
for path in settings.components_path
for path in settings.COMPONENTS_PATH
]
logger.info(f"Loading {len(custom_component_dicts)} custom components")
for custom_component_dict in custom_component_dicts:
custom_components_from_file = merge_nested_dicts(
custom_components_from_file, custom_component_dict
)
logger.info(f"Loaded {custom_component_dict}")
return merge_nested_dicts(native_components, custom_components_from_file)
@router.get("/load_custom_component_from_path")
def get_load_custom_component_from_path(path: str):
try:
data = build_langchain_custom_component_list_from_path(path)
except CustomComponentPathValueError as err:
raise HTTPException(
status_code=400,
detail={"error": type(err).__name__, "traceback": str(err)},
) from err
return data
@router.get("/load_custom_component_from_path_TEST")
def get_load_custom_component_from_path_test(path: str):
from langflow.interface.custom.directory_reader import (
DirectoryReader,
)
reader = DirectoryReader(path, False)
file_list = reader.get_files()
data = reader.build_component_menu_list(file_list)
return reader.filter_loaded_components(data, True)
# For backwards compatibility we will keep the old endpoint
@router.post("/predict/{flow_id}", response_model=ProcessResponse)
@router.post("/process/{flow_id}", response_model=ProcessResponse)

2
src/backend/langflow/api/v1/flows.py
View file

@ -61,7 +61,7 @@ def update_flow(
if not db_flow:
raise HTTPException(status_code=404, detail="Flow not found")
flow_data = flow.dict(exclude_unset=True)
if settings.remove_api_keys:
if settings.REMOVE_API_KEYS:
flow_data = remove_api_keys(flow_data)
for key, value in flow_data.items():
setattr(db_flow, key, value)

4
src/backend/langflow/components/__init__.py Normal file
View file

@ -0,0 +1,4 @@
from langflow.interface.custom.custom_component import CustomComponent
__all__ = ["CustomComponent"]

0
src/backend/langflow/components/chains/__init__.py Normal file
View file

33
src/backend/langflow/components/chains/prompt_runner.py Normal file
View file

@ -0,0 +1,33 @@
from langflow import CustomComponent
from langchain.llms.base import BaseLLM
from langchain import PromptTemplate
from langchain.schema import Document
class PromptRunner(CustomComponent):
display_name: str = "Prompt Runner"
description: str = "Run a Chain with the given PromptTemplate"
beta = True
field_config = {
"llm": {"display_name": "LLM"},
"prompt": {
"display_name": "Prompt Template",
"info": "Make sure the prompt has all variables filled.",
},
"code": {"show": False},
"inputs": {"field_type": "code"},
}
def build(
self,
llm: BaseLLM,
prompt: PromptTemplate,
) -> Document:
chain = prompt | llm
# The input is an empty dict because the prompt is already filled
result = chain.invoke({})
if hasattr(result, "content"):
result = result.content
self.repr_value = result
return Document(page_content=str(result))

49
src/backend/langflow/database/base.py
View file

@ -1,21 +1,48 @@
from contextlib import contextmanager
from langflow.settings import settings
import os
from sqlmodel import SQLModel, Session, create_engine
from langflow.utils.logger import logger
if settings.database_url and settings.database_url.startswith("sqlite"):
connect_args = {"check_same_thread": False}
else:
connect_args = {}
if not settings.database_url:
raise RuntimeError("No database_url provided")
engine = create_engine(settings.database_url, connect_args=connect_args)
class Engine:
_instance = None
@classmethod
def get(cls):
logger.debug("Getting database engine")
if cls._instance is None:
cls.create()
return cls._instance
@classmethod
def create(cls):
logger.debug("Creating database engine")
from langflow.settings import settings
if langflow_database_url := os.getenv("LANGFLOW_DATABASE_URL"):
settings.DATABASE_URL = langflow_database_url
logger.debug("Using LANGFLOW_DATABASE_URL")
if settings.DATABASE_URL and settings.DATABASE_URL.startswith("sqlite"):
connect_args = {"check_same_thread": False}
else:
connect_args = {}
if not settings.DATABASE_URL:
raise RuntimeError("No database_url provided")
cls._instance = create_engine(settings.DATABASE_URL, connect_args=connect_args)
@classmethod
def update(cls):
logger.debug("Updating database engine")
cls._instance = None
cls.create()
def create_db_and_tables():
logger.debug("Creating database and tables")
try:
SQLModel.metadata.create_all(engine)
SQLModel.metadata.create_all(Engine.get())
except Exception as exc:
logger.error(f"Error creating database and tables: {exc}")
raise RuntimeError("Error creating database and tables") from exc
@ -23,7 +50,7 @@ def create_db_and_tables():
# and we need to create the tables again.
from sqlalchemy import inspect
inspector = inspect(engine)
inspector = inspect(Engine.get())
if "flow" not in inspector.get_table_names():
logger.error("Something went wrong creating the database and tables.")
logger.error("Please check your database settings.")
@ -36,7 +63,7 @@ def create_db_and_tables():
@contextmanager
def session_getter():
try:
session = Session(engine)
session = Session(Engine.get())
yield session
except Exception as e:
print("Session rollback because of exception:", e)

20
src/backend/langflow/graph/vertex/base.py
View file

@ -1,3 +1,4 @@
import ast
from langflow.interface.initialize import loading
from langflow.interface.listing import ALL_TYPES_DICT
from langflow.utils.constants import DIRECT_TYPES
@ -100,7 +101,9 @@ class Vertex:
params[param_key] = edge.source
for key, value in template_dict.items():
if key == "_type" or not value.get("show"):
# Skip _type and any value that has show == False and is not code
# If we don't want to show code but we want to use it
if key == "_type" or (not value.get("show") and key != "code"):
continue
# If the type is not transformable to a python base class
# then we need to get the edge that connects to this node
@ -112,7 +115,14 @@ class Vertex:
params[key] = file_path
elif value.get("type") in DIRECT_TYPES and params.get(key) is None:
params[key] = value.get("value")
if value.get("type") == "code":
try:
params[key] = ast.literal_eval(value.get("value"))
except Exception as exc:
logger.debug(f"Error parsing code: {exc}")
params[key] = value.get("value")
else:
params[key] = value.get("value")
if not value.get("required") and params.get(key) is None:
if value.get("default"):
@ -259,4 +269,8 @@ class Vertex:
def _built_object_repr(self):
# Add a message with an emoji, stars for sucess,
return "Built sucessfully ✨" if self._built_object else "Failed to build 😵‍💫"
return (
"Built sucessfully ✨"
if self._built_object is not None
else "Failed to build 😵‍💫"
)

6
src/backend/langflow/graph/vertex/types.py
View file

@ -226,7 +226,11 @@ class PromptVertex(Vertex):
# so the prompt format doesn't break
artifacts.pop("handle_keys", None)
try:
template = self._built_object.format(**artifacts)
template = self._built_object.template
for key, value in artifacts.items():
if value:
replace_key = "{" + key + "}"
template = template.replace(replace_key, value)
return (
template
if isinstance(template, str)

2
src/backend/langflow/interface/agents/base.py
View file

@ -59,7 +59,7 @@ class AgentCreator(LangChainTypeCreator):
if hasattr(agent, "function_name")
else agent.__name__
)
if agent_name in settings.agents or settings.dev:
if agent_name in settings.AGENTS or settings.DEV:
names.append(agent_name)
return names

2
src/backend/langflow/interface/base.py
View file

@ -28,7 +28,7 @@ class LangChainTypeCreator(BaseModel, ABC):
"""A dict with the name of the component as key and the documentation link as value."""
if self.name_docs_dict is None:
try:
type_settings = getattr(settings, self.type_name)
type_settings = getattr(settings, self.type_name.upper())
self.name_docs_dict = {
name: value_dict["documentation"]
for name, value_dict in type_settings.items()

5
src/backend/langflow/interface/chains/base.py
View file

@ -8,6 +8,7 @@ from langflow.template.frontend_node.chains import ChainFrontendNode
from langflow.utils.logger import logger
from langflow.utils.util import build_template_from_class, build_template_from_method
from langchain import chains
from langchain_experimental.sql import SQLDatabaseChain # type: ignore
# Assuming necessary imports for Field, Template, and FrontendNode classes
@ -35,12 +36,14 @@ class ChainCreator(LangChainTypeCreator):
}
from langflow.interface.chains.custom import CUSTOM_CHAINS
self.type_dict["SQLDatabaseChain"] = SQLDatabaseChain
self.type_dict.update(CUSTOM_CHAINS)
# Filter according to settings.chains
self.type_dict = {
name: chain
for name, chain in self.type_dict.items()
if name in settings.chains or settings.dev
if name in settings.CHAINS or settings.DEV
}
return self.type_dict

8
src/backend/langflow/interface/custom/component.py
View file

@ -1,5 +1,5 @@
import ast
from typing import Optional
from typing import Any, Optional
from pydantic import BaseModel
from fastapi import HTTPException
@ -63,10 +63,10 @@ class Component(BaseModel):
elif "description" in item_name:
template_config["description"] = ast.literal_eval(item_value)
elif "field_config" in item_name:
template_config["field_config"] = ast.literal_eval(item_value)
elif "beta" in item_name:
template_config["beta"] = ast.literal_eval(item_value)
return template_config
def build(self):
def build(self, *args: Any, **kwargs: Any) -> Any:
raise NotImplementedError

2
src/backend/langflow/interface/custom/constants.py
View file

@ -7,6 +7,7 @@ from langchain.schema import BaseRetriever, Document
from langchain.text_splitter import TextSplitter
from langchain.tools import Tool
from langchain.vectorstores.base import VectorStore
from langchain.schema import BaseOutputParser
LANGCHAIN_BASE_TYPES = {
@ -20,6 +21,7 @@ LANGCHAIN_BASE_TYPES = {
"VectorStore": VectorStore,
"Embeddings": Embeddings,
"BaseRetriever": BaseRetriever,
"BaseOutputParser": BaseOutputParser,
}
# Langchain base types plus Python base types

7
src/backend/langflow/interface/custom/custom_component.py
View file

@ -9,6 +9,7 @@ from langflow.utils import validate
from langflow.database.base import session_getter
from langflow.database.models.flow import Flow
from pydantic import Extra
import yaml
class CustomComponent(Component, extra=Extra.allow):
@ -24,6 +25,10 @@ class CustomComponent(Component, extra=Extra.allow):
super().__init__(**data)
def custom_repr(self):
if isinstance(self.repr_value, dict):
return yaml.dump(self.repr_value)
if isinstance(self.repr_value, str):
return self.repr_value
return str(self.repr_value)
def build_config(self):
@ -190,5 +195,5 @@ class CustomComponent(Component, extra=Extra.allow):
raise ValueError(f"Flow {flow_name or flow_id} not found")
return self.load_flow(flow.id, tweaks)
def build(self):
def build(self, *args: Any, **kwargs: Any) -> Any:
raise NotImplementedError

34
src/backend/langflow/interface/custom/directory_reader.py
View file

@ -1,6 +1,7 @@
import os
import ast
import zlib
from langflow.utils.logger import logger
class CustomComponentPathValueError(ValueError):
@ -74,8 +75,11 @@ class DirectoryReader:
}
for menu in data["menu"]
]
filtred = [menu for menu in items if menu["components"]]
return {"menu": filtred}
filtered = [menu for menu in items if menu["components"]]
logger.debug(
f'Filtered components {"with errors" if with_errors else ""}: {filtered}'
)
return {"menu": filtered}
def validate_code(self, file_content):
"""
@ -116,7 +120,7 @@ class DirectoryReader:
file_list.extend(
os.path.join(root, filename)
for filename in files
if filename.endswith(".py")
if filename.endswith(".py") and not filename.startswith("__")
)
return file_list
@ -213,27 +217,47 @@ class DirectoryReader:
from the .py files in the directory.
"""
response = {"menu": []}
logger.debug(
"-------------------- Building component menu list --------------------"
)
for file_path in file_paths:
menu_name = os.path.basename(os.path.dirname(file_path))
logger.debug(f"Menu name: {menu_name}")
filename = os.path.basename(file_path)
validation_result, result_content = self.process_file(file_path)
logger.debug(f"Validation result: {validation_result}")
menu_result = self.find_menu(response, menu_name) or {
"name": menu_name,
"path": os.path.dirname(file_path),
"components": [],
}
component_name = filename.split(".")[0]
# This is the name of the file which will be displayed in the UI
# We need to change it from snake_case to CamelCase
# first check if it's already CamelCase
if "_" in component_name:
component_name_camelcase = " ".join(
word.title() for word in component_name.split("_")
)
else:
component_name_camelcase = component_name
component_info = {
"name": filename.split(".")[0],
"name": "CustomComponent",
"output_types": [component_name_camelcase],
"file": filename,
"code": result_content if validation_result else "",
"error": "" if validation_result else result_content,
}
menu_result["components"].append(component_info)
logger.debug(f"Component info: {component_info}")
if menu_result not in response["menu"]:
response["menu"].append(menu_result)
logger.debug(
"-------------------- Component menu list built --------------------"
)
return response

2
src/backend/langflow/interface/document_loaders/base.py
View file

@ -33,7 +33,7 @@ class DocumentLoaderCreator(LangChainTypeCreator):
return [
documentloader.__name__
for documentloader in self.type_to_loader_dict.values()
if documentloader.__name__ in settings.documentloaders or settings.dev
if documentloader.__name__ in settings.DOCUMENTLOADERS or settings.DEV
]

2
src/backend/langflow/interface/embeddings/base.py
View file

@ -35,7 +35,7 @@ class EmbeddingCreator(LangChainTypeCreator):
return [
embedding.__name__
for embedding in self.type_to_loader_dict.values()
if embedding.__name__ in settings.embeddings or settings.dev
if embedding.__name__ in settings.EMBEDDINGS or settings.DEV
]

4
src/backend/langflow/interface/importing/utils.py
View file

@ -61,9 +61,7 @@ def import_by_type(_type: str, name: str) -> Any:
def import_custom_component(custom_component: str) -> CustomComponent:
"""Import custom component from custom component name"""
return import_class(
f"langflow.interface.custom.custom_component.{custom_component}"
)
return import_class("langflow.interface.custom.custom_component.CustomComponent")
def import_output_parser(output_parser: str) -> Any:

12
src/backend/langflow/interface/initialize/loading.py
View file

@ -6,7 +6,11 @@ from langchain.agents.agent import AgentExecutor
from langchain.agents.agent_toolkits.base import BaseToolkit
from langchain.agents.tools import BaseTool
from langflow.interface.initialize.llm import initialize_vertexai
from langflow.interface.initialize.utils import handle_format_kwargs, handle_node_type
from langflow.interface.initialize.utils import (
handle_format_kwargs,
handle_node_type,
handle_partial_variables,
)
from langflow.interface.initialize.vector_store import vecstore_initializer
@ -29,6 +33,7 @@ from langflow.utils import validate
from langchain.chains.base import Chain
from langchain.vectorstores.base import VectorStore
from langchain.document_loaders.base import BaseLoader
from langflow.utils.logger import logger
def instantiate_class(node_type: str, base_type: str, params: Dict) -> Any:
@ -40,7 +45,7 @@ def instantiate_class(node_type: str, base_type: str, params: Dict) -> Any:
if hasattr(custom_node, "initialize"):
return custom_node.initialize(**params)
return custom_node(**params)
logger.debug(f"Instantiating {node_type} of type {base_type}")
class_object = import_by_type(_type=base_type, name=node_type)
return instantiate_based_on_type(class_object, base_type, node_type, params)
@ -217,6 +222,9 @@ def instantiate_agent(node_type, class_object: Type[agent_module.Agent], params:
def instantiate_prompt(node_type, class_object, params: Dict):
params, prompt = handle_node_type(node_type, class_object, params)
format_kwargs = handle_format_kwargs(prompt, params)
# Now we'll use partial_format to format the prompt
if format_kwargs:
prompt = handle_partial_variables(prompt, format_kwargs)
return prompt, format_kwargs

10
src/backend/langflow/interface/initialize/utils.py
View file

@ -44,6 +44,16 @@ def handle_format_kwargs(prompt, params: Dict):
return format_kwargs
def handle_partial_variables(prompt, format_kwargs: Dict):
partial_variables = format_kwargs.copy()
partial_variables = {
key: value for key, value in partial_variables.items() if value
}
# Remove handle_keys otherwise LangChain raises an error
partial_variables.pop("handle_keys", None)
return prompt.partial(**partial_variables)
def handle_variable(params: Dict, input_variable: str, format_kwargs: Dict):
variable = params[input_variable]
if isinstance(variable, str):

2
src/backend/langflow/interface/llms/base.py
View file

@ -36,7 +36,7 @@ class LLMCreator(LangChainTypeCreator):
return [
llm.__name__
for llm in self.type_to_loader_dict.values()
if llm.__name__ in settings.llms or settings.dev
if llm.__name__ in settings.LLMS or settings.DEV
]

2
src/backend/langflow/interface/memories/base.py
View file

@ -51,7 +51,7 @@ class MemoryCreator(LangChainTypeCreator):
return [
memory.__name__
for memory in self.type_to_loader_dict.values()
if memory.__name__ in settings.memories or settings.dev
if memory.__name__ in settings.MEMORIES or settings.DEV
]

2
src/backend/langflow/interface/output_parsers/base.py
View file

@ -33,7 +33,7 @@ class OutputParserCreator(LangChainTypeCreator):
self.type_dict = {
name: output_parser
for name, output_parser in self.type_dict.items()
if name in settings.output_parsers or settings.dev
if name in settings.OUTPUT_PARSERS or settings.DEV
}
return self.type_dict

2
src/backend/langflow/interface/prompts/base.py
View file

@ -34,7 +34,7 @@ class PromptCreator(LangChainTypeCreator):
self.type_dict = {
name: prompt
for name, prompt in self.type_dict.items()
if name in settings.prompts or settings.dev
if name in settings.PROMPTS or settings.DEV
}
return self.type_dict

2
src/backend/langflow/interface/retrievers/base.py
View file

@ -51,7 +51,7 @@ class RetrieverCreator(LangChainTypeCreator):
return [
retriever
for retriever in self.type_to_loader_dict.keys()
if retriever in settings.retrievers or settings.dev
if retriever in settings.RETRIEVERS or settings.DEV
]

2
src/backend/langflow/interface/text_splitters/base.py
View file

@ -33,7 +33,7 @@ class TextSplitterCreator(LangChainTypeCreator):
return [
textsplitter.__name__
for textsplitter in self.type_to_loader_dict.values()
if textsplitter.__name__ in settings.textsplitters or settings.dev
if textsplitter.__name__ in settings.TEXTSPLITTERS or settings.DEV
]

2
src/backend/langflow/interface/toolkits/base.py
View file

@ -35,7 +35,7 @@ class ToolkitCreator(LangChainTypeCreator):
)
# if toolkit_name is not lower case it is a class
for toolkit_name in agent_toolkits.__all__
if not toolkit_name.islower() and toolkit_name in settings.toolkits
if not toolkit_name.islower() and toolkit_name in settings.TOOLKITS
}
return self.type_dict

2
src/backend/langflow/interface/tools/base.py
View file

@ -74,7 +74,7 @@ class ToolCreator(LangChainTypeCreator):
tool_name = tool_params.get("name") or tool
if tool_name in settings.tools or settings.dev:
if tool_name in settings.TOOLS or settings.DEV:
if tool_name == "JsonSpec":
tool_params["path"] = tool_params.pop("dict_") # type: ignore
all_tools[tool_name] = {

57
src/backend/langflow/interface/types.py
View file

@ -1,3 +1,5 @@
import ast
import contextlib
from typing import Any
from langflow.interface.agents.base import agent_creator
from langflow.interface.chains.base import chain_creator
@ -145,7 +147,7 @@ def add_code_field(template, raw_code, field_config):
"dynamic": True,
"required": True,
"placeholder": "",
"show": True,
"show": field_config.pop("show", True),
"multiline": True,
"value": raw_code,
"password": False,
@ -186,7 +188,7 @@ def build_frontend_node(custom_component: CustomComponent):
return None
def update_display_name_and_description(frontend_node, template_config):
def update_attributes(frontend_node, template_config):
"""Update the display name and description of a frontend node"""
if "display_name" in template_config:
frontend_node["display_name"] = template_config["display_name"]
@ -194,6 +196,9 @@ def update_display_name_and_description(frontend_node, template_config):
if "description" in template_config:
frontend_node["description"] = template_config["description"]
if "beta" in template_config:
frontend_node["beta"] = template_config["beta"]
def build_field_config(custom_component: CustomComponent):
"""Build the field configuration for a custom component"""
@ -247,6 +252,8 @@ def get_field_properties(extra_field):
if not field_required:
field_type = extract_type_from_optional(field_type)
with contextlib.suppress(Exception):
field_value = ast.literal_eval(field_value)
return field_name, field_type, field_value, field_required
@ -274,28 +281,30 @@ def add_base_classes(frontend_node, return_type):
def build_langchain_template_custom_component(custom_component: CustomComponent):
"""Build a custom component template for the langchain"""
logger.debug("Building custom component template")
frontend_node = build_frontend_node(custom_component)
if frontend_node is None:
return None
logger.debug("Built base frontend node")
template_config = custom_component.build_template_config
update_display_name_and_description(frontend_node, template_config)
update_attributes(frontend_node, template_config)
logger.debug("Updated attributes")
field_config = build_field_config(custom_component)
logger.debug("Built field config")
add_extra_fields(
frontend_node, field_config, custom_component.get_function_entrypoint_args
)
logger.debug("Added extra fields")
frontend_node = add_code_field(
frontend_node, custom_component.code, field_config.get("code", {})
)
logger.debug("Added code field")
add_base_classes(
frontend_node, custom_component.get_function_entrypoint_return_type
)
logger.debug("Added base classes")
return frontend_node
@ -306,7 +315,7 @@ def load_files_from_path(path: str):
return reader.get_files()
def build_and_validate_all_files(reader, file_list):
def build_and_validate_all_files(reader: DirectoryReader, file_list):
"""Build and validate all files"""
data = reader.build_component_menu_list(file_list)
@ -319,31 +328,53 @@ def build_and_validate_all_files(reader, file_list):
def build_valid_menu(valid_components):
"""Build the valid menu"""
valid_menu = {}
logger.debug("------------------- VALID COMPONENTS -------------------")
for menu_item in valid_components["menu"]:
menu_name = menu_item["name"]
valid_menu[menu_name] = {}
for component in menu_item["components"]:
logger.debug(f"Building component: {component}")
try:
component_name = component["name"]
component_code = component["code"]
component_output_types = component["output_types"]
component_extractor = CustomComponent(code=component_code)
component_extractor.is_check_valid()
component_template = build_langchain_template_custom_component(
component_extractor
)
component_template["output_types"] = component_output_types
if len(component_output_types) == 1:
component_name = component_output_types[0]
else:
file_name = component.get("file").split(".")[0]
if "_" in file_name:
# turn .py file into camelcase
component_name = "".join(
[word.capitalize() for word in file_name.split("_")]
)
else:
component_name = file_name
valid_menu[menu_name][component_name] = component_template
logger.debug(f"Added {component_name} to valid menu to {menu_name}")
except Exception as exc:
logger.error(f"Error while building custom component: {exc}")
logger.error(f"Error loading Component: {component['output_types']}")
logger.exception(
f"Error while building custom component {component_output_types}: {exc}"
)
return valid_menu
def build_invalid_menu(invalid_components):
"""Build the invalid menu"""
if invalid_components.get("menu"):
logger.debug("------------------- INVALID COMPONENTS -------------------")
invalid_menu = {}
for menu_item in invalid_components["menu"]:
menu_name = menu_item["name"]
@ -364,12 +395,16 @@ def build_invalid_menu(invalid_components):
)
component_template["error"] = component.get("error", None)
logger.debug(component)
logger.debug(f"Component Path: {component.get('path', None)}")
logger.debug(f"Component Error: {component.get('error', None)}")
component_template.get("template").get("code")["value"] = component_code
invalid_menu[menu_name][component_name] = component_template
logger.debug(f"Added {component_name} to invalid menu to {menu_name}")
except Exception as exc:
logger.error(
logger.exception(
f"Error while creating custom component [{component_name}]: {str(exc)}"
)

2
src/backend/langflow/interface/utilities/base.py
View file

@ -35,7 +35,7 @@ class UtilityCreator(LangChainTypeCreator):
self.type_dict = {
name: utility
for name, utility in self.type_dict.items()
if name in settings.utilities or settings.dev
if name in settings.UTILITIES or settings.DEV
}
return self.type_dict

5
src/backend/langflow/interface/utils.py
View file

@ -69,18 +69,17 @@ def setup_llm_caching():
try:
set_langchain_cache(settings)
except ImportError:
logger.warning(f"Could not import {settings.cache}. ")
logger.warning(f"Could not import {settings.CACHE}. ")
except Exception as exc:
logger.warning(f"Could not setup LLM caching. Error: {exc}")
# TODO Rename this here and in `setup_llm_caching`
def set_langchain_cache(settings):
import langchain
from langflow.interface.importing.utils import import_class
cache_type = os.getenv("LANGFLOW_LANGCHAIN_CACHE")
cache_class = import_class(f"langchain.cache.{cache_type or settings.cache}")
cache_class = import_class(f"langchain.cache.{cache_type or settings.CACHE}")
logger.debug(f"Setting up LLM caching with {cache_class.__name__}")
langchain.llm_cache = cache_class()

2
src/backend/langflow/interface/vector_store/base.py
View file

@ -46,7 +46,7 @@ class VectorstoreCreator(LangChainTypeCreator):
return [
vectorstore
for vectorstore in self.type_to_loader_dict.keys()
if vectorstore in settings.vectorstores or settings.dev
if vectorstore in settings.VECTORSTORES or settings.DEV
]

3
src/backend/langflow/main.py
View file

@ -6,7 +6,7 @@ from fastapi.responses import FileResponse
from fastapi.staticfiles import StaticFiles
from langflow.api import router
from langflow.database.base import create_db_and_tables
from langflow.database.base import create_db_and_tables, Engine
from langflow.interface.utils import setup_llm_caching
from langflow.utils.logger import configure
@ -34,6 +34,7 @@ def create_app():
)
app.include_router(router)
app.on_event("startup")(Engine.update)
app.on_event("startup")(create_db_and_tables)
app.on_event("startup")(setup_llm_caching)
return app

171
src/backend/langflow/settings.py
View file

@ -1,71 +1,91 @@
import contextlib
import json
import os
from typing import Optional, List
from pathlib import Path
import yaml
from pydantic import BaseSettings, root_validator
from pydantic import BaseSettings, root_validator, validator
from langflow.utils.logger import logger
BASE_COMPONENTS_PATH = Path(__file__).parent / "components"
BASE_COMPONENTS_PATH = str(Path(__file__).parent / "components")
class Settings(BaseSettings):
chains: dict = {}
agents: dict = {}
prompts: dict = {}
llms: dict = {}
tools: dict = {}
memories: dict = {}
embeddings: dict = {}
vectorstores: dict = {}
documentloaders: dict = {}
wrappers: dict = {}
retrievers: dict = {}
toolkits: dict = {}
textsplitters: dict = {}
utilities: dict = {}
output_parsers: dict = {}
custom_components: dict = {}
CHAINS: dict = {}
AGENTS: dict = {}
PROMPTS: dict = {}
LLMS: dict = {}
TOOLS: dict = {}
MEMORIES: dict = {}
EMBEDDINGS: dict = {}
VECTORSTORES: dict = {}
DOCUMENTLOADERS: dict = {}
WRAPPERS: dict = {}
RETRIEVERS: dict = {}
TOOLKITS: dict = {}
TEXTSPLITTERS: dict = {}
UTILITIES: dict = {}
OUTPUT_PARSERS: dict = {}
CUSTOM_COMPONENTS: dict = {}
dev: bool = False
database_url: Optional[str] = None
cache: str = "InMemoryCache"
remove_api_keys: bool = False
components_path: List[Path]
DEV: bool = False
DATABASE_URL: Optional[str] = None
CACHE: str = "InMemoryCache"
REMOVE_API_KEYS: bool = False
COMPONENTS_PATH: List[str] = []
@root_validator(pre=True)
def set_env_variables(cls, values):
if "database_url" not in values:
@validator("DATABASE_URL", pre=True)
def set_database_url(cls, value):
if not value:
logger.debug(
"No database_url provided, trying LANGFLOW_DATABASE_URL env variable"
)
if langflow_database_url := os.getenv("LANGFLOW_DATABASE_URL"):
values["database_url"] = langflow_database_url
value = langflow_database_url
logger.debug("Using LANGFLOW_DATABASE_URL env variable.")
else:
logger.debug("No DATABASE_URL env variable, using sqlite database")
values["database_url"] = "sqlite:///./langflow.db"
value = "sqlite:///./langflow.db"
if not values.get("components_path"):
values["components_path"] = [BASE_COMPONENTS_PATH]
logger.debug("No components_path provided, using default components path")
elif BASE_COMPONENTS_PATH not in values["components_path"]:
values["components_path"].append(BASE_COMPONENTS_PATH)
logger.debug("Adding default components path to components_path")
return value
@validator("COMPONENTS_PATH", pre=True)
def set_components_path(cls, value):
if os.getenv("LANGFLOW_COMPONENTS_PATH"):
logger.debug("Adding LANGFLOW_COMPONENTS_PATH to components_path")
langflow_component_path = Path(os.getenv("LANGFLOW_COMPONENTS_PATH"))
langflow_component_path = os.getenv("LANGFLOW_COMPONENTS_PATH")
if (
langflow_component_path.exists()
and langflow_component_path not in values["components_path"]
Path(langflow_component_path).exists()
and langflow_component_path not in value
):
values["components_path"].append(langflow_component_path)
logger.debug(f"Adding {langflow_component_path} to components_path")
return values
if isinstance(langflow_component_path, list):
for path in langflow_component_path:
if path not in value:
value.append(path)
logger.debug(
f"Extending {langflow_component_path} to components_path"
)
elif langflow_component_path not in value:
value.append(langflow_component_path)
logger.debug(
f"Appending {langflow_component_path} to components_path"
)
if not value:
value = [BASE_COMPONENTS_PATH]
logger.debug("Setting default components path to components_path")
elif BASE_COMPONENTS_PATH not in value:
value.append(BASE_COMPONENTS_PATH)
logger.debug("Adding default components path to components_path")
logger.debug(f"Components path: {value}")
return value
class Config:
validate_assignment = True
extra = "ignore"
env_prefix = "LANGFLOW_"
@root_validator(allow_reuse=True)
def validate_lists(cls, values):
@ -76,35 +96,50 @@ class Settings(BaseSettings):
def update_from_yaml(self, file_path: str, dev: bool = False):
new_settings = load_settings_from_yaml(file_path)
self.chains = new_settings.chains or {}
self.agents = new_settings.agents or {}
self.prompts = new_settings.prompts or {}
self.llms = new_settings.llms or {}
self.tools = new_settings.tools or {}
self.memories = new_settings.memories or {}
self.wrappers = new_settings.wrappers or {}
self.toolkits = new_settings.toolkits or {}
self.textsplitters = new_settings.textsplitters or {}
self.utilities = new_settings.utilities or {}
self.embeddings = new_settings.embeddings or {}
self.vectorstores = new_settings.vectorstores or {}
self.documentloaders = new_settings.documentloaders or {}
self.retrievers = new_settings.retrievers or {}
self.output_parsers = new_settings.output_parsers or {}
self.custom_components = new_settings.custom_components or {}
self.components_path = new_settings.components_path or []
self.dev = dev
self.CHAINS = new_settings.CHAINS or {}
self.AGENTS = new_settings.AGENTS or {}
self.PROMPTS = new_settings.PROMPTS or {}
self.LLMS = new_settings.LLMS or {}
self.TOOLS = new_settings.TOOLS or {}
self.MEMORIES = new_settings.MEMORIES or {}
self.WRAPPERS = new_settings.WRAPPERS or {}
self.TOOLKITS = new_settings.TOOLKITS or {}
self.TEXTSPLITTERS = new_settings.TEXTSPLITTERS or {}
self.UTILITIES = new_settings.UTILITIES or {}
self.EMBEDDINGS = new_settings.EMBEDDINGS or {}
self.VECTORSTORES = new_settings.VECTORSTORES or {}
self.DOCUMENTLOADERS = new_settings.DOCUMENTLOADERS or {}
self.RETRIEVERS = new_settings.RETRIEVERS or {}
self.OUTPUT_PARSERS = new_settings.OUTPUT_PARSERS or {}
self.CUSTOM_COMPONENTS = new_settings.CUSTOM_COMPONENTS or {}
self.COMPONENTS_PATH = new_settings.COMPONENTS_PATH or []
self.DEV = dev
def update_settings(self, **kwargs):
logger.debug("Updating settings")
for key, value in kwargs.items():
if hasattr(self, key):
if isinstance(getattr(self, key), list):
if isinstance(value, list):
getattr(self, key).extend(value)
else:
getattr(self, key).append(value)
# value may contain sensitive information, so we don't want to log it
if not hasattr(self, key):
logger.debug(f"Key {key} not found in settings")
continue
logger.debug(f"Updating {key}")
if isinstance(getattr(self, key), list):
# value might be a '[something]' string
with contextlib.suppress(json.decoder.JSONDecodeError):
value = json.loads(str(value))
if isinstance(value, list):
for item in value:
if item not in getattr(self, key):
getattr(self, key).append(item)
logger.debug(f"Extended {key}")
else:
setattr(self, key, value)
getattr(self, key).append(value)
logger.debug(f"Appended {key}")
else:
setattr(self, key, value)
logger.debug(f"Updated {key}")
logger.debug(f"{key}: {getattr(self, key)}")
def save_settings_to_yaml(settings: Settings, file_path: str):
@ -123,6 +158,12 @@ def load_settings_from_yaml(file_path: str) -> Settings:
with open(file_path, "r") as f:
settings_dict = yaml.safe_load(f)
settings_dict = {k.upper(): v for k, v in settings_dict.items()}
for key in settings_dict:
if key not in Settings.__fields__.keys():
raise KeyError(f"Key {key} not found in settings")
logger.debug(f"Loading {len(settings_dict[key])} {key} from {file_path}")
return Settings(**settings_dict)

1
src/frontend/.dockerignore Normal file
View file

@ -0,0 +1 @@
**/node_modules

2
src/frontend/src/CustomNodes/GenericNode/components/parameterComponent/index.tsx
View file

@ -87,7 +87,7 @@ export default function ParameterComponent({
useEffect(() => {
if (name === "openai_api_base") console.log(info);
infoHtml.current = (
<div className="h-full w-full break-words">
<div className="h-full w-full word-break-break-word">
{info.split("\n").map((line, i) => (
<p key={i} className="block">
{line}

4
src/frontend/src/alerts/alertDropDown/components/singleAlertComponent/index.tsx
View file

@ -36,14 +36,14 @@ export default function SingleAlert({
/>
</div>
<div className="ml-3">
<h3 className="break-words text-sm font-medium text-error-foreground">
<h3 className="text-sm font-medium text-error-foreground word-break-break-word">
{dropItem.title}
</h3>
{dropItem.list ? (
<div className="mt-2 text-sm text-error-foreground">
<ul className="list-disc space-y-1 pl-5">
{dropItem.list.map((item, idx) => (
<li className="break-words" key={idx}>
<li className="word-break-break-word" key={idx}>
{item}
</li>
))}

4
src/frontend/src/alerts/notice/index.tsx
View file

@ -47,7 +47,9 @@ export default function NoticeAlert({
/>
</div>
<div className="ml-3 flex-1 md:flex md:justify-between">
<p className="text-sm text-info-foreground">{title}</p>
<p className="text-sm text-info-foreground word-break-break-word">
{title}
</p>
<p className="mt-3 text-sm md:ml-6 md:mt-0">
{link !== "" ? (
<Link

32
src/frontend/src/components/codeTabsComponent/index.tsx
View file

@ -132,21 +132,25 @@ export default function CodeTabsComponent({
}}
>
<div className="api-modal-tablist-div">
<TabsList>
{tabs.map((tab, index) => (
<TabsTrigger
className={
isMessage ? "data-[state=active]:bg-primary-foreground" : ""
}
key={index}
value={index.toString()}
>
{tab.name}
</TabsTrigger>
))}
</TabsList>
{tabs.length > 0 && tabs[0].name !== "" ? (
<TabsList>
{tabs.map((tab, index) => (
<TabsTrigger
className={
isMessage ? "data-[state=active]:bg-primary-foreground" : ""
}
key={index}
value={index.toString()}
>
{tab.name}
</TabsTrigger>
))}
</TabsList>
) : (
<div></div>
)}
{Number(activeTab) < 4 && (
<div className="float-right mx-1 flex gap-2">
<div className="float-right mx-1 mb-1 mt-2 flex gap-2">
<button
className="flex items-center gap-1.5 rounded bg-none p-1 text-xs text-gray-500 dark:text-gray-300"
onClick={copyToClipboard}

4
src/frontend/src/modals/codeAreaModal/index.tsx
View file

@ -180,8 +180,8 @@ export default function CodeAreaModal({
<h1 className="text-lg text-destructive">
{error?.detail?.error}
</h1>
<div className="ml-2 w-full break-all text-sm text-status-red">
<pre className="w-full whitespace-pre-wrap break-all">
<div className="ml-2 w-full text-sm text-status-red word-break-break-word">
<pre className="w-full word-break-break-word">
{error?.detail?.traceback}
</pre>
</div>

3
src/frontend/src/modals/exportModal/index.tsx
View file

@ -13,6 +13,7 @@ const ExportModal = forwardRef((props: { children: ReactNode }, ref) => {
useContext(TabsContext);
const [checked, setChecked] = useState(false);
const [name, setName] = useState(flows.find((f) => f.id === tabId).name);
const [invalidName, setInvalidName] = useState(false);
const [description, setDescription] = useState(
flows.find((f) => f.id === tabId).description
);
@ -30,6 +31,8 @@ const ExportModal = forwardRef((props: { children: ReactNode }, ref) => {
</BaseModal.Header>
<BaseModal.Content>
<EditFlowSettings
invalidName={invalidName}
setInvalidName={setInvalidName}
name={name}
description={description}
flows={flows}

6
src/frontend/src/modals/formModal/chatMessage/index.tsx
View file

@ -88,8 +88,8 @@ export default function ChatMessage({
<ReactMarkdown
remarkPlugins={[remarkGfm, remarkMath]}
rehypePlugins={[rehypeMathjax]}
className="markdown prose inline-block break-words text-primary dark:prose-invert
sm:w-[30vw] sm:max-w-[30vw] lg:w-[40vw] lg:max-w-[40vw]"
className="markdown prose min-w-full text-primary word-break-break-word
dark:prose-invert"
components={{
pre({ node, ...props }) {
return <>{props.children}</>;
@ -187,7 +187,7 @@ export default function ChatMessage({
}
/>
</button>
<span className="prose inline-block break-words text-primary dark:prose-invert">
<span className="prose text-primary word-break-break-word dark:prose-invert">
{promptOpen
? template?.split("\n")?.map((line, index) => {
const regex = /{([^}]+)}/g;

2
src/frontend/src/modals/importModal/buttonBox/index.tsx
View file

@ -96,7 +96,7 @@ export default function ButtonBox({
<div className="mb-auto mt-auto w-full">
<h3
className={classNames(
"w-full break-words font-semibold text-background truncate-multiline",
"w-full font-semibold text-background truncate-multiline word-break-break-word",
titleFontSize,
marginTop
)}

22
src/frontend/src/style/applies.css
View file

@ -301,13 +301,13 @@
}
.error-build-message {
@apply mt-6 w-96 cursor-pointer rounded-md bg-error-background p-4 shadow-xl;
@apply mt-6 w-96 cursor-pointer rounded-md bg-error-background p-4 shadow-xl;
}
.error-build-message-circle {
@apply alert-icon text-status-red;
}
.error-build-text {
@apply text-error-foreground;
@apply text-error-foreground word-break-break-word;
}
.error-build-foreground {
@apply error-build-text alert-font-size;
@ -326,7 +326,7 @@
@apply alert-icon text-status-green;
}
.success-alert-message {
@apply alert-font-size text-success-foreground;
@apply word-break-break-word alert-font-size text-success-foreground;
}
.card-component-title-display {
@ -336,7 +336,7 @@
@apply flex h-7 w-7 items-center justify-center rounded-full text-2xl;
}
.card-component-title-size {
@apply inline-block w-full flex-1 break-words truncate-doubleline;
@apply w-full flex-1 word-break-break-word truncate-doubleline;
}
.card-component-delete-button {
@apply flex self-start;
@ -544,7 +544,7 @@
@apply focus:outline-none focus:ring-1 focus:ring-primary focus:ring-offset-1;
}
.toggle-component-span {
@apply pointer-events-none relative inline-block h-5 w-5 transform rounded-full shadow ring-0 transition duration-200 ease-in-out;
@apply pointer-events-none relative h-5 w-5 transform rounded-full shadow ring-0 transition duration-200 ease-in-out;
}
.toggle-component-second-span {
@apply absolute inset-0 flex h-full w-full items-center justify-center transition-opacity;
@ -590,13 +590,13 @@
@apply flex-max-width items-center text-start;
}
.chat-message-modal-text {
@apply relative inline-block w-full text-start text-sm font-normal text-muted-foreground;
@apply relative w-full text-start text-sm font-normal text-muted-foreground;
}
.chat-message-modal-icon-div {
@apply absolute -left-2 -top-1 cursor-pointer;
}
.chat-message-modal-thought {
@apply chat-message-modal-thought-cursor ml-3 inline-block h-full w-[95%] rounded-md border border-ring bg-muted px-2 pb-3 pt-3 text-start text-muted-foreground;
@apply chat-message-modal-thought-cursor ml-3 h-full w-[95%] rounded-md border border-ring bg-muted px-2 pb-3 pt-3 text-start text-muted-foreground;
}
.chat-message-modal-thought-cursor {
@apply cursor-pointer overflow-scroll scrollbar-hide;
@ -608,7 +608,7 @@
@apply mt-1 animate-pulse cursor-default;
}
.chat-message-modal-alert {
@apply inline-block px-3 text-start text-muted-foreground;
@apply px-3 text-start text-muted-foreground;
}
.file-card-modal-image-div {
@ -852,11 +852,11 @@
@apply bg-white text-primary;
}
.code-highlight {
@apply block max-h-[64vh] w-full overflow-y-hidden break-all border-0 px-3 py-2 text-sm outline-0;
@apply block max-h-[64vh] w-full overflow-y-hidden word-break-break-word border-0 px-3 py-2 text-sm outline-0;
}
.code-nohighlight {
@apply block max-h-[70vh] w-full overflow-y-hidden break-all border-0 px-3 py-2 text-sm outline-0;
@apply block max-h-[70vh] w-full overflow-y-hidden word-break-break-word border-0 px-3 py-2 text-sm outline-0;
}
.form-modal-lockchat {
@apply form-input block w-full rounded-md border-border p-4 pr-16 custom-scroll focus:border-ring focus:ring-ring sm:text-sm;
@ -913,7 +913,7 @@
@apply rounded-md border border-ring/60;
}
.form-modal-chat-thought-size {
@apply inline-block h-full w-[95%];
@apply h-full w-[95%];
}
.form-modal-chat-thought {
@apply form-modal-chat-thought-border form-modal-chat-thought-size cursor-pointer overflow-scroll bg-background px-2 py-2 text-start text-primary scrollbar-hide;

4
src/frontend/tailwind.config.js
View file

@ -155,7 +155,9 @@ module.exports = {
overflow: "hidden",
"text-overflow": "ellipsis",
},
".word-break-break-word": {
wordBreak: "break-word",
},
".arrow-hide": {
"&::-webkit-inner-spin-button": {
"-webkit-appearance": "none",

11
tests/test_graph.py
View file

@ -12,7 +12,6 @@ from langflow.graph.vertex.types import (
FileToolVertex,
LLMVertex,
ToolkitVertex,
WrapperVertex,
)
from langflow.processing.process import get_result_and_thought
from langflow.utils.payload import get_root_node
@ -292,11 +291,11 @@ def test_file_tool_node_build(openapi_graph):
assert not Path(file_path).exists()
def test_wrapper_node_build(openapi_graph):
wrapper_node = get_node_by_type(openapi_graph, WrapperVertex)
assert wrapper_node is not None
built_object = wrapper_node.build()
assert built_object is not None
# def test_wrapper_node_build(openapi_graph):
# wrapper_node = get_node_by_type(openapi_graph, WrapperVertex)
# assert wrapper_node is not None
# built_object = wrapper_node.build()
# assert built_object is not None
def test_get_result_and_thought(basic_graph):

2
tests/test_llms_template.py
View file

@ -7,7 +7,7 @@ def test_llms_settings(client: TestClient):
assert response.status_code == 200
json_response = response.json()
llms = json_response["llms"]
assert set(llms.keys()) == set(settings.llms)
assert set(llms.keys()) == set(settings.LLMS)
# def test_hugging_face_hub(client: TestClient):

2
tests/test_prompts_template.py
View file

@ -7,7 +7,7 @@ def test_prompts_settings(client: TestClient):
assert response.status_code == 200
json_response = response.json()
prompts = json_response["prompts"]
assert set(prompts.keys()) == set(settings.prompts)
assert set(prompts.keys()) == set(settings.PROMPTS)
def test_prompt_template(client: TestClient):

2
tests/test_vectorstore_template.py
View file

@ -9,4 +9,4 @@ def test_vectorstores_settings(client: TestClient):
assert response.status_code == 200
json_response = response.json()
vectorstores = json_response["vectorstores"]
assert set(vectorstores.keys()) == set(settings.vectorstores)
assert set(vectorstores.keys()) == set(settings.VECTORSTORES)