jyapayne/langflow
1
0
Fork 0
Code Issues Pull requests Projects Releases 1 Packages Wiki Activity Actions

Merge branch 'feat-testComps' of https://github.com/logspace-ai/langflow into feat-testComps

Browse source
This commit is contained in:
cristhianzl 2023-10-18 15:53:25 -03:00
commit 1845149647
12 changed files with 4284 additions and 776 deletions
Download patch file Download diff file Expand all files Collapse all files

4
.gitignore vendored
View file

@ -254,4 +254,6 @@ langflow.db
/tmp/*
src/backend/langflow/frontend/
.docker
.docker
.idea

38
docs/docs/components/llms.mdx
View file

@ -217,4 +217,40 @@ Vertex AI is a cloud computing platform offered by Google Cloud Platform (GCP).
- **top_k:** How the model selects tokens for output, the next token is selected from – defaults to `40`.
- **top_p:** Tokens are selected from most probable to least until the sum of their – defaults to `0.95`.
- **tuned_model_name:** The name of a tuned model. If provided, model_name is ignored.
- **verbose:** This parameter is used to control the level of detail in the output of the chain. When set to True, it will print out some internal states of the chain while it is being run, which can help debug and understand the chain's behavior. If set to False, it will suppress the verbose output – defaults to `False`.
- **verbose:** This parameter is used to control the level of detail in the output of the chain. When set to True, it will print out some internal states of the chain while it is being run, which can help debug and understand the chain's behavior. If set to False, it will suppress the verbose output – defaults to `False`.
---
### QianfanLLMEndpoint
Wrapper around [Baidu Qianfan](https://cloud.baidu.com/doc/WENXINWORKSHOP/index.html) large language models.
:::info
The Qianfan Big Model Platform is a one-stop platform for enterprise developers to develop and operate large models and services. It provides data management based on ERNIE Bot's underlying model (Ernie Bot), automatic model customization and fine-tuning, and one-stop large-scale model customization services for cloud deployment of prediction services, and provides ERNIE Bot's enterprise level service API that can be quickly called, helping to implement the demand for generative AI applications in various industries.
:::
- **Model Name:** Model name. you could get from https://cloud.baidu.com/doc/WENXINWORKSHOP/s/Nlks5zkzu preset models are mapping to an endpoint. `Model Name` will be ignored if `Endpoint` is set.
- **Qianfan Ak:** which you could get from https://cloud.baidu.com/product/wenxinworkshop.
- **Qianfan Sk:** which you could get from https://cloud.baidu.com/product/wenxinworkshop.
- **Top p:** Model params, only supported in ERNIE-Bot and ERNIE-Bot-turbo. The diversity of the output text is affected, and the larger the value, the stronger the diversity of the generated text - defaults to `0.8`.
- **Temperature:** Model params, only supported in ERNIE-Bot and ERNIE-Bot-turbo. Higher values make the output more random, while lower values make it more concentrated and deterministic - defaults to `0.95`.
- **Penalty Score:** Model params, only supported in ERNIE-Bot and ERNIE-Bot-turbo. By increasing the penalty for generated tokens, the phenomenon of duplicate generation is reduced. A higher value indicates a higher penalty - defaults to `1.0`.
- **Endpoint:** Endpoint of the Qianfan LLM, required if custom model used.
---
### QianfanChatEndpoint
Wrapper around [Baidu Qianfan](https://cloud.baidu.com/doc/WENXINWORKSHOP/index.html) chat large language models. This component supports some of the LLMs (Large Language Models) available by Baidu qianfan and is used for tasks such as chatbots, Generative Question-Answering (GQA), and summarization.
:::info
The Qianfan Big Model Platform is a one-stop platform for enterprise developers to develop and operate large models and services. It provides data management based on ERNIE Bot's underlying model (Ernie Bot), automatic model customization and fine-tuning, and one-stop large-scale model customization services for cloud deployment of prediction services, and provides ERNIE Bot's enterprise level service API that can be quickly called, helping to implement the demand for generative AI applications in various industries.
:::
- **Model Name:** Model name. you could get from https://cloud.baidu.com/doc/WENXINWORKSHOP/s/Nlks5zkzu preset models are mapping to an endpoint. `Model Name` will be ignored if `Endpoint` is set.
- **Qianfan Ak:** which you could get from https://cloud.baidu.com/product/wenxinworkshop.
- **Qianfan Sk:** which you could get from https://cloud.baidu.com/product/wenxinworkshop.
- **Top p:** Model params, only supported in ERNIE-Bot and ERNIE-Bot-turbo. The diversity of the output text is affected, and the larger the value, the stronger the diversity of the generated text - defaults to `0.8`.
- **Temperature:** Model params, only supported in ERNIE-Bot and ERNIE-Bot-turbo. Higher values make the output more random, while lower values make it more concentrated and deterministic - defaults to `0.95`.
- **Penalty Score:** Model params, only supported in ERNIE-Bot and ERNIE-Bot-turbo. By increasing the penalty for generated tokens, the phenomenon of duplicate generation is reduced. A higher value indicates a higher penalty - defaults to `1.0`.
- **Endpoint:** Endpoint of the Qianfan LLM, required if custom model used.

658
poetry.lock generated
View file

File diff suppressed because it is too large Load diff

3
pyproject.toml
View file

@ -33,7 +33,7 @@ google-search-results = "^2.4.1"
google-api-python-client = "^2.79.0"
typer = "^0.9.0"
gunicorn = "^21.2.0"
langchain = "^0.0.308"
langchain = "^0.0.312"
openai = "^0.27.8"
pandas = "2.0.3"
chromadb = "^0.3.21"
@ -91,6 +91,7 @@ pillow = "^10.0.0"
metal-sdk = "^2.2.0"
markupsafe = "^2.1.3"
numexpr = "^2.8.6"
qianfan = "0.0.5"
[tool.poetry.group.dev.dependencies]

92
src/backend/langflow/components/llms/BaiduQianfanChatEndpoints.py Normal file
View file

@ -0,0 +1,92 @@
from typing import Optional
from langflow import CustomComponent
from langchain.chat_models.baidu_qianfan_endpoint import QianfanChatEndpoint
from langchain.llms.base import BaseLLM
class QianfanChatEndpointComponent(CustomComponent):
display_name: str = "QianfanChatEndpoint"
description: str = (
"Baidu Qianfan chat models. Get more detail from "
"https://python.langchain.com/docs/integrations/chat/baidu_qianfan_endpoint."
)
def build_config(self):
return {
"model": {
"display_name": "Model Name",
"options": [
"ERNIE-Bot",
"ERNIE-Bot-turbo",
"BLOOMZ-7B",
"Llama-2-7b-chat",
"Llama-2-13b-chat",
"Llama-2-70b-chat",
"Qianfan-BLOOMZ-7B-compressed",
"Qianfan-Chinese-Llama-2-7B",
"ChatGLM2-6B-32K",
"AquilaChat-7B",
],
"info": "https://python.langchain.com/docs/integrations/chat/baidu_qianfan_endpoint",
"required": True,
},
"qianfan_ak": {
"display_name": "Qianfan Ak",
"required": True,
"password": True,
"info": "which you could get from https://cloud.baidu.com/product/wenxinworkshop",
},
"qianfan_sk": {
"display_name": "Qianfan Sk",
"required": True,
"password": True,
"info": "which you could get from https://cloud.baidu.com/product/wenxinworkshop",
},
"top_p": {
"display_name": "Top p",
"field_type": "float",
"info": "Model params, only supported in ERNIE-Bot and ERNIE-Bot-turbo",
"value": 0.8,
},
"temperature": {
"display_name": "Temperature",
"field_type": "float",
"info": "Model params, only supported in ERNIE-Bot and ERNIE-Bot-turbo",
"value": 0.95,
},
"penalty_score": {
"display_name": "Penalty Score",
"field_type": "float",
"info": "Model params, only supported in ERNIE-Bot and ERNIE-Bot-turbo",
"value": 1.0,
},
"endpoint": {
"display_name": "Endpoint",
"info": "Endpoint of the Qianfan LLM, required if custom model used.",
},
"code": {"show": False},
}
def build(
self,
model: str = "ERNIE-Bot-turbo",
qianfan_ak: Optional[str] = None,
qianfan_sk: Optional[str] = None,
top_p: Optional[float] = None,
temperature: Optional[float] = None,
penalty_score: Optional[float] = None,
endpoint: Optional[str] = None,
) -> BaseLLM:
try:
output = QianfanChatEndpoint( # type: ignore
model=model,
qianfan_ak=qianfan_ak,
qianfan_sk=qianfan_sk,
top_p=top_p,
temperature=temperature,
penalty_score=penalty_score,
endpoint=endpoint,
)
except Exception as e:
raise ValueError("Could not connect to Baidu Qianfan API.") from e
return output # type: ignore

92
src/backend/langflow/components/llms/BaiduQianfanLLMEndpoints.py Normal file
View file

@ -0,0 +1,92 @@
from typing import Optional
from langflow import CustomComponent
from langchain.llms.baidu_qianfan_endpoint import QianfanLLMEndpoint
from langchain.llms.base import BaseLLM
class QianfanLLMEndpointComponent(CustomComponent):
display_name: str = "QianfanLLMEndpoint"
description: str = (
"Baidu Qianfan hosted open source or customized models. "
"Get more detail from https://python.langchain.com/docs/integrations/chat/baidu_qianfan_endpoint"
)
def build_config(self):
return {
"model": {
"display_name": "Model Name",
"options": [
"ERNIE-Bot",
"ERNIE-Bot-turbo",
"BLOOMZ-7B",
"Llama-2-7b-chat",
"Llama-2-13b-chat",
"Llama-2-70b-chat",
"Qianfan-BLOOMZ-7B-compressed",
"Qianfan-Chinese-Llama-2-7B",
"ChatGLM2-6B-32K",
"AquilaChat-7B",
],
"info": "https://python.langchain.com/docs/integrations/chat/baidu_qianfan_endpoint",
"required": True,
},
"qianfan_ak": {
"display_name": "Qianfan Ak",
"required": True,
"password": True,
"info": "which you could get from https://cloud.baidu.com/product/wenxinworkshop",
},
"qianfan_sk": {
"display_name": "Qianfan Sk",
"required": True,
"password": True,
"info": "which you could get from https://cloud.baidu.com/product/wenxinworkshop",
},
"top_p": {
"display_name": "Top p",
"field_type": "float",
"info": "Model params, only supported in ERNIE-Bot and ERNIE-Bot-turbo",
"value": 0.8,
},
"temperature": {
"display_name": "Temperature",
"field_type": "float",
"info": "Model params, only supported in ERNIE-Bot and ERNIE-Bot-turbo",
"value": 0.95,
},
"penalty_score": {
"display_name": "Penalty Score",
"field_type": "float",
"info": "Model params, only supported in ERNIE-Bot and ERNIE-Bot-turbo",
"value": 1.0,
},
"endpoint": {
"display_name": "Endpoint",
"info": "Endpoint of the Qianfan LLM, required if custom model used.",
},
"code": {"show": False},
}
def build(
self,
model: str = "ERNIE-Bot-turbo",
qianfan_ak: Optional[str] = None,
qianfan_sk: Optional[str] = None,
top_p: Optional[float] = None,
temperature: Optional[float] = None,
penalty_score: Optional[float] = None,
endpoint: Optional[str] = None,
) -> BaseLLM:
try:
output = QianfanLLMEndpoint( # type: ignore
model=model,
qianfan_ak=qianfan_ak,
qianfan_sk=qianfan_sk,
top_p=top_p,
temperature=temperature,
penalty_score=penalty_score,
endpoint=endpoint,
)
except Exception as e:
raise ValueError("Could not connect to Baidu Qianfan API.") from e
return output # type: ignore

2
src/backend/langflow/template/frontend_node/base.py
View file

@ -44,7 +44,7 @@ class FieldFormatters(BaseModel):
class FrontendNode(BaseModel):
template: Template
description: str
description: Optional[str] = None
base_classes: List[str]
name: str = ""
display_name: str = ""

1301
src/frontend/package-lock.json generated
View file

File diff suppressed because it is too large Load diff

3
src/frontend/run-tests.sh
View file

@ -30,6 +30,9 @@ terminate_process_by_port() {
# Trap signals to ensure cleanup on script termination
trap 'terminate_process_by_port 7860; terminate_process_by_port 3000' EXIT
# install playwright if there is not installed yet
npx playwright install
# Navigate to the project root directory (where the Makefile is located)
cd ../../

2684
src/frontend/tests/onlyFront/assets/collection.json Normal file
View file

File diff suppressed because it is too large Load diff

96
src/frontend/tests/onlyFront/assets/flow.json Normal file
View file

@ -0,0 +1,96 @@
{
"description": "Engineered for Excellence, Built for Business.",
"name": "Fluffy Sinoussi",
"data": {
"nodes": [
{
"id": "AgentInitializer-Zza0A",
"type": "genericNode",
"position": { "x": 595, "y": 224.25 },
"data": {
"type": "AgentInitializer",
"node": {
"template": {
"llm": {
"required": true,
"placeholder": "",
"show": true,
"multiline": false,
"password": false,
"name": "llm",
"display_name": "LLM",
"advanced": false,
"dynamic": false,
"info": "",
"type": "BaseLanguageModel",
"list": false
},
"memory": {
"required": false,
"placeholder": "",
"show": true,
"multiline": false,
"password": false,
"name": "memory",
"advanced": false,
"dynamic": false,
"info": "",
"type": "BaseChatMemory",
"list": false
},
"tools": {
"required": true,
"placeholder": "",
"show": true,
"multiline": false,
"password": false,
"name": "tools",
"advanced": false,
"dynamic": false,
"info": "",
"type": "Tool",
"list": true
},
"agent": {
"required": true,
"placeholder": "",
"show": true,
"multiline": false,
"value": "zero-shot-react-description",
"password": false,
"options": [
"zero-shot-react-description",
"react-docstore",
"self-ask-with-search",
"conversational-react-description",
"openai-functions",
"openai-multi-functions"
],
"name": "agent",
"advanced": false,
"dynamic": false,
"info": "",
"type": "str",
"list": true
},
"_type": "initialize_agent"
},
"description": "Construct a zero shot agent from an LLM and tools.",
"base_classes": ["AgentExecutor", "function"],
"display_name": "AgentInitializer",
"custom_fields": {},
"output_types": [],
"documentation": "https://python.langchain.com/docs/modules/agents/agent_types/",
"beta": false,
"error": null
},
"id": "AgentInitializer-Zza0A"
},
"positionAbsolute": { "x": 595, "y": 224.25 }
}
],
"edges": [],
"viewport": { "x": 0, "y": 0, "zoom": 1 }
},
"id": "84c4b46f-063b-4d48-bf7f-6c668013064f"
}

87
src/frontend/tests/onlyFront/dragAndDrop.spec.ts Normal file
View file

@ -0,0 +1,87 @@
import { expect, test } from "@playwright/test";
import { readFileSync } from "fs";
test.describe("drag and drop test", () => {
/// <reference lib="dom"/>
test("drop collection", async ({ page }) => {
await page.routeFromHAR("harFiles/langflow.har", {
url: "**/api/v1/**",
update: false,
});
await page.route("**/api/v1/flows/", async (route) => {
const json = {
id: "e9ac1bdc-429b-475d-ac03-d26f9a2a3210",
};
await route.fulfill({ json, status: 201 });
});
await page.goto("http:localhost:3000/");
await page.locator("span").filter({ hasText: "My Collection" }).isVisible();
// Read your file into a buffer.
const jsonContent = readFileSync(
"tests/onlyFront/assets/collection.json",
"utf-8"
);
// Create the DataTransfer and File
const dataTransfer = await page.evaluateHandle((data) => {
const dt = new DataTransfer();
// Convert the buffer to a hex array
const file = new File([data], "collection.json", {
type: "application/json",
});
dt.items.add(file);
return dt;
}, jsonContent);
// Now dispatch
await page.dispatchEvent('//*[@id="root"]/div/div[2]/div[2]', "drop", {
dataTransfer,
});
expect(
await page
.locator(".main-page-flows-display")
.evaluate((el) => el.children)
).toBeTruthy();
});
test("drop flow", async ({ page }) => {
await page.routeFromHAR("harFiles/langflow.har", {
url: "**/api/v1/**",
update: false,
});
await page.route("**/api/v1/flows/", async (route) => {
const json = {
id: "e9ac1bdc-429b-475d-ac03-d26f9a2a3210",
};
await route.fulfill({ json, status: 201 });
});
await page.goto("http:localhost:3000/");
await page.locator("span").filter({ hasText: "My Collection" }).isVisible();
// Read your file into a buffer.
const jsonContent = readFileSync(
"tests/onlyFront/assets/flow.json",
"utf-8"
);
// Create the DataTransfer and File
const dataTransfer = await page.evaluateHandle((data) => {
const dt = new DataTransfer();
// Convert the buffer to a hex array
const file = new File([data], "flow.json", {
type: "application/json",
});
dt.items.add(file);
return dt;
}, jsonContent);
// Now dispatch
await page.dispatchEvent('//*[@id="root"]/div/div[2]/div[2]', "drop", {
dataTransfer,
});
expect(
await page
.locator(".main-page-flows-display")
.evaluate((el) => el.children)
).toBeTruthy();
});
});