jyapayne/langflow
1
0
Fork 0
Code Issues Pull requests Projects Releases 1 Packages Wiki Activity Actions

tests: organize playwright tests by feature suites and optimize CI runtime (#4687)

Browse source 
* starting realocating components FE

* tagging tests

* 📝 (.github/workflows/typescript_test.yml): remove unnecessary comment before the Playwright test execution step to improve readability and maintainability

* 📝 (.github/changes-filter.yaml): Update paths in changes-filter.yaml to categorize different sections of the project for testing purposes
🔧 (.github/workflows/typescript_test.yml): Refactor workflow to use paths-filter action for determining test suites based on changes in different project sections

* ✨ (changes-filter.yaml): reorganize and update paths for test categories, starter projects, components, workspace, api, and database
✨ (components): add new components for GradientWrapper, pageLayout, and paginatorComponent
🔧 (MainPage): update import path for PageLayout component
🔧 (SettingsPage): update import path for PageLayout component
🔧 (StorePage): update import path for PageLayout component

* ✨ (frontend): Update import paths for components to use common directory instead of core directory for better organization and consistency
🔧 (frontend): Remove duplicate import of PaginatorComponent in various files and import it from the common directory to reduce redundancy and improve maintainability

* 🔧 (changes-filter.yaml): remove unnecessary release section from changes-filter.yaml
🔧 (typescript_test.yml): refactor workflow to support input suites and release flag for test runs

* 📝 (changes-filter.yaml): remove unnecessary entries from components section and add missing entries to workspace section for better organization and filtering of changes
🔧 (changes-filter.yaml): update changes-filter.yaml to include missing utils files in the frontend/src/utils directory
🔧 (typescript_test.yml): remove redundant code block that sets SUITES variable to default values as it is no longer needed due to changes in the changes-filter.yaml file

* ✨ (actionsMainPage-shard-1.spec.ts): Update test tags to remove unnecessary tags for better organization and clarity
📝 (store-shard-2.spec.ts): Update test tags to remove unnecessary tags for better organization and clarity
📝 (generalBugs-shard-4.spec.ts): Update test tags to remove unnecessary tags for better organization and clarity
📝 (generalBugs-shard-9.spec.ts): Update test tags to remove unnecessary tags for better organization and clarity
📝 (deleteComponents.spec.ts): Update test tags to remove unnecessary tags for better organization and clarity
📝 (deleteFlows.spec.ts): Update test tags to remove unnecessary tags for better organization and clarity

✨ (dragAndDrop.spec.ts): Refactor drag and drop test to improve readability and maintainability
✨ (flowPage.spec.ts): Refactor save test in Flow Page to improve readability and maintainability
✨ (store-shard-0.spec.ts): Refactor Store test to improve readability and maintainability
✨ (store-shard-1.spec.ts): Refactor Store test to improve readability and maintainability

✨ (store-shard-3.spec.ts): Remove unnecessary tags from test descriptions to improve clarity and consistency
✨ (userSettings.spec.ts): Remove unnecessary tags from test descriptions to improve clarity and consistency
✨ (general-bugs-shard-3909.spec.ts): Remove unnecessary tags from test descriptions to improve clarity and consistency

📝 (frontend): Update test tags in regression tests to remove unnecessary tags for better organization and clarity
📝 (frontend): Update test file paths in tsconfig.json to match the new file structure for consistency and maintainability

* [autofix.ci] apply automated fixes

* [autofix.ci] apply automated fixes

* Update paths-filter action to v3 and adjust filter configuration file

* Add logging to matrix setup in TypeScript test workflow

* Refactor decisionFlow.spec.ts to improve test setup and execution flow

* Refactor import paths for genericIconComponent and shadTooltipComponent in PlaygroundButton component

* feat: add release input to CI workflows for enhanced deployment control

* Fix: Correct 'release' input type in CI workflow configuration

* [autofix.ci] apply automated fixes

* Refactor Blog Writer test: streamline setup and interaction steps

* Refactor test cases to improve modal handling and streamline API key input logic

* Add test grep pattern to GitHub Actions workflow for targeted test execution

- Introduced `test_grep` output in the `determine-test-suite` job to generate a grep pattern based on changed files or input suites.
- Updated Playwright test execution to use the generated grep pattern, allowing for more targeted test runs.
- Enhanced the `Set Matrix` step to include tag processing for test filtering.

* Enhance tag filtering in test workflow with regex lookaheads

* Fix syntax for default suite input in GitHub Actions workflow

* Update GitHub Actions to handle empty suite input for release builds

* Escape regex lookaheads and quotes for GitHub Actions in test workflow

* Update regex pattern and quote escaping in TypeScript test workflow

* Use single quotes for echo command in GitHub Actions script

* Refactor tag handling in GitHub Actions to use OR logic for grep patterns

* 🔧 (FlowPage/index.tsx): Remove unnecessary empty line to improve code readability and consistency

* 📝 (FlowPage/index.tsx): add missing line break for better code readability

* Refactor import path and reorder CSS classes for consistency in UpdateAllComponents component

* [autofix.ci] apply automated fixes

* Update import path for PaginatorComponent in AdminPage index.tsx

* ✨ (logs.spec.ts): introduce a new variable 'modalCount' to keep track of the number of modals opened in the test suite

* 📝 (filterSidebar.spec.ts): remove unnecessary test steps related to filter removal and sidebar interactions to simplify and improve test focus on main functionality

* ✅ (logs.spec.ts): add tests to ensure proper interaction with logs feature in the frontend application

* ✅ (similarity.spec.ts): add a test tag for release and components to categorize the test case
💡 (similarity.spec.ts): add a comment to explain the purpose of the test case and the condition for skipping the test

* ✅ (decisionFlow.spec.ts): add missing newline before async function declaration to improve code readability

* ✅ (textInputOutput.spec.ts): skip the test for TextInputOutputComponent to exclude it from the test suite temporarily

---------

Co-authored-by: Gabriel Luiz Freitas Almeida <gabriel@langflow.org>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
This commit is contained in:
Cristhian Zanforlin Lousa 2024-11-25 17:55:03 -03:00 • committed by GitHub
commit 989cefd0ac
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
331 changed files with 10386 additions and 9694 deletions
Download patch file Download diff file Expand all files Collapse all files

202
src/frontend/tests/core/integrations/Basic Prompting.spec.ts
View file

@ -2,126 +2,132 @@ import { test } from "@playwright/test";
import * as dotenv from "dotenv";
import path from "path";
test("Basic Prompting (Hello, World)", async ({ page }) => {
test.skip(
!process?.env?.OPENAI_API_KEY,
"OPENAI_API_KEY required to run this test",
);
test(
"Basic Prompting (Hello, World)",
{ tag: ["@release", "@starter-project"] },
async ({ page }) => {
test.skip(
!process?.env?.OPENAI_API_KEY,
"OPENAI_API_KEY required to run this test",
);
if (!process.env.CI) {
dotenv.config({ path: path.resolve(__dirname, "../../.env") });
}
await page.goto("/");
await page.waitForSelector('[data-testid="mainpage_title"]', {
timeout: 30000,
});
await page.waitForSelector('[id="new-project-btn"]', {
timeout: 30000,
});
let modalCount = 0;
try {
const modalTitleElement = await page?.getByTestId("modal-title");
if (modalTitleElement) {
modalCount = await modalTitleElement.count();
if (!process.env.CI) {
dotenv.config({ path: path.resolve(__dirname, "../../.env") });
}
} catch (error) {
modalCount = 0;
}
while (modalCount === 0) {
await page.getByText("New Flow", { exact: true }).click();
await page.waitForSelector('[data-testid="modal-title"]', {
timeout: 3000,
await page.goto("/");
await page.waitForSelector('[data-testid="mainpage_title"]', {
timeout: 30000,
});
modalCount = await page.getByTestId("modal-title")?.count();
}
await page.getByTestId("side_nav_options_all-templates").click();
await page.getByRole("heading", { name: "Basic Prompting" }).click();
await page.waitForSelector('[id="new-project-btn"]', {
timeout: 30000,
});
await page.waitForSelector('[data-testid="fit_view"]', {
timeout: 100000,
});
let modalCount = 0;
try {
const modalTitleElement = await page?.getByTestId("modal-title");
if (modalTitleElement) {
modalCount = await modalTitleElement.count();
}
} catch (error) {
modalCount = 0;
}
await page.getByTestId("fit_view").click();
await page.getByTestId("zoom_out").click();
await page.getByTestId("zoom_out").click();
await page.getByTestId("zoom_out").click();
while (modalCount === 0) {
await page.getByText("New Flow", { exact: true }).click();
await page.waitForSelector('[data-testid="modal-title"]', {
timeout: 3000,
});
modalCount = await page.getByTestId("modal-title")?.count();
}
let outdatedComponents = await page.getByTestId("icon-AlertTriangle").count();
await page.getByTestId("side_nav_options_all-templates").click();
await page.getByRole("heading", { name: "Basic Prompting" }).click();
while (outdatedComponents > 0) {
await page.getByTestId("icon-AlertTriangle").first().click();
outdatedComponents = await page.getByTestId("icon-AlertTriangle").count();
}
await page.waitForSelector('[data-testid="fit_view"]', {
timeout: 100000,
});
let filledApiKey = await page.getByTestId("remove-icon-badge").count();
while (filledApiKey > 0) {
await page.getByTestId("remove-icon-badge").first().click();
filledApiKey = await page.getByTestId("remove-icon-badge").count();
}
await page.getByTestId("fit_view").click();
await page.getByTestId("zoom_out").click();
await page.getByTestId("zoom_out").click();
await page.getByTestId("zoom_out").click();
const apiKeyInput = page.getByTestId("popover-anchor-input-api_key");
const isApiKeyInputVisible = await apiKeyInput.isVisible();
let outdatedComponents = await page
.getByTestId("icon-AlertTriangle")
.count();
if (isApiKeyInputVisible) {
await apiKeyInput.fill(process.env.OPENAI_API_KEY ?? "");
}
while (outdatedComponents > 0) {
await page.getByTestId("icon-AlertTriangle").first().click();
outdatedComponents = await page.getByTestId("icon-AlertTriangle").count();
}
await page.getByTestId("dropdown_str_model_name").click();
await page.getByTestId("gpt-4o-1-option").click();
let filledApiKey = await page.getByTestId("remove-icon-badge").count();
while (filledApiKey > 0) {
await page.getByTestId("remove-icon-badge").first().click();
filledApiKey = await page.getByTestId("remove-icon-badge").count();
}
await page.getByTestId("button_run_chat output").click();
await page.waitForSelector("text=built successfully", { timeout: 30000 });
const apiKeyInput = page.getByTestId("popover-anchor-input-api_key");
const isApiKeyInputVisible = await apiKeyInput.isVisible();
await page.getByText("built successfully").last().click({
timeout: 15000,
});
if (isApiKeyInputVisible) {
await apiKeyInput.fill(process.env.OPENAI_API_KEY ?? "");
}
await page.getByText("Playground", { exact: true }).last().click();
await page
.getByText("No input message provided.", { exact: true })
.last()
.isVisible();
await page.getByTestId("dropdown_str_model_name").click();
await page.getByTestId("gpt-4o-1-option").click();
await page.waitForSelector('[data-testid="input-chat-playground"]', {
timeout: 100000,
});
await page.getByTestId("button_run_chat output").click();
await page.waitForSelector("text=built successfully", { timeout: 30000 });
await page
.getByTestId("input-chat-playground")
.last()
.fill("Say hello as a pirate");
await page.getByText("built successfully").last().click({
timeout: 15000,
});
await page.waitForSelector('[data-testid="button-send"]', {
timeout: 100000,
});
await page.getByText("Playground", { exact: true }).last().click();
await page
.getByText("No input message provided.", { exact: true })
.last()
.isVisible();
await page.getByTestId("button-send").last().click();
await page.waitForSelector('[data-testid="input-chat-playground"]', {
timeout: 100000,
});
await page.waitForSelector("text=matey", {
timeout: 100000,
});
await page
.getByTestId("input-chat-playground")
.last()
.fill("Say hello as a pirate");
await page.getByText("matey").last().isVisible();
await page.getByText("Default Session").last().click();
await page.waitForSelector('[data-testid="button-send"]', {
timeout: 100000,
});
await page.getByText("timestamp", { exact: true }).last().isVisible();
await page.getByText("text", { exact: true }).last().isVisible();
await page.getByText("sender", { exact: true }).last().isVisible();
await page.getByText("sender_name", { exact: true }).last().isVisible();
await page.getByText("session_id", { exact: true }).last().isVisible();
await page.getByText("files", { exact: true }).last().isVisible();
await page.getByTestId("button-send").last().click();
await page.getByRole("gridcell").last().isVisible();
await page.getByRole("combobox").click();
await page.getByLabel("Delete").click();
await page.waitForSelector('[data-testid="input-chat-playground"]', {
timeout: 100000,
});
await page.waitForSelector("text=matey", {
timeout: 100000,
});
await page.getByTestId("input-chat-playground").last().isVisible();
});
await page.getByText("matey").last().isVisible();
await page.getByText("Default Session").last().click();
await page.getByText("timestamp", { exact: true }).last().isVisible();
await page.getByText("text", { exact: true }).last().isVisible();
await page.getByText("sender", { exact: true }).last().isVisible();
await page.getByText("sender_name", { exact: true }).last().isVisible();
await page.getByText("session_id", { exact: true }).last().isVisible();
await page.getByText("files", { exact: true }).last().isVisible();
await page.getByRole("gridcell").last().isVisible();
await page.getByRole("combobox").click();
await page.getByLabel("Delete").click();
await page.waitForSelector('[data-testid="input-chat-playground"]', {
timeout: 100000,
});
await page.getByTestId("input-chat-playground").last().isVisible();
},
);

188
src/frontend/tests/core/integrations/Blog Writer.spec.ts
View file

@ -1,113 +1,119 @@
import { expect, test } from "@playwright/test";
import { test } from "@playwright/test";
import * as dotenv from "dotenv";
import path from "path";
test("Blog Writer", async ({ page }) => {
test.skip(
!process?.env?.OPENAI_API_KEY,
"OPENAI_API_KEY required to run this test",
);
test(
"Blog Writer",
{ tag: ["@release", "@starter-project"] },
async ({ page }) => {
test.skip(
!process?.env?.OPENAI_API_KEY,
"OPENAI_API_KEY required to run this test",
);
if (!process.env.CI) {
dotenv.config({ path: path.resolve(__dirname, "../../.env") });
}
await page.goto("/");
await page.waitForSelector('[data-testid="mainpage_title"]', {
timeout: 30000,
});
await page.waitForSelector('[id="new-project-btn"]', {
timeout: 30000,
});
let modalCount = 0;
try {
const modalTitleElement = await page?.getByTestId("modal-title");
if (modalTitleElement) {
modalCount = await modalTitleElement.count();
if (!process.env.CI) {
dotenv.config({ path: path.resolve(__dirname, "../../.env") });
}
} catch (error) {
modalCount = 0;
}
while (modalCount === 0) {
await page.getByText("New Flow", { exact: true }).click();
await page.waitForSelector('[data-testid="modal-title"]', {
timeout: 3000,
await page.goto("/");
await page.waitForSelector('[data-testid="mainpage_title"]', {
timeout: 30000,
});
modalCount = await page.getByTestId("modal-title")?.count();
}
await page.getByTestId("side_nav_options_all-templates").click();
await page.getByRole("heading", { name: "Blog Writer" }).click();
await page.waitForSelector('[data-testid="fit_view"]', {
timeout: 5000,
});
await page.waitForSelector('[id="new-project-btn"]', {
timeout: 30000,
});
await page.getByTestId("fit_view").click();
await page.getByTestId("zoom_out").click();
await page.getByTestId("zoom_out").click();
await page.getByTestId("zoom_out").click();
let modalCount = 0;
try {
const modalTitleElement = await page?.getByTestId("modal-title");
if (modalTitleElement) {
modalCount = await modalTitleElement.count();
}
} catch (error) {
modalCount = 0;
}
let outdatedComponents = await page.getByTestId("icon-AlertTriangle").count();
while (modalCount === 0) {
await page.getByText("New Flow", { exact: true }).click();
await page.waitForSelector('[data-testid="modal-title"]', {
timeout: 3000,
});
modalCount = await page.getByTestId("modal-title")?.count();
}
while (outdatedComponents > 0) {
await page.getByTestId("icon-AlertTriangle").first().click();
outdatedComponents = await page.getByTestId("icon-AlertTriangle").count();
}
await page.getByTestId("side_nav_options_all-templates").click();
await page.getByRole("heading", { name: "Blog Writer" }).click();
await page.waitForSelector('[data-testid="fit_view"]', {
timeout: 5000,
});
let filledApiKey = await page.getByTestId("remove-icon-badge").count();
while (filledApiKey > 0) {
await page.getByTestId("remove-icon-badge").first().click();
filledApiKey = await page.getByTestId("remove-icon-badge").count();
}
await page.getByTestId("fit_view").click();
await page.getByTestId("zoom_out").click();
await page.getByTestId("zoom_out").click();
await page.getByTestId("zoom_out").click();
const apiKeyInput = page.getByTestId("popover-anchor-input-api_key");
const isApiKeyInputVisible = await apiKeyInput.isVisible();
let outdatedComponents = await page
.getByTestId("icon-AlertTriangle")
.count();
if (isApiKeyInputVisible) {
await apiKeyInput.fill(process.env.OPENAI_API_KEY ?? "");
}
while (outdatedComponents > 0) {
await page.getByTestId("icon-AlertTriangle").first().click();
outdatedComponents = await page.getByTestId("icon-AlertTriangle").count();
}
await page.getByTestId("dropdown_str_model_name").click();
await page.getByTestId("gpt-4o-1-option").click();
let filledApiKey = await page.getByTestId("remove-icon-badge").count();
while (filledApiKey > 0) {
await page.getByTestId("remove-icon-badge").first().click();
filledApiKey = await page.getByTestId("remove-icon-badge").count();
}
await page
.getByTestId("inputlist_str_urls_0")
.nth(0)
.fill(
"https://www.natgeokids.com/uk/discover/animals/sea-life/turtle-facts/",
);
await page
.getByTestId("inputlist_str_urls_1")
.nth(0)
.fill("https://www.originaldiving.com/blog/top-ten-turtle-facts");
const apiKeyInput = page.getByTestId("popover-anchor-input-api_key");
const isApiKeyInputVisible = await apiKeyInput.isVisible();
await page
.getByTestId("textarea_str_input_value")
.fill(
"Use the references above for style to write a new blog/tutorial about turtles. Suggest non-covered topics.",
);
if (isApiKeyInputVisible) {
await apiKeyInput.fill(process.env.OPENAI_API_KEY ?? "");
}
await page.getByTestId("button_run_chat output").click();
await page.getByTestId("dropdown_str_model_name").click();
await page.getByTestId("gpt-4o-1-option").click();
await page.waitForSelector("text=built successfully", { timeout: 30000 });
await page
.getByTestId("inputlist_str_urls_0")
.nth(0)
.fill(
"https://www.natgeokids.com/uk/discover/animals/sea-life/turtle-facts/",
);
await page
.getByTestId("inputlist_str_urls_1")
.nth(0)
.fill("https://www.originaldiving.com/blog/top-ten-turtle-facts");
await page.getByText("built successfully").last().click({
timeout: 30000,
});
await page
.getByTestId("textarea_str_input_value")
.fill(
"Use the references above for style to write a new blog/tutorial about turtles. Suggest non-covered topics.",
);
await page.getByText("Playground", { exact: true }).last().click();
await page
.getByPlaceholder(
"No chat input variables found. Click to run your flow.",
{ exact: true },
)
.last()
.isVisible();
await page.getByTestId("button_run_chat output").click();
await page.getByText("turtles").last().isVisible();
await page.getByText("sea").last().isVisible();
await page.getByText("survival").last().isVisible();
});
await page.waitForSelector("text=built successfully", { timeout: 30000 });
await page.getByText("built successfully").last().click({
timeout: 30000,
});
await page.getByText("Playground", { exact: true }).last().click();
await page
.getByPlaceholder(
"No chat input variables found. Click to run your flow.",
{ exact: true },
)
.last()
.isVisible();
await page.getByText("turtles").last().isVisible();
await page.getByText("sea").last().isVisible();
await page.getByText("survival").last().isVisible();
},
);

216
src/frontend/tests/core/integrations/Document QA.spec.ts
View file

@ -2,132 +2,138 @@ import { test } from "@playwright/test";
import * as dotenv from "dotenv";
import path from "path";
test("Document Q&A", async ({ page }) => {
test.skip(
!process?.env?.OPENAI_API_KEY,
"OPENAI_API_KEY required to run this test",
);
test(
"Document Q&A",
{ tag: ["@release", "@starter-project"] },
async ({ page }) => {
test.skip(
!process?.env?.OPENAI_API_KEY,
"OPENAI_API_KEY required to run this test",
);
if (!process.env.CI) {
dotenv.config({ path: path.resolve(__dirname, "../../.env") });
}
await page.goto("/");
await page.waitForSelector('[data-testid="mainpage_title"]', {
timeout: 30000,
});
await page.waitForSelector('[id="new-project-btn"]', {
timeout: 30000,
});
let modalCount = 0;
try {
const modalTitleElement = await page?.getByTestId("modal-title");
if (modalTitleElement) {
modalCount = await modalTitleElement.count();
if (!process.env.CI) {
dotenv.config({ path: path.resolve(__dirname, "../../.env") });
}
} catch (error) {
modalCount = 0;
}
while (modalCount === 0) {
await page.getByText("New Flow", { exact: true }).click();
await page.waitForSelector('[data-testid="modal-title"]', {
await page.goto("/");
await page.waitForSelector('[data-testid="mainpage_title"]', {
timeout: 30000,
});
await page.waitForSelector('[id="new-project-btn"]', {
timeout: 30000,
});
let modalCount = 0;
try {
const modalTitleElement = await page?.getByTestId("modal-title");
if (modalTitleElement) {
modalCount = await modalTitleElement.count();
}
} catch (error) {
modalCount = 0;
}
while (modalCount === 0) {
await page.getByText("New Flow", { exact: true }).click();
await page.waitForSelector('[data-testid="modal-title"]', {
timeout: 3000,
});
modalCount = await page.getByTestId("modal-title")?.count();
}
await page.getByTestId("side_nav_options_all-templates").click();
await page.getByRole("heading", { name: "Document Q&A" }).click();
await page.waitForSelector('[data-testid="fit_view"]', {
timeout: 3000,
});
modalCount = await page.getByTestId("modal-title")?.count();
}
await page.getByTestId("side_nav_options_all-templates").click();
await page.getByRole("heading", { name: "Document Q&A" }).click();
await page.waitForSelector('[data-testid="fit_view"]', {
timeout: 3000,
});
await page.getByTestId("fit_view").click();
await page.getByTestId("zoom_out").click();
await page.getByTestId("zoom_out").click();
await page.getByTestId("zoom_out").click();
await page.getByTestId("fit_view").click();
await page.getByTestId("zoom_out").click();
await page.getByTestId("zoom_out").click();
await page.getByTestId("zoom_out").click();
let outdatedComponents = await page
.getByTestId("icon-AlertTriangle")
.count();
let outdatedComponents = await page.getByTestId("icon-AlertTriangle").count();
while (outdatedComponents > 0) {
await page.getByTestId("icon-AlertTriangle").first().click();
outdatedComponents = await page.getByTestId("icon-AlertTriangle").count();
}
while (outdatedComponents > 0) {
await page.getByTestId("icon-AlertTriangle").first().click();
outdatedComponents = await page.getByTestId("icon-AlertTriangle").count();
}
let filledApiKey = await page.getByTestId("remove-icon-badge").count();
while (filledApiKey > 0) {
await page.getByTestId("remove-icon-badge").first().click();
filledApiKey = await page.getByTestId("remove-icon-badge").count();
}
let filledApiKey = await page.getByTestId("remove-icon-badge").count();
while (filledApiKey > 0) {
await page.getByTestId("remove-icon-badge").first().click();
filledApiKey = await page.getByTestId("remove-icon-badge").count();
}
const apiKeyInput = page.getByTestId("popover-anchor-input-api_key");
const isApiKeyInputVisible = await apiKeyInput.isVisible();
const apiKeyInput = page.getByTestId("popover-anchor-input-api_key");
const isApiKeyInputVisible = await apiKeyInput.isVisible();
if (isApiKeyInputVisible) {
await apiKeyInput.fill(process.env.OPENAI_API_KEY ?? "");
}
if (isApiKeyInputVisible) {
await apiKeyInput.fill(process.env.OPENAI_API_KEY ?? "");
}
await page.getByTestId("dropdown_str_model_name").click();
await page.getByTestId("gpt-4o-1-option").click();
await page.getByTestId("dropdown_str_model_name").click();
await page.getByTestId("gpt-4o-1-option").click();
const fileChooserPromise = page.waitForEvent("filechooser");
await page.getByTestId("button_upload_file").click();
const fileChooser = await fileChooserPromise;
await fileChooser.setFiles(
path.join(__dirname, "../../assets/test_file.txt"),
);
await page.getByText("test_file.txt").isVisible();
const fileChooserPromise = page.waitForEvent("filechooser");
await page.getByTestId("button_upload_file").click();
const fileChooser = await fileChooserPromise;
await fileChooser.setFiles(
path.join(__dirname, "../../assets/test_file.txt"),
);
await page.getByText("test_file.txt").isVisible();
await page.waitForSelector('[data-testid="button_run_chat output"]', {
timeout: 3000,
});
await page.waitForSelector('[data-testid="button_run_chat output"]', {
timeout: 3000,
});
await page.getByTestId("button_run_chat output").click();
await page.waitForSelector("text=built successfully", { timeout: 30000 });
await page.getByTestId("button_run_chat output").click();
await page.waitForSelector("text=built successfully", { timeout: 30000 });
await page.getByText("built successfully").last().click({
timeout: 15000,
});
await page.getByText("built successfully").last().click({
timeout: 15000,
});
await page.getByText("Playground", { exact: true }).last().click();
await page
.getByText("No input message provided.", { exact: true })
.last()
.isVisible();
await page.getByText("Playground", { exact: true }).last().click();
await page
.getByText("No input message provided.", { exact: true })
.last()
.isVisible();
await page.waitForSelector('[data-testid="input-chat-playground"]', {
timeout: 100000,
});
await page
.getByTestId("input-chat-playground")
.last()
.fill("whats the text in the file?");
await page.getByTestId("button-send").last().click();
await page.waitForSelector('[data-testid="input-chat-playground"]', {
timeout: 100000,
});
await page
.getByTestId("input-chat-playground")
.last()
.fill("whats the text in the file?");
await page.getByTestId("button-send").last().click();
await page.waitForSelector("text=this is a test file", {
timeout: 10000,
});
await page.waitForSelector("text=this is a test file", {
timeout: 10000,
});
await page.getByText("this is a test file").last().isVisible();
await page.getByText("this is a test file").last().isVisible();
await page.getByText("Default Session").last().click();
await page.getByText("Default Session").last().click();
await page.getByText("timestamp", { exact: true }).last().isVisible();
await page.getByText("text", { exact: true }).last().isVisible();
await page.getByText("sender", { exact: true }).last().isVisible();
await page.getByText("sender_name", { exact: true }).last().isVisible();
await page.getByText("session_id", { exact: true }).last().isVisible();
await page.getByText("files", { exact: true }).last().isVisible();
await page.getByText("timestamp", { exact: true }).last().isVisible();
await page.getByText("text", { exact: true }).last().isVisible();
await page.getByText("sender", { exact: true }).last().isVisible();
await page.getByText("sender_name", { exact: true }).last().isVisible();
await page.getByText("session_id", { exact: true }).last().isVisible();
await page.getByText("files", { exact: true }).last().isVisible();
await page.getByRole("gridcell").last().isVisible();
await page.getByRole("combobox").click();
await page.getByLabel("Delete").click();
await page.waitForSelector('[data-testid="input-chat-playground"]', {
timeout: 100000,
});
await page.getByRole("gridcell").last().isVisible();
await page.getByRole("combobox").click();
await page.getByLabel("Delete").click();
await page.waitForSelector('[data-testid="input-chat-playground"]', {
timeout: 100000,
});
await page.getByTestId("input-chat-playground").last().isVisible();
});
await page.getByTestId("input-chat-playground").last().isVisible();
},
);

190
src/frontend/tests/core/integrations/Dynamic Agent.spec.ts
View file

@ -2,99 +2,107 @@ import { expect, test } from "@playwright/test";
import * as dotenv from "dotenv";
import path from "path";
test.skip("Dynamic Agent", async ({ page }) => {
test.skip(
!process?.env?.OPENAI_API_KEY,
"OPENAI_API_KEY required to run this test",
);
test.skip(
!process?.env?.SEARCH_API_KEY,
"SEARCH_API_KEY required to run this test",
);
if (!process.env.CI) {
dotenv.config({ path: path.resolve(__dirname, "../../.env") });
}
await page.goto("/");
await page.waitForSelector('[data-testid="mainpage_title"]', {
timeout: 30000,
});
await page.waitForSelector('[id="new-project-btn"]', {
timeout: 30000,
});
let modalCount = 0;
try {
const modalTitleElement = await page?.getByTestId("modal-title");
if (modalTitleElement) {
modalCount = await modalTitleElement.count();
test.skip(
"Dynamic Agent",
{ tag: ["@release", "@starter-project"] },
async ({ page }) => {
test.skip(
!process?.env?.OPENAI_API_KEY,
"OPENAI_API_KEY required to run this test",
);
test.skip(
!process?.env?.SEARCH_API_KEY,
"SEARCH_API_KEY required to run this test",
);
if (!process.env.CI) {
dotenv.config({ path: path.resolve(__dirname, "../../.env") });
}
await page.goto("/");
await page.waitForSelector('[data-testid="mainpage_title"]', {
timeout: 30000,
});
await page.waitForSelector('[id="new-project-btn"]', {
timeout: 30000,
});
let modalCount = 0;
try {
const modalTitleElement = await page?.getByTestId("modal-title");
if (modalTitleElement) {
modalCount = await modalTitleElement.count();
}
} catch (error) {
modalCount = 0;
}
while (modalCount === 0) {
await page.getByText("New Flow", { exact: true }).click();
await page.waitForTimeout(3000);
modalCount = await page.getByTestId("modal-title")?.count();
}
await page.getByTestId("side_nav_options_all-templates").click();
await page.getByRole("heading", { name: "Dynamic Agent" }).last().click();
await page.waitForSelector('[data-testid="fit_view"]', {
timeout: 100000,
});
await page.getByTestId("fit_view").click();
await page.getByTestId("zoom_out").click();
await page.getByTestId("zoom_out").click();
await page.getByTestId("zoom_out").click();
let outdatedComponents = await page
.getByTestId("icon-AlertTriangle")
.count();
while (outdatedComponents > 0) {
await page.getByTestId("icon-AlertTriangle").first().click();
await page.waitForTimeout(1000);
outdatedComponents = await page.getByTestId("icon-AlertTriangle").count();
}
let filledApiKey = await page.getByTestId("remove-icon-badge").count();
while (filledApiKey > 0) {
await page.getByTestId("remove-icon-badge").first().click();
await page.waitForTimeout(1000);
filledApiKey = await page.getByTestId("remove-icon-badge").count();
}
} catch (error) {
modalCount = 0;
}
while (modalCount === 0) {
await page.getByText("New Flow", { exact: true }).click();
await page.waitForTimeout(3000);
modalCount = await page.getByTestId("modal-title")?.count();
}
await page.getByTestId("side_nav_options_all-templates").click();
await page.getByRole("heading", { name: "Dynamic Agent" }).last().click();
await page.waitForSelector('[data-testid="fit_view"]', {
timeout: 100000,
});
await page.getByTestId("fit_view").click();
await page.getByTestId("zoom_out").click();
await page.getByTestId("zoom_out").click();
await page.getByTestId("zoom_out").click();
let outdatedComponents = await page.getByTestId("icon-AlertTriangle").count();
while (outdatedComponents > 0) {
await page.getByTestId("icon-AlertTriangle").first().click();
await page.waitForTimeout(1000);
outdatedComponents = await page.getByTestId("icon-AlertTriangle").count();
}
let filledApiKey = await page.getByTestId("remove-icon-badge").count();
while (filledApiKey > 0) {
await page.getByTestId("remove-icon-badge").first().click();
await page.waitForTimeout(1000);
filledApiKey = await page.getByTestId("remove-icon-badge").count();
}
await page
.getByTestId("popover-anchor-input-api_key")
.last()
.fill(process.env.SEARCH_API_KEY ?? "");
await page.waitForTimeout(1000);
let openAiLlms = await page.getByText("OpenAI", { exact: true }).count();
await page.waitForSelector('[data-testid="fit_view"]', {
timeout: 100000,
});
for (let i = 0; i < openAiLlms; i++) {
await page
.getByTestId("popover-anchor-input-api_key")
.nth(i)
.fill(process.env.OPENAI_API_KEY ?? "");
await page.getByTestId("zoom_in").click();
await page.getByTestId("dropdown_str_model_name").nth(i).click();
await page.getByTestId("gpt-4o-1-option").last().click();
.last()
.fill(process.env.SEARCH_API_KEY ?? "");
await page.waitForTimeout(1000);
}
await page
.getByTestId("textarea_str_input_value")
.first()
.fill("how much is an apple stock today");
await page.getByTestId("button_run_chat output").click();
let openAiLlms = await page.getByText("OpenAI", { exact: true }).count();
await page.waitForSelector('[data-testid="fit_view"]', {
timeout: 100000,
});
for (let i = 0; i < openAiLlms; i++) {
await page
.getByTestId("popover-anchor-input-api_key")
.nth(i)
.fill(process.env.OPENAI_API_KEY ?? "");
await page.getByTestId("zoom_in").click();
await page.getByTestId("dropdown_str_model_name").nth(i).click();
await page.getByTestId("gpt-4o-1-option").last().click();
await page.waitForTimeout(1000);
}
await page
.getByTestId("textarea_str_input_value")
.first()
.fill("how much is an apple stock today");
await page.getByTestId("button_run_chat output").click();
await page.waitForSelector("text=built successfully", { timeout: 60000 * 3 });
await page.getByText("built successfully").last().click({
timeout: 15000,
});
await page.getByText("Playground", { exact: true }).last().click();
await page.waitForTimeout(1000);
expect(page.getByText("apple").last()).toBeVisible();
const textContents = await page
.getByTestId("div-chat-message")
.allTextContents();
const concatAllText = textContents.join(" ");
expect(concatAllText.toLocaleLowerCase()).toContain("apple");
expect(concatAllText.toLocaleLowerCase()).not.toContain("error");
expect(concatAllText.toLocaleLowerCase()).not.toContain("apologize");
const allTextLength = concatAllText.length;
expect(allTextLength).toBeGreaterThan(100);
});
await page.waitForSelector("text=built successfully", {
timeout: 60000 * 3,
});
await page.getByText("built successfully").last().click({
timeout: 15000,
});
await page.getByText("Playground", { exact: true }).last().click();
await page.waitForTimeout(1000);
expect(page.getByText("apple").last()).toBeVisible();
const textContents = await page
.getByTestId("div-chat-message")
.allTextContents();
const concatAllText = textContents.join(" ");
expect(concatAllText.toLocaleLowerCase()).toContain("apple");
expect(concatAllText.toLocaleLowerCase()).not.toContain("error");
expect(concatAllText.toLocaleLowerCase()).not.toContain("apologize");
const allTextLength = concatAllText.length;
expect(allTextLength).toBeGreaterThan(100);
},
);

194
src/frontend/tests/core/integrations/Hierarchical Agent.spec.ts
View file

@ -3,115 +3,121 @@ import * as dotenv from "dotenv";
import path from "path";
import uaParser from "ua-parser-js";
test.skip("Hierarchical Tasks Agent", async ({ page }) => {
test.skip(
!process?.env?.OPENAI_API_KEY,
"OPENAI_API_KEY required to run this test",
);
test.skip(
"Hierarchical Tasks Agent",
{ tag: ["@release", "@starter-project"] },
async ({ page }) => {
test.skip(
!process?.env?.OPENAI_API_KEY,
"OPENAI_API_KEY required to run this test",
);
test.skip(
!process?.env?.SEARCH_API_KEY,
"SEARCH_API_KEY required to run this test",
);
test.skip(
!process?.env?.SEARCH_API_KEY,
"SEARCH_API_KEY required to run this test",
);
if (!process.env.CI) {
dotenv.config({ path: path.resolve(__dirname, "../../.env") });
}
await page.goto("/");
await page.waitForSelector('[data-testid="mainpage_title"]', {
timeout: 30000,
});
await page.waitForSelector('[id="new-project-btn"]', {
timeout: 30000,
});
let modalCount = 0;
try {
const modalTitleElement = await page?.getByTestId("modal-title");
if (modalTitleElement) {
modalCount = await modalTitleElement.count();
if (!process.env.CI) {
dotenv.config({ path: path.resolve(__dirname, "../../.env") });
}
} catch (error) {
modalCount = 0;
}
while (modalCount === 0) {
await page.getByText("New Flow", { exact: true }).click();
await page.waitForTimeout(3000);
modalCount = await page.getByTestId("modal-title")?.count();
}
await page.goto("/");
await page.waitForSelector('[data-testid="mainpage_title"]', {
timeout: 30000,
});
const getUA = await page.evaluate(() => navigator.userAgent);
const userAgentInfo = uaParser(getUA);
await page.waitForSelector('[id="new-project-btn"]', {
timeout: 30000,
});
await page.getByTestId("side_nav_options_all-templates").click();
await page
.getByRole("heading", { name: "Hierarchical Tasks Agent" })
.first()
.click();
let modalCount = 0;
try {
const modalTitleElement = await page?.getByTestId("modal-title");
if (modalTitleElement) {
modalCount = await modalTitleElement.count();
}
} catch (error) {
modalCount = 0;
}
await page.waitForSelector('[data-testid="fit_view"]', {
timeout: 100000,
});
while (modalCount === 0) {
await page.getByText("New Flow", { exact: true }).click();
await page.waitForTimeout(3000);
modalCount = await page.getByTestId("modal-title")?.count();
}
await page.getByTestId("fit_view").click();
await page.getByTestId("zoom_out").click();
await page.getByTestId("zoom_out").click();
await page.getByTestId("zoom_out").click();
const getUA = await page.evaluate(() => navigator.userAgent);
const userAgentInfo = uaParser(getUA);
let outdatedComponents = await page.getByTestId("icon-AlertTriangle").count();
await page.getByTestId("side_nav_options_all-templates").click();
await page
.getByRole("heading", { name: "Hierarchical Tasks Agent" })
.first()
.click();
await page.waitForSelector('[data-testid="fit_view"]', {
timeout: 100000,
});
await page.getByTestId("fit_view").click();
await page.getByTestId("zoom_out").click();
await page.getByTestId("zoom_out").click();
await page.getByTestId("zoom_out").click();
let outdatedComponents = await page
.getByTestId("icon-AlertTriangle")
.count();
while (outdatedComponents > 0) {
await page.getByTestId("icon-AlertTriangle").first().click();
await page.waitForTimeout(1000);
outdatedComponents = await page.getByTestId("icon-AlertTriangle").count();
}
let filledApiKey = await page.getByTestId("remove-icon-badge").count();
while (filledApiKey > 0) {
await page.getByTestId("remove-icon-badge").first().click();
await page.waitForTimeout(1000);
filledApiKey = await page.getByTestId("remove-icon-badge").count();
}
await page
.getByTestId("popover-anchor-input-api_key")
.nth(1)
.fill(process.env.OPENAI_API_KEY ?? "");
await page
.getByTestId("popover-anchor-input-api_key")
.nth(0)
.fill(process.env.OPENAI_API_KEY ?? "");
await page.getByTestId("dropdown_str_model_name").nth(1).click();
await page.getByTestId("gpt-4o-1-option").last().click();
await page.getByTestId("dropdown_str_model_name").nth(0).click();
await page.getByTestId("gpt-4o-1-option").first().click();
await page
.getByTestId("popover-anchor-input-api_key")
.last()
.fill(process.env.SEARCH_API_KEY ?? "");
while (outdatedComponents > 0) {
await page.getByTestId("icon-AlertTriangle").first().click();
await page.waitForTimeout(1000);
outdatedComponents = await page.getByTestId("icon-AlertTriangle").count();
}
let filledApiKey = await page.getByTestId("remove-icon-badge").count();
while (filledApiKey > 0) {
await page.getByTestId("remove-icon-badge").first().click();
await page.getByTestId("button_run_chat output").click();
await page.waitForTimeout(1000);
filledApiKey = await page.getByTestId("remove-icon-badge").count();
}
await page
.getByTestId("popover-anchor-input-api_key")
.nth(1)
.fill(process.env.OPENAI_API_KEY ?? "");
await page.waitForSelector("text=built successfully", { timeout: 30000 });
await page
.getByTestId("popover-anchor-input-api_key")
.nth(0)
.fill(process.env.OPENAI_API_KEY ?? "");
await page.getByText("built successfully").last().click({
timeout: 15000,
});
await page.getByTestId("dropdown_str_model_name").nth(1).click();
await page.getByTestId("gpt-4o-1-option").last().click();
await page.getByText("Playground", { exact: true }).last().click();
await page.getByTestId("dropdown_str_model_name").nth(0).click();
await page.getByTestId("gpt-4o-1-option").first().click();
expect(await page.locator(".markdown").count()).toBeGreaterThan(0);
await page
.getByTestId("popover-anchor-input-api_key")
.last()
.fill(process.env.SEARCH_API_KEY ?? "");
await page.waitForTimeout(1000);
await page.getByTestId("button_run_chat output").click();
await page.waitForTimeout(1000);
await page.waitForSelector("text=built successfully", { timeout: 30000 });
await page.getByText("built successfully").last().click({
timeout: 15000,
});
await page.getByText("Playground", { exact: true }).last().click();
expect(await page.locator(".markdown").count()).toBeGreaterThan(0);
expect(await page.getByText("Langflow").count()).toBeGreaterThan(2);
});
expect(await page.getByText("Langflow").count()).toBeGreaterThan(2);
},
);

214
src/frontend/tests/core/integrations/Memory Chatbot.spec.ts
View file

@ -2,134 +2,140 @@ import { test } from "@playwright/test";
import * as dotenv from "dotenv";
import path from "path";
test("Memory Chatbot", async ({ page }) => {
test.skip(
!process?.env?.OPENAI_API_KEY,
"OPENAI_API_KEY required to run this test",
);
test(
"Memory Chatbot",
{ tag: ["@release", "@starter-project"] },
async ({ page }) => {
test.skip(
!process?.env?.OPENAI_API_KEY,
"OPENAI_API_KEY required to run this test",
);
if (!process.env.CI) {
dotenv.config({ path: path.resolve(__dirname, "../../.env") });
}
await page.goto("/");
await page.waitForSelector('[data-testid="mainpage_title"]', {
timeout: 30000,
});
await page.waitForSelector('[id="new-project-btn"]', {
timeout: 30000,
});
let modalCount = 0;
try {
const modalTitleElement = await page?.getByTestId("modal-title");
if (modalTitleElement) {
modalCount = await modalTitleElement.count();
if (!process.env.CI) {
dotenv.config({ path: path.resolve(__dirname, "../../.env") });
}
} catch (error) {
modalCount = 0;
}
while (modalCount === 0) {
await page.getByText("New Flow", { exact: true }).click();
await page.waitForTimeout(3000);
modalCount = await page.getByTestId("modal-title")?.count();
}
await page.goto("/");
await page.waitForSelector('[data-testid="mainpage_title"]', {
timeout: 30000,
});
await page.getByTestId("side_nav_options_all-templates").click();
await page.getByRole("heading", { name: "Memory Chatbot" }).click();
await page.waitForTimeout(1000);
await page.waitForSelector('[id="new-project-btn"]', {
timeout: 30000,
});
await page.getByTestId("fit_view").click();
await page.getByTestId("zoom_out").click();
await page.getByTestId("zoom_out").click();
await page.getByTestId("zoom_out").click();
let modalCount = 0;
try {
const modalTitleElement = await page?.getByTestId("modal-title");
if (modalTitleElement) {
modalCount = await modalTitleElement.count();
}
} catch (error) {
modalCount = 0;
}
await page.waitForTimeout(1000);
while (modalCount === 0) {
await page.getByText("New Flow", { exact: true }).click();
await page.waitForTimeout(3000);
modalCount = await page.getByTestId("modal-title")?.count();
}
let outdatedComponents = await page.getByTestId("icon-AlertTriangle").count();
while (outdatedComponents > 0) {
await page.getByTestId("icon-AlertTriangle").first().click();
await page.getByTestId("side_nav_options_all-templates").click();
await page.getByRole("heading", { name: "Memory Chatbot" }).click();
await page.waitForTimeout(1000);
outdatedComponents = await page.getByTestId("icon-AlertTriangle").count();
}
let filledApiKey = await page.getByTestId("remove-icon-badge").count();
while (filledApiKey > 0) {
await page.getByTestId("remove-icon-badge").first().click();
await page.getByTestId("fit_view").click();
await page.getByTestId("zoom_out").click();
await page.getByTestId("zoom_out").click();
await page.getByTestId("zoom_out").click();
await page.waitForTimeout(1000);
filledApiKey = await page.getByTestId("remove-icon-badge").count();
}
const apiKeyInput = page.getByTestId("popover-anchor-input-api_key");
const isApiKeyInputVisible = await apiKeyInput.isVisible();
let outdatedComponents = await page
.getByTestId("icon-AlertTriangle")
.count();
if (isApiKeyInputVisible) {
await apiKeyInput.fill(process.env.OPENAI_API_KEY ?? "");
}
while (outdatedComponents > 0) {
await page.getByTestId("icon-AlertTriangle").first().click();
await page.waitForTimeout(1000);
outdatedComponents = await page.getByTestId("icon-AlertTriangle").count();
}
await page.getByTestId("dropdown_str_model_name").click();
await page.getByTestId("gpt-4o-1-option").click();
let filledApiKey = await page.getByTestId("remove-icon-badge").count();
while (filledApiKey > 0) {
await page.getByTestId("remove-icon-badge").first().click();
await page.waitForTimeout(1000);
filledApiKey = await page.getByTestId("remove-icon-badge").count();
}
await page.waitForTimeout(1000);
const apiKeyInput = page.getByTestId("popover-anchor-input-api_key");
const isApiKeyInputVisible = await apiKeyInput.isVisible();
await page.getByTestId("button_run_chat output").click();
await page.waitForSelector("text=built successfully", { timeout: 30000 });
if (isApiKeyInputVisible) {
await apiKeyInput.fill(process.env.OPENAI_API_KEY ?? "");
}
await page.getByText("built successfully").last().click({
timeout: 15000,
});
await page.getByTestId("dropdown_str_model_name").click();
await page.getByTestId("gpt-4o-1-option").click();
await page.getByText("Playground", { exact: true }).last().click();
await page.waitForTimeout(1000);
await page
.getByText("No input message provided.", { exact: true })
.last()
.isVisible();
await page.getByTestId("button_run_chat output").click();
await page.waitForSelector("text=built successfully", { timeout: 30000 });
await page.waitForSelector('[data-testid="input-chat-playground"]', {
timeout: 100000,
});
await page.getByText("built successfully").last().click({
timeout: 15000,
});
await page
.getByTestId("input-chat-playground")
.last()
.fill("Remember that I'm a lion");
await page.getByTestId("button-send").last().click();
await page.getByText("Playground", { exact: true }).last().click();
await page.waitForSelector('[data-testid="input-chat-playground"]', {
timeout: 100000,
});
await page
.getByText("No input message provided.", { exact: true })
.last()
.isVisible();
await page
.getByTestId("input-chat-playground")
.last()
.fill("try reproduce the sound I made in words");
await page.waitForSelector('[data-testid="input-chat-playground"]', {
timeout: 100000,
});
await page.waitForSelector('[data-testid="button-send"]', {
timeout: 100000,
});
await page
.getByTestId("input-chat-playground")
.last()
.fill("Remember that I'm a lion");
await page.getByTestId("button-send").last().click();
await page.getByTestId("button-send").last().click();
await page.waitForSelector('[data-testid="input-chat-playground"]', {
timeout: 100000,
});
await page.waitForSelector("text=roar", { timeout: 30000 });
await page.getByText("roar").last().isVisible();
await page.getByText("Default Session").last().click();
await page
.getByTestId("input-chat-playground")
.last()
.fill("try reproduce the sound I made in words");
await page.getByText("timestamp", { exact: true }).last().isVisible();
await page.getByText("text", { exact: true }).last().isVisible();
await page.getByText("sender", { exact: true }).last().isVisible();
await page.getByText("sender_name", { exact: true }).last().isVisible();
await page.getByText("session_id", { exact: true }).last().isVisible();
await page.getByText("files", { exact: true }).last().isVisible();
await page.waitForSelector('[data-testid="button-send"]', {
timeout: 100000,
});
await page.getByRole("gridcell").last().isVisible();
await page.getByRole("combobox").click();
await page.getByLabel("Delete").click();
await page.waitForSelector('[data-testid="input-chat-playground"]', {
timeout: 100000,
});
await page.getByTestId("input-chat-playground").last().isVisible();
});
await page.getByTestId("button-send").last().click();
await page.waitForSelector("text=roar", { timeout: 30000 });
await page.getByText("roar").last().isVisible();
await page.getByText("Default Session").last().click();
await page.getByText("timestamp", { exact: true }).last().isVisible();
await page.getByText("text", { exact: true }).last().isVisible();
await page.getByText("sender", { exact: true }).last().isVisible();
await page.getByText("sender_name", { exact: true }).last().isVisible();
await page.getByText("session_id", { exact: true }).last().isVisible();
await page.getByText("files", { exact: true }).last().isVisible();
await page.getByRole("gridcell").last().isVisible();
await page.getByRole("combobox").click();
await page.getByLabel("Delete").click();
await page.waitForSelector('[data-testid="input-chat-playground"]', {
timeout: 100000,
});
await page.getByTestId("input-chat-playground").last().isVisible();
},
);

158
src/frontend/tests/core/integrations/Sequential Task Agent.spec.ts
View file

@ -3,96 +3,102 @@ import * as dotenv from "dotenv";
import path from "path";
import uaParser from "ua-parser-js";
test.skip("Sequential Task Agent", async ({ page }) => {
test.skip(
!process?.env?.OPENAI_API_KEY,
"OPENAI_API_KEY required to run this test",
);
test.skip(
"Sequential Task Agent",
{ tag: ["@release", "@starter-project"] },
async ({ page }) => {
test.skip(
!process?.env?.OPENAI_API_KEY,
"OPENAI_API_KEY required to run this test",
);
if (!process.env.CI) {
dotenv.config({ path: path.resolve(__dirname, "../../.env") });
}
await page.goto("/");
await page.waitForSelector('[data-testid="mainpage_title"]', {
timeout: 30000,
});
await page.waitForSelector('[id="new-project-btn"]', {
timeout: 30000,
});
let modalCount = 0;
try {
const modalTitleElement = await page?.getByTestId("modal-title");
if (modalTitleElement) {
modalCount = await modalTitleElement.count();
if (!process.env.CI) {
dotenv.config({ path: path.resolve(__dirname, "../../.env") });
}
} catch (error) {
modalCount = 0;
}
while (modalCount === 0) {
await page.getByText("New Flow", { exact: true }).click();
await page.waitForTimeout(3000);
modalCount = await page.getByTestId("modal-title")?.count();
}
await page.goto("/");
await page.waitForSelector('[data-testid="mainpage_title"]', {
timeout: 30000,
});
const getUA = await page.evaluate(() => navigator.userAgent);
const userAgentInfo = uaParser(getUA);
await page.waitForSelector('[id="new-project-btn"]', {
timeout: 30000,
});
await page.getByTestId("side_nav_options_all-templates").click();
await page
.getByRole("heading", { name: "Sequential Tasks Agent" })
.first()
.click();
let modalCount = 0;
try {
const modalTitleElement = await page?.getByTestId("modal-title");
if (modalTitleElement) {
modalCount = await modalTitleElement.count();
}
} catch (error) {
modalCount = 0;
}
await page.waitForSelector('[data-testid="fit_view"]', {
timeout: 100000,
});
while (modalCount === 0) {
await page.getByText("New Flow", { exact: true }).click();
await page.waitForTimeout(3000);
modalCount = await page.getByTestId("modal-title")?.count();
}
await page.getByTestId("fit_view").click();
await page.getByTestId("zoom_out").click();
await page.getByTestId("zoom_out").click();
await page.getByTestId("zoom_out").click();
const getUA = await page.evaluate(() => navigator.userAgent);
const userAgentInfo = uaParser(getUA);
let outdatedComponents = await page.getByTestId("icon-AlertTriangle").count();
await page.getByTestId("side_nav_options_all-templates").click();
await page
.getByRole("heading", { name: "Sequential Tasks Agent" })
.first()
.click();
await page.waitForSelector('[data-testid="fit_view"]', {
timeout: 100000,
});
await page.getByTestId("fit_view").click();
await page.getByTestId("zoom_out").click();
await page.getByTestId("zoom_out").click();
await page.getByTestId("zoom_out").click();
let outdatedComponents = await page
.getByTestId("icon-AlertTriangle")
.count();
while (outdatedComponents > 0) {
await page.getByTestId("icon-AlertTriangle").first().click();
await page.waitForTimeout(1000);
outdatedComponents = await page.getByTestId("icon-AlertTriangle").count();
}
let filledApiKey = await page.getByTestId("remove-icon-badge").count();
while (filledApiKey > 0) {
await page.getByTestId("remove-icon-badge").first().click();
await page.waitForTimeout(1000);
filledApiKey = await page.getByTestId("remove-icon-badge").count();
}
await page
.getByTestId("popover-anchor-input-api_key")
.fill(process.env.OPENAI_API_KEY ?? "");
await page.getByTestId("dropdown_str_model_name").click();
await page.getByTestId("gpt-4o-1-option").click();
while (outdatedComponents > 0) {
await page.getByTestId("icon-AlertTriangle").first().click();
await page.waitForTimeout(1000);
outdatedComponents = await page.getByTestId("icon-AlertTriangle").count();
}
let filledApiKey = await page.getByTestId("remove-icon-badge").count();
while (filledApiKey > 0) {
await page.getByTestId("remove-icon-badge").first().click();
await page.getByTestId("button_run_chat output").click();
await page.waitForTimeout(1000);
filledApiKey = await page.getByTestId("remove-icon-badge").count();
}
await page
.getByTestId("popover-anchor-input-api_key")
.fill(process.env.OPENAI_API_KEY ?? "");
await page.waitForSelector("text=built successfully", { timeout: 30000 });
await page.getByTestId("dropdown_str_model_name").click();
await page.getByTestId("gpt-4o-1-option").click();
await page.getByText("built successfully").last().click({
timeout: 15000,
});
await page.waitForTimeout(1000);
await page.getByText("Playground", { exact: true }).last().click();
await page.getByTestId("button_run_chat output").click();
expect(await page.locator(".markdown").count()).toBeGreaterThan(0);
await page.waitForTimeout(1000);
await page.waitForSelector("text=built successfully", { timeout: 30000 });
await page.getByText("built successfully").last().click({
timeout: 15000,
});
await page.getByText("Playground", { exact: true }).last().click();
expect(await page.locator(".markdown").count()).toBeGreaterThan(0);
expect(await page.getByText("Agile").count()).toBeGreaterThan(0);
});
expect(await page.getByText("Agile").count()).toBeGreaterThan(0);
},
);

151
src/frontend/tests/core/integrations/Simple Agent.spec.ts
View file

@ -3,96 +3,103 @@ import * as dotenv from "dotenv";
import path from "path";
import uaParser from "ua-parser-js";
test("Simple Agent", async ({ page }) => {
test.skip(
!process?.env?.OPENAI_API_KEY,
"OPENAI_API_KEY required to run this test",
);
test(
"Simple Agent",
{ tag: ["@release", "@starter-project"] },
if (!process.env.CI) {
dotenv.config({ path: path.resolve(__dirname, "../../.env") });
}
async ({ page }) => {
test.skip(
!process?.env?.OPENAI_API_KEY,
"OPENAI_API_KEY required to run this test",
);
await page.goto("/");
await page.waitForSelector('[data-testid="mainpage_title"]', {
timeout: 30000,
});
await page.waitForSelector('[id="new-project-btn"]', {
timeout: 30000,
});
let modalCount = 0;
try {
const modalTitleElement = await page?.getByTestId("modal-title");
if (modalTitleElement) {
modalCount = await modalTitleElement.count();
if (!process.env.CI) {
dotenv.config({ path: path.resolve(__dirname, "../../.env") });
}
} catch (error) {
modalCount = 0;
}
while (modalCount === 0) {
await page.getByText("New Flow", { exact: true }).click();
await page.waitForTimeout(3000);
modalCount = await page.getByTestId("modal-title")?.count();
}
await page.goto("/");
await page.waitForSelector('[data-testid="mainpage_title"]', {
timeout: 30000,
});
const getUA = await page.evaluate(() => navigator.userAgent);
const userAgentInfo = uaParser(getUA);
await page.waitForSelector('[id="new-project-btn"]', {
timeout: 30000,
});
await page.getByTestId("side_nav_options_all-templates").click();
await page.getByRole("heading", { name: "Simple Agent" }).first().click();
let modalCount = 0;
try {
const modalTitleElement = await page?.getByTestId("modal-title");
if (modalTitleElement) {
modalCount = await modalTitleElement.count();
}
} catch (error) {
modalCount = 0;
}
await page.waitForSelector('[data-testid="fit_view"]', {
timeout: 100000,
});
while (modalCount === 0) {
await page.getByText("New Flow", { exact: true }).click();
await page.waitForTimeout(3000);
modalCount = await page.getByTestId("modal-title")?.count();
}
await page.getByTestId("fit_view").click();
await page.getByTestId("zoom_out").click();
await page.getByTestId("zoom_out").click();
await page.getByTestId("zoom_out").click();
const getUA = await page.evaluate(() => navigator.userAgent);
const userAgentInfo = uaParser(getUA);
let outdatedComponents = await page.getByTestId("icon-AlertTriangle").count();
await page.getByTestId("side_nav_options_all-templates").click();
await page.getByRole("heading", { name: "Simple Agent" }).first().click();
while (outdatedComponents > 0) {
await page.getByTestId("icon-AlertTriangle").first().click();
await page.waitForTimeout(1000);
outdatedComponents = await page.getByTestId("icon-AlertTriangle").count();
}
await page.waitForSelector('[data-testid="fit_view"]', {
timeout: 100000,
});
let filledApiKey = await page.getByTestId("remove-icon-badge").count();
while (filledApiKey > 0) {
await page.getByTestId("remove-icon-badge").first().click();
await page.waitForTimeout(1000);
filledApiKey = await page.getByTestId("remove-icon-badge").count();
}
await page.getByTestId("fit_view").click();
await page.getByTestId("zoom_out").click();
await page.getByTestId("zoom_out").click();
await page.getByTestId("zoom_out").click();
await page
.getByTestId("popover-anchor-input-api_key")
.fill(process.env.OPENAI_API_KEY ?? "");
let outdatedComponents = await page
.getByTestId("icon-AlertTriangle")
.count();
await page.getByTestId("fit_view").click();
while (outdatedComponents > 0) {
await page.getByTestId("icon-AlertTriangle").first().click();
await page.waitForTimeout(1000);
outdatedComponents = await page.getByTestId("icon-AlertTriangle").count();
}
await page.getByTestId("dropdown_str_model_name").click();
await page.getByTestId("gpt-4o-1-option").click();
let filledApiKey = await page.getByTestId("remove-icon-badge").count();
while (filledApiKey > 0) {
await page.getByTestId("remove-icon-badge").first().click();
await page.waitForTimeout(1000);
filledApiKey = await page.getByTestId("remove-icon-badge").count();
}
await page.waitForTimeout(500);
await page
.getByTestId("popover-anchor-input-api_key")
.fill(process.env.OPENAI_API_KEY ?? "");
await page.getByTestId("button_run_chat output").last().click();
await page.getByTestId("fit_view").click();
await page.waitForSelector("text=built successfully", {
timeout: 10000 * 60 * 3,
});
await page.getByTestId("dropdown_str_model_name").click();
await page.getByTestId("gpt-4o-1-option").click();
await page.getByTestId("playground-btn-flow-io").click();
await page.waitForTimeout(500);
const textContents = await page
.getByTestId("div-chat-message")
.allTextContents();
await page.getByTestId("button_run_chat output").last().click();
const concatAllText = textContents.join(" ").toLowerCase();
await page.waitForSelector("text=built successfully", {
timeout: 10000 * 60 * 3,
});
expect(concatAllText).toContain("hello! how can i assist you today?");
expect(concatAllText.length).toBeGreaterThan(20);
});
await page.getByTestId("playground-btn-flow-io").click();
const textContents = await page
.getByTestId("div-chat-message")
.allTextContents();
const concatAllText = textContents.join(" ").toLowerCase();
expect(concatAllText).toContain("hello! how can i assist you today?");
expect(concatAllText.length).toBeGreaterThan(20);
},
);

250
src/frontend/tests/core/integrations/Travel Planning Agent.spec.ts
View file

@ -2,145 +2,151 @@ import { expect, Page, test } from "@playwright/test";
import * as dotenv from "dotenv";
import path from "path";
test("Travel Planning Agent", async ({ page }) => {
test.skip(
!process?.env?.OPENAI_API_KEY,
"OPENAI_API_KEY required to run this test",
);
test.skip(
!process?.env?.SEARCH_API_KEY,
"SEARCH_API_KEY required to run this test",
);
if (!process.env.CI) {
dotenv.config({ path: path.resolve(__dirname, "../../.env") });
}
await page.goto("/");
await page.waitForSelector('[data-testid="mainpage_title"]', {
timeout: 30000,
});
await page.waitForSelector('[id="new-project-btn"]', {
timeout: 30000,
});
let modalCount = 0;
try {
const modalTitleElement = await page?.getByTestId("modal-title");
if (modalTitleElement) {
modalCount = await modalTitleElement.count();
}
} catch (error) {
modalCount = 0;
}
while (modalCount === 0) {
await page.getByText("New Flow", { exact: true }).click();
await page.waitForTimeout(3000);
modalCount = await page.getByTestId("modal-title")?.count();
}
await page.getByTestId("side_nav_options_all-templates").click();
await page
.getByRole("heading", { name: "Travel Planning Agents" })
.last()
.click();
await page.waitForSelector('[data-testid="fit_view"]', {
timeout: 100000,
});
await page.getByTestId("fit_view").click();
await page.getByTestId("zoom_out").click();
await page.getByTestId("zoom_out").click();
await page.getByTestId("zoom_out").click();
let outdatedComponents = await page.getByTestId("icon-AlertTriangle").count();
while (outdatedComponents > 0) {
await page.getByTestId("icon-AlertTriangle").first().click();
await page.waitForTimeout(1000);
outdatedComponents = await page.getByTestId("icon-AlertTriangle").count();
}
let filledApiKey = await page.getByTestId("remove-icon-badge").count();
while (filledApiKey > 0) {
await page.getByTestId("remove-icon-badge").first().click();
await page.waitForTimeout(1000);
filledApiKey = await page.getByTestId("remove-icon-badge").count();
}
const randomCity = cities[Math.floor(Math.random() * cities.length)];
const randomCity2 = cities[Math.floor(Math.random() * cities.length)];
const randomFood = foods[Math.floor(Math.random() * foods.length)];
await page
.getByTestId("textarea_str_input_value")
.first()
.fill(
`Create a travel plan from ${randomCity} to ${randomCity2} with ${randomFood}`,
test(
"Travel Planning Agent",
{ tag: ["@release", "@starter-project"] },
async ({ page }) => {
test.skip(
!process?.env?.OPENAI_API_KEY,
"OPENAI_API_KEY required to run this test",
);
let openAiLlms = await page.getByText("OpenAI", { exact: true }).count();
await page.waitForSelector('[data-testid="fit_view"]', {
timeout: 100000,
});
test.skip(
!process?.env?.SEARCH_API_KEY,
"SEARCH_API_KEY required to run this test",
);
if (!process.env.CI) {
dotenv.config({ path: path.resolve(__dirname, "../../.env") });
}
await page.goto("/");
await page.waitForSelector('[data-testid="mainpage_title"]', {
timeout: 30000,
});
await page.waitForSelector('[id="new-project-btn"]', {
timeout: 30000,
});
let modalCount = 0;
try {
const modalTitleElement = await page?.getByTestId("modal-title");
if (modalTitleElement) {
modalCount = await modalTitleElement.count();
}
} catch (error) {
modalCount = 0;
}
while (modalCount === 0) {
await page.getByText("New Flow", { exact: true }).click();
await page.waitForTimeout(3000);
modalCount = await page.getByTestId("modal-title")?.count();
}
await page.getByTestId("side_nav_options_all-templates").click();
await page
.getByRole("heading", { name: "Travel Planning Agents" })
.last()
.click();
await page.waitForSelector('[data-testid="fit_view"]', {
timeout: 100000,
});
await page.getByTestId("fit_view").click();
await page.getByTestId("zoom_out").click();
await page.getByTestId("zoom_out").click();
await page.getByTestId("zoom_out").click();
let outdatedComponents = await page
.getByTestId("icon-AlertTriangle")
.count();
while (outdatedComponents > 0) {
await page.getByTestId("icon-AlertTriangle").first().click();
await page.waitForTimeout(1000);
outdatedComponents = await page.getByTestId("icon-AlertTriangle").count();
}
let filledApiKey = await page.getByTestId("remove-icon-badge").count();
while (filledApiKey > 0) {
await page.getByTestId("remove-icon-badge").first().click();
await page.waitForTimeout(1000);
filledApiKey = await page.getByTestId("remove-icon-badge").count();
}
const randomCity = cities[Math.floor(Math.random() * cities.length)];
const randomCity2 = cities[Math.floor(Math.random() * cities.length)];
const randomFood = foods[Math.floor(Math.random() * foods.length)];
await page
.getByTestId("textarea_str_input_value")
.first()
.fill(
`Create a travel plan from ${randomCity} to ${randomCity2} with ${randomFood}`,
);
let openAiLlms = await page.getByText("OpenAI", { exact: true }).count();
await page.waitForSelector('[data-testid="fit_view"]', {
timeout: 100000,
});
for (let i = 0; i < openAiLlms; i++) {
await page
.getByTestId("popover-anchor-input-api_key")
.nth(i + 1)
.fill(process.env.OPENAI_API_KEY ?? "");
await page.getByTestId("zoom_in").click();
await page.getByTestId("dropdown_str_model_name").nth(i).click();
await page.getByTestId("gpt-4o-1-option").last().click();
await page.waitForTimeout(1000);
}
for (let i = 0; i < openAiLlms; i++) {
await page
.getByTestId("popover-anchor-input-api_key")
.nth(i + 1)
.fill(process.env.OPENAI_API_KEY ?? "");
await page.getByTestId("zoom_in").click();
await page.getByTestId("dropdown_str_model_name").nth(i).click();
await page.getByTestId("gpt-4o-1-option").last().click();
.first()
.fill(process.env.SEARCH_API_KEY ?? "");
await page.waitForTimeout(1000);
}
await page
.getByTestId("popover-anchor-input-api_key")
.first()
.fill(process.env.SEARCH_API_KEY ?? "");
await page.getByTestId("button_run_chat output").click();
await page.waitForTimeout(1000);
await page.getByTestId("button_run_chat output").last().click();
await page.getByTestId("button_run_chat output").click();
if (await checkRateLimit(page)) {
console.log("Rate limit detected, skipping test");
test.skip();
}
await page.getByTestId("button_run_chat output").last().click();
await page.waitForSelector("text=built successfully", {
timeout: 60000 * 3,
});
if (await checkRateLimit(page)) {
console.log("Rate limit detected, skipping test");
test.skip();
}
await page.getByText("built successfully").last().click({
timeout: 15000,
});
await page.waitForSelector("text=built successfully", {
timeout: 60000 * 3,
});
await page.getByText("Playground", { exact: true }).last().click();
await page.getByText("built successfully").last().click({
timeout: 15000,
});
await page.waitForSelector("text=default session", {
timeout: 30000,
});
await page.getByText("Playground", { exact: true }).last().click();
await page.waitForTimeout(1000);
await page.waitForSelector("text=default session", {
timeout: 30000,
});
const output = await page.getByTestId("div-chat-message").allTextContents();
const outputText = output.join("\n");
await page.waitForTimeout(1000);
expect(outputText.toLowerCase()).toContain("weather");
expect(outputText.toLowerCase()).toContain("budget");
const output = await page.getByTestId("div-chat-message").allTextContents();
const outputText = output.join("\n");
expect(outputText.toLowerCase()).toContain("weather");
expect(outputText.toLowerCase()).toContain("budget");
expect(outputText.toLowerCase()).toContain(randomCity.toLowerCase());
expect(outputText.toLowerCase()).toContain(randomCity2.toLowerCase());
expect(outputText.toLowerCase()).toContain(randomFood.toLowerCase());
});
expect(outputText.toLowerCase()).toContain(randomCity.toLowerCase());
expect(outputText.toLowerCase()).toContain(randomCity2.toLowerCase());
expect(outputText.toLowerCase()).toContain(randomFood.toLowerCase());
},
);
async function checkRateLimit(page: Page): Promise<boolean> {
try {

359
src/frontend/tests/core/integrations/Vector Store.spec.ts
View file

@ -2,181 +2,196 @@ import { Page, test } from "@playwright/test";
import path from "path";
import uaParser from "ua-parser-js";
test("Vector Store RAG", async ({ page }) => {
test.skip(
!process?.env?.OPENAI_API_KEY,
"OPENAI_API_KEY required to run this test",
);
test.skip(
!process?.env?.ASTRA_DB_API_ENDPOINT,
"ASTRA_DB_API_ENDPOINT required to run this test",
);
test.skip(
!process?.env?.ASTRA_DB_APPLICATION_TOKEN,
"ASTRA_DB_APPLICATION_TOKEN required to run this test",
);
await page.goto("/");
await page.waitForSelector('[data-testid="mainpage_title"]', {
timeout: 30000,
});
await page.waitForSelector('[id="new-project-btn"]', {
timeout: 30000,
});
let modalCount = 0;
try {
const modalTitleElement = await page?.getByTestId("modal-title");
if (modalTitleElement) {
modalCount = await modalTitleElement.count();
}
} catch (error) {
modalCount = 0;
}
while (modalCount === 0) {
await page.getByText("New Flow", { exact: true }).click();
await page.waitForTimeout(3000);
modalCount = await page.getByTestId("modal-title")?.count();
}
await page.getByTestId("side_nav_options_all-templates").click();
await page.getByRole("heading", { name: "Vector Store RAG" }).first().click();
await page.waitForSelector('[title="fit view"]', {
timeout: 100000,
});
await page.getByTitle("fit view").click();
await page.getByTitle("zoom out").click();
await page.getByTitle("zoom out").click();
await page.getByTitle("zoom out").click();
let outdatedComponents = await page.getByTestId("icon-AlertTriangle").count();
while (outdatedComponents > 0) {
await page.getByTestId("icon-AlertTriangle").first().click();
await page.waitForTimeout(1000);
outdatedComponents = await page.getByTestId("icon-AlertTriangle").count();
}
let filledApiKey = await page.getByTestId("remove-icon-badge").count();
while (filledApiKey > 0) {
await page.getByTestId("remove-icon-badge").first().click();
await page.waitForTimeout(1000);
filledApiKey = await page.getByTestId("remove-icon-badge").count();
}
if (process?.env?.ASTRA_DB_API_ENDPOINT?.includes("astra-dev")) {
const getUA = await page.evaluate(() => navigator.userAgent);
const userAgentInfo = uaParser(getUA);
await page.getByTestId("title-Astra DB").first().click();
await page.waitForTimeout(500);
await page.getByTestId("code-button-modal").click();
await page.waitForTimeout(500);
let cleanCode = await extractAndCleanCode(page);
cleanCode = cleanCode!.replace(
'"pre_delete_collection": self.pre_delete_collection or False,',
'"pre_delete_collection": self.pre_delete_collection or False,\n "environment": "dev",',
test(
"Vector Store RAG",
{ tag: ["@release", "@starter-project"] },
async ({ page }) => {
test.skip(
!process?.env?.OPENAI_API_KEY,
"OPENAI_API_KEY required to run this test",
);
await page.locator("textarea").last().press(`ControlOrMeta+a`);
await page.keyboard.press("Backspace");
await page.locator("textarea").last().fill(cleanCode);
await page.locator('//*[@id="checkAndSaveBtn"]').click();
await page.waitForTimeout(500);
await page.getByTestId("title-Astra DB").last().click();
await page.waitForTimeout(500);
await page.getByTestId("code-button-modal").click();
await page.waitForTimeout(500);
await page.locator("textarea").last().press(`ControlOrMeta+a`);
await page.keyboard.press("Backspace");
await page.locator("textarea").last().fill(cleanCode);
await page.locator('//*[@id="checkAndSaveBtn"]').click();
await page.waitForTimeout(500);
}
const apiKeyInput = page.getByTestId("popover-anchor-input-api_key");
const isApiKeyInputVisible = await apiKeyInput.isVisible();
if (isApiKeyInputVisible) {
await apiKeyInput.fill(process.env.OPENAI_API_KEY ?? "");
}
await page
.getByTestId("popover-anchor-input-api_key") // input ID without "anchor-"
.nth(0)
.fill(process.env.OPENAI_API_KEY ?? "");
await page
.getByTestId("popover-anchor-input-openai_api_key")
.nth(1)
.fill(process.env.OPENAI_API_KEY ?? "");
await page
.getByTestId("popover-anchor-input-openai_api_key")
.nth(0)
.fill(process.env.OPENAI_API_KEY ?? "");
// Astra DB tokens
await page
.getByTestId("popover-anchor-input-token")
.nth(0)
.fill(process.env.ASTRA_DB_APPLICATION_TOKEN ?? "");
await page
.getByTestId("popover-anchor-input-token")
.nth(1)
.fill(process.env.ASTRA_DB_APPLICATION_TOKEN ?? "");
// Astra DB endpoints
await page
.getByTestId("popover-anchor-input-api_endpoint")
.nth(0)
.fill(process.env.ASTRA_DB_API_ENDPOINT ?? "");
await page
.getByTestId("popover-anchor-input-api_endpoint")
.nth(1)
.fill(process.env.ASTRA_DB_API_ENDPOINT ?? "");
test.skip(
!process?.env?.ASTRA_DB_API_ENDPOINT,
"ASTRA_DB_API_ENDPOINT required to run this test",
);
test.skip(
!process?.env?.ASTRA_DB_APPLICATION_TOKEN,
"ASTRA_DB_APPLICATION_TOKEN required to run this test",
);
await page.goto("/");
await page.waitForSelector('[data-testid="mainpage_title"]', {
timeout: 30000,
});
await page.waitForSelector('[id="new-project-btn"]', {
timeout: 30000,
});
let modalCount = 0;
try {
const modalTitleElement = await page?.getByTestId("modal-title");
if (modalTitleElement) {
modalCount = await modalTitleElement.count();
}
} catch (error) {
modalCount = 0;
}
while (modalCount === 0) {
await page.getByText("New Flow", { exact: true }).click();
await page.waitForTimeout(3000);
modalCount = await page.getByTestId("modal-title")?.count();
}
await page.getByTestId("side_nav_options_all-templates").click();
await page
.getByRole("heading", { name: "Vector Store RAG" })
.first()
.click();
await page.waitForSelector('[title="fit view"]', {
timeout: 100000,
});
await page.getByTitle("fit view").click();
await page.getByTitle("zoom out").click();
await page.getByTitle("zoom out").click();
await page.getByTitle("zoom out").click();
let outdatedComponents = await page
.getByTestId("icon-AlertTriangle")
.count();
while (outdatedComponents > 0) {
await page.getByTestId("icon-AlertTriangle").first().click();
await page.waitForTimeout(1000);
outdatedComponents = await page.getByTestId("icon-AlertTriangle").count();
}
let filledApiKey = await page.getByTestId("remove-icon-badge").count();
while (filledApiKey > 0) {
await page.getByTestId("remove-icon-badge").first().click();
await page.waitForTimeout(1000);
filledApiKey = await page.getByTestId("remove-icon-badge").count();
}
if (process?.env?.ASTRA_DB_API_ENDPOINT?.includes("astra-dev")) {
const getUA = await page.evaluate(() => navigator.userAgent);
const userAgentInfo = uaParser(getUA);
await page.getByTestId("title-Astra DB").first().click();
await page.waitForTimeout(500);
await page.getByTestId("code-button-modal").click();
await page.waitForTimeout(500);
let cleanCode = await extractAndCleanCode(page);
cleanCode = cleanCode!.replace(
'"pre_delete_collection": self.pre_delete_collection or False,',
'"pre_delete_collection": self.pre_delete_collection or False,\n "environment": "dev",',
);
await page.locator("textarea").last().press(`ControlOrMeta+a`);
await page.keyboard.press("Backspace");
await page.locator("textarea").last().fill(cleanCode);
await page.locator('//*[@id="checkAndSaveBtn"]').click();
await page.waitForTimeout(500);
await page.getByTestId("title-Astra DB").last().click();
await page.waitForTimeout(500);
await page.getByTestId("code-button-modal").click();
await page.waitForTimeout(500);
await page.locator("textarea").last().press(`ControlOrMeta+a`);
await page.keyboard.press("Backspace");
await page.locator("textarea").last().fill(cleanCode);
await page.locator('//*[@id="checkAndSaveBtn"]').click();
await page.waitForTimeout(500);
}
const apiKeyInput = page.getByTestId("popover-anchor-input-api_key");
const isApiKeyInputVisible = await apiKeyInput.isVisible();
if (isApiKeyInputVisible) {
await apiKeyInput.fill(process.env.OPENAI_API_KEY ?? "");
}
await page
.getByTestId("popover-anchor-input-api_key") // input ID without "anchor-"
.nth(0)
.fill(process.env.OPENAI_API_KEY ?? "");
await page
.getByTestId("popover-anchor-input-openai_api_key")
.nth(1)
.fill(process.env.OPENAI_API_KEY ?? "");
await page
.getByTestId("popover-anchor-input-openai_api_key")
.nth(0)
.fill(process.env.OPENAI_API_KEY ?? "");
// Astra DB tokens
await page
.getByTestId("popover-anchor-input-token")
.nth(0)
.fill(process.env.ASTRA_DB_APPLICATION_TOKEN ?? "");
await page
.getByTestId("popover-anchor-input-token")
.nth(1)
.fill(process.env.ASTRA_DB_APPLICATION_TOKEN ?? "");
// Astra DB endpoints
await page
.getByTestId("popover-anchor-input-api_endpoint")
.nth(0)
.fill(process.env.ASTRA_DB_API_ENDPOINT ?? "");
await page
.getByTestId("popover-anchor-input-api_endpoint")
.nth(1)
.fill(process.env.ASTRA_DB_API_ENDPOINT ?? "");
await page
.getByTestId("popover-anchor-input-collection_name")
.nth(0)
.fill("test");
const fileChooserPromise = page.waitForEvent("filechooser");
await page.getByTestId("icon-Upload").last().click();
const fileChooser = await fileChooserPromise;
await fileChooser.setFiles(
path.join(__dirname, "../../assets/test_file.txt"),
);
await page.getByText("test_file.txt").isVisible();
await page.waitForTimeout(1000);
await page.getByTestId("button_run_astra db").last().click();
await page.waitForSelector("text=built successfully", { timeout: 60000 * 2 });
await page.getByText("built successfully").last().click({
timeout: 30000,
});
await page.getByTestId("button_run_chat output").click();
await page.waitForSelector("text=built successfully", { timeout: 60000 * 2 });
await page.getByText("built successfully").last().click({
timeout: 30000,
});
await page.getByTestId("button_run_astra db").first().click();
await page.waitForSelector("text=built successfully", { timeout: 60000 * 2 });
await page.getByText("built successfully").last().click({
timeout: 30000,
});
await page.getByText("Playground", { exact: true }).last().click();
await page.waitForSelector('[data-testid="input-chat-playground"]', {
timeout: 100000,
});
await page.getByTestId("input-chat-playground").last().fill("hello");
await page.getByTestId("input-chat-playground").last().click();
await page.keyboard.press("Enter");
await page
.getByTestId("popover-anchor-input-collection_name")
.nth(0)
.fill("test");
const fileChooserPromise = page.waitForEvent("filechooser");
await page.getByTestId("icon-Upload").last().click();
const fileChooser = await fileChooserPromise;
await fileChooser.setFiles(
path.join(__dirname, "../../assets/test_file.txt"),
);
await page.getByText("test_file.txt").isVisible();
await page.waitForTimeout(1000);
await page.getByTestId("button_run_astra db").last().click();
await page.waitForSelector("text=built successfully", {
timeout: 60000 * 2,
});
await page.getByText("built successfully").last().click({
timeout: 30000,
});
await page.getByTestId("button_run_chat output").click();
await page.waitForSelector("text=built successfully", {
timeout: 60000 * 2,
});
await page.getByText("built successfully").last().click({
timeout: 30000,
});
await page.getByTestId("button_run_astra db").first().click();
await page.waitForSelector("text=built successfully", {
timeout: 60000 * 2,
});
await page.getByText("built successfully").last().click({
timeout: 30000,
});
await page.getByText("Playground", { exact: true }).last().click();
await page.waitForSelector('[data-testid="input-chat-playground"]', {
timeout: 100000,
});
await page.getByTestId("input-chat-playground").last().fill("hello");
await page.getByTestId("input-chat-playground").last().click();
await page.keyboard.press("Enter");
await page
.getByText("This is a test file.", { exact: true })
.last()
.isVisible();
await page.getByText("Chat", { exact: true }).last().click();
await page.getByText("Default Session").last().click();
await page.getByRole("combobox").click();
await page.getByLabel("Message logs").click();
await page.getByText("timestamp", { exact: true }).last().isVisible();
await page.getByText("text", { exact: true }).last().isVisible();
await page.getByText("sender", { exact: true }).last().isVisible();
await page.getByText("sender_name", { exact: true }).last().isVisible();
await page.getByText("session_id", { exact: true }).last().isVisible();
await page.getByText("files", { exact: true }).last().isVisible();
await page.getByRole("gridcell").last().isVisible();
await page.getByRole("combobox").click();
await page.getByLabel("Delete").click();
await page.waitForSelector('[data-testid="input-chat-playground"]', {
timeout: 100000,
});
await page.getByTestId("input-chat-playground").last().isVisible();
});
await page
.getByText("This is a test file.", { exact: true })
.last()
.isVisible();
await page.getByText("Chat", { exact: true }).last().click();
await page.getByText("Default Session").last().click();
await page.getByRole("combobox").click();
await page.getByLabel("Message logs").click();
await page.getByText("timestamp", { exact: true }).last().isVisible();
await page.getByText("text", { exact: true }).last().isVisible();
await page.getByText("sender", { exact: true }).last().isVisible();
await page.getByText("sender_name", { exact: true }).last().isVisible();
await page.getByText("session_id", { exact: true }).last().isVisible();
await page.getByText("files", { exact: true }).last().isVisible();
await page.getByRole("gridcell").last().isVisible();
await page.getByRole("combobox").click();
await page.getByLabel("Delete").click();
await page.waitForSelector('[data-testid="input-chat-playground"]', {
timeout: 100000,
});
await page.getByTestId("input-chat-playground").last().isVisible();
},
);
async function extractAndCleanCode(page: Page): Promise<string> {
const outerHTML = await page

669
src/frontend/tests/core/integrations/decisionFlow.spec.ts
View file

@ -8,225 +8,237 @@ async function zoomOut(page: Page, times: number = 4) {
}
}
test("should create a flow with decision", async ({ page }) => {
test.skip(
!process?.env?.OPENAI_API_KEY,
"OPENAI_API_KEY required to run this test",
);
if (!process.env.CI) {
dotenv.config({ path: path.resolve(__dirname, "../../.env") });
}
await page.goto("/");
await page.waitForSelector('[data-testid="mainpage_title"]', {
timeout: 30000,
});
await page.waitForSelector('[id="new-project-btn"]', {
timeout: 30000,
});
let modalCount = 0;
try {
const modalTitleElement = await page?.getByTestId("modal-title");
if (modalTitleElement) {
modalCount = await modalTitleElement.count();
test(
"should create a flow with decision",
{ tag: ["@release", "@components"] },
async ({ page }) => {
test.skip(
!process?.env?.OPENAI_API_KEY,
"OPENAI_API_KEY required to run this test",
);
if (!process.env.CI) {
dotenv.config({ path: path.resolve(__dirname, "../../.env") });
}
} catch (error) {
modalCount = 0;
}
while (modalCount === 0) {
await page.getByText("New Flow", { exact: true }).click();
await page.waitForSelector('[data-testid="modal-title"]', {
timeout: 3000,
await page.goto("/");
await page.waitForSelector('[data-testid="mainpage_title"]', {
timeout: 30000,
});
modalCount = await page.getByTestId("modal-title")?.count();
}
await page.waitForSelector('[data-testid="blank-flow"]', {
timeout: 30000,
});
await page.getByTestId("blank-flow").click();
await page.waitForSelector('[id="new-project-btn"]', {
timeout: 30000,
});
let modalCount = 0;
try {
const modalTitleElement = await page?.getByTestId("modal-title");
if (modalTitleElement) {
modalCount = await modalTitleElement.count();
}
} catch (error) {
modalCount = 0;
}
while (modalCount === 0) {
await page.getByText("New Flow", { exact: true }).click();
await page.waitForSelector('[data-testid="modal-title"]', {
timeout: 3000,
});
modalCount = await page.getByTestId("modal-title")?.count();
}
await page.waitForSelector('[data-testid="blank-flow"]', {
timeout: 30000,
});
await page.getByTestId("blank-flow").click();
await page.getByTestId("sidebar-options-trigger").click();
await page.getByTestId("sidebar-legacy-switch").isVisible({ timeout: 5000 });
await page.getByTestId("sidebar-legacy-switch").click();
await page.getByTestId("sidebar-options-trigger").click();
await page
.getByTestId("sidebar-legacy-switch")
.isVisible({ timeout: 5000 });
await page.getByTestId("sidebar-legacy-switch").click();
//---------------------------------- CHAT INPUT
await page.getByTestId("sidebar-search-input").click();
await page.getByTestId("sidebar-search-input").fill("chat input");
await page.waitForSelector('[data-testid="inputsChat Input"]', {
timeout: 500,
});
await page
.getByTestId("inputsChat Input")
.dragTo(page.locator('//*[@id="react-flow-id"]'));
await zoomOut(page);
//---------------------------------- CHAT INPUT
await page.getByTestId("sidebar-search-input").click();
await page.getByTestId("sidebar-search-input").fill("chat input");
await page.waitForSelector('[data-testid="inputsChat Input"]', {
timeout: 500,
});
await page
.getByTestId("inputsChat Input")
.dragTo(page.locator('//*[@id="react-flow-id"]'));
await zoomOut(page);
//---------------------------------- CREATE LIST
await page.getByTestId("sidebar-search-input").click();
await page.getByTestId("sidebar-search-input").fill("list");
await page.waitForSelector('[data-testid="helpersCreate List"]', {
timeout: 500,
});
await page
.getByTestId("helpersCreate List")
.dragTo(page.locator('//*[@id="react-flow-id"]'), {
targetPosition: { x: 100, y: 100 },
//---------------------------------- CREATE LIST
await page.getByTestId("sidebar-search-input").click();
await page.getByTestId("sidebar-search-input").fill("list");
await page.waitForSelector('[data-testid="helpersCreate List"]', {
timeout: 500,
});
await page
.getByTestId("helpersCreate List")
.dragTo(page.locator('//*[@id="react-flow-id"]'), {
targetPosition: { x: 100, y: 100 },
});
await page.getByTestId("input-list-plus-btn_texts-0").first().click();
await page.getByTestId("input-list-plus-btn_texts-0").first().click();
await page.getByTestId("input-list-plus-btn_texts-0").first().click();
await page
.getByTestId("inputlist_str_texts_0")
.first()
.fill("big news! langflow 1.0 is out");
await page
.getByTestId("inputlist_str_texts_1")
.first()
.fill("uhul that movie was awesome");
await page.getByTestId("inputlist_str_texts_2").first().fill("love you babe");
await page
.getByTestId("helpersCreate List")
.dragTo(page.locator('//*[@id="react-flow-id"]'), {
targetPosition: { x: 300, y: 300 },
await page.getByTestId("input-list-plus-btn_texts-0").first().click();
await page.getByTestId("input-list-plus-btn_texts-0").first().click();
await page.getByTestId("input-list-plus-btn_texts-0").first().click();
await page
.getByTestId("inputlist_str_texts_0")
.first()
.fill("big news! langflow 1.0 is out");
await page
.getByTestId("inputlist_str_texts_1")
.first()
.fill("uhul that movie was awesome");
await page
.getByTestId("inputlist_str_texts_2")
.first()
.fill("love you babe");
await page
.getByTestId("helpersCreate List")
.dragTo(page.locator('//*[@id="react-flow-id"]'), {
targetPosition: { x: 300, y: 300 },
});
await page.getByTestId("input-list-plus-btn_texts-0").last().click();
await page.getByTestId("input-list-plus-btn_texts-0").last().click();
await page.getByTestId("input-list-plus-btn_texts-0").last().click();
await page
.getByTestId("inputlist_str_texts_0")
.last()
.fill("oh my cat died");
await page
.getByTestId("inputlist_str_texts_1")
.last()
.fill("No one loves me");
await page.getByTestId("inputlist_str_texts_2").last().fill("not cool..");
//---------------------------------- PARSE DATA
await page.getByTestId("sidebar-search-input").click();
await page.getByTestId("sidebar-search-input").fill("parse data");
await page.waitForSelector('[data-testid="processingParse Data"]', {
timeout: 500,
});
await page.getByTestId("input-list-plus-btn_texts-0").last().click();
await page.getByTestId("input-list-plus-btn_texts-0").last().click();
await page.getByTestId("input-list-plus-btn_texts-0").last().click();
await page.getByTestId("inputlist_str_texts_0").last().fill("oh my cat died");
await page
.getByTestId("inputlist_str_texts_1")
.last()
.fill("No one loves me");
await page.getByTestId("inputlist_str_texts_2").last().fill("not cool..");
//---------------------------------- PARSE DATA
await page.getByTestId("sidebar-search-input").click();
await page.getByTestId("sidebar-search-input").fill("parse data");
await page.waitForSelector('[data-testid="processingParse Data"]', {
timeout: 500,
});
await page
.getByTestId("processingParse Data")
.dragTo(page.locator('//*[@id="react-flow-id"]'), {
targetPosition: { x: 350, y: 100 },
});
await zoomOut(page, 1);
await page
.getByTestId("processingParse Data")
.dragTo(page.locator('//*[@id="react-flow-id"]'), {
targetPosition: { x: 50, y: 300 },
});
await zoomOut(page, 2);
await page
.getByTestId("processingParse Data")
.dragTo(page.locator('//*[@id="react-flow-id"]'), {
targetPosition: { x: 350, y: 100 },
});
await zoomOut(page, 1);
await page
.getByTestId("processingParse Data")
.dragTo(page.locator('//*[@id="react-flow-id"]'), {
targetPosition: { x: 50, y: 300 },
});
await zoomOut(page, 2);
//---------------------------------- PASS
await page.getByTestId("sidebar-search-input").click();
await page.getByTestId("sidebar-search-input").fill("pass");
await page.waitForSelector('[data-testid="logicPass"]', {
timeout: 500,
});
await page
.getByTestId("logicPass")
.dragTo(page.locator('//*[@id="react-flow-id"]'), {
targetPosition: { x: 400, y: 100 },
//---------------------------------- PASS
await page.getByTestId("sidebar-search-input").click();
await page.getByTestId("sidebar-search-input").fill("pass");
await page.waitForSelector('[data-testid="logicPass"]', {
timeout: 500,
});
await page.waitForSelector('[data-testid="logicPass"]', {
timeout: 500,
});
//---------------------------------- PASS
await page.getByTestId("sidebar-search-input").click();
await page.getByTestId("sidebar-search-input").fill("pass");
await page.waitForSelector('[data-testid="logicPass"]', {
timeout: 500,
});
await page
.getByTestId("logicPass")
.dragTo(page.locator('//*[@id="react-flow-id"]'), {
targetPosition: { x: 600, y: 200 },
await page
.getByTestId("logicPass")
.dragTo(page.locator('//*[@id="react-flow-id"]'), {
targetPosition: { x: 400, y: 100 },
});
await page.waitForSelector('[data-testid="logicPass"]', {
timeout: 500,
});
await page.waitForSelector('[data-testid="logicPass"]', {
timeout: 500,
});
//---------------------------------- PASS
await page.getByTestId("sidebar-search-input").click();
await page.getByTestId("sidebar-search-input").fill("pass");
await page.waitForSelector('[data-testid="logicPass"]', {
timeout: 500,
});
await page
.getByTestId("logicPass")
.dragTo(page.locator('//*[@id="react-flow-id"]'), {
targetPosition: { x: 650, y: 350 },
//---------------------------------- PASS
await page.getByTestId("sidebar-search-input").click();
await page.getByTestId("sidebar-search-input").fill("pass");
await page.waitForSelector('[data-testid="logicPass"]', {
timeout: 500,
});
await page.waitForSelector('[data-testid="logicPass"]', {
timeout: 500,
});
zoomOut(page, 2);
//---------------------------------- PROMPT
await page.getByTestId("sidebar-search-input").click();
await page.getByTestId("sidebar-search-input").fill("prompt");
await page.waitForSelector('[data-testid="promptsPrompt"]', {
timeout: 500,
});
await page
.getByTestId("promptsPrompt")
.dragTo(page.locator('//*[@id="react-flow-id"]'), {
targetPosition: { x: 50, y: 150 },
await page
.getByTestId("logicPass")
.dragTo(page.locator('//*[@id="react-flow-id"]'), {
targetPosition: { x: 600, y: 200 },
});
await page.waitForSelector('[data-testid="logicPass"]', {
timeout: 500,
});
//---------------------------------- PASS
await page.getByTestId("sidebar-search-input").click();
await page.getByTestId("sidebar-search-input").fill("pass");
await page.waitForSelector('[data-testid="logicPass"]', {
timeout: 500,
});
await page
.getByTestId("logicPass")
.dragTo(page.locator('//*[@id="react-flow-id"]'), {
targetPosition: { x: 650, y: 350 },
});
await page.waitForSelector('[data-testid="logicPass"]', {
timeout: 500,
});
zoomOut(page, 2);
//---------------------------------- PROMPT
await page.getByTestId("sidebar-search-input").click();
await page.getByTestId("sidebar-search-input").fill("prompt");
await page.waitForSelector('[data-testid="promptsPrompt"]', {
timeout: 500,
});
await page
.getByTestId("promptsPrompt")
.dragTo(page.locator('//*[@id="react-flow-id"]'), {
targetPosition: { x: 50, y: 150 },
});
//---------------------------------- OPENAI
await page.getByTestId("sidebar-search-input").click();
await page.getByTestId("sidebar-search-input").fill("openai");
await page.waitForSelector('[data-testid="modelsOpenAI"]', {
timeout: 500,
});
await page
.getByTestId("modelsOpenAI")
.dragTo(page.locator('//*[@id="react-flow-id"]'), {
targetPosition: { x: 50, y: 300 },
//---------------------------------- OPENAI
await page.getByTestId("sidebar-search-input").click();
await page.getByTestId("sidebar-search-input").fill("openai");
await page.waitForSelector('[data-testid="modelsOpenAI"]', {
timeout: 500,
});
await page
.getByTestId("modelsOpenAI")
.dragTo(page.locator('//*[@id="react-flow-id"]'), {
targetPosition: { x: 50, y: 300 },
});
//---------------------------------- CONDITIONAL ROUTER
await page.getByTestId("sidebar-search-input").click();
await page.getByTestId("sidebar-search-input").fill("if else");
await page.waitForSelector('[data-testid="logicIf-Else"]', {
timeout: 500,
});
await page
.getByTestId("logicIf-Else")
.dragTo(page.locator('//*[@id="react-flow-id"]'), {
targetPosition: { x: 750, y: 150 },
//---------------------------------- CONDITIONAL ROUTER
await page.getByTestId("sidebar-search-input").click();
await page.getByTestId("sidebar-search-input").fill("if else");
await page.waitForSelector('[data-testid="logicIf-Else"]', {
timeout: 500,
});
//---------------------------------- CHAT OUTPUT
await page.getByTestId("sidebar-search-input").click();
await page.getByTestId("sidebar-search-input").fill("chat output");
await page.waitForSelector('[data-testid="outputsChat Output"]', {
timeout: 500,
});
await page
.getByTestId("outputsChat Output")
.dragTo(page.locator('//*[@id="react-flow-id"]'), {
targetPosition: { x: 100, y: 75 },
await page
.getByTestId("logicIf-Else")
.dragTo(page.locator('//*[@id="react-flow-id"]'), {
targetPosition: { x: 750, y: 150 },
});
//---------------------------------- CHAT OUTPUT
await page.getByTestId("sidebar-search-input").click();
await page.getByTestId("sidebar-search-input").fill("chat output");
await page.waitForSelector('[data-testid="outputsChat Output"]', {
timeout: 500,
});
await page.waitForSelector('[data-testid="outputsChat Output"]', {
timeout: 500,
});
//---------------------------------- CHAT OUTPUT
await page.getByTestId("sidebar-search-input").click();
await page.getByTestId("sidebar-search-input").fill("chat output");
await page.waitForSelector('[data-testid="outputsChat Output"]', {
timeout: 500,
});
await page
.getByTestId("outputsChat Output")
.dragTo(page.locator('//*[@id="react-flow-id"]'), {
targetPosition: { x: 250, y: 75 },
await page
.getByTestId("outputsChat Output")
.dragTo(page.locator('//*[@id="react-flow-id"]'), {
targetPosition: { x: 100, y: 75 },
});
await page.waitForSelector('[data-testid="outputsChat Output"]', {
timeout: 500,
});
await page.waitForSelector('[data-testid="outputsChat Output"]', {
timeout: 500,
});
//----------------------------------
await page.getByTestId("fit_view").click();
//---------------------------------- EDIT PROMPT
await page.getByTestId("promptarea_prompt_template").first().click();
await page.getByTestId("modal-promptarea_prompt_template").first().fill(`
//---------------------------------- CHAT OUTPUT
await page.getByTestId("sidebar-search-input").click();
await page.getByTestId("sidebar-search-input").fill("chat output");
await page.waitForSelector('[data-testid="outputsChat Output"]', {
timeout: 500,
});
await page
.getByTestId("outputsChat Output")
.dragTo(page.locator('//*[@id="react-flow-id"]'), {
targetPosition: { x: 250, y: 75 },
});
await page.waitForSelector('[data-testid="outputsChat Output"]', {
timeout: 500,
});
//----------------------------------
await page.getByTestId("fit_view").click();
//---------------------------------- EDIT PROMPT
await page.getByTestId("promptarea_prompt_template").first().click();
await page.getByTestId("modal-promptarea_prompt_template").first().fill(`
{Condition}
Answer with either TRUE or FALSE (and nothing else).
TRUE Examples:
@ -236,123 +248,142 @@ test("should create a flow with decision", async ({ page }) => {
User: {user_message}
AI:
`);
await page.getByText("Check & Save").last().click();
//---------------------------------- MAKE CONNECTIONS
await page
.getByTestId("handle-createlist-shownode-data list-right")
.nth(0)
.click();
await page.getByTestId("handle-parsedata-shownode-data-left").nth(0).click();
await page
.getByTestId("handle-createlist-shownode-data list-right")
.nth(2)
.click();
await page.getByTestId("handle-parsedata-shownode-data-left").nth(1).click();
await page
.getByTestId("handle-chatinput-shownode-message-right")
.nth(0)
.click();
await page
.getByTestId("handle-pass-shownode-input message-left")
.nth(2)
.click();
await page.getByTestId("handle-parsedata-shownode-text-right").nth(0).click();
//quebrando aqui
await page
.getByTestId("handle-prompt-shownode-true_examples-left")
.nth(0)
.click();
await page.getByTestId("handle-parsedata-shownode-text-right").nth(2).click();
await page
.getByTestId("handle-prompt-shownode-false_examples-left")
.nth(0)
.click();
await page
.getByTestId("handle-pass-shownode-output message-right")
.nth(4)
.click();
await page
.getByTestId("handle-prompt-shownode-user_message-left")
.nth(0)
.click();
await page
.getByTestId("handle-prompt-shownode-prompt message-right")
.first()
.click();
await page
.getByTestId("handle-openaimodel-shownode-input-left")
.nth(0)
.click();
await page
.getByTestId("handle-openaimodel-shownode-text-right")
.nth(0)
.click();
await page
.getByTestId("handle-conditionalrouter-shownode-text input-left")
.nth(0)
.click();
await page.getByTestId("popover-anchor-input-match_text").fill("TRUE");
await page.getByTestId("title-Pass").nth(1).click();
await page.getByTestId("edit-button-modal").click();
await page
.getByTestId("popover-anchor-input-input_message-edit")
.nth(0)
.fill("You're Happy! 🤪");
await page.getByTestId("showignored_message").last().click();
await page.getByText("Close").last().click();
await page.getByTestId("title-Pass").nth(0).click();
await page.getByTestId("edit-button-modal").click();
await page
.getByTestId("popover-anchor-input-input_message-edit")
.nth(0)
.fill("You're Sad! 🥲");
await page.getByTestId("showignored_message").last().click();
await page.getByText("Close").last().click();
await page
.getByTestId("handle-conditionalrouter-shownode-true-right")
.nth(0)
.click();
await page
.getByTestId("handle-pass-shownode-ignored message-left")
.nth(1)
.click();
await page
.getByTestId("handle-conditionalrouter-shownode-false-right")
.nth(0)
.click();
await page
.getByTestId("handle-pass-shownode-ignored message-left")
.nth(0)
.click();
await page
.getByTestId("handle-pass-shownode-output message-right")
.nth(2)
.click();
await page.getByTestId("handle-chatoutput-shownode-text-left").nth(0).click();
await page
.getByTestId("handle-pass-shownode-output message-right")
.nth(0)
.click();
await page.getByTestId("handle-chatoutput-shownode-text-left").nth(1).click();
const apiKeyInput = page.getByTestId("popover-anchor-input-api_key");
const isApiKeyInputVisible = await apiKeyInput.isVisible();
if (isApiKeyInputVisible) {
await apiKeyInput.fill(process.env.OPENAI_API_KEY ?? "");
}
await page.getByTestId("dropdown_str_model_name").click();
await page.getByTestId("gpt-4o-1-option").click();
await page.getByTestId("fit_view").click();
await page.getByText("Playground", { exact: true }).last().click();
await page.waitForSelector('[data-testid="input-chat-playground"]', {
timeout: 100000,
});
await page.getByTestId("input-chat-playground").click();
await page
.getByTestId("input-chat-playground")
.fill("my dog is alive and happy!");
await page.waitForSelector('[data-testid="button-send"]', {
timeout: 100000,
});
await page.getByText("Check & Save").last().click();
//---------------------------------- MAKE CONNECTIONS
await page
.getByTestId("handle-createlist-shownode-data list-right")
.nth(0)
.click();
await page
.getByTestId("handle-parsedata-shownode-data-left")
.nth(0)
.click();
await page
.getByTestId("handle-createlist-shownode-data list-right")
.nth(2)
.click();
await page
.getByTestId("handle-parsedata-shownode-data-left")
.nth(1)
.click();
await page
.getByTestId("handle-chatinput-shownode-message-right")
.nth(0)
.click();
await page
.getByTestId("handle-pass-shownode-input message-left")
.nth(2)
.click();
await page
.getByTestId("handle-parsedata-shownode-text-right")
.nth(0)
.click();
//quebrando aqui
await page
.getByTestId("handle-prompt-shownode-true_examples-left")
.nth(0)
.click();
await page
.getByTestId("handle-parsedata-shownode-text-right")
.nth(2)
.click();
await page
.getByTestId("handle-prompt-shownode-false_examples-left")
.nth(0)
.click();
await page
.getByTestId("handle-pass-shownode-output message-right")
.nth(4)
.click();
await page
.getByTestId("handle-prompt-shownode-user_message-left")
.nth(0)
.click();
await page
.getByTestId("handle-prompt-shownode-prompt message-right")
.first()
.click();
await page
.getByTestId("handle-openaimodel-shownode-input-left")
.nth(0)
.click();
await page
.getByTestId("handle-openaimodel-shownode-text-right")
.nth(0)
.click();
await page
.getByTestId("handle-conditionalrouter-shownode-text input-left")
.nth(0)
.click();
await page.getByTestId("popover-anchor-input-match_text").fill("TRUE");
await page.getByTestId("title-Pass").nth(1).click();
await page.getByTestId("edit-button-modal").click();
await page
.getByTestId("popover-anchor-input-input_message-edit")
.nth(0)
.fill("You're Happy! 🤪");
await page.getByTestId("showignored_message").last().click();
await page.getByText("Close").last().click();
await page.getByTestId("title-Pass").nth(0).click();
await page.getByTestId("edit-button-modal").click();
await page
.getByTestId("popover-anchor-input-input_message-edit")
.nth(0)
.fill("You're Sad! 🥲");
await page.getByTestId("showignored_message").last().click();
await page.getByText("Close").last().click();
await page
.getByTestId("handle-conditionalrouter-shownode-true-right")
.nth(0)
.click();
await page
.getByTestId("handle-pass-shownode-ignored message-left")
.nth(1)
.click();
await page
.getByTestId("handle-conditionalrouter-shownode-false-right")
.nth(0)
.click();
await page
.getByTestId("handle-pass-shownode-ignored message-left")
.nth(0)
.click();
await page
.getByTestId("handle-pass-shownode-output message-right")
.nth(2)
.click();
await page
.getByTestId("handle-chatoutput-shownode-text-left")
.nth(0)
.click();
await page
.getByTestId("handle-pass-shownode-output message-right")
.nth(0)
.click();
await page
.getByTestId("handle-chatoutput-shownode-text-left")
.nth(1)
.click();
const apiKeyInput = page.getByTestId("popover-anchor-input-api_key");
const isApiKeyInputVisible = await apiKeyInput.isVisible();
if (isApiKeyInputVisible) {
await apiKeyInput.fill(process.env.OPENAI_API_KEY ?? "");
}
await page.getByTestId("dropdown_str_model_name").click();
await page.getByTestId("gpt-4o-1-option").click();
await page.getByTestId("fit_view").click();
await page.getByText("Playground", { exact: true }).last().click();
await page.waitForSelector('[data-testid="input-chat-playground"]', {
timeout: 100000,
});
await page.getByTestId("input-chat-playground").click();
await page
.getByTestId("input-chat-playground")
.fill("my dog is alive and happy!");
await page.waitForSelector('[data-testid="button-send"]', {
timeout: 100000,
});
await page.getByTestId("button-send").last().click();
});
await page.getByTestId("button-send").last().click();
},
);

603
src/frontend/tests/core/integrations/similarity.spec.ts
View file

@ -1,336 +1,341 @@
import { expect, test } from "@playwright/test";
test("user must be able to check similarity between embedding texts", async ({
page,
}) => {
test.skip(
!process?.env?.OPENAI_API_KEY,
"OPENAI_API_KEY required to run this test",
);
test(
"user must be able to check similarity between embedding texts",
{ tag: ["@release", "@components"] },
await page.goto("/");
await page.waitForTimeout(500);
async ({ page }) => {
test.skip(
!process?.env?.OPENAI_API_KEY,
"OPENAI_API_KEY required to run this test",
);
let modalCount = 0;
try {
const modalTitleElement = await page?.getByTestId("modal-title");
if (modalTitleElement) {
modalCount = await modalTitleElement.count();
await page.goto("/");
await page.waitForTimeout(500);
let modalCount = 0;
try {
const modalTitleElement = await page?.getByTestId("modal-title");
if (modalTitleElement) {
modalCount = await modalTitleElement.count();
}
} catch (error) {
modalCount = 0;
}
} catch (error) {
modalCount = 0;
}
while (modalCount === 0) {
await page.getByText("New Flow", { exact: true }).click();
await page.waitForTimeout(3000);
modalCount = await page.getByTestId("modal-title")?.count();
}
while (modalCount === 0) {
await page.getByText("New Flow", { exact: true }).click();
await page.waitForTimeout(3000);
modalCount = await page.getByTestId("modal-title")?.count();
}
await page.getByTestId("blank-flow").click();
await page.getByTestId("blank-flow").click();
//first component
await page.getByTestId("sidebar-search-input").click();
await page.getByTestId("sidebar-search-input").fill("openai embedding");
await page.waitForTimeout(500);
//first component
await page.getByTestId("sidebar-search-input").click();
await page.getByTestId("sidebar-search-input").fill("openai embedding");
await page.waitForTimeout(500);
await page
.getByText("OpenAI Embeddings", { exact: true })
.dragTo(page.locator('//*[@id="react-flow-id"]'));
await page
.getByText("OpenAI Embeddings", { exact: true })
.dragTo(page.locator('//*[@id="react-flow-id"]'));
await page.getByTestId("zoom_out").click();
await page
.locator('//*[@id="react-flow-id"]')
.hover()
.then(async () => {
await page.mouse.down();
await page.mouse.move(-50, 50);
});
await page.getByTestId("zoom_out").click();
await page
.locator('//*[@id="react-flow-id"]')
.hover()
.then(async () => {
await page.mouse.down();
await page.mouse.move(-50, 50);
});
await page.mouse.up();
await page.mouse.up();
await page.waitForTimeout(1000);
await page.getByTestId("sidebar-search-input").click();
await page.getByTestId("sidebar-search-input").fill("text embedder");
await page.waitForTimeout(500);
await page
.getByTestId("embeddingsText Embedder")
.dragTo(page.locator('//*[@id="react-flow-id"]'));
await page.getByTestId("zoom_out").click();
await page
.locator('//*[@id="react-flow-id"]')
.hover()
.then(async () => {
await page.mouse.down();
await page.mouse.move(-50, 50);
});
await page.mouse.up();
//fourth component
await page
.getByTestId("embeddingsText Embedder")
.dragTo(page.locator('//*[@id="react-flow-id"]'));
await page.getByTestId("zoom_out").click();
await page
.locator('//*[@id="react-flow-id"]')
.hover()
.then(async () => {
await page.mouse.down();
await page.mouse.move(-50, 50);
});
await page.mouse.up();
//fifth component
await page.getByTestId("sidebar-search-input").click();
await page.getByTestId("sidebar-search-input").fill("embedding similarity");
await page.waitForTimeout(500);
await page
.getByTestId("embeddingsEmbedding Similarity")
.dragTo(page.locator('//*[@id="react-flow-id"]'));
await page.getByTestId("zoom_out").click();
await page
.locator('//*[@id="react-flow-id"]')
.hover()
.then(async () => {
await page.mouse.down();
await page.mouse.move(-50, 50);
});
await page.mouse.up();
//sisxth component
await page.getByTestId("sidebar-search-input").click();
await page.getByTestId("sidebar-search-input").fill("parse data");
await page.waitForTimeout(500);
await page
.getByTestId("processingParse Data")
.dragTo(page.locator('//*[@id="react-flow-id"]'));
await page.getByTestId("zoom_out").click();
await page
.locator('//*[@id="react-flow-id"]')
.hover()
.then(async () => {
await page.mouse.down();
await page.mouse.move(-50, 50);
});
await page.mouse.up();
//seventh component
await page.getByTestId("sidebar-search-input").click();
await page.getByTestId("sidebar-search-input").fill("text output");
await page.waitForTimeout(500);
await page
.getByTestId("outputsText Output")
.dragTo(page.locator('//*[@id="react-flow-id"]'));
await page.getByTestId("zoom_out").click();
await page
.locator('//*[@id="react-flow-id"]')
.hover()
.then(async () => {
await page.mouse.down();
await page.mouse.move(-50, 50);
});
await page.mouse.up();
await page.getByTestId("sidebar-search-input").click();
await page.getByTestId("sidebar-search-input").fill("filter data");
await page.waitForTimeout(500);
await page
.getByTestId("processingFilter Data")
.dragTo(page.locator('//*[@id="react-flow-id"]'));
await page.getByTestId("zoom_out").click();
await page
.locator('//*[@id="react-flow-id"]')
.hover()
.then(async () => {
await page.mouse.down();
await page.mouse.move(-50, 50);
});
await page.mouse.up();
let outdatedComponents = await page.getByTestId("icon-AlertTriangle").count();
while (outdatedComponents > 0) {
await page.getByTestId("icon-AlertTriangle").first().click();
// await page.waitForTimeout(1000);
outdatedComponents = await page.getByTestId("icon-AlertTriangle").count();
}
let filledApiKey = await page.getByTestId("remove-icon-badge").count();
while (filledApiKey > 0) {
await page.getByTestId("remove-icon-badge").first().click();
await page.waitForTimeout(1000);
filledApiKey = await page.getByTestId("remove-icon-badge").count();
}
await page.getByTestId("fit_view").click();
await page.getByTestId("sidebar-search-input").click();
await page.getByTestId("sidebar-search-input").fill("text embedder");
await page.waitForTimeout(500);
await page
.getByTestId("textarea_str_template")
.last()
.fill("{similarity_score}");
await page
.getByTestId("embeddingsText Embedder")
.dragTo(page.locator('//*[@id="react-flow-id"]'));
await page
.getByTestId("popover-anchor-input-message")
.last()
.fill("datastax");
await page
.getByTestId("popover-anchor-input-message")
.first()
.fill("langflow");
await page.getByTestId("zoom_out").click();
await page
.locator('//*[@id="react-flow-id"]')
.hover()
.then(async () => {
await page.mouse.down();
await page.mouse.move(-50, 50);
});
const firstApiKeyInput = page
.getByTestId("popover-anchor-input-openai_api_key")
.nth(0);
const secondApiKeyInput = page
.getByTestId("popover-anchor-input-openai_api_key")
.nth(1);
await page.mouse.up();
const isFirstInputVisible = await firstApiKeyInput.isVisible();
const isSecondInputVisible = await secondApiKeyInput.isVisible();
//fourth component
if (isFirstInputVisible) {
await firstApiKeyInput.fill(process.env.OPENAI_API_KEY ?? "");
}
await page
.getByTestId("embeddingsText Embedder")
.dragTo(page.locator('//*[@id="react-flow-id"]'));
if (isSecondInputVisible) {
await secondApiKeyInput.fill(process.env.OPENAI_API_KEY ?? "");
}
await page.getByTestId("zoom_out").click();
await page
.locator('//*[@id="react-flow-id"]')
.hover()
.then(async () => {
await page.mouse.down();
await page.mouse.move(-50, 50);
});
await page
.getByTestId("inputlist_str_filter_criteria_0")
.nth(0)
.fill("similarity_score");
await page.mouse.up();
await page.getByTestId("fit_view").click();
await page.mouse.wheel(0, 500);
//fifth component
await page.locator(".react-flow__pane").click();
await page.getByTestId("sidebar-search-input").click();
await page.getByTestId("sidebar-search-input").fill("embedding similarity");
await page.waitForTimeout(500);
await page.getByTestId("fit_view").click();
await page
.getByTestId("embeddingsEmbedding Similarity")
.dragTo(page.locator('//*[@id="react-flow-id"]'));
//connection 1
const openAiEmbeddingOutput_0 = await page
.getByTestId("handle-openaiembeddings-shownode-embeddings-right")
.nth(0);
await openAiEmbeddingOutput_0.hover();
await page.mouse.down();
await page.getByTestId("zoom_out").click();
await page
.locator('//*[@id="react-flow-id"]')
.hover()
.then(async () => {
await page.mouse.down();
await page.mouse.move(-50, 50);
});
const textEmbedderInput_0 = await page
.getByTestId("handle-textembeddercomponent-shownode-embedding model-left")
.nth(0);
await textEmbedderInput_0.hover();
await page.mouse.up();
await page.mouse.up();
//connection 2
const openAiEmbeddingOutput_1 = await page
.getByTestId("handle-openaiembeddings-shownode-embeddings-right")
.nth(0);
await openAiEmbeddingOutput_1.hover();
await page.mouse.down();
const textEmbedderInput_1 = await page
.getByTestId("handle-textembeddercomponent-shownode-embedding model-left")
.nth(1);
await textEmbedderInput_1.hover();
await page.mouse.up();
//sisxth component
//connection 3
const textEmbedderOutput_0 = await page
.getByTestId("handle-textembeddercomponent-shownode-embedding data-right")
.nth(0);
await textEmbedderOutput_0.hover();
await page.mouse.down();
const embeddingSimilarityInput = await page
.getByTestId(
"handle-embeddingsimilaritycomponent-shownode-embedding vectors-left",
)
.nth(0);
await embeddingSimilarityInput.hover();
await page.mouse.up();
await page.getByTestId("sidebar-search-input").click();
await page.getByTestId("sidebar-search-input").fill("parse data");
await page.waitForTimeout(500);
//connection 4
const textEmbedderOutput_1 = await page
.getByTestId("handle-textembeddercomponent-shownode-embedding data-right")
.nth(2);
await textEmbedderOutput_1.hover();
await page.mouse.down();
await embeddingSimilarityInput.hover();
await page.mouse.up();
await page
.getByTestId("processingParse Data")
.dragTo(page.locator('//*[@id="react-flow-id"]'));
//connection 5
const embeddingSimilarityOutput = await page
.getByTestId(
"handle-embeddingsimilaritycomponent-shownode-similarity data-right",
)
.nth(0);
await embeddingSimilarityOutput.hover();
await page.mouse.down();
const filterDataInput = await page
.getByTestId("handle-filterdata-shownode-data-left")
.nth(0);
await filterDataInput.hover();
await page.mouse.up();
await page.getByTestId("zoom_out").click();
await page
.locator('//*[@id="react-flow-id"]')
.hover()
.then(async () => {
await page.mouse.down();
await page.mouse.move(-50, 50);
});
//connection 6
const filterDataOutput = await page
.getByTestId("handle-filterdata-shownode-filtered data-right")
.nth(0);
await filterDataOutput.hover();
await page.mouse.down();
const parseDataInput = await page
.getByTestId("handle-parsedata-shownode-data-left")
.nth(0);
await parseDataInput.hover();
await page.mouse.up();
await page.mouse.up();
//connection 7
const parseDataOutput = await page
.getByTestId("handle-parsedata-shownode-text-right")
.nth(0);
await parseDataOutput.hover();
await page.mouse.down();
const textOutputInput = await page
.getByTestId("handle-textoutput-shownode-text-left")
.nth(0);
await textOutputInput.hover();
await page.mouse.up();
//seventh component
await page.waitForTimeout(3000);
await page.getByTestId("sidebar-search-input").click();
await page.getByTestId("sidebar-search-input").fill("text output");
await page.waitForTimeout(500);
await page.getByTestId("button_run_text output").click();
await page
.getByTestId("outputsText Output")
.dragTo(page.locator('//*[@id="react-flow-id"]'));
await page.waitForSelector("text=built successfully", { timeout: 30000 });
await page.getByTestId("zoom_out").click();
await page
.locator('//*[@id="react-flow-id"]')
.hover()
.then(async () => {
await page.mouse.down();
await page.mouse.move(-50, 50);
});
await page.waitForTimeout(1000);
await page
.getByTestId(/rf__node-TextOutput-[a-zA-Z0-9]{5}/)
.getByTestId("output-inspection-text")
.first()
.click();
const valueSimilarity = await page.getByTestId("textarea").textContent();
await page.mouse.up();
expect(valueSimilarity).toContain("cosine_similarity");
const valueLength = valueSimilarity!.length;
expect(valueLength).toBeGreaterThan(20);
});
await page.getByTestId("sidebar-search-input").click();
await page.getByTestId("sidebar-search-input").fill("filter data");
await page.waitForTimeout(500);
await page
.getByTestId("processingFilter Data")
.dragTo(page.locator('//*[@id="react-flow-id"]'));
await page.getByTestId("zoom_out").click();
await page
.locator('//*[@id="react-flow-id"]')
.hover()
.then(async () => {
await page.mouse.down();
await page.mouse.move(-50, 50);
});
await page.mouse.up();
let outdatedComponents = await page
.getByTestId("icon-AlertTriangle")
.count();
while (outdatedComponents > 0) {
await page.getByTestId("icon-AlertTriangle").first().click();
// await page.waitForTimeout(1000);
outdatedComponents = await page.getByTestId("icon-AlertTriangle").count();
}
let filledApiKey = await page.getByTestId("remove-icon-badge").count();
while (filledApiKey > 0) {
await page.getByTestId("remove-icon-badge").first().click();
await page.waitForTimeout(1000);
filledApiKey = await page.getByTestId("remove-icon-badge").count();
}
await page.getByTestId("fit_view").click();
await page
.getByTestId("textarea_str_template")
.last()
.fill("{similarity_score}");
await page
.getByTestId("popover-anchor-input-message")
.last()
.fill("datastax");
await page
.getByTestId("popover-anchor-input-message")
.first()
.fill("langflow");
const firstApiKeyInput = page
.getByTestId("popover-anchor-input-openai_api_key")
.nth(0);
const secondApiKeyInput = page
.getByTestId("popover-anchor-input-openai_api_key")
.nth(1);
const isFirstInputVisible = await firstApiKeyInput.isVisible();
const isSecondInputVisible = await secondApiKeyInput.isVisible();
if (isFirstInputVisible) {
await firstApiKeyInput.fill(process.env.OPENAI_API_KEY ?? "");
}
if (isSecondInputVisible) {
await secondApiKeyInput.fill(process.env.OPENAI_API_KEY ?? "");
}
await page
.getByTestId("inputlist_str_filter_criteria_0")
.nth(0)
.fill("similarity_score");
await page.getByTestId("fit_view").click();
await page.mouse.wheel(0, 500);
await page.locator(".react-flow__pane").click();
await page.getByTestId("fit_view").click();
//connection 1
const openAiEmbeddingOutput_0 = await page
.getByTestId("handle-openaiembeddings-shownode-embeddings-right")
.nth(0);
await openAiEmbeddingOutput_0.hover();
await page.mouse.down();
const textEmbedderInput_0 = await page
.getByTestId("handle-textembeddercomponent-shownode-embedding model-left")
.nth(0);
await textEmbedderInput_0.hover();
await page.mouse.up();
//connection 2
const openAiEmbeddingOutput_1 = await page
.getByTestId("handle-openaiembeddings-shownode-embeddings-right")
.nth(0);
await openAiEmbeddingOutput_1.hover();
await page.mouse.down();
const textEmbedderInput_1 = await page
.getByTestId("handle-textembeddercomponent-shownode-embedding model-left")
.nth(1);
await textEmbedderInput_1.hover();
await page.mouse.up();
//connection 3
const textEmbedderOutput_0 = await page
.getByTestId("handle-textembeddercomponent-shownode-embedding data-right")
.nth(0);
await textEmbedderOutput_0.hover();
await page.mouse.down();
const embeddingSimilarityInput = await page
.getByTestId(
"handle-embeddingsimilaritycomponent-shownode-embedding vectors-left",
)
.nth(0);
await embeddingSimilarityInput.hover();
await page.mouse.up();
//connection 4
const textEmbedderOutput_1 = await page
.getByTestId("handle-textembeddercomponent-shownode-embedding data-right")
.nth(2);
await textEmbedderOutput_1.hover();
await page.mouse.down();
await embeddingSimilarityInput.hover();
await page.mouse.up();
//connection 5
const embeddingSimilarityOutput = await page
.getByTestId(
"handle-embeddingsimilaritycomponent-shownode-similarity data-right",
)
.nth(0);
await embeddingSimilarityOutput.hover();
await page.mouse.down();
const filterDataInput = await page
.getByTestId("handle-filterdata-shownode-data-left")
.nth(0);
await filterDataInput.hover();
await page.mouse.up();
//connection 6
const filterDataOutput = await page
.getByTestId("handle-filterdata-shownode-filtered data-right")
.nth(0);
await filterDataOutput.hover();
await page.mouse.down();
const parseDataInput = await page
.getByTestId("handle-parsedata-shownode-data-left")
.nth(0);
await parseDataInput.hover();
await page.mouse.up();
//connection 7
const parseDataOutput = await page
.getByTestId("handle-parsedata-shownode-text-right")
.nth(0);
await parseDataOutput.hover();
await page.mouse.down();
const textOutputInput = await page
.getByTestId("handle-textoutput-shownode-text-left")
.nth(0);
await textOutputInput.hover();
await page.mouse.up();
await page.waitForTimeout(3000);
await page.getByTestId("button_run_text output").click();
await page.waitForSelector("text=built successfully", { timeout: 30000 });
await page.waitForTimeout(1000);
await page
.getByTestId(/rf__node-TextOutput-[a-zA-Z0-9]{5}/)
.getByTestId("output-inspection-text")
.first()
.click();
const valueSimilarity = await page.getByTestId("textarea").textContent();
expect(valueSimilarity).toContain("cosine_similarity");
const valueLength = valueSimilarity!.length;
expect(valueLength).toBeGreaterThan(20);
},
);

164
src/frontend/tests/core/integrations/starter-projects.spec.ts
View file

@ -1,92 +1,96 @@
const { test, expect } = require("@playwright/test");
test("vector store from starter projects should have its connections and nodes on the flow", async ({
page,
request,
}) => {
const response = await request.get("/api/v1/starter-projects");
expect(response.status()).toBe(200);
const responseBody = await response.json();
test(
"vector store from starter projects should have its connections and nodes on the flow",
{ tag: ["@release", "@starter-project"] },
async ({ page, request }) => {
const response = await request.get("/api/v1/starter-projects");
expect(response.status()).toBe(200);
const responseBody = await response.json();
const astraStarterProject = responseBody.find((project) => {
if (project.data.nodes) {
return project.data.nodes.some((node) => node.id.includes("Astra"));
}
});
await page.route("**/api/v1/flows/", async (route) => {
if (route.request().method() === "GET") {
try {
const response = await route.fetch();
const flowsData = await response.json();
const modifiedFlows = flowsData.map((flow) => {
if (flow.name === "Vector Store RAG" && flow.user_id === null) {
return {
...flow,
data: astraStarterProject?.data,
};
}
return flow;
});
const modifiedResponse = JSON.stringify(modifiedFlows);
route.fulfill({
status: response.status(),
headers: response.headers(),
body: modifiedResponse,
});
} catch (error) {
console.error("Error in route handler:", error);
const astraStarterProject = responseBody.find((project) => {
if (project.data.nodes) {
return project.data.nodes.some((node) => node.id.includes("Astra"));
}
} else {
// If not a GET request, continue without modifying
await route.continue();
});
await page.route("**/api/v1/flows/", async (route) => {
if (route.request().method() === "GET") {
try {
const response = await route.fetch();
const flowsData = await response.json();
const modifiedFlows = flowsData.map((flow) => {
if (flow.name === "Vector Store RAG" && flow.user_id === null) {
return {
...flow,
data: astraStarterProject?.data,
};
}
return flow;
});
const modifiedResponse = JSON.stringify(modifiedFlows);
route.fulfill({
status: response.status(),
headers: response.headers(),
body: modifiedResponse,
});
} catch (error) {
console.error("Error in route handler:", error);
}
} else {
// If not a GET request, continue without modifying
await route.continue();
}
});
await page.goto("/");
await page.waitForSelector('[data-testid="mainpage_title"]', {
timeout: 30000,
});
await page.waitForSelector('[id="new-project-btn"]', {
timeout: 30000,
});
let modalCount = 0;
try {
const modalTitleElement = await page?.getByTestId("modal-title");
if (modalTitleElement) {
modalCount = await modalTitleElement.count();
}
} catch (error) {
modalCount = 0;
}
});
await page.goto("/");
await page.waitForSelector('[data-testid="mainpage_title"]', {
timeout: 30000,
});
await page.waitForSelector('[id="new-project-btn"]', {
timeout: 30000,
});
let modalCount = 0;
try {
const modalTitleElement = await page?.getByTestId("modal-title");
if (modalTitleElement) {
modalCount = await modalTitleElement.count();
while (modalCount === 0) {
await page.getByText("New Flow", { exact: true }).click();
await page.waitForTimeout(3000);
modalCount = await page.getByTestId("modal-title")?.count();
}
} catch (error) {
modalCount = 0;
}
while (modalCount === 0) {
await page.getByText("New Flow", { exact: true }).click();
await page.waitForTimeout(3000);
modalCount = await page.getByTestId("modal-title")?.count();
}
await page.getByTestId("side_nav_options_all-templates").click();
await page
.getByRole("heading", { name: "Vector Store RAG" })
.first()
.click();
await page.waitForSelector('[data-testid="fit_view"]', {
timeout: 100000,
});
await page.getByTestId("side_nav_options_all-templates").click();
await page.getByRole("heading", { name: "Vector Store RAG" }).first().click();
await page.waitForSelector('[data-testid="fit_view"]', {
timeout: 100000,
});
await page.getByTestId("fit_view").click();
await page.getByTestId("zoom_out").click();
await page.getByTestId("fit_view").click();
await page.getByTestId("zoom_out").click();
const edges = await page.locator(".react-flow__edge-interaction").count();
const nodes = await page.getByTestId("div-generic-node").count();
const edges = await page.locator(".react-flow__edge-interaction").count();
const nodes = await page.getByTestId("div-generic-node").count();
const edgesFromServer = astraStarterProject?.data.edges.length;
const nodesFromServer = astraStarterProject?.data.nodes.length;
const edgesFromServer = astraStarterProject?.data.edges.length;
const nodesFromServer = astraStarterProject?.data.nodes.length;
expect(edges).toBe(edgesFromServer);
expect(nodes).toBe(nodesFromServer);
});
expect(edges).toBe(edgesFromServer);
expect(nodes).toBe(nodesFromServer);
},
);

382
src/frontend/tests/core/integrations/textInputOutput.spec.ts
View file

@ -2,194 +2,204 @@ import { expect, test } from "@playwright/test";
import * as dotenv from "dotenv";
import path from "path";
test.skip("TextInputOutputComponent", async ({ page }) => {
// commented out because new playground does not support text io yet
test.skip(
!process?.env?.OPENAI_API_KEY,
"OPENAI_API_KEY required to run this test",
);
if (!process.env.CI) {
dotenv.config({ path: path.resolve(__dirname, "../../.env") });
}
await page.goto("/");
await page.waitForSelector('[data-testid="mainpage_title"]', {
timeout: 30000,
});
await page.waitForSelector('[id="new-project-btn"]', {
timeout: 30000,
});
let modalCount = 0;
try {
const modalTitleElement = await page?.getByTestId("modal-title");
if (modalTitleElement) {
modalCount = await modalTitleElement.count();
test.skip(
"TextInputOutputComponent",
{ tag: ["@release", "@components"] },
async ({ page }) => {
// commented out because new playground does not support text io yet
test.skip(
!process?.env?.OPENAI_API_KEY,
"OPENAI_API_KEY required to run this test",
);
if (!process.env.CI) {
dotenv.config({ path: path.resolve(__dirname, "../../.env") });
}
} catch (error) {
modalCount = 0;
}
while (modalCount === 0) {
await page.getByText("New Flow", { exact: true }).click();
await page.waitForTimeout(3000);
modalCount = await page.getByTestId("modal-title")?.count();
}
await page.waitForSelector('[data-testid="blank-flow"]', {
timeout: 30000,
});
await page.getByTestId("blank-flow").click();
await page.getByTestId("sidebar-search-input").click();
await page.getByTestId("sidebar-search-input").fill("text input");
await page.waitForTimeout(1000);
await page
.getByTestId("inputsText Input")
.dragTo(page.locator('//*[@id="react-flow-id"]'));
await page.mouse.up();
await page.mouse.down();
await page.getByTestId("sidebar-search-input").click();
await page.getByTestId("sidebar-search-input").fill("openai");
await page.waitForTimeout(1000);
await page
.getByTestId("modelsOpenAI")
.dragTo(page.locator('//*[@id="react-flow-id"]'));
await page.mouse.up();
await page.mouse.down();
await page.waitForSelector('[data-testid="fit_view"]', {
timeout: 100000,
});
await page.getByTestId("fit_view").click();
await page.getByTestId("zoom_out").click();
await page.getByTestId("zoom_out").click();
await page.getByTestId("zoom_out").click();
let visibleElementHandle;
const elementsTextInputOutput = await page
.getByTestId("handle-textinput-shownode-text-right")
.all();
for (const element of elementsTextInputOutput) {
if (await element.isVisible()) {
visibleElementHandle = element;
break;
await page.goto("/");
await page.waitForSelector('[data-testid="mainpage_title"]', {
timeout: 30000,
});
await page.waitForSelector('[id="new-project-btn"]', {
timeout: 30000,
});
let modalCount = 0;
try {
const modalTitleElement = await page?.getByTestId("modal-title");
if (modalTitleElement) {
modalCount = await modalTitleElement.count();
}
} catch (error) {
modalCount = 0;
}
}
await visibleElementHandle.waitFor({
state: "visible",
timeout: 30000,
});
await visibleElementHandle.hover();
await page.mouse.down();
for (const element of elementsTextInputOutput) {
if (await element.isVisible()) {
visibleElementHandle = element;
break;
while (modalCount === 0) {
await page.getByText("New Flow", { exact: true }).click();
await page.waitForTimeout(3000);
modalCount = await page.getByTestId("modal-title")?.count();
}
}
await visibleElementHandle.waitFor({
state: "visible",
timeout: 30000,
});
// Move to the second element
await visibleElementHandle.hover();
// Release the mouse
await page.mouse.up();
await page.getByTestId("sidebar-search-input").click();
await page.getByTestId("sidebar-search-input").fill("text output");
await page
.getByTestId("outputsText Output")
.dragTo(page.locator('//*[@id="react-flow-id"]'));
await page.mouse.up();
await page.mouse.down();
await page.waitForSelector('[data-testid="fit_view"]', {
timeout: 100000,
});
await page.getByTestId("zoom_out").click();
await page.getByTestId("zoom_out").click();
await page.getByTestId("zoom_out").click();
await page.getByTestId("zoom_out").click();
await page.getByTestId("zoom_out").click();
await page.getByTestId("zoom_out").click();
const elementsOpenAiOutput = await page
.getByTestId("handle-openaimodel-shownode-text-right")
.all();
for (const element of elementsOpenAiOutput) {
if (await element.isVisible()) {
visibleElementHandle = element;
break;
}
}
await visibleElementHandle.waitFor({
state: "visible",
timeout: 30000,
});
// Click and hold on the first element
await visibleElementHandle.hover();
await page.mouse.down();
const elementTextOutputInput = await page
.getByTestId("handle-textoutput-shownode-text-left")
.all();
for (const element of elementTextOutputInput) {
if (await element.isVisible()) {
visibleElementHandle = element;
break;
}
}
await visibleElementHandle.waitFor({
state: "visible",
timeout: 30000,
});
// Move to the second element
await visibleElementHandle.hover();
// Release the mouse
await page.mouse.up();
await page
.getByTestId(/^rf__node-TextInput-[a-zA-Z0-9]+$/)
.getByTestId("textarea_str_input_value")
.fill("This is a test!");
let outdatedComponents = await page.getByTestId("icon-AlertTriangle").count();
while (outdatedComponents > 0) {
await page.getByTestId("icon-AlertTriangle").first().click();
await page.waitForSelector('[data-testid="blank-flow"]', {
timeout: 30000,
});
await page.getByTestId("blank-flow").click();
await page.getByTestId("sidebar-search-input").click();
await page.getByTestId("sidebar-search-input").fill("text input");
await page.waitForTimeout(1000);
outdatedComponents = await page.getByTestId("icon-AlertTriangle").count();
}
let filledApiKey = await page.getByTestId("remove-icon-badge").count();
while (filledApiKey > 0) {
await page.getByTestId("remove-icon-badge").first().click();
await page
.getByTestId("inputsText Input")
.dragTo(page.locator('//*[@id="react-flow-id"]'));
await page.mouse.up();
await page.mouse.down();
await page.getByTestId("sidebar-search-input").click();
await page.getByTestId("sidebar-search-input").fill("openai");
await page.waitForTimeout(1000);
filledApiKey = await page.getByTestId("remove-icon-badge").count();
}
const apiKeyInput = page.getByTestId("popover-anchor-input-api_key");
const isApiKeyInputVisible = await apiKeyInput.isVisible();
if (isApiKeyInputVisible) {
await apiKeyInput.fill(process.env.OPENAI_API_KEY ?? "");
}
await page.getByTestId("dropdown_str_model_name").click();
await page.getByTestId("gpt-4o-1-option").click();
await page.waitForTimeout(1000);
await page.getByText("Playground", { exact: true }).last().click();
await page.getByTestId("button_run_text_output").click();
await page
.getByTestId(/^rf__node-TextOutput-[a-zA-Z0-9]+$/)
.getByTestId("output-inspection-text")
.click();
await page.getByText("Run Flow", { exact: true }).click();
await page.waitForTimeout(5000);
let textInputContent = await page
.getByPlaceholder("Enter text...")
.textContent();
expect(textInputContent).toBe("This is a test!");
await page.getByText("Outputs", { exact: true }).nth(1).click();
await page.getByText("Text Output", { exact: true }).nth(2).click();
let contentOutput = await page.getByPlaceholder("Enter text...").inputValue();
expect(contentOutput).not.toBe(null);
await page.keyboard.press("Escape");
await page
.getByTestId(/^rf__node-TextInput-[a-zA-Z0-9]+$/)
.getByTestId("textarea_str_input_value")
.fill("This is a test, again just to be sure!");
await page.getByText("Playground", { exact: true }).last().click();
await page.getByText("Run Flow", { exact: true }).click();
await page.waitForTimeout(5000);
textInputContent = await page.getByPlaceholder("Enter text...").textContent();
expect(textInputContent).toBe("This is a test, again just to be sure!");
await page.getByText("Outputs", { exact: true }).nth(1).click();
await page.getByText("Text Output", { exact: true }).nth(2).click();
contentOutput = await page.getByPlaceholder("Enter text...").inputValue();
expect(contentOutput).not.toBe(null);
});
await page
.getByTestId("modelsOpenAI")
.dragTo(page.locator('//*[@id="react-flow-id"]'));
await page.mouse.up();
await page.mouse.down();
await page.waitForSelector('[data-testid="fit_view"]', {
timeout: 100000,
});
await page.getByTestId("fit_view").click();
await page.getByTestId("zoom_out").click();
await page.getByTestId("zoom_out").click();
await page.getByTestId("zoom_out").click();
let visibleElementHandle;
const elementsTextInputOutput = await page
.getByTestId("handle-textinput-shownode-text-right")
.all();
for (const element of elementsTextInputOutput) {
if (await element.isVisible()) {
visibleElementHandle = element;
break;
}
}
await visibleElementHandle.waitFor({
state: "visible",
timeout: 30000,
});
await visibleElementHandle.hover();
await page.mouse.down();
for (const element of elementsTextInputOutput) {
if (await element.isVisible()) {
visibleElementHandle = element;
break;
}
}
await visibleElementHandle.waitFor({
state: "visible",
timeout: 30000,
});
// Move to the second element
await visibleElementHandle.hover();
// Release the mouse
await page.mouse.up();
await page.getByTestId("sidebar-search-input").click();
await page.getByTestId("sidebar-search-input").fill("text output");
await page
.getByTestId("outputsText Output")
.dragTo(page.locator('//*[@id="react-flow-id"]'));
await page.mouse.up();
await page.mouse.down();
await page.waitForSelector('[data-testid="fit_view"]', {
timeout: 100000,
});
await page.getByTestId("zoom_out").click();
await page.getByTestId("zoom_out").click();
await page.getByTestId("zoom_out").click();
await page.getByTestId("zoom_out").click();
await page.getByTestId("zoom_out").click();
await page.getByTestId("zoom_out").click();
const elementsOpenAiOutput = await page
.getByTestId("handle-openaimodel-shownode-text-right")
.all();
for (const element of elementsOpenAiOutput) {
if (await element.isVisible()) {
visibleElementHandle = element;
break;
}
}
await visibleElementHandle.waitFor({
state: "visible",
timeout: 30000,
});
// Click and hold on the first element
await visibleElementHandle.hover();
await page.mouse.down();
const elementTextOutputInput = await page
.getByTestId("handle-textoutput-shownode-text-left")
.all();
for (const element of elementTextOutputInput) {
if (await element.isVisible()) {
visibleElementHandle = element;
break;
}
}
await visibleElementHandle.waitFor({
state: "visible",
timeout: 30000,
});
// Move to the second element
await visibleElementHandle.hover();
// Release the mouse
await page.mouse.up();
await page
.getByTestId(/^rf__node-TextInput-[a-zA-Z0-9]+$/)
.getByTestId("textarea_str_input_value")
.fill("This is a test!");
let outdatedComponents = await page
.getByTestId("icon-AlertTriangle")
.count();
while (outdatedComponents > 0) {
await page.getByTestId("icon-AlertTriangle").first().click();
await page.waitForTimeout(1000);
outdatedComponents = await page.getByTestId("icon-AlertTriangle").count();
}
let filledApiKey = await page.getByTestId("remove-icon-badge").count();
while (filledApiKey > 0) {
await page.getByTestId("remove-icon-badge").first().click();
await page.waitForTimeout(1000);
filledApiKey = await page.getByTestId("remove-icon-badge").count();
}
const apiKeyInput = page.getByTestId("popover-anchor-input-api_key");
const isApiKeyInputVisible = await apiKeyInput.isVisible();
if (isApiKeyInputVisible) {
await apiKeyInput.fill(process.env.OPENAI_API_KEY ?? "");
}
await page.getByTestId("dropdown_str_model_name").click();
await page.getByTestId("gpt-4o-1-option").click();
await page.waitForTimeout(1000);
await page.getByText("Playground", { exact: true }).last().click();
await page.getByTestId("button_run_text_output").click();
await page
.getByTestId(/^rf__node-TextOutput-[a-zA-Z0-9]+$/)
.getByTestId("output-inspection-text")
.click();
await page.getByText("Run Flow", { exact: true }).click();
await page.waitForTimeout(5000);
let textInputContent = await page
.getByPlaceholder("Enter text...")
.textContent();
expect(textInputContent).toBe("This is a test!");
await page.getByText("Outputs", { exact: true }).nth(1).click();
await page.getByText("Text Output", { exact: true }).nth(2).click();
let contentOutput = await page
.getByPlaceholder("Enter text...")
.inputValue();
expect(contentOutput).not.toBe(null);
await page.keyboard.press("Escape");
await page
.getByTestId(/^rf__node-TextInput-[a-zA-Z0-9]+$/)
.getByTestId("textarea_str_input_value")
.fill("This is a test, again just to be sure!");
await page.getByText("Playground", { exact: true }).last().click();
await page.getByText("Run Flow", { exact: true }).click();
await page.waitForTimeout(5000);
textInputContent = await page
.getByPlaceholder("Enter text...")
.textContent();
expect(textInputContent).toBe("This is a test, again just to be sure!");
await page.getByText("Outputs", { exact: true }).nth(1).click();
await page.getByText("Text Output", { exact: true }).nth(2).click();
contentOutput = await page.getByPlaceholder("Enter text...").inputValue();
expect(contentOutput).not.toBe(null);
},
);