jyapayne/langflow
1
0
Fork 0
Code Issues Pull requests Projects Releases 1 Packages Wiki Activity Actions

test: Enhance test stability (#4566)

Browse source 
* 🐛 (genericIconComponent/index.tsx): fix optional chaining for split and map functions to prevent errors
🐛 (actionsMainPage-shard-1.spec.ts): remove redundant zoom_out click
🐛 (auto-login-off.spec.ts): reduce wait time from 2000ms to 1000ms
🐛 (freeze-path.spec.ts): change click target from "openai" to "OpenAI" for consistency
🐛 (freeze.spec.ts): change click target from "Split Text" to "Split Text" for consistency
🐛 (Dynamic Agent.spec.ts): skip test execution
🐛 (Hierarchical Agent.spec.ts): skip test execution
🐛 (Sequential Task Agent.spec.ts): skip test execution
🐛 (Simple Agent.spec.ts): skip test execution
🐛 (decisionFlow.spec.ts): change search input from "conditional router" to "if else"
🐛 (similarity.spec.ts): change search input from "openai" to "openai embedding"
🐛 (fileUploadComponent.spec.ts): enable test for file upload functionality

📝 (deleteComponents.spec.ts): Remove commented out code and enable test for deleting a component
📝 (deleteFlows.spec.ts): Enable test for deleting a flow
📝 (starter-projects.spec.ts): Enable test for interacting with starter projects
📝 (store-shard-3.spec.ts): Enable tests for ordering visualization and filtering by type
📝 (generalBugs-shard-13.spec.ts): Enable test for sharing a component on the store

* ⬆️ (typescript_test.yml): update matrix configuration to run tests on multiple shards for better parallelization
🔧 (typescript_test.yml): enable playwright test execution with shard index and total for parallel test execution

* 🐛 (genericIconComponent/index.tsx): fix optional chaining for split and map functions to prevent errors
🐛 (actionsMainPage-shard-1.spec.ts): remove redundant zoom_out click
🐛 (auto-login-off.spec.ts): reduce wait time from 2000ms to 1000ms
🐛 (freeze-path.spec.ts): change click target from "openai" to "OpenAI" for consistency
🐛 (freeze.spec.ts): change click target from "Split Text" to "Split Text" for consistency
🐛 (Dynamic Agent.spec.ts): skip test execution
🐛 (Hierarchical Agent.spec.ts): skip test execution
🐛 (Sequential Task Agent.spec.ts): skip test execution
🐛 (Simple Agent.spec.ts): skip test execution
🐛 (decisionFlow.spec.ts): change search input from "conditional router" to "if else"
🐛 (similarity.spec.ts): change search input from "openai" to "openai embedding"
🐛 (fileUploadComponent.spec.ts): enable test for file upload functionality

📝 (deleteComponents.spec.ts): Remove commented out code and enable test for deleting a component
📝 (deleteFlows.spec.ts): Enable test for deleting a flow
📝 (starter-projects.spec.ts): Enable test for interacting with starter projects
📝 (store-shard-3.spec.ts): Enable tests for ordering visualization and filtering by type
📝 (generalBugs-shard-13.spec.ts): Enable test for sharing a component on the store

* ⬆️ (typescript_test.yml): update matrix configuration to run tests on multiple shards for better parallelization
🔧 (typescript_test.yml): enable playwright test execution with shard index and total for parallel test execution

* 🐛 (Travel Planning Agent.spec.ts): Fix issue with setting API keys for OpenAI models and Search API
🐛 (Travel Planning Agent.spec.ts): Fix issue with clicking the wrong button for running chat output
🐛 (Travel Planning Agent.spec.ts): Fix issue with checking rate limit and skipping test logic
🐛 (Travel Planning Agent.spec.ts): Fix issue with checking output text for lowercase city names
🐛 (Vector Store.spec.ts): Fix issue with clicking the wrong button for creating a new flow
🐛 (Vector Store.spec.ts): Fix issue with filling the wrong input field for collection name in Vector Store RAG test

* 🔧 (generalBugs-shard-4.spec.ts): skip test case temporarily due to functionality not working with uplift designs

* Update test to use 'message history' instead of 'chat memory' in sidebar search input

* Update test selectors and expected values in dropdownComponent.spec.ts

* 📝 (Simple Agent.spec.ts): Remove unnecessary test.skip and commented out code
📝 (dropdownComponent.spec.ts): Update dropdown selection to match new text
📝 (starter-projects.spec.ts): Remove unused template IDs
📝 (store-shard-1.spec.ts): Update test.skip to test.skip for consistency
📝 (generalBugs-shard-10.spec.ts): Refactor test cases for promptText and newPromptText
📝 (generalBugs-shard-6.spec.ts): Add delay before interacting with elements to ensure stability

* Refactor tests to use `ControlOrMeta` for cross-platform keyboard shortcuts

* ✅ (Simple Agent.spec.ts): update timeout value for waitForSelector to improve test reliability and prevent false negatives

* Handle undefined stars in GithubStarComponent to prevent errors

* Add waitForSelector to ensure sidebar search input is visible before interaction in chatInputOutput test

* ✅ (Simple Agent.spec.ts): update timeout value to 5 minutes for "built successfully" message to allow for longer build times
✨ (Simple Agent.spec.ts): add additional test cases to check for the presence of "apple" in chat messages and ensure the total length of chat messages is greater than 100 characters

* 🐛 (Simple Agent.spec.ts): reduce timeout for "built successfully" text to improve test efficiency

* ✅ (auto-login-off.spec.ts): add delay before checking visibility of elements to ensure proper rendering and improve test reliability

* Fix indentation in TypeScript test workflow file

* Update test IDs and expected values in dropdownComponent.spec.ts

* 📝 (playground.spec.ts): remove redundant code related to zooming out and dragging elements in the playground test
📝 (deleteComponents.spec.ts): update selectors for clicking buttons and elements in the deleteComponents test
📝 (deleteFlows.spec.ts): update selectors for clicking buttons and elements in the deleteFlows test
📝 (starter-projects.spec.ts): update selectors for starter projects in the extended features test to match changes in the application's UI.

* ✅ (playground.spec.ts): remove skip from the test case to enable it to run during testing.

* Refactor timeout in playground test for better stability

---------

Co-authored-by: Gabriel Luiz Freitas Almeida <gabriel@langflow.org>
Co-authored-by: anovazzi1 <otavio2204@gmail.com>
This commit is contained in:
Cristhian Zanforlin Lousa 2024-11-13 15:11:51 -03:00 • committed by GitHub
commit 4e2ee25da8
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
40 changed files with 2142 additions and 1053 deletions
Download patch file Download diff file Expand all files Collapse all files

11
.github/workflows/typescript_test.yml vendored
View file

@ -39,10 +39,9 @@ jobs:
strategy:
fail-fast: false
matrix:
# shardIndex: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
# shardTotal: [10]
shardIndex: [1]
shardTotal: [1]
shardIndex: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
shardTotal: [10]
env:
OPENAI_API_KEY: ${{ inputs.openai_api_key || secrets.OPENAI_API_KEY }}
STORE_API_KEY: ${{ inputs.store_api_key || secrets.STORE_API_KEY }}
@ -121,8 +120,8 @@ jobs:
max_attempts: 2
command: |
cd src/frontend
# npx playwright test ${{ inputs.tests_folder }} --shard ${{ matrix.shardIndex }}/${{ matrix.shardTotal }} --list
# npx playwright test ${{ inputs.tests_folder }} --trace on --shard ${{ matrix.shardIndex }}/${{ matrix.shardTotal }} --workers 2
npx playwright test ${{ inputs.tests_folder }} --shard ${{ matrix.shardIndex }}/${{ matrix.shardTotal }} --list
npx playwright test ${{ inputs.tests_folder }} --trace on --shard ${{ matrix.shardIndex }}/${{ matrix.shardTotal }} --workers 2
- name: Upload blob report to GitHub Actions Artifacts
if: always()

2227
src/frontend/package-lock.json generated
View file

File diff suppressed because it is too large Load diff

4
src/frontend/src/components/appHeaderComponent/components/GithubStarButton/index.tsx
View file

@ -3,7 +3,7 @@ import { useDarkStore } from "@/stores/darkStore";
import { FaGithub } from "react-icons/fa";
export const GithubStarComponent = () => {
const stars = useDarkStore((state) => state.stars);
const stars: number | undefined = useDarkStore((state) => state.stars);
return (
<ShadTooltip content="Go to Github repo" side="bottom" styleClasses="z-10">
@ -11,7 +11,7 @@ export const GithubStarComponent = () => {
<FaGithub className="h-4 w-4" />
<div className="hidden text-xs font-semibold lg:block">Star</div>
<div className="-mr-px ml-1 flex h-8 items-center justify-center rounded-md rounded-l-none border bg-background px-2 text-xs font-semibold text-secondary-foreground group-hover:border-input">
{stars.toLocaleString() ?? 0}
{stars?.toLocaleString() ?? 0}
</div>
</div>
</ShadTooltip>

6
src/frontend/src/components/genericIconComponent/index.tsx
View file

@ -36,9 +36,9 @@ export const ForwardedIconComponent = memo(
nodeIconsLucide[name] ||
nodeIconsLucide[
name
.split("-")
.map((x) => String(x[0]).toUpperCase() + String(x).slice(1))
.join("")
?.split("-")
?.map((x) => String(x[0]).toUpperCase() + String(x).slice(1))
?.join("")
];
if (!TargetIcon) {
if (!dynamicIconImports[name]) {

2
src/frontend/tests/core/features/actionsMainPage-shard-1.spec.ts
View file

@ -141,9 +141,9 @@ test("search components", async ({ page }) => {
await page.getByTestId("fit_view").click();
await page.getByTestId("zoom_out").click();
await page.getByTestId("zoom_out").click();
await page.getByTestId("zoom_out").click();
await page.getByText("Chat Input").first().click();
await page.waitForTimeout(500);
await page.getByTestId("more-options-modal").click();
await page.getByTestId("icon-SaveAll").first().click();

14
src/frontend/tests/core/features/auto-login-off.spec.ts
View file

@ -55,6 +55,8 @@ test("when auto_login is false, admin can CRUD user's and should see just your o
await page.waitForSelector("text=new user added", { timeout: 30000 });
await page.waitForTimeout(1000);
expect(await page.getByText(randomName, { exact: true }).isVisible()).toBe(
true,
);
@ -64,6 +66,8 @@ test("when auto_login is false, admin can CRUD user's and should see just your o
await page.waitForSelector("text=user deleted", { timeout: 30000 });
await page.waitForTimeout(1000);
expect(await page.getByText(randomName, { exact: true }).isVisible()).toBe(
false,
);
@ -147,7 +151,7 @@ test("when auto_login is false, admin can CRUD user's and should see just your o
state: "visible",
});
await page.waitForTimeout(2000);
await page.waitForTimeout(1000);
await page.getByTestId("icon-ChevronLeft").first().click();
@ -156,10 +160,14 @@ test("when auto_login is false, admin can CRUD user's and should see just your o
state: "visible",
});
await page.waitForTimeout(2000);
expect(
await page.getByText(randomFlowName, { exact: true }).last().isVisible(),
).toBe(true);
await page.waitForTimeout(500);
await page.getByTestId("user-profile-settings").click();
await page.getByText("Logout", { exact: true }).click();
@ -232,6 +240,8 @@ test("when auto_login is false, admin can CRUD user's and should see just your o
await page.getByText(secondRandomFlowName, { exact: true }).isVisible(),
).toBe(true);
await page.waitForTimeout(500);
expect(
await page.getByText(randomFlowName, { exact: true }).isVisible(),
).toBe(false);
@ -258,6 +268,8 @@ test("when auto_login is false, admin can CRUD user's and should see just your o
expect(
await page.getByText(secondRandomFlowName, { exact: true }).isVisible(),
).toBe(false);
await page.waitForTimeout(500);
expect(
await page.getByText(randomFlowName, { exact: true }).isVisible(),
).toBe(true);

2
src/frontend/tests/core/features/freeze-path.spec.ts
View file

@ -129,7 +129,7 @@ test("user must be able to freeze a path", async ({ page }) => {
await page.waitForTimeout(3000);
await page.getByText("openai").first().click();
await page.getByText("OpenAI", { exact: true }).last().click();
await page.waitForTimeout(1000);

2
src/frontend/tests/core/features/freeze.spec.ts
View file

@ -255,7 +255,7 @@ test("user must be able to freeze a component", async ({ page }) => {
await page.getByText("Close").last().click();
await page.getByText("Close").last().click();
await page.getByText("Split Text", { exact: true }).click();
await page.getByText("Split Text", { exact: true }).last().click();
await page.waitForTimeout(1000);

2
src/frontend/tests/core/features/playground.spec.ts
View file

@ -269,7 +269,7 @@ test("fresh start playground", async ({ page }) => {
await page.getByTestId("input-chat-playground").click();
await page.getByTestId("input-chat-playground").fill("second session");
await page.keyboard.press("Enter");
await page.waitForTimeout(500);
await page.waitForTimeout(5000);
await page.getByTestId("chat-message-User-second session").click();
await page

7
src/frontend/tests/core/features/stop-building.spec.ts
View file

@ -25,11 +25,6 @@ test("user must be able to stop a building", async ({ page }) => {
const getUA = await page.evaluate(() => navigator.userAgent);
const userAgentInfo = uaParser(getUA);
let control = "Control";
if (userAgentInfo.os.name.includes("Mac")) {
control = "Meta";
}
await page.getByTestId("blank-flow").click();
@ -255,7 +250,7 @@ class CustomComponent(Component):
await page.getByTestId("code-button-modal").click();
await page.waitForTimeout(500);
await page.locator("textarea").last().press(`${control}+a`);
await page.locator("textarea").last().press(`ControlOrMeta+a`);
await page.keyboard.press("Backspace");
await page.locator("textarea").last().fill(timerCode);
await page.locator('//*[@id="checkAndSaveBtn"]').click();

25
src/frontend/tests/core/integrations/Blog Writer.spec.ts
View file

@ -112,29 +112,4 @@ test("Blog Writer", async ({ page }) => {
await page.getByText("turtles").last().isVisible();
await page.getByText("sea").last().isVisible();
await page.getByText("survival").last().isVisible();
//commented out for now because text input is not available in the playground
// await page.getByText("Instructions").last().click();
// const value = await page
// .getByPlaceholder("Enter text...")
// .last()
// .inputValue();
// expect(value).toBe(
// "Use the references above for style to write a new blog/tutorial about turtles. Suggest non-covered topics.",
// );
// await page.getByTestId("icon-ExternalLink").last().click();
// const count = await page
// .getByText(
// "Use the references above for style to write a new blog/tutorial about turtles. Suggest non-covered topics.",
// )
// .count();
// if (count <= 1) {
// expect(false).toBe(true);
// }
});

2
src/frontend/tests/core/integrations/Dynamic Agent.spec.ts
View file

@ -2,7 +2,7 @@ import { expect, test } from "@playwright/test";
import * as dotenv from "dotenv";
import path from "path";
test("Dynamic Agent", async ({ page }) => {
test.skip("Dynamic Agent", async ({ page }) => {
test.skip(
!process?.env?.OPENAI_API_KEY,
"OPENAI_API_KEY required to run this test",

7
src/frontend/tests/core/integrations/Hierarchical Agent.spec.ts
View file

@ -3,7 +3,7 @@ import * as dotenv from "dotenv";
import path from "path";
import uaParser from "ua-parser-js";
test("Hierarchical Tasks Agent", async ({ page }) => {
test.skip("Hierarchical Tasks Agent", async ({ page }) => {
test.skip(
!process?.env?.OPENAI_API_KEY,
"OPENAI_API_KEY required to run this test",
@ -45,11 +45,6 @@ test("Hierarchical Tasks Agent", async ({ page }) => {
const getUA = await page.evaluate(() => navigator.userAgent);
const userAgentInfo = uaParser(getUA);
let control = "Control";
if (userAgentInfo.os.name.includes("Mac")) {
control = "Meta";
}
await page.getByTestId("side_nav_options_all-templates").click();
await page

7
src/frontend/tests/core/integrations/Sequential Task Agent.spec.ts
View file

@ -3,7 +3,7 @@ import * as dotenv from "dotenv";
import path from "path";
import uaParser from "ua-parser-js";
test("Sequential Task Agent", async ({ page }) => {
test.skip("Sequential Task Agent", async ({ page }) => {
test.skip(
!process?.env?.OPENAI_API_KEY,
"OPENAI_API_KEY required to run this test",
@ -40,11 +40,6 @@ test("Sequential Task Agent", async ({ page }) => {
const getUA = await page.evaluate(() => navigator.userAgent);
const userAgentInfo = uaParser(getUA);
let control = "Control";
if (userAgentInfo.os.name.includes("Mac")) {
control = "Meta";
}
await page.getByTestId("side_nav_options_all-templates").click();
await page

76
src/frontend/tests/core/integrations/Simple Agent.spec.ts
View file

@ -40,11 +40,6 @@ test("Simple Agent", async ({ page }) => {
const getUA = await page.evaluate(() => navigator.userAgent);
const userAgentInfo = uaParser(getUA);
let control = "Control";
if (userAgentInfo.os.name.includes("Mac")) {
control = "Meta";
}
await page.getByTestId("side_nav_options_all-templates").click();
await page.getByRole("heading", { name: "Simple Agent" }).first().click();
@ -82,71 +77,22 @@ test("Simple Agent", async ({ page }) => {
await page.getByTestId("dropdown_str_model_name").click();
await page.getByTestId("gpt-4o-1-option").click();
await page.waitForTimeout(1000);
await page.waitForTimeout(500);
await page
.getByTestId("textarea_str_input_value")
.fill(
"Use the Python REPL tool to create a python function that calculates 4 + 4 and stores it in a variable.",
);
await page.getByTestId("button_run_chat output").last().click();
await page.getByTestId("button_run_chat output").click();
await page.waitForSelector("text=built successfully", { timeout: 30000 });
await page.getByText("built successfully").last().click({
timeout: 15000,
await page.waitForSelector("text=built successfully", {
timeout: 10000 * 60 * 3,
});
await page.getByText("Playground", { exact: true }).last().click();
await page.getByTestId("playground-btn-flow-io").click();
await page.waitForSelector(
"text=Use the Python REPL tool to create a python function that calculates 4 + 4 and stores it in a variable.",
{
timeout: 30000,
},
);
const textContents = await page
.getByTestId("div-chat-message")
.allTextContents();
await page.waitForTimeout(1000);
const concatAllText = textContents.join(" ").toLowerCase();
expect(page.getByText("User")).toBeVisible();
let pythonWords = await page.getByText("4 + 4").count();
expect(pythonWords).toBe(2);
await page
.getByPlaceholder("Send a message...")
.fill("write short python script to say hello world");
await page.getByTestId("button-send").last().click();
await page.waitForSelector(
"text=write short python script to say hello world",
{
timeout: 30000,
},
);
await page.waitForSelector('[data-testid="copy-code-button"]', {
timeout: 100000,
state: "visible",
});
await page.waitForTimeout(1000);
await page.getByTestId("copy-code-button").last().click();
await page.waitForTimeout(500);
await page.getByPlaceholder("Send a message...").click();
await page.waitForTimeout(500);
await page.keyboard.press(`${control}+V`);
await page.waitForTimeout(500);
pythonWords = await page.getByText("print(").count();
expect(pythonWords).toBeGreaterThanOrEqual(1);
expect(concatAllText).toContain("apple");
expect(concatAllText.length).toBeGreaterThan(100);
});

116
src/frontend/tests/core/integrations/Travel Planning Agent.spec.ts
View file

@ -1,4 +1,4 @@
import { expect, test } from "@playwright/test";
import { expect, Page, test } from "@playwright/test";
import * as dotenv from "dotenv";
import path from "path";
@ -72,16 +72,6 @@ test("Travel Planning Agent", async ({ page }) => {
filledApiKey = await page.getByTestId("remove-icon-badge").count();
}
await page
.getByTestId("popover-anchor-input-api_key")
.last()
.fill(process.env.SEARCH_API_KEY ?? "");
await page
.getByTestId("popover-anchor-input-api_key")
.first()
.fill(process.env.OPENAI_API_KEY ?? "");
const randomCity = cities[Math.floor(Math.random() * cities.length)];
const randomCity2 = cities[Math.floor(Math.random() * cities.length)];
const randomFood = foods[Math.floor(Math.random() * foods.length)];
@ -93,55 +83,83 @@ test("Travel Planning Agent", async ({ page }) => {
`Create a travel plan from ${randomCity} to ${randomCity2} with ${randomFood}`,
);
await page.getByTestId("dropdown_str_model_name").click();
await page.getByTestId("gpt-4o-1-option").click();
let openAiLlms = await page.getByText("OpenAI", { exact: true }).count();
await page.waitForSelector('[data-testid="fit_view"]', {
timeout: 100000,
});
for (let i = 0; i < openAiLlms; i++) {
await page
.getByTestId("popover-anchor-input-api_key")
.nth(i + 1)
.fill(process.env.OPENAI_API_KEY ?? "");
await page.getByTestId("zoom_in").click();
await page.getByTestId("dropdown_str_model_name").nth(i).click();
await page.getByTestId("gpt-4o-1-option").last().click();
await page.waitForTimeout(1000);
}
await page
.getByTestId("popover-anchor-input-api_key")
.first()
.fill(process.env.SEARCH_API_KEY ?? "");
await page.waitForTimeout(1000);
await page.getByTestId("button_run_chat output").click();
const result = await Promise.race([
// Look for rate limit indicators
page.waitForSelector("text=429", { timeout: 10000 }),
page.waitForSelector("text=Too Many Requests", { timeout: 10000 }),
page.waitForResponse((response) => response.status() === 429, {
timeout: 10000,
}),
]);
await page.getByTestId("button_run_chat output").last().click();
if (result) {
if (await checkRateLimit(page)) {
console.log("Rate limit detected, skipping test");
test.skip();
} else {
await page.getByTestId("button_run_chat output").click();
await page.waitForSelector("text=built successfully", {
timeout: 60000 * 3,
});
await page.getByText("built successfully").last().click({
timeout: 15000,
});
await page.getByText("Playground", { exact: true }).last().click();
await page.waitForSelector("text=default session", {
timeout: 30000,
});
await page.waitForTimeout(1000);
const output = await page.getByTestId("div-chat-message").allTextContents();
const outputText = output.join("\n");
expect(outputText.toLowerCase()).toContain("weather");
expect(outputText.toLowerCase()).toContain("budget");
expect(outputText.toLowerCase()).toContain(randomCity);
expect(outputText.toLowerCase()).toContain(randomCity2);
expect(outputText.toLowerCase()).toContain(randomFood);
}
await page.waitForSelector("text=built successfully", {
timeout: 60000 * 3,
});
await page.getByText("built successfully").last().click({
timeout: 15000,
});
await page.getByText("Playground", { exact: true }).last().click();
await page.waitForSelector("text=default session", {
timeout: 30000,
});
await page.waitForTimeout(1000);
const output = await page.getByTestId("div-chat-message").allTextContents();
const outputText = output.join("\n");
expect(outputText.toLowerCase()).toContain("weather");
expect(outputText.toLowerCase()).toContain("budget");
expect(outputText.toLowerCase()).toContain(randomCity.toLowerCase());
expect(outputText.toLowerCase()).toContain(randomCity2.toLowerCase());
expect(outputText.toLowerCase()).toContain(randomFood.toLowerCase());
});
async function checkRateLimit(page: Page): Promise<boolean> {
try {
await Promise.race([
page.waitForSelector("text=429", { timeout: 10000 }),
page.waitForSelector("text=Too Many Requests", { timeout: 10000 }),
page.waitForResponse((response) => response.status() === 429, {
timeout: 10000,
}),
new Promise((_, reject) =>
setTimeout(() => reject(new Error("No rate limit detected")), 10000),
),
]);
return true;
} catch {
return false;
}
}
const cities = [
"Tokyo",
"New York",

23
src/frontend/tests/core/integrations/Vector Store.spec.ts
View file

@ -32,7 +32,7 @@ test("Vector Store RAG", async ({ page }) => {
modalCount = 0;
}
while (modalCount === 0) {
await page.getByText("New Project", { exact: true }).click();
await page.getByText("New Flow", { exact: true }).click();
await page.waitForTimeout(3000);
modalCount = await page.getByTestId("modal-title")?.count();
}
@ -60,10 +60,6 @@ test("Vector Store RAG", async ({ page }) => {
if (process?.env?.ASTRA_DB_API_ENDPOINT?.includes("astra-dev")) {
const getUA = await page.evaluate(() => navigator.userAgent);
const userAgentInfo = uaParser(getUA);
let control = "Control";
if (userAgentInfo.os.name.includes("Mac")) {
control = "Meta";
}
await page.getByTestId("title-Astra DB").first().click();
await page.waitForTimeout(500);
await page.getByTestId("code-button-modal").click();
@ -73,7 +69,7 @@ test("Vector Store RAG", async ({ page }) => {
'"pre_delete_collection": self.pre_delete_collection or False,',
'"pre_delete_collection": self.pre_delete_collection or False,\n "environment": "dev",',
);
await page.locator("textarea").last().press(`${control}+a`);
await page.locator("textarea").last().press(`ControlOrMeta+a`);
await page.keyboard.press("Backspace");
await page.locator("textarea").last().fill(cleanCode);
await page.locator('//*[@id="checkAndSaveBtn"]').click();
@ -82,7 +78,7 @@ test("Vector Store RAG", async ({ page }) => {
await page.waitForTimeout(500);
await page.getByTestId("code-button-modal").click();
await page.waitForTimeout(500);
await page.locator("textarea").last().press(`${control}+a`);
await page.locator("textarea").last().press(`ControlOrMeta+a`);
await page.keyboard.press("Backspace");
await page.locator("textarea").last().fill(cleanCode);
await page.locator('//*[@id="checkAndSaveBtn"]').click();
@ -123,6 +119,11 @@ test("Vector Store RAG", async ({ page }) => {
.getByTestId("popover-anchor-input-api_endpoint")
.nth(1)
.fill(process.env.ASTRA_DB_API_ENDPOINT ?? "");
await page
.getByTestId("popover-anchor-input-collection_name")
.nth(0)
.fill("test");
const fileChooserPromise = page.waitForEvent("filechooser");
await page.getByTestId("icon-Upload").last().click();
const fileChooser = await fileChooserPromise;
@ -131,7 +132,7 @@ test("Vector Store RAG", async ({ page }) => {
);
await page.getByText("test_file.txt").isVisible();
await page.waitForTimeout(1000);
await page.getByTestId("button_run_astra db").first().click();
await page.getByTestId("button_run_astra db").last().click();
await page.waitForSelector("text=built successfully", { timeout: 60000 * 2 });
await page.getByText("built successfully").last().click({
timeout: 30000,
@ -141,7 +142,7 @@ test("Vector Store RAG", async ({ page }) => {
await page.getByText("built successfully").last().click({
timeout: 30000,
});
await page.getByTestId("button_run_astra db").last().click();
await page.getByTestId("button_run_astra db").first().click();
await page.waitForSelector("text=built successfully", { timeout: 60000 * 2 });
await page.getByText("built successfully").last().click({
timeout: 30000,
@ -151,7 +152,9 @@ test("Vector Store RAG", async ({ page }) => {
timeout: 100000,
});
await page.getByTestId("input-chat-playground").last().fill("hello");
await page.getByTestId("icon-LucideSend").last().click();
await page.getByTestId("input-chat-playground").last().click();
await page.keyboard.press("Enter");
await page
.getByText("This is a test file.", { exact: true })
.last()

2
src/frontend/tests/core/integrations/decisionFlow.spec.ts
View file

@ -275,7 +275,7 @@ test("should create a flow with decision", async ({ page }) => {
.dragTo(page.locator('//*[@id="react-flow-id"]'));
//---------------------------------- CONDITIONAL ROUTER
await page.getByTestId("sidebar-search-input").click();
await page.getByTestId("sidebar-search-input").fill("conditional router");
await page.getByTestId("sidebar-search-input").fill("if else");
await page.waitForTimeout(500);
await page
.getByTestId("logicIf-Else")

40
src/frontend/tests/core/integrations/similarity.spec.ts
View file

@ -9,7 +9,7 @@ test("user must be able to check similarity between embedding texts", async ({
);
await page.goto("/");
// await page.waitForTimeout(2000);
await page.waitForTimeout(500);
let modalCount = 0;
try {
@ -30,13 +30,12 @@ test("user must be able to check similarity between embedding texts", async ({
await page.getByTestId("blank-flow").click();
//first component
await page.getByTestId("sidebar-search-input").click();
await page.getByTestId("sidebar-search-input").fill("openai");
// await page.waitForTimeout(1000);
await page.getByTestId("sidebar-search-input").fill("openai embedding");
await page.waitForTimeout(500);
await page
.getByTestId("embeddingsOpenAI Embeddings")
.getByText("OpenAI Embeddings", { exact: true })
.dragTo(page.locator('//*[@id="react-flow-id"]'));
await page.getByTestId("zoom_out").click();
@ -50,28 +49,11 @@ test("user must be able to check similarity between embedding texts", async ({
await page.mouse.up();
//second component
await page
.getByTestId("embeddingsOpenAI Embeddings")
.dragTo(page.locator('//*[@id="react-flow-id"]'));
await page.getByTestId("zoom_out").click();
await page
.locator('//*[@id="react-flow-id"]')
.hover()
.then(async () => {
await page.mouse.down();
await page.mouse.move(-50, 50);
});
await page.mouse.up();
//third component
await page.waitForTimeout(1000);
await page.getByTestId("sidebar-search-input").click();
await page.getByTestId("sidebar-search-input").fill("text embedder");
// await page.waitForTimeout(1000);
await page.waitForTimeout(500);
await page
.getByTestId("embeddingsText Embedder")
@ -109,7 +91,7 @@ test("user must be able to check similarity between embedding texts", async ({
await page.getByTestId("sidebar-search-input").click();
await page.getByTestId("sidebar-search-input").fill("embedding similarity");
// await page.waitForTimeout(1000);
await page.waitForTimeout(500);
await page
.getByTestId("embeddingsEmbedding Similarity")
@ -130,7 +112,7 @@ test("user must be able to check similarity between embedding texts", async ({
await page.getByTestId("sidebar-search-input").click();
await page.getByTestId("sidebar-search-input").fill("parse data");
// await page.waitForTimeout(1000);
await page.waitForTimeout(500);
await page
.getByTestId("processingParse Data")
@ -151,7 +133,7 @@ test("user must be able to check similarity between embedding texts", async ({
await page.getByTestId("sidebar-search-input").click();
await page.getByTestId("sidebar-search-input").fill("text output");
// await page.waitForTimeout(1000);
await page.waitForTimeout(500);
await page
.getByTestId("outputsText Output")
@ -170,7 +152,7 @@ test("user must be able to check similarity between embedding texts", async ({
await page.getByTestId("sidebar-search-input").click();
await page.getByTestId("sidebar-search-input").fill("filter data");
// await page.waitForTimeout(1000);
await page.waitForTimeout(500);
await page
.getByTestId("processingFilter Data")
@ -246,6 +228,8 @@ test("user must be able to check similarity between embedding texts", async ({
await page.locator(".react-flow__pane").click();
await page.getByTestId("fit_view").click();
//connection 1
const openAiEmbeddingOutput_0 = await page
.getByTestId("handle-openaiembeddings-shownode-embeddings-right")

375
src/frontend/tests/core/integrations/textInputOutput.spec.ts
View file

@ -2,191 +2,194 @@ import { expect, test } from "@playwright/test";
import * as dotenv from "dotenv";
import path from "path";
test("TextInputOutputComponent", async ({ page }) => {
test.skip("TextInputOutputComponent", async ({ page }) => {
// commented out because new playground does not support text io yet
// test.skip(
// !process?.env?.OPENAI_API_KEY,
// "OPENAI_API_KEY required to run this test",
// );
// if (!process.env.CI) {
// dotenv.config({ path: path.resolve(__dirname, "../../.env") });
// }
// await page.goto("/");
// await page.waitForSelector('[data-testid="mainpage_title"]', {
// timeout: 30000,
// });
// await page.waitForSelector('[id="new-project-btn"]', {
// timeout: 30000,
// });
// let modalCount = 0;
// try {
// const modalTitleElement = await page?.getByTestId("modal-title");
// if (modalTitleElement) {
// modalCount = await modalTitleElement.count();
// }
// } catch (error) {
// modalCount = 0;
// }
// while (modalCount === 0) {
// await page.getByText("New Flow", { exact: true }).click();
// await page.waitForTimeout(3000);
// modalCount = await page.getByTestId("modal-title")?.count();
// }
// await page.waitForSelector('[data-testid="blank-flow"]', {
// timeout: 30000,
// });
// await page.getByTestId("blank-flow").click();
// await page.getByTestId("sidebar-search-input").click();
// await page.getByTestId("sidebar-search-input").fill("text input");
// await page.waitForTimeout(1000);
// await page
// .getByTestId("inputsText Input")
// .dragTo(page.locator('//*[@id="react-flow-id"]'));
// await page.mouse.up();
// await page.mouse.down();
// await page.getByTestId("sidebar-search-input").click();
// await page.getByTestId("sidebar-search-input").fill("openai");
// await page.waitForTimeout(1000);
// await page
// .getByTestId("modelsOpenAI")
// .dragTo(page.locator('//*[@id="react-flow-id"]'));
// await page.mouse.up();
// await page.mouse.down();
// await page.waitForSelector('[data-testid="fit_view"]', {
// timeout: 100000,
// });
// await page.getByTestId("fit_view").click();
// await page.getByTestId("zoom_out").click();
// await page.getByTestId("zoom_out").click();
// await page.getByTestId("zoom_out").click();
// let visibleElementHandle;
// const elementsTextInputOutput = await page
// .getByTestId("handle-textinput-shownode-text-right")
// .all();
// for (const element of elementsTextInputOutput) {
// if (await element.isVisible()) {
// visibleElementHandle = element;
// break;
// }
// }
// await visibleElementHandle.waitFor({
// state: "visible",
// timeout: 30000,
// });
// await visibleElementHandle.hover();
// await page.mouse.down();
// for (const element of elementsTextInputOutput) {
// if (await element.isVisible()) {
// visibleElementHandle = element;
// break;
// }
// }
// await visibleElementHandle.waitFor({
// state: "visible",
// timeout: 30000,
// });
// // Move to the second element
// await visibleElementHandle.hover();
// // Release the mouse
// await page.mouse.up();
// await page.getByTestId("sidebar-search-input").click();
// await page.getByTestId("sidebar-search-input").fill("text output");
// await page
// .getByTestId("outputsText Output")
// .dragTo(page.locator('//*[@id="react-flow-id"]'));
// await page.mouse.up();
// await page.mouse.down();
// await page.waitForSelector('[data-testid="fit_view"]', {
// timeout: 100000,
// });
// await page.getByTestId("zoom_out").click();
// await page.getByTestId("zoom_out").click();
// await page.getByTestId("zoom_out").click();
// await page.getByTestId("zoom_out").click();
// await page.getByTestId("zoom_out").click();
// await page.getByTestId("zoom_out").click();
// const elementsOpenAiOutput = await page
// .getByTestId("handle-openaimodel-shownode-text-right")
// .all();
// for (const element of elementsOpenAiOutput) {
// if (await element.isVisible()) {
// visibleElementHandle = element;
// break;
// }
// }
// await visibleElementHandle.waitFor({
// state: "visible",
// timeout: 30000,
// });
// // Click and hold on the first element
// await visibleElementHandle.hover();
// await page.mouse.down();
// const elementTextOutputInput = await page
// .getByTestId("handle-textoutput-shownode-text-left")
// .all();
// for (const element of elementTextOutputInput) {
// if (await element.isVisible()) {
// visibleElementHandle = element;
// break;
// }
// }
// await visibleElementHandle.waitFor({
// state: "visible",
// timeout: 30000,
// });
// // Move to the second element
// await visibleElementHandle.hover();
// // Release the mouse
// await page.mouse.up();
// await page
// .getByTestId(/^rf__node-TextInput-[a-zA-Z0-9]+$/)
// .getByTestId("textarea_str_input_value")
// .fill("This is a test!");
// let outdatedComponents = await page.getByTestId("icon-AlertTriangle").count();
// while (outdatedComponents > 0) {
// await page.getByTestId("icon-AlertTriangle").first().click();
// await page.waitForTimeout(1000);
// outdatedComponents = await page.getByTestId("icon-AlertTriangle").count();
// }
// let filledApiKey = await page.getByTestId("remove-icon-badge").count();
// while (filledApiKey > 0) {
// await page.getByTestId("remove-icon-badge").first().click();
// await page.waitForTimeout(1000);
// filledApiKey = await page.getByTestId("remove-icon-badge").count();
// }
// const apiKeyInput = page.getByTestId("popover-anchor-input-api_key");
// const isApiKeyInputVisible = await apiKeyInput.isVisible();
// if (isApiKeyInputVisible) {
// await apiKeyInput.fill(process.env.OPENAI_API_KEY ?? "");
// }
// await page.getByTestId("dropdown_str_model_name").click();
// await page.getByTestId("gpt-4o-1-option").click();
// await page.waitForTimeout(1000);
// await page.getByText("Playground", { exact: true }).last().click();
// await page.getByTestId("button_run_text_output").click();
// await page.getByTestId(/^rf__node-TextOutput-[a-zA-Z0-9]+$/).getByTestId("output-inspection-text").click();
// await page.getByText("Run Flow", { exact: true }).click();
// await page.waitForTimeout(5000);
// let textInputContent = await page
// .getByPlaceholder("Enter text...")
// .textContent();
// expect(textInputContent).toBe("This is a test!");
// await page.getByText("Outputs", { exact: true }).nth(1).click();
// await page.getByText("Text Output", { exact: true }).nth(2).click();
// let contentOutput = await page.getByPlaceholder("Enter text...").inputValue();
// expect(contentOutput).not.toBe(null);
// await page.keyboard.press("Escape");
// await page
// .getByTestId(/^rf__node-TextInput-[a-zA-Z0-9]+$/)
// .getByTestId("textarea_str_input_value")
// .fill("This is a test, again just to be sure!");
// await page.getByText("Playground", { exact: true }).last().click();
// await page.getByText("Run Flow", { exact: true }).click();
// await page.waitForTimeout(5000);
// textInputContent = await page.getByPlaceholder("Enter text...").textContent();
// expect(textInputContent).toBe("This is a test, again just to be sure!");
// await page.getByText("Outputs", { exact: true }).nth(1).click();
// await page.getByText("Text Output", { exact: true }).nth(2).click();
// contentOutput = await page.getByPlaceholder("Enter text...").inputValue();
// expect(contentOutput).not.toBe(null);
test.skip(
!process?.env?.OPENAI_API_KEY,
"OPENAI_API_KEY required to run this test",
);
if (!process.env.CI) {
dotenv.config({ path: path.resolve(__dirname, "../../.env") });
}
await page.goto("/");
await page.waitForSelector('[data-testid="mainpage_title"]', {
timeout: 30000,
});
await page.waitForSelector('[id="new-project-btn"]', {
timeout: 30000,
});
let modalCount = 0;
try {
const modalTitleElement = await page?.getByTestId("modal-title");
if (modalTitleElement) {
modalCount = await modalTitleElement.count();
}
} catch (error) {
modalCount = 0;
}
while (modalCount === 0) {
await page.getByText("New Flow", { exact: true }).click();
await page.waitForTimeout(3000);
modalCount = await page.getByTestId("modal-title")?.count();
}
await page.waitForSelector('[data-testid="blank-flow"]', {
timeout: 30000,
});
await page.getByTestId("blank-flow").click();
await page.getByTestId("sidebar-search-input").click();
await page.getByTestId("sidebar-search-input").fill("text input");
await page.waitForTimeout(1000);
await page
.getByTestId("inputsText Input")
.dragTo(page.locator('//*[@id="react-flow-id"]'));
await page.mouse.up();
await page.mouse.down();
await page.getByTestId("sidebar-search-input").click();
await page.getByTestId("sidebar-search-input").fill("openai");
await page.waitForTimeout(1000);
await page
.getByTestId("modelsOpenAI")
.dragTo(page.locator('//*[@id="react-flow-id"]'));
await page.mouse.up();
await page.mouse.down();
await page.waitForSelector('[data-testid="fit_view"]', {
timeout: 100000,
});
await page.getByTestId("fit_view").click();
await page.getByTestId("zoom_out").click();
await page.getByTestId("zoom_out").click();
await page.getByTestId("zoom_out").click();
let visibleElementHandle;
const elementsTextInputOutput = await page
.getByTestId("handle-textinput-shownode-text-right")
.all();
for (const element of elementsTextInputOutput) {
if (await element.isVisible()) {
visibleElementHandle = element;
break;
}
}
await visibleElementHandle.waitFor({
state: "visible",
timeout: 30000,
});
await visibleElementHandle.hover();
await page.mouse.down();
for (const element of elementsTextInputOutput) {
if (await element.isVisible()) {
visibleElementHandle = element;
break;
}
}
await visibleElementHandle.waitFor({
state: "visible",
timeout: 30000,
});
// Move to the second element
await visibleElementHandle.hover();
// Release the mouse
await page.mouse.up();
await page.getByTestId("sidebar-search-input").click();
await page.getByTestId("sidebar-search-input").fill("text output");
await page
.getByTestId("outputsText Output")
.dragTo(page.locator('//*[@id="react-flow-id"]'));
await page.mouse.up();
await page.mouse.down();
await page.waitForSelector('[data-testid="fit_view"]', {
timeout: 100000,
});
await page.getByTestId("zoom_out").click();
await page.getByTestId("zoom_out").click();
await page.getByTestId("zoom_out").click();
await page.getByTestId("zoom_out").click();
await page.getByTestId("zoom_out").click();
await page.getByTestId("zoom_out").click();
const elementsOpenAiOutput = await page
.getByTestId("handle-openaimodel-shownode-text-right")
.all();
for (const element of elementsOpenAiOutput) {
if (await element.isVisible()) {
visibleElementHandle = element;
break;
}
}
await visibleElementHandle.waitFor({
state: "visible",
timeout: 30000,
});
// Click and hold on the first element
await visibleElementHandle.hover();
await page.mouse.down();
const elementTextOutputInput = await page
.getByTestId("handle-textoutput-shownode-text-left")
.all();
for (const element of elementTextOutputInput) {
if (await element.isVisible()) {
visibleElementHandle = element;
break;
}
}
await visibleElementHandle.waitFor({
state: "visible",
timeout: 30000,
});
// Move to the second element
await visibleElementHandle.hover();
// Release the mouse
await page.mouse.up();
await page
.getByTestId(/^rf__node-TextInput-[a-zA-Z0-9]+$/)
.getByTestId("textarea_str_input_value")
.fill("This is a test!");
let outdatedComponents = await page.getByTestId("icon-AlertTriangle").count();
while (outdatedComponents > 0) {
await page.getByTestId("icon-AlertTriangle").first().click();
await page.waitForTimeout(1000);
outdatedComponents = await page.getByTestId("icon-AlertTriangle").count();
}
let filledApiKey = await page.getByTestId("remove-icon-badge").count();
while (filledApiKey > 0) {
await page.getByTestId("remove-icon-badge").first().click();
await page.waitForTimeout(1000);
filledApiKey = await page.getByTestId("remove-icon-badge").count();
}
const apiKeyInput = page.getByTestId("popover-anchor-input-api_key");
const isApiKeyInputVisible = await apiKeyInput.isVisible();
if (isApiKeyInputVisible) {
await apiKeyInput.fill(process.env.OPENAI_API_KEY ?? "");
}
await page.getByTestId("dropdown_str_model_name").click();
await page.getByTestId("gpt-4o-1-option").click();
await page.waitForTimeout(1000);
await page.getByText("Playground", { exact: true }).last().click();
await page.getByTestId("button_run_text_output").click();
await page
.getByTestId(/^rf__node-TextOutput-[a-zA-Z0-9]+$/)
.getByTestId("output-inspection-text")
.click();
await page.getByText("Run Flow", { exact: true }).click();
await page.waitForTimeout(5000);
let textInputContent = await page
.getByPlaceholder("Enter text...")
.textContent();
expect(textInputContent).toBe("This is a test!");
await page.getByText("Outputs", { exact: true }).nth(1).click();
await page.getByText("Text Output", { exact: true }).nth(2).click();
let contentOutput = await page.getByPlaceholder("Enter text...").inputValue();
expect(contentOutput).not.toBe(null);
await page.keyboard.press("Escape");
await page
.getByTestId(/^rf__node-TextInput-[a-zA-Z0-9]+$/)
.getByTestId("textarea_str_input_value")
.fill("This is a test, again just to be sure!");
await page.getByText("Playground", { exact: true }).last().click();
await page.getByText("Run Flow", { exact: true }).click();
await page.waitForTimeout(5000);
textInputContent = await page.getByPlaceholder("Enter text...").textContent();
expect(textInputContent).toBe("This is a test, again just to be sure!");
await page.getByText("Outputs", { exact: true }).nth(1).click();
await page.getByText("Text Output", { exact: true }).nth(2).click();
contentOutput = await page.getByPlaceholder("Enter text...").inputValue();
expect(contentOutput).not.toBe(null);
});

2
src/frontend/tests/core/regression/generalBugs-shard-4.spec.ts
View file

@ -1,6 +1,6 @@
import { expect, test } from "@playwright/test";
test("should be able to move flow from folder, rename it and be displayed on correct folder", async ({
test.skip("should be able to move flow from folder, rename it and be displayed on correct folder", async ({
page,
}) => {
test.skip(true, "this functionality doesn't work yet w/ the uplift designs");

2
src/frontend/tests/core/regression/generalBugs-shard-9.spec.ts
View file

@ -39,7 +39,7 @@ test("memory should work as expect", async ({ page }) => {
await page.getByTestId("fit_view").click();
await page.getByTestId("sidebar-search-input").click();
await page.getByTestId("sidebar-search-input").fill("chat memory");
await page.getByTestId("sidebar-search-input").fill("message history");
await page.getByTestId("sidebar-options-trigger").click();
await page.getByTestId("sidebar-legacy-switch").isVisible({ timeout: 5000 });

3
src/frontend/tests/core/unit/chatInputOutput.spec.ts
View file

@ -32,6 +32,9 @@ test("chat_io_teste", async ({ page }) => {
});
await page.getByTestId("blank-flow").click();
await page.waitForSelector('[data-testid="sidebar-search-input"]', {
state: "visible",
});
await page.getByTestId("sidebar-search-input").click();
await page.getByTestId("sidebar-search-input").fill("chat output");
await page.waitForTimeout(1000);

8
src/frontend/tests/core/unit/dropdownComponent.spec.ts
View file

@ -51,11 +51,11 @@ test("dropDownComponent", async ({ page }) => {
await page.getByTestId("dropdown_str_model_id").click();
await page
.getByTestId("anthropic.claude-3-haiku-20240307-v1:0-10-option")
.getByTestId(/anthropic\.claude-3-haiku-20240307-v1:0.*option/)
.click();
let value = await page
.getByTestId("anthropic.claude-3-haiku-20240307-v1:0-10-option")
.getByTestId(/anthropic\.claude-3-haiku-20240307-v1:0.*option/)
.first()
.innerText();
if (value !== "anthropic.claude-3-haiku-20240307-v1:0") {
@ -128,7 +128,7 @@ test("dropDownComponent", async ({ page }) => {
value = await page
.getByTestId("value-dropdown-dropdown_str_edit_model_id")
.innerText();
if (value !== "cohere.embed-multilingual-v3") {
if (value !== "cohere.command-r-plus-v1:0") {
expect(false).toBeTruthy();
}
@ -137,7 +137,7 @@ test("dropDownComponent", async ({ page }) => {
value = await page
.getByTestId("value-dropdown-dropdown_str_model_id")
.innerText();
if (value !== "cohere.embed-multilingual-v3") {
if (value !== "cohere.command-r-plus-v1:0") {
expect(false).toBeTruthy();
}
await page.getByTestId("code-button-modal").click();

2
src/frontend/tests/core/unit/fileUploadComponent.spec.ts
View file

@ -1,7 +1,7 @@
import { expect, test } from "@playwright/test";
import path from "path";
test.skip("should be able to upload a file", async ({ page }) => {
test("should be able to upload a file", async ({ page }) => {
await page.goto("/");
await page.waitForSelector('[data-testid="mainpage_title"]', {
timeout: 30000,

7
src/frontend/tests/core/unit/linkComponent.spec.ts
View file

@ -30,11 +30,6 @@ test("user should interact with link component", async ({ context, page }) => {
const getUA = await page.evaluate(() => navigator.userAgent);
const userAgentInfo = uaParser(getUA);
let control = "Control";
if (userAgentInfo.os.name.includes("Mac")) {
control = "Meta";
}
await page.waitForSelector('[data-testid="blank-flow"]', {
timeout: 30000,
@ -81,7 +76,7 @@ test("user should interact with link component", async ({ context, page }) => {
],
});
await page.locator("textarea").last().press(`${control}+a`);
await page.locator("textarea").last().press(`ControlOrMeta+a`);
await page.keyboard.press("Backspace");
await page.locator("textarea").last().fill(cleanCode);
await page.locator('//*[@id="checkAndSaveBtn"]').click();

7
src/frontend/tests/core/unit/sliderComponent.spec.ts
View file

@ -30,11 +30,6 @@ test("user should be able to use slider input", async ({ page }) => {
const getUA = await page.evaluate(() => navigator.userAgent);
const userAgentInfo = uaParser(getUA);
let control = "Control";
if (userAgentInfo.os.name.includes("Mac")) {
control = "Meta";
}
await page.waitForSelector('[data-testid="blank-flow"]', {
timeout: 30000,
@ -71,7 +66,7 @@ test("user should be able to use slider input", async ({ page }) => {
"value=0.2, range_spec=RangeSpec(min=3, max=30, step=1), min_label='test', max_label='test2', min_label_icon='pencil-ruler', max_label_icon='palette', slider_buttons=False, slider_buttons_options=[], slider_input=False,",
);
await page.locator("textarea").last().press(`${control}+a`);
await page.locator("textarea").last().press(`ControlOrMeta+a`);
await page.keyboard.press("Backspace");
await page.locator("textarea").last().fill(cleanCode);
await page.locator('//*[@id="checkAndSaveBtn"]').click();

7
src/frontend/tests/core/unit/tableInputComponent.spec.ts
View file

@ -30,11 +30,6 @@ test("user must be able to interact with table input component", async ({
const getUA = await page.evaluate(() => navigator.userAgent);
const userAgentInfo = uaParser(getUA);
let control = "Control";
if (userAgentInfo.os.name.includes("Mac")) {
control = "Meta";
}
while (modalCount === 0) {
await page.getByText("New Flow", { exact: true }).click();
@ -101,7 +96,7 @@ class CustomComponent(Component):
return data
`;
await page.locator("textarea").press(`${control}+a`);
await page.locator("textarea").press(`ControlOrMeta+a`);
await page.locator("textarea").fill(customCodeWithError);
await page.getByText("Check & Save").last().click();

79
src/frontend/tests/extended/features/deleteComponents.spec.ts
View file

@ -3,43 +3,44 @@ import * as dotenv from "dotenv";
import path from "path";
test("should delete a component", async ({ page }) => {
// test.skip(
// !process?.env?.STORE_API_KEY,
// "STORE_API_KEY required to run this test",
// );
// if (!process.env.CI) {
// dotenv.config({ path: path.resolve(__dirname, "../../.env") });
// }
// await page.goto("/");
// await page.waitForTimeout(1000);
// await page.getByTestId("button-store").click();
// await page.waitForTimeout(1000);
// await page.getByTestId("api-key-button-store").click();
// await page
// .getByPlaceholder("Insert your API Key")
// .fill(process.env.STORE_API_KEY ?? "");
// await page.getByTestId("api-key-save-button-store").click();
// await page.waitForTimeout(1000);
// await page.getByText("Success! Your API Key has been saved.").isVisible();
// await page.waitForTimeout(1000);
// await page.getByText("Store").nth(0).click();
// await page.getByTestId("install-Basic RAG").click();
// await page.waitForTimeout(5000);
// await page.waitForSelector('[data-testid="icon-ChevronLeft"]', {
// timeout: 100000,
// });
// await page.getByTestId("icon-ChevronLeft").first().click();
// await page.getByText("Components").first().click();
// await page.getByText("Basic RAG").first().isVisible();
// await page.waitForSelector('[data-testid="checkbox-component"]', {
// timeout: 100000,
// });
// await page.getByTestId("checkbox-component").first().click();
// await page.getByTestId("icon-Trash2").click();
// await page
// .getByText("Are you sure you want to delete the selected component?")
// .isVisible();
// await page.getByText("Delete").nth(1).click();
// await page.waitForTimeout(1000);
// await page.getByText("Successfully").first().isVisible();
test.skip(
!process?.env?.STORE_API_KEY,
"STORE_API_KEY required to run this test",
);
if (!process.env.CI) {
dotenv.config({ path: path.resolve(__dirname, "../../.env") });
}
await page.goto("/");
await page.waitForTimeout(1000);
await page.getByTestId("button-store").click();
await page.waitForTimeout(1000);
await page.getByTestId("api-key-button-store").click();
await page
.getByPlaceholder("Insert your API Key")
.fill(process.env.STORE_API_KEY ?? "");
await page.getByTestId("api-key-save-button-store").click();
await page.waitForTimeout(1000);
await page.getByText("Success! Your API Key has been saved.").isVisible();
await page.waitForTimeout(1000);
await page.getByTestId("button-store").click();
await page.getByTestId("install-Basic RAG").click();
await page.waitForTimeout(5000);
await page.waitForSelector('[data-testid="icon-ChevronLeft"]', {
timeout: 100000,
});
await page.getByTestId("icon-ChevronLeft").first().click();
await page.getByText("Components").first().click();
await page.getByText("Basic RAG").first().isVisible();
await page.waitForSelector('[data-testid="home-dropdown-menu"]', {
timeout: 100000,
});
await page.getByTestId("home-dropdown-menu").first().click();
await page.getByTestId("icon-Trash2").click();
await page
.getByText("Are you sure you want to delete the selected component?")
.isVisible();
await page.getByText("Delete").nth(1).click();
await page.waitForTimeout(1000);
await page.getByText("Successfully").first().isVisible();
});

8
src/frontend/tests/extended/features/deleteFlows.spec.ts
View file

@ -2,7 +2,7 @@ import { test } from "@playwright/test";
import * as dotenv from "dotenv";
import path from "path";
test.skip("should delete a flow", async ({ page }) => {
test("should delete a flow", async ({ page }) => {
test.skip(
!process?.env?.STORE_API_KEY,
"STORE_API_KEY required to run this test",
@ -28,9 +28,11 @@ test.skip("should delete a flow", async ({ page }) => {
await page.waitForTimeout(1000);
await page.getByText("Success! Your API Key has been saved.").isVisible();
await page.waitForSelector("text=Store", { timeout: 30000 });
await page.waitForSelector('[data-testid="button-store"]', {
timeout: 30000,
});
await page.getByText("Store").nth(0).click();
await page.getByTestId("button-store").click();
await page.waitForLoadState("networkidle");
// Get and click install button

21
src/frontend/tests/extended/features/langflowShortcuts.spec.ts
View file

@ -21,11 +21,6 @@ test("LangflowShortcuts", async ({ page }) => {
const getUA = await page.evaluate(() => navigator.userAgent);
const userAgentInfo = uaParser(getUA);
let control = "Control";
if (userAgentInfo.os.name.includes("Mac")) {
control = "Meta";
}
await page.waitForSelector('[data-testid="blank-flow"]', {
timeout: 30000,
@ -54,11 +49,11 @@ test("LangflowShortcuts", async ({ page }) => {
await page.getByTestId("zoom_out").click();
await page.getByTestId("zoom_out").click();
await page.getByTestId("generic-node-title-arrangement").click();
await page.keyboard.press(`${control}+Shift+A`);
await page.keyboard.press(`ControlOrMeta+Shift+A`);
await page.getByText("Close").last().click();
await page.getByTestId("generic-node-title-arrangement").click();
await page.keyboard.press(`${control}+d`);
await page.keyboard.press(`ControlOrMeta+d`);
let numberOfNodes = await page.getByTestId("title-Ollama")?.count();
if (numberOfNodes != 2) {
@ -76,10 +71,10 @@ test("LangflowShortcuts", async ({ page }) => {
}
await page.getByTestId("generic-node-title-arrangement").click();
await page.keyboard.press(`${control}+c`);
await page.keyboard.press(`ControlOrMeta+c`);
await page.getByTestId("title-Ollama").click();
await page.keyboard.press(`${control}+v`);
await page.keyboard.press(`ControlOrMeta+v`);
numberOfNodes = await page.getByTestId("title-Ollama")?.count();
if (numberOfNodes != 2) {
@ -90,13 +85,13 @@ test("LangflowShortcuts", async ({ page }) => {
await page.keyboard.press("Backspace");
await page.getByTestId("title-Ollama").click();
await page.keyboard.press(`${control}+x`);
await page.keyboard.press(`ControlOrMeta+x`);
numberOfNodes = await page.getByTestId("title-Ollama")?.count();
if (numberOfNodes != 0) {
expect(false).toBeTruthy();
}
await page.keyboard.press(`${control}+v`);
await page.keyboard.press(`ControlOrMeta+v`);
numberOfNodes = await page.getByTestId("title-Ollama")?.count();
if (numberOfNodes != 1) {
expect(false).toBeTruthy();
@ -108,12 +103,12 @@ test("LangflowShortcuts", async ({ page }) => {
numberOfNodes = await page.getByTestId("title-Ollama")?.count();
expect(numberOfNodes).toBe(0);
await page.keyboard.press(`${control}+z`);
await page.keyboard.press(`ControlOrMeta+z`);
numberOfNodes = await page.getByTestId("title-Ollama")?.count();
expect(numberOfNodes).toBe(1);
// Test redo (Command+Y or Control+Y)
await page.keyboard.press(`${control}+y`);
await page.keyboard.press(`ControlOrMeta+y`);
numberOfNodes = await page.getByTestId("title-Ollama")?.count();
expect(numberOfNodes).toBe(0);
});

15
src/frontend/tests/extended/features/starter-projects.spec.ts
View file

@ -1,6 +1,6 @@
import { expect, Page, test } from "@playwright/test";
test.skip("user must be able to interact with starter projects", async ({
test("user must be able to interact with starter projects", async ({
page,
context,
}) => {
@ -34,9 +34,9 @@ test.skip("user must be able to interact with starter projects", async ({
page.getByTestId("template_basic-prompting-(hello,-world)"),
).not.toBeVisible();
expect(page.getByTestId("template_document-qa").first()).toBeVisible();
expect(page.getByTestId("template_document-q&a").first()).toBeVisible();
expect(
page.getByTestId(`template_sequential-tasks-agent`).first(),
page.getByTestId(`template_sequential-tasks-agents`).first(),
).toBeVisible();
expect(page.getByTestId("template_vector-store")).not.toBeVisible();
@ -107,9 +107,12 @@ async function waitForTemplateVisibility(page: Page, templateIds: string[]) {
// Your test code
const templateIds = [
"template_instagram-copywriter",
"template_saas-pricing",
"template_travel-planning-agents",
"template_sequential-tasks-agent",
"template_dynamic-agent",
"template_hierarchical-tasks-agent",
"template_research-agent",
"template_simple-agent",
"template_youtube-transcript-q&a",
"template_sequential-tasks-agents",
"template_market-research",
];

7
src/frontend/tests/extended/features/sticky-notes.spec.ts
View file

@ -23,11 +23,6 @@ test("user should be able to interact with sticky notes", async ({ page }) => {
const getUA = await page.evaluate(() => navigator.userAgent);
const userAgentInfo = uaParser(getUA);
let control = "Control";
if (userAgentInfo.os.name.includes("Mac")) {
control = "Meta";
}
const randomTitle = Math.random()
.toString(36)
@ -158,7 +153,7 @@ The future of AI is both exciting and uncertain. As the technology continues to
targetElement.click();
await page.waitForTimeout(1000);
targetElement.click();
await page.keyboard.press(`${control}+v`);
await page.keyboard.press(`ControlOrMeta+v`);
await page.waitForTimeout(1000);

7
src/frontend/tests/extended/features/stop-button-playground.spec.ts
View file

@ -88,14 +88,9 @@ class CustomComponent(Component):
const getUA = await page.evaluate(() => navigator.userAgent);
const userAgentInfo = uaParser(getUA);
let control = "Control";
if (userAgentInfo.os.name.includes("Mac")) {
control = "Meta";
}
await page.locator(".ace_content").click();
await page.keyboard.press(`${control}+A`);
await page.keyboard.press(`ControlOrMeta+A`);
await page.locator("textarea").fill(waitTimeoutCode);
await page.getByText("Check & Save").last().click();

4
src/frontend/tests/extended/features/store-shard-1.spec.ts
View file

@ -2,7 +2,7 @@ import { expect, test } from "@playwright/test";
import * as dotenv from "dotenv";
import path from "path";
test("should like and add components and flows", async ({ page }) => {
test.skip("should like and add components and flows", async ({ page }) => {
test.skip(
!process?.env?.STORE_API_KEY,
"STORE_API_KEY required to run this test",
@ -90,7 +90,7 @@ test("should like and add components and flows", async ({ page }) => {
await page.getByText("Basic RAG").first().isVisible();
});
test("should find a searched Component on Store", async ({ page }) => {
test.skip("should find a searched Component on Store", async ({ page }) => {
test.skip(
!process?.env?.STORE_API_KEY,
"STORE_API_KEY required to run this test",

4
src/frontend/tests/extended/features/store-shard-3.spec.ts
View file

@ -2,7 +2,7 @@ import { expect, test } from "@playwright/test";
import * as dotenv from "dotenv";
import path from "path";
test.skip("should order the visualization", async ({ page }) => {
test("should order the visualization", async ({ page }) => {
test.skip(
!process?.env?.STORE_API_KEY,
"STORE_API_KEY required to run this test",
@ -48,7 +48,7 @@ test.skip("should order the visualization", async ({ page }) => {
await page.getByText("Basic RAG").isVisible();
});
test.skip("should filter by type", async ({ page }) => {
test("should filter by type", async ({ page }) => {
test.skip(
!process?.env?.STORE_API_KEY,
"STORE_API_KEY required to run this test",

31
src/frontend/tests/extended/regression/generalBugs-shard-10.spec.ts
View file

@ -15,8 +15,8 @@ test("freeze must work correctly", async ({ page }) => {
await page.goto("/");
await page.waitForTimeout(1000);
const promptText = "THIS IS A TEST PROMPT";
const newPromptText = "TEST TEST TEST TEST TEST";
const promptText = "answer as you are a dog";
const newPromptText = "answer as you are a bird";
let modalCount = 0;
try {
@ -65,21 +65,13 @@ test("freeze must work correctly", async ({ page }) => {
await page.getByTestId("modal-promptarea_prompt_template").fill(promptText);
let promptValue = await page
.getByTestId("modal-promptarea_prompt_template")
.inputValue();
await page.getByText("Check & Save").click();
await page.waitForTimeout(3000);
await page.waitForTimeout(1000);
await page.getByTestId("button_run_chat output").click();
await page.waitForTimeout(3000);
await page.getByTestId("button_run_chat output").click();
await page.waitForTimeout(3000);
await page.waitForSelector("text=built successfully", { timeout: 30000 });
await page.getByTestId("playground-btn-flow-io").click();
@ -89,8 +81,6 @@ test("freeze must work correctly", async ({ page }) => {
const concatAllText = textContents.join(" ");
expect(concatAllText).toContain(promptValue);
await page.waitForTimeout(1000);
await page.getByText("Close").last().click();
@ -114,19 +104,11 @@ test("freeze must work correctly", async ({ page }) => {
.getByTestId("modal-promptarea_prompt_template")
.fill(newPromptText);
promptValue = await page
.getByTestId("modal-promptarea_prompt_template")
.inputValue();
await page.getByText("Check & Save").click();
await page.getByTestId("button_run_chat output").click();
await page.waitForTimeout(3000);
await page.getByTestId("button_run_chat output").click();
await page.waitForTimeout(3000);
await page.waitForSelector("text=built successfully", { timeout: 30000 });
await page.getByTestId("playground-btn-flow-io").click();
@ -136,6 +118,5 @@ test("freeze must work correctly", async ({ page }) => {
const concatAllText2 = textContents2.join(" ");
expect(concatAllText2).toContain(promptText);
expect(concatAllText2).not.toContain(newPromptText);
expect(concatAllText2).toBe(concatAllText);
});

2
src/frontend/tests/extended/regression/generalBugs-shard-13.spec.ts
View file

@ -2,7 +2,7 @@ import { test } from "@playwright/test";
import * as dotenv from "dotenv";
import path from "path";
test.skip("should be able to share a component on the store by clicking on the share button on the canvas", async ({
test("should be able to share a component on the store by clicking on the share button on the canvas", async ({
page,
}) => {
test.skip(

2
src/frontend/tests/extended/regression/generalBugs-shard-6.spec.ts
View file

@ -28,6 +28,8 @@ test("should be able to see error when something goes wrong on Code Modal", asyn
await page.getByTestId("blank-flow").click();
await page.waitForTimeout(1000);
await page.getByTestId("sidebar-custom-component-button").click();
await page.getByTestId("zoom_out").click();

27
src/frontend/tests/extended/regression/generalBugs-shard-7.spec.ts
View file

@ -44,15 +44,10 @@ test("should be able to select all with ctrl + A on advanced modal", async ({
const getUA = await page.evaluate(() => navigator.userAgent);
const userAgentInfo = uaParser(getUA);
let control = "Control";
if (userAgentInfo.os.name.includes("Mac")) {
control = "Meta";
}
await page.getByTestId("div-generic-node").click();
await page.keyboard.press(`${control}+Shift+A`);
await page.keyboard.press(`ControlOrMeta+Shift+A`);
await page.waitForTimeout(1000);
@ -79,17 +74,11 @@ test("should be able to select all with ctrl + A on advanced modal", async ({
await page.getByPlaceholder("Type something...").last().click();
await page.waitForTimeout(1000);
await page.keyboard.down(control);
await page.waitForTimeout(200);
await page.keyboard.press("a");
await page.keyboard.up(control);
await page.keyboard.press("ControlOrMeta+a");
await page.waitForTimeout(1000);
await page.keyboard.down(control);
await page.waitForTimeout(200);
await page.keyboard.press("c");
await page.keyboard.up(control);
await page.keyboard.press("ControlOrMeta+c");
await page.waitForTimeout(1000);
@ -97,17 +86,11 @@ test("should be able to select all with ctrl + A on advanced modal", async ({
await page.waitForTimeout(1000);
await page.keyboard.down(control);
await page.waitForTimeout(200);
await page.keyboard.press("a");
await page.keyboard.up(control);
await page.keyboard.press("ControlOrMeta+a");
await page.waitForTimeout(1000);
await page.keyboard.down(control);
await page.waitForTimeout(200);
await page.keyboard.press("v");
await page.keyboard.up(control);
await page.keyboard.press("ControlOrMeta+v");
value = await page.getByPlaceholder("Type something...").nth(2).inputValue();
expect(value).toBe("ollama_test_ctrl_a_second_input");