test: Enhance test stability (#4566)

* 🐛 (genericIconComponent/index.tsx): fix optional chaining for split and map functions to prevent errors
🐛 (actionsMainPage-shard-1.spec.ts): remove redundant zoom_out click
🐛 (auto-login-off.spec.ts): reduce wait time from 2000ms to 1000ms
🐛 (freeze-path.spec.ts): change click target from "openai" to "OpenAI" for consistency
🐛 (freeze.spec.ts): change click target from "Split Text" to "Split Text" for consistency
🐛 (Dynamic Agent.spec.ts): skip test execution
🐛 (Hierarchical Agent.spec.ts): skip test execution
🐛 (Sequential Task Agent.spec.ts): skip test execution
🐛 (Simple Agent.spec.ts): skip test execution
🐛 (decisionFlow.spec.ts): change search input from "conditional router" to "if else"
🐛 (similarity.spec.ts): change search input from "openai" to "openai embedding"
🐛 (fileUploadComponent.spec.ts): enable test for file upload functionality

📝 (deleteComponents.spec.ts): Remove commented out code and enable test for deleting a component
📝 (deleteFlows.spec.ts): Enable test for deleting a flow
📝 (starter-projects.spec.ts): Enable test for interacting with starter projects
📝 (store-shard-3.spec.ts): Enable tests for ordering visualization and filtering by type
📝 (generalBugs-shard-13.spec.ts): Enable test for sharing a component on the store

* ⬆️ (typescript_test.yml): update matrix configuration to run tests on multiple shards for better parallelization
🔧 (typescript_test.yml): enable playwright test execution with shard index and total for parallel test execution

* 🐛 (genericIconComponent/index.tsx): fix optional chaining for split and map functions to prevent errors
🐛 (actionsMainPage-shard-1.spec.ts): remove redundant zoom_out click
🐛 (auto-login-off.spec.ts): reduce wait time from 2000ms to 1000ms
🐛 (freeze-path.spec.ts): change click target from "openai" to "OpenAI" for consistency
🐛 (freeze.spec.ts): change click target from "Split Text" to "Split Text" for consistency
🐛 (Dynamic Agent.spec.ts): skip test execution
🐛 (Hierarchical Agent.spec.ts): skip test execution
🐛 (Sequential Task Agent.spec.ts): skip test execution
🐛 (Simple Agent.spec.ts): skip test execution
🐛 (decisionFlow.spec.ts): change search input from "conditional router" to "if else"
🐛 (similarity.spec.ts): change search input from "openai" to "openai embedding"
🐛 (fileUploadComponent.spec.ts): enable test for file upload functionality

📝 (deleteComponents.spec.ts): Remove commented out code and enable test for deleting a component
📝 (deleteFlows.spec.ts): Enable test for deleting a flow
📝 (starter-projects.spec.ts): Enable test for interacting with starter projects
📝 (store-shard-3.spec.ts): Enable tests for ordering visualization and filtering by type
📝 (generalBugs-shard-13.spec.ts): Enable test for sharing a component on the store

* ⬆️ (typescript_test.yml): update matrix configuration to run tests on multiple shards for better parallelization
🔧 (typescript_test.yml): enable playwright test execution with shard index and total for parallel test execution

* 🐛 (Travel Planning Agent.spec.ts): Fix issue with setting API keys for OpenAI models and Search API
🐛 (Travel Planning Agent.spec.ts): Fix issue with clicking the wrong button for running chat output
🐛 (Travel Planning Agent.spec.ts): Fix issue with checking rate limit and skipping test logic
🐛 (Travel Planning Agent.spec.ts): Fix issue with checking output text for lowercase city names
🐛 (Vector Store.spec.ts): Fix issue with clicking the wrong button for creating a new flow
🐛 (Vector Store.spec.ts): Fix issue with filling the wrong input field for collection name in Vector Store RAG test

* 🔧 (generalBugs-shard-4.spec.ts): skip test case temporarily due to functionality not working with uplift designs

* Update test to use 'message history' instead of 'chat memory' in sidebar search input

* Update test selectors and expected values in dropdownComponent.spec.ts

* 📝 (Simple Agent.spec.ts): Remove unnecessary test.skip and commented out code
📝 (dropdownComponent.spec.ts): Update dropdown selection to match new text
📝 (starter-projects.spec.ts): Remove unused template IDs
📝 (store-shard-1.spec.ts): Update test.skip to test.skip for consistency
📝 (generalBugs-shard-10.spec.ts): Refactor test cases for promptText and newPromptText
📝 (generalBugs-shard-6.spec.ts): Add delay before interacting with elements to ensure stability

* Refactor tests to use `ControlOrMeta` for cross-platform keyboard shortcuts

*  (Simple Agent.spec.ts): update timeout value for waitForSelector to improve test reliability and prevent false negatives

* Handle undefined stars in GithubStarComponent to prevent errors

* Add waitForSelector to ensure sidebar search input is visible before interaction in chatInputOutput test

*  (Simple Agent.spec.ts): update timeout value to 5 minutes for "built successfully" message to allow for longer build times
 (Simple Agent.spec.ts): add additional test cases to check for the presence of "apple" in chat messages and ensure the total length of chat messages is greater than 100 characters

* 🐛 (Simple Agent.spec.ts): reduce timeout for "built successfully" text to improve test efficiency

*  (auto-login-off.spec.ts): add delay before checking visibility of elements to ensure proper rendering and improve test reliability

* Fix indentation in TypeScript test workflow file

* Update test IDs and expected values in dropdownComponent.spec.ts

* 📝 (playground.spec.ts): remove redundant code related to zooming out and dragging elements in the playground test
📝 (deleteComponents.spec.ts): update selectors for clicking buttons and elements in the deleteComponents test
📝 (deleteFlows.spec.ts): update selectors for clicking buttons and elements in the deleteFlows test
📝 (starter-projects.spec.ts): update selectors for starter projects in the extended features test to match changes in the application's UI.

*  (playground.spec.ts): remove skip from the test case to enable it to run during testing.

* Refactor timeout in playground test for better stability

---------

Co-authored-by: Gabriel Luiz Freitas Almeida <gabriel@langflow.org>
Co-authored-by: anovazzi1 <otavio2204@gmail.com>
This commit is contained in:
Cristhian Zanforlin Lousa 2024-11-13 15:11:51 -03:00 committed by GitHub
commit 4e2ee25da8
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
40 changed files with 2142 additions and 1053 deletions

View file

@ -39,10 +39,9 @@ jobs:
strategy:
fail-fast: false
matrix:
# shardIndex: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
# shardTotal: [10]
shardIndex: [1]
shardTotal: [1]
shardIndex: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
shardTotal: [10]
env:
OPENAI_API_KEY: ${{ inputs.openai_api_key || secrets.OPENAI_API_KEY }}
STORE_API_KEY: ${{ inputs.store_api_key || secrets.STORE_API_KEY }}
@ -121,8 +120,8 @@ jobs:
max_attempts: 2
command: |
cd src/frontend
# npx playwright test ${{ inputs.tests_folder }} --shard ${{ matrix.shardIndex }}/${{ matrix.shardTotal }} --list
# npx playwright test ${{ inputs.tests_folder }} --trace on --shard ${{ matrix.shardIndex }}/${{ matrix.shardTotal }} --workers 2
npx playwright test ${{ inputs.tests_folder }} --shard ${{ matrix.shardIndex }}/${{ matrix.shardTotal }} --list
npx playwright test ${{ inputs.tests_folder }} --trace on --shard ${{ matrix.shardIndex }}/${{ matrix.shardTotal }} --workers 2
- name: Upload blob report to GitHub Actions Artifacts
if: always()