* refactor: simplify TaskService by removing Celery integration
* refactor: enhance AnyIO backend task management and error handling
* refactor: restructure flow building process and enhance event handling
* feat: implement QueueService for managing job queues and tasks
* feat: Add QueueService for managing job queues and task lifecycle
* feat!: Enhance flow building with QueueService integration for job management
* revert changes to async session
* feat: Integrate QueueService into lifespan management for task handling
* refactor: Enhance QueueService with robust lifecycle management and cleanup mechanisms
* refactor: Simplify docstring for get_queue_service function
* refactor: Update import statements in queue factory for clarity
* test: Improve chat endpoint tests with comprehensive build flow scenarios
* refactor: Streamline flow build event generation and queue management
* refactor: Improve flow build process with modular event handling and URL parameter management
* test: Enhance loop component tests with flow build and event handling utilities
* fix: Improve transaction logging with null flow_id handling and debug logging
* fix: Remove unnecessary db.refresh() in transaction logging
* feat: Add polling mode for build events with optional streaming
* feat: Add event delivery configuration option to settings
* feat: Implement polling for build events with configuration-driven streaming
* refactor: Enhance queue cleanup with improved logging and error handling
* test: Improve build event stream assertion with detailed error reporting
* test: Add benchmark test for build flow polling mechanism
* refactor: Remove redundant end event in flow event generation
* test: Update test_component_tool_with_api_key to use async flow
* Convert test method to async
* Update graph start method to use async_start()
* Add client parameter to test method
* Modify tool retrieval to use async method
* test: Add session ID generation in agent component test
* fix: Enhance error handling in LCAgentComponent for ExceptionWithMessageError
Add null checks to prevent potential AttributeError when handling agent message deletion
* test: Refactor test assertion for model name options in agent component
Simplify the assertion for checking "sonnet" in model name options by extracting the options to a variable first
* refactor: Improve type hints and imports in AnyIOTaskResult
* Add `__future__` import for type annotations
* Use conditional import for `Callable` and `TracebackType`
* Explicitly type `_traceback` attribute with `TracebackType | None`
* fix: Add null checks for event task in build events streaming
* refactor: Extract build and disconnect handling into separate modules
This commit separates the build and disconnect handling logic from the chat API module into dedicated files:
- Created `langflow/api/build.py` to house the flow generation and vertex building logic
- Created `langflow/api/disconnect.py` to manage the custom streaming response with disconnect handling
- Removed duplicate code from `langflow/api/v1/chat.py`
- Improved code organization and modularity
* refactor: Extract flow build and event handling methods in build API
This commit introduces two new methods in the build API to improve code organization and reusability:
- `start_flow_build()`: Centralizes the logic for creating a queue and starting a flow build task
- `get_flow_events_response()`: Handles retrieving flow events for both streaming and polling modes
The changes simplify the chat API endpoint implementations and reduce code duplication
* refactor: Deprecate /task endpoint and add informative error message
This commit marks the /task endpoint as deprecated and raises an HTTP 400 error with a clear message indicating the endpoint will be removed in a future version. The changes improve API clarity and guide users towards the recommended /run endpoint.
* refactor: Update log_transaction function return type annotation
Modify the return type hint for log_transaction to allow for potential None return, improving type safety and clarity in the transactions logging method.
* feat: Add configurable event delivery streaming option
Introduce support for configurable event delivery mode (streaming/polling) across frontend components:
- Add `shouldStreamEvents()` method in NodeStatus and IOModal
- Update flowStore to accept a `stream` parameter
- Modify buildFlowVerticesWithFallback to use the stream parameter directly
- Remove hardcoded polling logic in buildUtils
* fix: Add null check for flow_id in log_transaction function
Prevent logging transactions without a flow_id by returning None, ensuring data integrity and avoiding potential database errors
* chore: Update changes-filter to include chat API path
* fix: Add error handling and null checks in flow transaction deletion
Improve robustness of transaction and vertex build deletion by:
- Adding a null check for flow_id in delete_transactions_by_flow_id
- Wrapping deletion operations in try-except blocks to prevent failures
- Logging debug messages for any deletion errors
* fix: Remove redundant commit in vertex builds deletion
Remove unnecessary db.commit() from delete_vertex_builds_by_flow_id to prevent duplicate commits and simplify database transaction handling
* fix: Improve transaction logging with debug message and return value
Move debug logging from CRUD layer to utils to provide more context about logged transactions and ensure proper error handling
* fix: Improve error handling and transaction cleanup in active_user fixture
Enhance user and transaction cleanup process in test fixture by:
- Adding separate try-except blocks for transaction/vertex build deletion and user deletion
- Adding debug logging for potential errors during cleanup
- Ensuring proper session commits for each operation
* propagate parent task CancelledError instead of supressing it
Co-authored-by: Christophe Bornet <cbornet@hotmail.com>
* refactor: Rename queue service to job queue service
Restructure job queue management by:
- Renaming QueueService to JobQueueService
- Moving queue-related files to a new job_queue directory
- Updating import paths and service type references
- Enhancing job queue service with more robust async job management
* fix: Correct ServiceType enum reference for job queue service
Update the service type constant to match the recently renamed JobQueueService, ensuring consistent service type referencing across the application.
* refactor: Enhance JobQueueService with comprehensive logging and documentation
Improve the JobQueueService implementation by:
- Adding detailed docstrings with clear explanations of methods and attributes
- Implementing comprehensive logging throughout the service
- Enhancing error handling and logging for queue and task management
- Providing more context in log messages for debugging and monitoring
* docs: Improve stop method docstring for JobQueueService
Enhance the documentation for the stop method by:
- Providing a more detailed and precise description of the shutdown process
- Clarifying the steps involved in gracefully stopping the service
- Improving the explanation of resource cleanup and task cancellation
* fix: Properly handle task cancellation and propagate exceptions in JobQueueService
* fix: Enhance error handling and raise appropriate exceptions in JobQueueService methods
* improve docstring
Co-authored-by: Christophe Bornet <cbornet@hotmail.com>
* fix: Remove redundant exception raises in JobQueueService methods and improve cleanup logic
* fix: Improve logging during job cleanup and handle exceptions more appropriately
* feat: add utility to run tests with multiple event delivery modes
* feat: integrate withEventDeliveryModes utility in multiple test files
* refactor: replace status code assertions with httpx codes for clarity
* remove noqa comment and change argument name
Co-authored-by: Christophe Bornet <cbornet@hotmail.com>
* refactor: streamline event polling logic in get_flow_events_response
* refactor: use getattr for safer session attribute access
Co-author: @cbornet
* feat: add is_started method to JobQueueService
* refactor: modify JobQueueService start method and main.py queue service initialization
* feat: add ready state and teardown method to JobQueueService
* refactor: simplify job queue cleanup logic in JobQueueService
* refactor: improve error logging in active_user fixture
* refactor: improve AnyIO task management with TaskGroup and CancelScope
* refactor: Implement LimitedBackgroundTasks for controlled vertex build logging (#6312)
* feat: implement LimitedBackgroundTasks for controlled vertex build logging
* refactor: replace BackgroundTasks with LimitedBackgroundTasks in build_flow endpoint
* refactor: improve task cancellation error handling in JobQueueService
* refactor: Rename LimitedBackgroundTasks to LimitVertexBuildBackgroundTasks
* feat: Add EventDeliveryType enum for event delivery methods
* feat: Add polling constants for endpoint and streaming status
* refactor: Update buildFlowVerticesWithFallback to use polling constants
* refactor: Update event delivery handling to use EventDeliveryType enum
* [autofix.ci] apply automated fixes
* fix: Improve error handling for cancelled build tasks
Refactor generate_flow_events to properly handle and propagate CancelledError
- Remove unnecessary task creation and manual cancellation
- Directly await _build_vertex instead of creating a separate task
- Improve exception logging for cancelled tasks
- Ensure CancelledError is raised instead of being silently handled
* fix: trigger event_manager.on_end after error handling in generate_flow_events
---------
Co-authored-by: Christophe Bornet <cbornet@hotmail.com>
Co-authored-by: anovazzi1 <otavio2204@gmail.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
✨ (validate-raise-errors-components.spec.ts): add test to validate error messages on popups when an error is raised in custom components
📝 (add-custom-component.ts): add utility function to add custom components in tests for easier testing and validation of error messages
* Added xyflow and updated imports
* Fix changing node.width to node.measured.width
* Updated NodeType to follow new API
* Fixed note node data type
* Created AllNodeType to contain note node type
* Changed flow types to follow new type from reactflow 12
* Updated flowStore to work with new types
* Updated flowStore and reactFlowUtils to work with custom edge type
* Updated updateAllNodes to use new type
* Updated PageComponent to follow new names for node dragging and edge reconnect
* Made selected prop be optional
* Changed reactFlowInstance to have nodes and edges type
* Updated updateAllComponent to follow new types
* Updated ReactFlowUtils with new types
* Updated reactFlowUtils type
* Updated all reactFlowUtils to be generic
* Updated handleRenderComponent with Connection type
* Updated node description and name with null checks for names
* Added check if node is genericNode on nodeOutputField
* Updated note node type
* Updated nodestatus with selected null check
* Updated notenode with new node type
* Update NodeType imports to be AllNodeType
* Fix more lint issues
* Fixed react flow button css
* ✨ (freeze.spec.ts): add zoomOut utility function to handle zooming out in tests for better code reusability and readability
📝 (decisionFlow.spec.ts): import zoomOut utility function to handle zooming out in tests for better code reusability and readability
📝 (zoom-out.ts): create zoomOut utility function to handle zooming out in tests for better code reusability and readability
* 🐛 (generalBugs-shard-10.spec.ts): fix test to wait for the element with class "border-ring-frozen" to be visible before asserting its count
* 📝 (generalBugs-shard-10.spec.ts): remove unnecessary locator click on element with id 'react-flow-id' to improve test reliability and maintainability
* 📝 (SelectionMenuComponent): add data-testid attribute to differentiate error-group-node and group-node components
📝 (similarity.spec.ts): refactor dragTo calls to use targetPosition for better accuracy, replace zoom_out clicks with zoomOut function, updateOldComponents function to handle outdated components
📝 (generalBugs-shard-5.spec.ts): replace zoom_out clicks with zoomOut function, add waitForSelector for group-node before clicking, remove unnecessary mouse interactions
📝 (intComponent.spec.ts): replace zoom_out clicks with zoomOut function, click on div-generic-node component
📝 (keyPairListComponent.spec.ts): click on div-generic-node component, adjustScreenView function call
📝 (nestedComponent.spec.ts): remove unnecessary click on react-flow-id element
📝 (generalBugs-shard-7.spec.ts): replace zoom_out clicks with zoomOut function
* ✨ (stop-building.spec.ts): Add new utility functions to improve code modularity and readability
🔧 (stop-building.spec.ts): Refactor drag and drop operations to use utility functions for better maintainability
🔧 (stop-building.spec.ts): Refactor zoom out operations to use utility function for consistency
🔧 (stop-building.spec.ts): Refactor component positioning operations to use utility functions for clarity
🔧 (stop-building.spec.ts): Refactor outdated components and filled API keys handling to use utility functions for reusability
🔧 (stop-building.spec.ts): Refactor fit view operation to use utility function for consistency
* ✨ (generalBugs-shard-9.spec.ts): refactor test to use initialGPTsetup function for GPT setup instead of manual steps to improve code readability and maintainability
* ✨ (store-shard-2.spec.ts): Increase timeout for clicking "api-key-button-store" to 200000ms for better test stability
✨ (deleteComponents.spec.ts, deleteFlows.spec.ts, store-shard-1.spec.ts, store-shard-3.spec.ts): Increase timeout for clicking "api-key-button-store" to 200000ms for better test stability
✨ (store-shard-1.spec.ts, store-shard-3.spec.ts): Remove unnecessary waitForSelector and add timeout of 200000ms for clicking "api-key-button-store" for better test stability
* ✨ (fileUploadComponent.spec.ts): update dragTo method calls with targetPosition option to specify the position of the drag action
* ✨ (decisionFlow.spec.ts): add explicit wait for an element to be attached before interacting with it to improve test reliability
🐛 (sticky-notes.spec.ts): fix test by adding keyboard press to simulate pressing the Escape key to close a modal before checking text length
* ✅ (sticky-notes.spec.ts): update assertion to use 'toHaveCount' matcher for improved test readability and reliability
* Added function to make controls horizontal
* ✨ (sticky-notes.spec.ts): update selector for textMarkdown to improve test reliability and readability
---------
Co-authored-by: cristhianzl <cristhian.lousa@gmail.com>
* change tests to use utils libs
* 🔧 (frontend): add data-testid attribute to buttons in baseModal and secretKeyModal components
🔧 (frontend): add data-testid attribute to buttons in various test files for improved testing
🔧 (frontend): refactor awaitBootstrapTest function to remove skipNewFlow option and improve modal handling
* ✅ (userSettings.spec.ts): add a 1-second timeout before checking for the "Please save" text to ensure it is rendered on the page before interacting with it
* ✅ (userSettings.spec.ts): add comment to clarify waiting for API key creation to complete before proceeding to the next form element
* ✨ (freeze-path.spec.ts): refactor adjustScreenView function to accept a parameter for the number of zoom outs to perform, improving flexibility and reusability in test scenarios.
🔧 (freeze-path.spec.ts): replace adjustScreenView function with initialGPTsetup function for setting up initial GPT configuration in the test scenario.
* refactor: Improve wait time for modal in drag and drop test
* ✨ (dragAndDrop.spec.ts): update file path to read collection.json from the correct location
🐛 (dragAndDrop.spec.ts): increase timeout for waiting for "uploaded successfully" text to prevent test failures
* ✨ (dragAndDrop.spec.ts): Add tests for drag and drop functionality to ensure users can drag and drop collections and flows without crashing the application
📝 (simulate-drag-and-drop.ts): Add utility function to simulate drag and drop events in tests for improved test coverage and functionality
* add debouncing time on file upload
* [autofix.ci] apply automated fixes
---------
Co-authored-by: cristhianzl <cristhian.lousa@gmail.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
* ✨ (ChatCodeTabComponent.tsx): add data-testid attribute to ChatCodeTabComponent for testing purposes
🔧 (TemplateCardComponent/index.tsx): add data-testid attribute with dynamic value for testing
🔧 (sidebarDraggableComponent/index.tsx): import convertTestName function and add data-testid attribute with dynamic value for testing
🔧 (Custom Component Generator.spec.ts): add test for Custom Component Generator with Playwright
🔧 (Image Sentiment Analysis.spec.ts): add test for Image Sentiment Analysis with Playwright
✅ (Instagram Copywriter.spec.ts, Market Research.spec.ts): Add integration tests for Instagram Copywriter and Market Research functionalities.
✨ (Prompt Chaining.spec.ts): Add integration test for Prompt Chaining feature
✨ (SEO Keyword Generator.spec.ts): Add integration test for SEO Keyword Generator feature
✨ (SaaS Pricing.spec.ts): Add integration test for SaaS Pricing feature
✨ (Twitter Thread Generator.spec.ts): Add integration test for Twitter Thread Generator feature
* ✨ (Custom Component Generator.spec.ts): Update environment variable name from OPENAI_API_KEY to ANTHROPIC_API_KEY for consistency and clarity
📝 (Custom Component Generator.spec.ts): Update test descriptions to reflect the correct environment variable names and requirements
📝 (Instagram Copywriter.spec.ts): Add support for TAVILY_API_KEY environment variable in tests and update test descriptions to reflect the change
* 📝 Add test utilities for updating components, adding new API keys, adjusting screen view, awaiting bootstrap test, and getting all response messages.
* 📝 (Custom Component Generator.spec.ts): Add waitForOpenModalWithChatInput utility function for better code organization and readability
📝 (Image Sentiment Analysis.spec.ts): Add buildDataTransfer utility function for better code organization and readability
📝 (Instagram Copywriter.spec.ts): Add waitForOpenModalWithChatInput utility function for better code organization and readability
📝 (Market Research.spec.ts): Add waitForOpenModalWithChatInput utility function for better code organization and readability
📝 (Prompt Chaining.spec.ts): Add waitForOpenModalWithChatInput utility function for better code organization and readability
📝 (SEO Keyword Generator.spec.ts): Add waitForOpenModalWithoutChatInput utility function for better code organization and readability
📝 (SaaS Pricing.spec.ts): Add waitForOpenModalWithoutChatInput utility function for better code organization and readability
📝 (Twitter Thread Generator.spec.ts): Add waitForOpenModalWithoutChatInput utility function for better code organization and readability
📝 (build-data-transfer.ts): Add buildDataTransfer utility function for better code organization and readability
📝 (wait-for-open-modal.ts): Add waitForOpenModalWithChatInput and waitForOpenModalWithoutChatInput utility functions for better code organization and readability
* 🔧 (ci.yml): add ANTHROPIC_API_KEY and TAVILY_API_KEY secrets for CI workflows
🔧 (nightly_build.yml): add ANTHROPIC_API_KEY and TAVILY_API_KEY secrets for nightly build jobs
🔧 (typescript_test.yml): add ANTHROPIC_API_KEY and TAVILY_API_KEY secrets for TypeScript test workflows
* feat: Add initialGPTsetup utility function for setting up GPT environment
This commit adds the initialGPTsetup utility function to the test/utils directory. The function is responsible for setting up the GPT environment by performing tasks such as adjusting the screen view, updating old components, removing old API keys, adding new API keys, and selecting the GPT model. The function accepts optional parameters to skip specific tasks if needed.
* change to utility function
* change to utility function
* change to utility function
* change to utility function
* change to utility function
* change to utility function
* change to utility function
* change to utility function
---------
Co-authored-by: anovazzi1 <otavio2204@gmail.com>
✨ (index.tsx): Add convertTestName function to generate data-testid attribute dynamically based on display_name
🔧 (use-shortcuts.tsx): Pass hasToolMode parameter to handleToolModeWShortcut function to conditionally activate tool mode
🔧 (index.tsx): Pass hasToolMode prop to NodeToolbarComponent to enable/disable tool mode functionality
🔧 (Vector Store.spec.ts): Import and use extractAndCleanCode function to extract and clean code content
🔧 (general-bugs-component-as-tool-shortcut.spec.ts): Import and use extractAndCleanCode function to extract and clean code content
📝 (extract-and-clean-code.ts): Add utility function extractAndCleanCode to extract and clean code content from HTML page
Gabriel Luiz Freitas Almeida