🐳 chore(deploy): update traefik image version to v3.0 for improved functionality and compatibility
🔧 chore(deploy): remove unnecessary port mappings for backend, queue, pgadmin, redis, prometheus, and grafana services
This commit adds a new file `docker-compose.override.yml` to the `deploy` directory. The file contains configuration for the Traefik proxy and services used for local development. The configuration includes:
- Enabling Docker in Traefik to read labels from Docker services
- Adding a constraint to only use services with a specific label for this stack
- Disabling the exposure of all Docker services by default
- Enabling the access log and Traefik log
- Enabling the Traefik Dashboard and API in insecure mode for local development
The file also includes configuration for the following services:
- `proxy`: Configured to listen on ports 80 and 8090, with labels for routing and load balancing
- `pgadmin`: Configured to listen on port 5050
- `flower`: Configured to listen on port 5555
- `backend`: Configured with labels for routing and load balancing on paths `/api/v1`, `/docs`, and `/health`
- `frontend`: Configured with labels for routing and load balancing on the root path
The `traefik-public` network is also defined for use by the Traefik proxy.
This file is intended to be used as an override for the main `docker-compose.yml` file during local development.
🐳 chore(docker-compose.yml): add Traefik reverse proxy configuration for services
🔧 fix(docker-compose.yml): fix path to startup-backend.sh script
🚀 feat(docker-compose.yml): add support for deploying backend, celery worker, flower, frontend, prometheus, and grafana services
🔧 fix(docker-compose.yml): fix volume configuration for db service
🔧 fix(docker-compose.yml): fix network configuration for pgadmin service
🔧 fix(docker-compose.yml): fix network configuration for traefik proxy service
🔧 fix(docker-compose.yml): fix network configuration for flower service
🔧 fix(docker-compose.yml): fix network configuration for frontend service
🔧 fix(docker-compose.yml): fix network configuration for prometheus service
🔧 fix(docker-compose.yml): fix network configuration for grafana service
🔧 chore(docker-compose.yml): update backend.env file path to remove leading './' for consistency
🔧 chore(docker-compose.yml): remove explicit port mapping for backend service to use the default port
🔧 chore(docker-compose.yml): add deploy configuration for celery worker service to ensure only one replica is deployed
🔧 chore(docker-compose.yml): add prometheus and grafana services with necessary configurations and dependencies
📝 docs(prometheus.yml): add prometheus configuration file with scrape targets for prometheus and flower services
🔧 chore(locustfile.py): update host URL to remove unnecessary colon in the port number
🔧 chore(locustfile.py): refactor send_name_and_check task to use a more efficient polling mechanism for task status
🔧 chore(locustfile.py): update flow_id and session_id generation logic to use more meaningful values
🔧 chore(locustfile.py): refactor send_name_and_check task to use a separate process function for better code organization and readability
🔧 chore(locustfile.py): update payload2 to include a more descriptive text for the input
🔧 chore(locustfile.py): update assertion in send_name_and_check task to check if the name is present in the result2 response
✨ feat(endpoints.py): add new endpoint to get task status by task ID
🔧 fix(schemas.py): update ProcessResponse schema to use 'id' instead of 'session_id' for consistency
✨ feat(schemas.py): add new schema for TaskStatusResponse
📝 docs(custom_component.py): update docstring for better readability and understanding
🔧 chore(custom_component.py): add new file names.txt for locust tests with a list of names for load testing
✨ feat(process.py): introduce SessionManager to handle loading and updating langchain_object sessions
🐛 fix(process.py): update cache with the updated langchain_object after processing graph
🐛 fix(celeryconfig.py): fix condition to check if BROKER_URL and RESULT_BACKEND are present in os.environ before using them, fallback to langflow_redis_host and langflow_redis_port if not present
🔧 fix(worker.py): refactor process_graph_cached into process_graph_cached_task and update function signature and implementation to use SessionManager for loading and updating the graph
✨ feat(manager.py): add initialization for session manager and its dependencies (cache manager and settings manager) to enable session management functionality
🐛 fix(test_process.py): refactor test_load_langchain_object_with_cached_session to use session_manager
🐛 fix(test_process.py): refactor test_load_langchain_object_with_no_cached_session to use session_manager
🐛 fix(test_process.py): refactor test_load_langchain_object_without_session_id to use session_manager
✨ feat(base.Dockerfile): add healthcheck for queue and celeryworker services to ensure their availability
✨ feat(docker-compose.yml): add healthcheck for queue, celeryworker, and flower services to ensure their availability
🐛 fix(docker-compose.yml): fix the path in the command for backend service startup
🐛 fix(flower.env): add missing double quote at the end of C_FORCE_ROOT value
🐛 fix(startup-backend.sh): remove unnecessary --reload flag from uvicorn command and add --workers -1 flag to utilize all available CPU cores
🔧 chore(docker-compose.yml): update Redis image version to 7.2.0 for the queue service
🔧 chore(docker-compose.yml): update command for the flower service to use environment variables for Redis connection details
🔧 chore(flower.env): add LANGFLOW_REDIS_PASSWORD environment variable
✨ feat(manager.py): add support for handling service dependencies when creating services
🔧 chore(manager.py): comment out validation for creating services before the settings service
🔧 chore(utils.py): add get_session_manager() function to retrieve the session manager service
🔀 merge(manager.py): refactor _get method to extract logic into _get_without_lock method for code reuse and readability
🔀 merge(manager.py): refactor set method to use _get_without_lock method for code reuse and readability
🔀 merge(manager.py): add lock acquisition in delete method to ensure thread safety
🔀 merge(utils.py): add update_cache method to Memoize class for updating cache values
- Added `SessionManagerFactory` class in `src/backend/langflow/services/session/factory.py` to create instances of `SessionManager`.
- Added `SessionManager` class in `src/backend/langflow/services/session/manager.py` to manage sessions.
- Added `load_session`, `generate_key`, `update_session`, and `clear_session` methods to `SessionManager` to handle session-related logic.
These changes were made to introduce session management functionality to the application.
🐛 fix(cache/factory.py): change variable name from settings_service to settings_manager for consistency and clarity
🐛 fix(database/factory.py): change variable name from settings_service to settings_manager for consistency and clarity
✨ feat(worker.py): add import statement for SoftTimeLimitExceeded exception from celery.exceptions module to handle SoftTimeLimitExceeded error in build_vertex task
🔀 refactor(base.py): refactor the build method in Vertex class to improve readability and remove redundant code
✨ feat(base.py): add get_task method to Vertex class to retrieve the task from celery using task_id
✨ feat(manager.py): add check to prevent duplicate registration of service factories in ServiceManager class
✨ feat(manager.py): add initialize_settings_manager() function to initialize the settings manager
🐛 fix(test_cache.py): fix clear_cache function call by passing session_id parameter
🐛 fix(test_cache_manager.py): import CacheManager from langflow.services.chat.cache module instead of langflow.services.cache.manager
🐛 fix(test_cli.py): convert temp_dir to string before checking if it is in COMPONENTS_PATH
🐛 fix(test_process.py): fix clear_cache function call by passing session_id parameter
🔧 chore(process.py): replace deprecated build_sorted_vertices_with_caching.hash with build_sorted_vertices_with_caching.session_id to fix incorrect session_id assignment
🔧 chore(utils.py): add get_cache_manager function to retrieve the cache manager from the service manager
🐛 fix(schemas.py): change type annotation of session_id field in ProcessResponse schema to Optional[str] to allow for None value
🐛 fix(run.py): change return type annotation of build_sorted_vertices_with_caching function to Tuple[Any, Dict] to specify the return types
🐛 fix(process.py): change return type annotation of load_langchain_object function to Tuple[Union[Chain, VectorStore], Dict[str, Any], str] to include the session_id value in the return tuple
✨ feat(process.py): add clear_caches_if_needed function to clear cache if clear_cache flag is set
✨ feat(process.py): add load_langchain_object function to load langchain_object and artifacts from data_graph
✨ feat(process.py): add process_inputs function to process inputs and add artifacts to inputs
✨ feat(process.py): add generate_result function to generate result and thought based on langchain_object and inputs
