chore: Enhance Locust load testing and optimize database settings (#6265)

* feat: Enhance Locust load testing for Langflow run endpoint

Refactor locustfile to provide more robust and configurable load testing:
- Add dynamic configuration via environment variables
- Improve error handling and logging
- Implement realistic flow run simulation
- Add connection and timeout handling
- Support API key authentication
- Enhance stats tracking and error reporting

* fix: Improve transaction logging error handling and performance

- Add `no_autoflush` context to prevent unnecessary database operations
- Change transaction logging error from exception to error level logging
- Simplify error handling in log_transaction function

* chore: Add Locust to development dependencies

Update project dependencies by adding Locust (version 2.32.9) to the development requirements, supporting load testing capabilities

* feat: Optimize database connection settings for improved performance and scalability

- Increase default pool_size from 10 to 20 for better connection handling
- Adjust max_overflow to 40 to support higher concurrent connections
- Extend db_connect_timeout from 20 to 30 seconds
- Add pool_recycle and echo settings to db_connection_settings
- Enhance documentation for database connection settings, highlighting SQLite limitations

* feat: Add Locust load testing configuration to Makefile

- Introduce comprehensive Locust load testing target with configurable parameters
- Support flexible testing scenarios with customizable users, spawn rate, and host
- Enable headless and interactive testing modes
- Add environment variable support for API key, flow ID, and other testing parameters
- Provide sensible default values for load testing configuration

* refactor: Remove unused retry configuration in Locust load testing

- Remove RETRY_DELAY and MAX_RETRIES environment variables
- Simplify FlowRunUser configuration by eliminating unused retry settings
- Maintain existing wait time configuration for load testing

* feat: Enforce FLOW_ID requirement for Locust load testing

- Add mandatory validation for FLOW_ID environment variable
- Raise a clear ValueError if FLOW_ID is not provided
- Remove default flow ID to ensure explicit configuration
- Improve load testing configuration robustness

* feat: Add configurable request timeout for Locust load testing

- Introduce `locust_request_timeout` parameter in Makefile
- Update locustfile to use configurable request timeout from environment variable
- Set dynamic connection and network timeout based on REQUEST_TIMEOUT
- Improve request handling with flexible timeout configuration

* revert change to database connection retry

Makefile

@@ -463,3 +463,51 @@ alembic-check: ## check migration status
 alembic-stamp: ## stamp the database with a specific revision
 	@echo 'Stamping the database with revision $(revision)'
 	cd src/backend/base/langflow/ && uv run alembic stamp $(revision)
+
+######################
+# LOAD TESTING
+######################
+
+# Default values for locust configuration
+locust_users ?= 10
+locust_spawn_rate ?= 1
+locust_host ?= http://localhost:7860
+locust_headless ?= true
+locust_time ?= 300s
+locust_api_key ?= your-api-key
+locust_flow_id ?= your-flow-id
+locust_file ?= src/backend/tests/locust/locustfile.py
+locust_min_wait ?= 2000
+locust_max_wait ?= 5000
+locust_request_timeout ?= 30.0
+
+locust: ## run locust load tests (options: locust_users=10 locust_spawn_rate=1 locust_host=http://localhost:7860 locust_headless=true locust_time=300s locust_api_key=your-api-key locust_flow_id=your-flow-id locust_file=src/backend/tests/locust/locustfile.py locust_min_wait=2000 locust_max_wait=5000 locust_request_timeout=30.0)
+	@if [ ! -f "$(locust_file)" ]; then \
+		echo "$(RED)Error: Locustfile not found at $(locust_file)$(NC)"; \
+		exit 1; \
+	fi
+	@echo "Starting Locust with $(locust_users) users, spawn rate of $(locust_spawn_rate)"
+	@echo "Testing host: $(locust_host)"
+	@echo "Using locustfile: $(locust_file)"
+	@export API_KEY=$(locust_api_key) && \
+	export FLOW_ID=$(locust_flow_id) && \
+	export LANGFLOW_HOST=$(locust_host) && \
+	export MIN_WAIT=$(locust_min_wait) && \
+	export MAX_WAIT=$(locust_max_wait) && \
+	export REQUEST_TIMEOUT=$(locust_request_timeout) && \
+	cd $$(dirname "$(locust_file)") && \
+	if [ "$(locust_headless)" = "true" ]; then \
+		uv run locust \
+			--headless \
+			-u $(locust_users) \
+			-r $(locust_spawn_rate) \
+			--run-time $(locust_time) \
+			--host $(locust_host) \
+			-f $$(basename "$(locust_file)"); \
+	else \
+		uv run locust \
+			-u $(locust_users) \
+			-r $(locust_spawn_rate) \
+			--host $(locust_host) \
+			-f $$(basename "$(locust_file)"); \
+	fi