🔧 chore(deploy): update docker-compose.override.yml to version 3.8
📦 chore(docker-compose.with_tests.yml): add docker-compose file with tests configuration
This commit adds a new docker-compose file named `docker-compose.with_tests.yml` which includes the configuration for running tests. The file includes the following services:
- `proxy`: Configures Traefik as a reverse proxy with Docker integration and enables access logs, the Traefik dashboard, and API.
- `backend`: Sets up the backend service with dependencies on a database, message broker, and result backend. It also includes labels for Traefik routing.
- `db`: Configures a PostgreSQL database with a volume for data persistence.
- `pgadmin`: Sets up pgAdmin for managing the PostgreSQL database.
- `result_backend`: Configures a Redis instance for the result backend.
- `celeryworker`: Sets up a Celery worker for background task processing.
- `flower`: Configures Flower for monitoring and managing Celery workers.
- `frontend`: Sets up the frontend service with labels for Traefik routing.
- `broker`: Configures RabbitMQ with the management console.
- `prometheus`: Sets up Prometheus for monitoring.
- `grafana`: Configures Grafana for visualization and monitoring.
- `tests`: Extends the `backend` service and runs pytest for running tests.
This file allows running the application with the necessary services for testing and monitoring.
🔧 chore(docker-compose.yml): add missing volumes and networks for services
🔧 chore(docker-compose.yml): add traefik-public network with configurable external setting for flexibility in testing
📝 docs(async-tasks.mdx): update docker-compose command to use the correct file name for running tests
- Rename volume `./pgadmin` to `pgadmin-data` for better naming convention and clarity.
- Rename volumes for RabbitMQ:
- `./.docker/rabbitmq/etc/` to `rabbitmq_data` for consistency and clarity.
- `./.docker/rabbitmq/data/` to `rabbitmq_data` for consistency and clarity.
- `./.docker/rabbitmq/logs/` to `rabbitmq_log` for consistency and clarity.
No functional changes, only volume name updates.
- Update RabbitMQ configuration by setting the default username and password to "langflow"
- Add OpenAI API key for integration with OpenAI services
- Add superuser credentials for the Langflow application
✨ feat(docker-compose.override.yml): add configuration for the celeryworker service to enable Traefik routing and expose port 7860 for API endpoints
🐛 fix(docker-compose.yml): remove parallel test execution flag from the command for the test service
🔧 chore(docker-compose.yml): add support for loading environment variables from .env file for the proxy service
🔧 chore(docker-compose.yml): add support for loading environment variables from .env file for the backend service
🔧 chore(docker-compose.yml): add support for loading environment variables from .env file for the pgadmin service
🔧 chore(docker-compose.yml): add support for loading environment variables from .env file for the queue service
🔧 chore(docker-compose.yml): add support for loading environment variables from .env file for the celeryworker service
🔧 chore(docker-compose.yml): add support for loading environment variables from .env file for the flower service
🔧 chore(docker-compose.yml): add support for loading environment variables from .env file for the frontend service
🔧 chore(docker-compose.yml): add support for loading environment variables from .env file for the prometheus service
🔧 chore(docker-compose.yml): add support for loading environment variables from .env file for the grafana service
🔧 chore(docker-compose.yml): add build configuration to flower service to use base.Dockerfile for building the image
🔧 chore(docker-compose.yml): update command for flower service to use the correct broker URL and set the port to 5555
🔧 chore(docker-compose.yml): remove unnecessary comment at the end of the file
🔧 chore(terraform): modify user data script for manager instance to fetch instance metadata with token and start a netcat server on port 8080 to provide join token
🔧 chore(terraform): modify user data script for worker instance to fetch manager IP and join the swarm using the join token obtained from the manager on port 8080
feat(terraform): add aws_subnet resource for swarm-public-subnet to create a public subnet for the swarm cluster
feat(terraform): add aws_internet_gateway resource to create an internet gateway for the swarm VPC
feat(terraform): add aws_route_table resource to create a route table for the swarm VPC
feat(terraform): add aws_route_table_association resource to associate the swarm-public-subnet with the public route table
🐳 chore(deploy): update traefik image version to v3.0 for improved functionality and compatibility
🔧 chore(deploy): remove unnecessary port mappings for backend, queue, pgadmin, redis, prometheus, and grafana services
This commit adds a new file `docker-compose.override.yml` to the `deploy` directory. The file contains configuration for the Traefik proxy and services used for local development. The configuration includes:
- Enabling Docker in Traefik to read labels from Docker services
- Adding a constraint to only use services with a specific label for this stack
- Disabling the exposure of all Docker services by default
- Enabling the access log and Traefik log
- Enabling the Traefik Dashboard and API in insecure mode for local development
The file also includes configuration for the following services:
- `proxy`: Configured to listen on ports 80 and 8090, with labels for routing and load balancing
- `pgadmin`: Configured to listen on port 5050
- `flower`: Configured to listen on port 5555
- `backend`: Configured with labels for routing and load balancing on paths `/api/v1`, `/docs`, and `/health`
- `frontend`: Configured with labels for routing and load balancing on the root path
The `traefik-public` network is also defined for use by the Traefik proxy.
This file is intended to be used as an override for the main `docker-compose.yml` file during local development.
🐳 chore(docker-compose.yml): add Traefik reverse proxy configuration for services
🔧 fix(docker-compose.yml): fix path to startup-backend.sh script
🚀 feat(docker-compose.yml): add support for deploying backend, celery worker, flower, frontend, prometheus, and grafana services
🔧 fix(docker-compose.yml): fix volume configuration for db service
🔧 fix(docker-compose.yml): fix network configuration for pgadmin service
🔧 fix(docker-compose.yml): fix network configuration for traefik proxy service
🔧 fix(docker-compose.yml): fix network configuration for flower service
🔧 fix(docker-compose.yml): fix network configuration for frontend service
🔧 fix(docker-compose.yml): fix network configuration for prometheus service
🔧 fix(docker-compose.yml): fix network configuration for grafana service
🔧 chore(docker-compose.yml): update backend.env file path to remove leading './' for consistency
🔧 chore(docker-compose.yml): remove explicit port mapping for backend service to use the default port
🔧 chore(docker-compose.yml): add deploy configuration for celery worker service to ensure only one replica is deployed
🔧 chore(docker-compose.yml): add prometheus and grafana services with necessary configurations and dependencies
📝 docs(prometheus.yml): add prometheus configuration file with scrape targets for prometheus and flower services
✨ feat(base.Dockerfile): add healthcheck for queue and celeryworker services to ensure their availability
✨ feat(docker-compose.yml): add healthcheck for queue, celeryworker, and flower services to ensure their availability
🐛 fix(docker-compose.yml): fix the path in the command for backend service startup
🐛 fix(flower.env): add missing double quote at the end of C_FORCE_ROOT value
🐛 fix(startup-backend.sh): remove unnecessary --reload flag from uvicorn command and add --workers -1 flag to utilize all available CPU cores
🔧 chore(docker-compose.yml): update Redis image version to 7.2.0 for the queue service
🔧 chore(docker-compose.yml): update command for the flower service to use environment variables for Redis connection details
🔧 chore(flower.env): add LANGFLOW_REDIS_PASSWORD environment variable
