* improving development environment and instructions
* minor edits as requested
* fix typo in DEVELOPMENT.md
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
---------
Co-authored-by: Gabriel Luiz Freitas Almeida <gabriel@langflow.org>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Add pyproject.toml config flag to ruff commands in Makefile
* Update ruff exclude path in pyproject.toml
* Remove unused TYPE_CHECKING import from api_key.py
* Remove redundant config flag from 'ruff check' command in Makefile
* Remove Poetry references from pyproject.toml and update project metadata
* Remove '--no-update' flag from 'uv lock' commands in Makefile
* Add script metadata with Python version and dependencies
* Remove Poetry references and integrate UV setup in CI workflow
* Remove Poetry setup and installation from integration tests workflow
* Remove Poetry references and update workflow to use custom setup action
* Remove references to Poetry from configuration files
* Update scripts
* update the base dep in uv deps
* update nightly scripts
* Add uv creds for publish
* skip tests for now
* fix version
* only build the wheel
* try again
* add uv to python run
* [autofix.ci] apply automated fixes
* use uv cache
* more version fixe
* fixing versions
* fix base version
* Try no frozen?
* skip everything to try docker build
* tag
* frozen
* separate script for updating uv dep
* [autofix.ci] apply automated fixes
* hardcoded versions
* hardcoded versions
* add version to editable package
* build project before docker file runs
* try again
* fix uv patht o build
* don't know why this would mkae a difference
* debug statements
* debug statements
* debug statements
* change path to whl 🤷
* manually move the wheel...
* make dir
* try no sources
* add back tests
* refactor uv to action
* add uv action
* Update nightly build workflow to include uv lock files in version update commit
* Update lint-py workflow to use specific ref for setup-uv action
* Add checkout step to style-check-py GitHub Actions workflow
* Remove redundant GitHub ref syntax in lint-py.yml workflow
* Update lint-py.yml to use specific ref for setup-uv action
* Update action.yml: standardize quotes and remove redundant checkout step
* Add checkout step to GitHub Actions workflows for specific ref handling
- Introduced `actions/checkout@v4` step to multiple workflows to ensure code is checked out at a specific ref.
- Updated `.github/workflows/docker-build.yml`, `.github/workflows/release_nightly.yml`, `.github/workflows/lint-py.yml`, and `.github/workflows/style-check-py.yml` to include the new checkout step.
- Ensured credentials are persisted during the checkout process.
* Add checkout step to Python test workflow with specific ref
---------
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Gabriel Luiz Freitas Almeida <gabriel@langflow.org>
* fix dev dependencies
* fix dev dependencies
* update lock
* Add langchain-unstructured dependency to pyproject.toml
* Switch to uv-based Docker image and streamline build process
- Use `ghcr.io/astral-sh/uv:python3.12-bookworm-slim` as the base image
- Simplify environment variable setup and remove unnecessary ones
- Install project dependencies using `uv sync` with cache mounts
- Add and copy necessary files for the build process
- Update npm installation and frontend build steps
- Adjust entrypoint and runtime configurations
- Remove redundant user setup and streamline CMD execution
* Refactor Dockerfile to use multi-stage builds for optimized image size
- Introduce a builder stage to install dependencies and build the frontend.
- Use npm ci instead of npm install for more reliable builds.
- Copy built frontend assets to the final image.
- Transition to a runtime stage with a slimmer Python base image.
- Ensure the final image only contains necessary runtime dependencies.
* Switch base image to `ghcr.io/astral-sh/uv:python3.12-bookworm-slim` and update Dockerfile
- Replace `python:3.12.3-slim` with `ghcr.io/astral-sh/uv:python3.12-bookworm-slim` as the base image.
- Enable bytecode compilation and set link mode to copy.
- Update dependency installation process using `uv sync`.
- Simplify frontend build and copy process.
- Add OCI labels for image metadata.
- Update CMD to use `langflow-base` for running the application.
* Update Dockerfile to use uv base image and optimize dependency installation
- Switch base image to `ghcr.io/astral-sh/uv:python3.12-bookworm-slim`
- Combine apt-get commands and clean up to reduce image size
- Replace Poetry with uv for dependency management and caching
* Refactor Dockerfile to use multi-stage build for optimized image size
- Introduce a builder stage using `ghcr.io/astral-sh/uv:python3.12-bookworm-slim`
- Add a runtime stage using `python:3.12.3-slim`
- Copy the virtual environment from the builder stage to the runtime stage
- Remove the `uv` entrypoint to avoid invoking it by default
* Update Dockerfile: Fix indentation and remove redundant ENTRYPOINT reset
* ci: update release workflows to use uv (#3919)
* Update nightly build workflow to use 'uv' for dependency management and caching
- Replace Poetry with 'uv' for dependency installation and project setup
- Add steps to install 'uv' and set up caching for 'uv.lock'
- Modify Python setup to use version specified in 'pyproject.toml'
- Remove Node.js setup steps
* Update release workflow to use uv for dependency management
- Replace Poetry with uv for dependency installation and caching
- Update Python setup to use version specified in pyproject.toml
- Add steps to restore uv cache and install project dependencies using uv
- Adjust publish steps to maintain functionality with new setup
* Replace 'poetry publish' with 'uv publish' in Makefile for consistency
* Update nightly build workflow to use 'uv' for dependency management and caching
* Set up Node.js 20 in release_nightly workflow and update version verification logic
* Update Makefile to use `uv sync --frozen` for backend dependencies installation
* Update Makefile to pass arguments to 'uv build' and remove '--skip-existing' from publish commands
- Modified 'build_langflow' target to accept arguments.
- Removed '--skip-existing' from 'publish_base' and 'publish_base_testpypi' commands.
- Added TODO comments for updating test-pypi repository usage.
* Remove --skip-existing flag from uv publish commands and add TODO comments for test-pypi updates
* new lock
* Update CI workflow to remove version prefix and add build args
- Removed 'v' prefix from version extraction in nightly release workflow.
- Added '--no-sources' argument to the build command in the distribution step.
* Update CI workflow to use 'uv' for versioning and publishing
- Replace 'poetry' with 'uv' for version extraction in release checks
* Update CI workflow to use UV_PUBLISH_TOKEN for PyPI publishing
* Add verification step for 'langflow-nightly' name and version in CI workflow
- Corrected awk commands for extracting 'langflow-base' name and version.
- Added a new step to verify 'langflow-nightly' name and version against expected values.
* feat: Update dev.Dockerfile to use 'uv' for dependency management and caching
* update dockerfiles with --no-editable
---------
Co-authored-by: phact <estevezsebastian@gmail.com>
Co-authored-by: Jordan Frazier <jordan.frazier@datastax.com>
* Remove Poetry check, update uv command, and remove dev target from Makefile
* Update CONTRIBUTING.md to reflect new development setup and remove docker-compose instructions
* uv sync works
* fist stab at Makefile
* uv treatment for langflow-base
* sqlmodel to 0.0.18
* add reinstall_backend to Makefile
* makefile - reinstall_backend fix and unit_test dependency
* fix dev dependencies
* fix dev dependencies
* fix dev dependencies
* lock
* Makefile
* [autofix.ci] apply automated fixes
* Update Makefile
Co-authored-by: Gabriel Luiz Freitas Almeida <gabriel@langflow.org>
* delete update_dependencies
* fix lint
* Remove Poetry lock check from GitHub Actions workflow
* Switch to 'uv' for dependency management and caching in style-check workflow
* Update style-check workflow to use '--only-dev' flag for Ruff check
* Integrate 'uv' package setup and caching in GitHub Actions workflows
* Update version check in GitHub Actions to use 'uv tree' for langflow-base
* Remove redundant poetry environment setup in GitHub Actions workflow
* Add step to minimize uv cache in GitHub Actions workflow
* Update GitHub Actions workflow to use 'uv' for dependency management and caching
* Remove redundant script execution from build_langflow target in Makefile
* [autofix.ci] apply automated fixes
* Switch build system from Poetry to Hatchling and update dependencies
- Replace `poetry-core` with `hatchling` in build-system requirements
- Update `langflow-base` dependency to version `0.0.96`
- Add `tool.hatch.build.targets.wheel` configuration
- Adjust `tool.uv.sources` paths for `langflow-frontend` and `langflow-base`
* update lock
* Switch build system from Poetry to Hatchling in pyproject.toml
* Add langchain-unstructured dependency to pyproject.toml
---------
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Gabriel Luiz Freitas Almeida <gabriel@langflow.org>
* Refactor test fixtures and imports for optimization
- Reordered and optimized imports in `conftest.py`.
- Changed return statements to yield in `basic_graph` and `create_webhook` fixtures for better resource management.
- Added cleanup step to `create_webhook` fixture to delete created webhook after test completion.
* Refactor imports and add placeholder client fixture in test_ChatOllama_component.py
* Add pytest fixture 'client' to test_graph.py
* Add pytest fixture 'client' to test_callback_graph.py for test optimization
* Add 'unit_tests_looponfail' target to Makefile for running unit tests with loop-on-fail option
* Optimize test durations and update test cases in various modules
* chore(Makefile): redirect output of 'poetry install' and 'npm install' commands to /dev/null to reduce noise
chore(Makefile): redirect output of 'npm run build' command to /dev/null to improve build process efficiency
feat(Makefile): update build_frontend target to include copying built files to backend directory for deployment
* chore(Makefile): refactor lint target to depend on install_backend task for better dependency management
chore(Makefile): refactor frontend, frontendc, backend, build_and_run, build_and_install, and build targets to depend on setup_env task for consistent environment setup before execution
* chore(Makefile): update Makefile commands to use @ symbol for silent execution and improve readability
* chore(Makefile): update Makefile commands to use "@" to suppress command output and improve readability
* chore(Makefile): update run_cli target to include dependencies installation and frontend build steps for better clarity and consistency
* chore(Makefile): add command to delete empty directories in clean_python_cache target
* Updates and changes to the Makefile:
1. Added removal of `frontend` directory inside `src/backend/base/langflow/` and `build` directory inside `src/frontend/` to the `clean_npm_cache` target.
2. Added descriptive comments for the `build_and_install`, `build_and_run`, `fix_codespell`, `setup_poetry`, `unit_tests`, `integration_tests`, and `tests_frontend` targets.
Looking forward to your feedback.
* Improvements: Structure and Functionality Improvements
- Reorganized commands to facilitate understanding of the structure
- Enhanced check_tools to detect the Python version
- Added a multi-environment script to support check_tools
- make init now builds the frontend and runs the application
- Aesthetic improvements in output messages
TO-DO:
- Reorganize container-related commands
- Reorganize other miscellaneous utilities
- Document usage in the application docs
- Prepare the dev environment following the maintainers' recommended practices
* Removed pre-commit as it is no longer used.
* Restored 'patch' command in Makefile, it updates the 'pyproject.toml' with the new project version.
* fix: update PYTHON_REQUIRED extraction to correctly capture Python version
Changed the method of extracting the required Python version from `pyproject.toml`:
- Old method:
`PYTHON_REQUIRED=$(shell grep "^python" pyproject.toml | sed -n 's/.*"\(.*\)"$$/\1/p')`
- New method:
`PYTHON_REQUIRED=$(shell grep '^python[[:space:]]*=' pyproject.toml | sed -n 's/.*"\([^"]*\)".*/\1/p')`
The old method of capturing the Python version was too broad and could inadvertently match other dependencies starting with "python" (e.g., `python-socketio`, `python-dotenv`). This could lead to incorrect extraction of the required Python version, potentially causing version mismatches and failures in environment setup.
* fix: update PYTHON_REQUIRED extraction to correctly capture Python version
Changed the method of extracting the required Python version from `pyproject.toml`:
- Old method:
`PYTHON_REQUIRED=$(shell grep "^python" pyproject.toml | sed -n 's/.*"\(.*\)"$$/\1/p')`
- New method:
`PYTHON_REQUIRED=$(shell grep '^python[[:space:]]*=' pyproject.toml | sed -n 's/.*"\([^"]*\)".*/\1/p')`
The old method of capturing the Python version was too broad and could inadvertently match other dependencies starting with "python" (e.g., `python-socketio`, `python-dotenv`). This could lead to incorrect extraction of the required Python version, potentially causing version mismatches and failures in environment setup.
---------
Co-authored-by: Gabriel Luiz Freitas Almeida <gabriel@langflow.org>
* feat(utils.py): add escape_json_dump function to escape JSON strings for Edge dictionaries
* refactor(Output): streamline add_types method to prevent duplicate entries in types list for improved type management
* feat(data.py): add classmethod decorator to validate_data for enhanced validation logic when checking data types
* feat(setup.py): implement retry logic for loading starter projects to enhance robustness against JSON decode errors
* fix(input_mixin.py): improve model_config formatting and update field_type alias for clarity and consistency in field definitions
* feat(types.py): refactor vertex constructors to use NodeData and add input/output methods for better component interaction
* feat(schema.py): add NodeData and Position TypedDicts for improved type safety and structure in vertex data handling
* feat(base.py): update Vertex to use NodeData type and add to_data method for better data management and access
* refactor(schema.py): update TargetHandle and SourceHandle models to include model_config attribute
* Add TypedDict classes for graph schema serialization in `schema.py`
* Refactor `Edge` class to improve handle validation and data handling
- Consolidated imports and removed redundant `BaseModel` definitions for `SourceHandle` and `TargetHandle`.
- Added `valid_handles`, `target_param`, and `_target_handle` attributes to `Edge` class.
- Enhanced handle validation logic to distinguish between dictionary and string types.
- Introduced `to_data` method to return edge data.
- Updated attribute names to follow consistent naming conventions (`base_classes`, `input_types`, `field_name`).
* Refactor `Edge` class to improve handle validation and data handling
* Refactor: Standardize attribute naming and add `to_data` method in Edge class
- Renamed attributes to use snake_case consistently (`baseClasses` to `base_classes`, `inputTypes` to `input_types`, `fieldName` to `field_name`).
- Added `to_data` method to return `_data` attribute.
- Updated validation methods to use new attribute names.
* Refactor: Update Edge class to consistently use snake_case for attributes and improve validation logic for handles
* Refactor: Change node argument type in add_node and _create_vertex methods to NodeData for better type safety and clarity
* Refactor: Implement JSON serialization for graph data with `dumps` and `dump` methods, enhancing data export capabilities
* Refactor: Add pytest fixtures for ingestion and RAG graphs, enhance test structure for better clarity and organization
* Refactor: Add pytest fixtures for memory_chatbot_graph tests and improve test structure
* Refactor: Remove unused methods in ComponentVertex class to streamline code and improve readability
* Refactor: Remove unnecessary line in ComponentVertex class to enhance code clarity and maintainability
* Refactor: Update import path for DefaultPromptField to improve code organization and maintainability in api_utils.py
* Refactor: Update import path for DefaultPromptField to enhance code organization and maintainability in prompt.py
* fix: Remove fixture in test_memory_chatbot.py that blocked db setup
* Refactor: Add durations path for unit tests to improve test reporting
* Refactor: Add splitting algorithm option for unit tests
* Add async option to Makefile for unit tests and update GitHub Actions workflow
- Introduced `async` variable in Makefile to conditionally run unit tests with or without parallel execution.
- Updated `unit_tests` target in Makefile to handle `async` flag.
- Modified GitHub Actions workflow to set `async=false` for unit tests.
* Updates and changes to the Makefile:
1. Added removal of `frontend` directory inside `src/backend/base/langflow/` and `build` directory inside `src/frontend/` to the `clean_npm_cache` target.
2. Added descriptive comments for the `build_and_install`, `build_and_run`, `fix_codespell`, `setup_poetry`, `unit_tests`, `integration_tests`, and `tests_frontend` targets.
Looking forward to your feedback.
* Improvements: Structure and Functionality Improvements
- Reorganized commands to facilitate understanding of the structure
- Enhanced check_tools to detect the Python version
- Added a multi-environment script to support check_tools
- make init now builds the frontend and runs the application
- Aesthetic improvements in output messages
TO-DO:
- Reorganize container-related commands
- Reorganize other miscellaneous utilities
- Document usage in the application docs
- Prepare the dev environment following the maintainers' recommended practices
* Removed pre-commit as it is no longer used.
* Restored 'patch' command in Makefile, it updates the 'pyproject.toml' with the new project version.
---------
Co-authored-by: Gabriel Luiz Freitas Almeida <gabriel@langflow.org>
* feat(Makefile): add 'patch' target to bump the version in langflow and langflow-base
* chore: bump langflow and langflow-base versions to 1.0.13 and 0.0.89 respectively
* chore: lock dependencies without updating in Makefile
Lock the dependencies without updating them in the Makefile. This ensures that the dependencies remain at their current versions and prevents any unintended changes in compatibility or functionality.
* chore: update dependencies and add lock step in py_autofix.yml
Update the dependencies in the py_autofix.yml workflow file to include a new step for checking the Poetry lock file. This ensures that the dependencies are locked and prevents any unintended changes in compatibility or functionality. The lock step is added to the existing workflow to ensure that the Poetry lock file is checked and updated if necessary.
* refactor: move tests folder to src/backend
* chore(Makefile): update pytest commands to run tests from the correct directory paths for unit and integration tests
* refactor: update file path in test_custom_component.py
The file path in the test_custom_component.py file has been updated to use the correct relative path to the component_multiple_outputs.py file. This change ensures that the test code can access the correct file and improves the reliability of the test.
Updates and changes to the Makefile:
1. Added removal of `frontend` directory inside `src/backend/base/langflow/` and `build` directory inside `src/frontend/` to the `clean_npm_cache` target.
2. Added descriptive comments for the `build_and_install`, `build_and_run`, `fix_codespell`, `setup_poetry`, `unit_tests`, `integration_tests`, and `tests_frontend` targets.
Looking forward to your feedback.
* feat: Add Gemma 2 to Groq model list (#2586)
Add gemma2 to groq_constants.py
* Improve Makefile: color output, clean commands, dependency checks
- Add colored output to 'make help' for various shells.
- Add 'clean_all', 'clean_python_cache', and 'clean_npm_cache' commands.
- Check for installed tools: 'poetry', 'docker', 'pipx', and 'npm'.
* Added success message at the end of the check_tools target to indicate all required tools are installed.
* Update Makefile to use command substitution for killing process
* break long lines into shorter ones
* replace fixed values with variables
* correct the unit test coverage report
---------
Co-authored-by: italojohnny <italojohnnydosanjos@gmail.com>
Co-authored-by: Gabriel Luiz Freitas Almeida <gabriel@langflow.org>
The Makefile has been updated to ignore the integration tests when running the unit_tests target. This change improves the efficiency of the unit testing process by excluding unnecessary tests.
The call to build_frontend in the Makefile has been removed as it is no longer needed. This change improves the build process by eliminating unnecessary steps.
🔧 (pyproject.toml): update version to "1.0.0rc0" for the 'langflow' package
🔧 (pyproject.toml): update version to "0.0.73" for the 'langflow-base' package
🔧 (pyproject.toml): remove 'python-socketio' dependency and update other dependencies to specific versions
* add first astra integ test framework
* use fixtures
* remove old tests from merge
* Add correct sender type
* chore: Update unit test command in GitHub workflow
---------
Co-authored-by: ogabrielluiz <gabriel@langflow.org>
* ignore sqlite tmp files
* update many dependencies to be optional
* improve makefile help messagem
* chore: Update langflow-base dependency to use local path for development
* change postgres to mandatory dependency
* improve makefile readability
* Revert update many dependencies to be optional
This reverts commit 80780565df5bc63632e6971054670cee467542f9.
---------
Co-authored-by: ogabrielluiz <gabriel@langflow.org>
* refactor: Update pytest arguments in Makefile for better test execution
The Makefile has been modified to update the pytest arguments in the `tests` target. The `--instafail` flag has been replaced with `-ra -n auto -m "not api_key_required"`. This change allows for better test execution by including additional options for reporting, parallelization, and test selection.
Note: The commit message has been generated based on the provided code changes and recent commits.
* ✨ (test_endpoints.py): add pytest marker 'api_key_required' to mark tests that require an API key for authorization. This helps in easily identifying and managing tests that need special permissions.
