chore: update docker images to use uv (#3916)

* fix dev dependencies * fix dev dependencies * update lock * Add langchain-unstructured dependency to pyproject.toml * Switch to uv-based Docker image and streamline build process - Use `ghcr.io/astral-sh/uv:python3.12-bookworm-slim` as the base image - Simplify environment variable setup and remove unnecessary ones - Install project dependencies using `uv sync` with cache mounts - Add and copy necessary files for the build process - Update npm installation and frontend build steps - Adjust entrypoint and runtime configurations - Remove redundant user setup and streamline CMD execution * Refactor Dockerfile to use multi-stage builds for optimized image size - Introduce a builder stage to install dependencies and build the frontend. - Use npm ci instead of npm install for more reliable builds. - Copy built frontend assets to the final image. - Transition to a runtime stage with a slimmer Python base image. - Ensure the final image only contains necessary runtime dependencies. * Switch base image to `ghcr.io/astral-sh/uv:python3.12-bookworm-slim` and update Dockerfile - Replace `python:3.12.3-slim` with `ghcr.io/astral-sh/uv:python3.12-bookworm-slim` as the base image. - Enable bytecode compilation and set link mode to copy. - Update dependency installation process using `uv sync`. - Simplify frontend build and copy process. - Add OCI labels for image metadata. - Update CMD to use `langflow-base` for running the application. * Update Dockerfile to use uv base image and optimize dependency installation - Switch base image to `ghcr.io/astral-sh/uv:python3.12-bookworm-slim` - Combine apt-get commands and clean up to reduce image size - Replace Poetry with uv for dependency management and caching * Refactor Dockerfile to use multi-stage build for optimized image size - Introduce a builder stage using `ghcr.io/astral-sh/uv:python3.12-bookworm-slim` - Add a runtime stage using `python:3.12.3-slim` - Copy the virtual environment from the builder stage to the runtime stage - Remove the `uv` entrypoint to avoid invoking it by default * Update Dockerfile: Fix indentation and remove redundant ENTRYPOINT reset * ci: update release workflows to use uv (#3919) * Update nightly build workflow to use 'uv' for dependency management and caching - Replace Poetry with 'uv' for dependency installation and project setup - Add steps to install 'uv' and set up caching for 'uv.lock' - Modify Python setup to use version specified in 'pyproject.toml' - Remove Node.js setup steps * Update release workflow to use uv for dependency management - Replace Poetry with uv for dependency installation and caching - Update Python setup to use version specified in pyproject.toml - Add steps to restore uv cache and install project dependencies using uv - Adjust publish steps to maintain functionality with new setup * Replace 'poetry publish' with 'uv publish' in Makefile for consistency * Update nightly build workflow to use 'uv' for dependency management and caching * Set up Node.js 20 in release_nightly workflow and update version verification logic * Update Makefile to use `uv sync --frozen` for backend dependencies installation * Update Makefile to pass arguments to 'uv build' and remove '--skip-existing' from publish commands - Modified 'build_langflow' target to accept arguments. - Removed '--skip-existing' from 'publish_base' and 'publish_base_testpypi' commands. - Added TODO comments for updating test-pypi repository usage. * Remove --skip-existing flag from uv publish commands and add TODO comments for test-pypi updates * new lock * Update CI workflow to remove version prefix and add build args - Removed 'v' prefix from version extraction in nightly release workflow. - Added '--no-sources' argument to the build command in the distribution step. * Update CI workflow to use 'uv' for versioning and publishing - Replace 'poetry' with 'uv' for version extraction in release checks * Update CI workflow to use UV_PUBLISH_TOKEN for PyPI publishing * Add verification step for 'langflow-nightly' name and version in CI workflow - Corrected awk commands for extracting 'langflow-base' name and version. - Added a new step to verify 'langflow-nightly' name and version against expected values. * feat: Update dev.Dockerfile to use 'uv' for dependency management and caching * update dockerfiles with --no-editable --------- Co-authored-by: phact <estevezsebastian@gmail.com> Co-authored-by: Jordan Frazier <jordan.frazier@datastax.com>
2024-09-30 19:49:25 -03:00 · 2024-09-30 19:49:25 -03:00 · 43ad226ddb
commit 43ad226ddb
parent 855694208e
8 changed files with 231 additions and 239 deletions
--- a/docker/build_and_push.Dockerfile
+++ b/docker/build_and_push.Dockerfile
@ -1,7 +1,6 @@
 # syntax=docker/dockerfile:1
 # Keep this syntax directive! It's used to enable Docker BuildKit

-
 ################################
 # BUILDER-BASE
 # Used to build deps + create our virtual environment
@ -9,76 +8,66 @@

 # 1. use python:3.12.3-slim as the base image until https://github.com/pydantic/pydantic-core/issues/1292 gets resolved
 # 2. do not add --platform=$BUILDPLATFORM because the pydantic binaries must be resolved for the final architecture
-FROM python:3.12.3-slim as builder-base
+# Use a Python image with uv pre-installed
+FROM ghcr.io/astral-sh/uv:python3.12-bookworm-slim AS builder

-ENV PYTHONDONTWRITEBYTECODE=1 \
-    \
-    # pip
-    PIP_DISABLE_PIP_VERSION_CHECK=on \
-    PIP_DEFAULT_TIMEOUT=100 \
-    \
-    # poetry
-    # https://python-poetry.org/docs/configuration/#using-environment-variables
-    POETRY_VERSION=1.8.2 \
-    # make poetry install to this location
-    POETRY_HOME="/opt/poetry" \
-    # make poetry create the virtual environment in the project's root
-    # it gets named `.venv`
-    POETRY_VIRTUALENVS_IN_PROJECT=true \
-    # do not ask any interactive question
-    POETRY_NO_INTERACTION=1 \
-    \
-    # paths
-    # this is where our requirements + virtual environment will live
-    PYSETUP_PATH="/opt/pysetup" \
-    VENV_PATH="/opt/pysetup/.venv"
+# Install the project into `/app`
+WORKDIR /app
+
+# Enable bytecode compilation
+ENV UV_COMPILE_BYTECODE=1
+
+# Copy from the cache instead of linking since it's a mounted volume
+ENV UV_LINK_MODE=copy

 RUN apt-get update \
    && apt-get install --no-install-recommends -y \
-    # deps for installing poetry
-    curl \
    # deps for building python deps
-    build-essential npm \
+    build-essential \
+    # npm
+    npm \
    # gcc
    gcc \
    && apt-get clean \
    && rm -rf /var/lib/apt/lists/*

-RUN --mount=type=cache,target=/root/.cache \
-    curl -sSL https://install.python-poetry.org | python3 -
+RUN --mount=type=cache,target=/root/.cache/uv \
+    --mount=type=bind,source=uv.lock,target=uv.lock \
+    --mount=type=bind,source=README.md,target=README.md \
+    --mount=type=bind,source=pyproject.toml,target=pyproject.toml \
+    --mount=type=bind,source=src/backend/base/README.md,target=src/backend/base/README.md \
+    --mount=type=bind,source=src/backend/base/uv.lock,target=src/backend/base/uv.lock \
+    --mount=type=bind,source=src/backend/base/pyproject.toml,target=src/backend/base/pyproject.toml \
+    uv sync --frozen --no-install-project --no-editable
+
+ADD ./src /app/src
+
+COPY src/frontend /tmp/src/frontend
+WORKDIR /tmp/src/frontend
+RUN --mount=type=cache,target=/root/.npm \
+    npm ci \
+    && npm run build \
+    && cp -r build /app/src/backend/langflow/frontend \
+    && rm -rf /tmp/src/frontend

 WORKDIR /app
-COPY pyproject.toml poetry.lock README.md ./
-COPY src/ ./src
-COPY scripts/ ./scripts
-RUN python -m pip install requests --user && cd ./scripts && python update_dependencies.py
+ADD ./pyproject.toml /app/pyproject.toml
+ADD ./uv.lock /app/uv.lock

-# 1. Install the dependencies using the current poetry.lock file to create reproducible builds
-# 2. Do not install dev dependencies
-# 3. Install all the extras to ensure all optionals are installed as well
-# 4. --sync to ensure nothing else is in the environment
-# 5. Build the wheel and install "langflow" package (mainly for version)
-
-# Note: moving to build and installing the wheel will make the docker images not reproducible.
-RUN $POETRY_HOME/bin/poetry lock --no-update \
-      # install current lock file with fixed dependencies versions \
-      # do not install dev dependencies \
-      && $POETRY_HOME/bin/poetry install --without dev --sync -E deploy -E couchbase -E cassio \
-      && $POETRY_HOME/bin/poetry build -f wheel \
-      && $POETRY_HOME/bin/poetry run pip install dist/*.whl
+RUN --mount=type=cache,target=/root/.cache/uv \
+    uv sync --frozen --no-editable

 ################################
 # RUNTIME
 # Setup user, utilities and copy the virtual environment only
 ################################
-# 1. use python:3.12.3-slim as the base image until https://github.com/pydantic/pydantic-core/issues/1292 gets resolved
-FROM python:3.12.3-slim as runtime
+FROM python:3.12.3-slim AS runtime

-RUN apt-get -y update \
-    && apt-get install --no-install-recommends -y \
-    curl \
-    && apt-get clean \
-    && rm -rf /var/lib/apt/lists/*
+RUN useradd user -u 1000 -g 0 --no-create-home --home-dir /app/data
+COPY --from=builder --chown=1000 /app/.venv /app/.venv
+
+# Place executables in the environment at the front of the path
+ENV PATH="/app/.venv/bin:$PATH"

 LABEL org.opencontainers.image.title=langflow
 LABEL org.opencontainers.image.authors=['Langflow']
@ -86,14 +75,10 @@ LABEL org.opencontainers.image.licenses=MIT
 LABEL org.opencontainers.image.url=https://github.com/langflow-ai/langflow
 LABEL org.opencontainers.image.source=https://github.com/langflow-ai/langflow

-RUN useradd user -u 1000 -g 0 --no-create-home --home-dir /app/data
-COPY --from=builder-base --chown=1000 /app/.venv /app/.venv
-ENV PATH="/app/.venv/bin:${PATH}"
-
 USER user
 WORKDIR /app

 ENV LANGFLOW_HOST=0.0.0.0
 ENV LANGFLOW_PORT=7860

-CMD ["python", "-m", "langflow", "run"]
+CMD ["langflow", "run"]
--- a/docker/build_and_push_base.Dockerfile
+++ b/docker/build_and_push_base.Dockerfile
@ -1,58 +1,28 @@
-
-
 # syntax=docker/dockerfile:1
 # Keep this syntax directive! It's used to enable Docker BuildKit

-# Based on https://github.com/python-poetry/poetry/discussions/1879?sort=top#discussioncomment-216865
-# but I try to keep it updated (see history)
-
-################################
-# PYTHON-BASE
-# Sets up all our shared environment variables
-################################
-
-# use python:3.12.3-slim as the base image until https://github.com/pydantic/pydantic-core/issues/1292 gets resolved
-FROM python:3.12.3-slim as python-base
-
-# python
-ENV PYTHONUNBUFFERED=1 \
-    # prevents python creating .pyc files
-    PYTHONDONTWRITEBYTECODE=1 \
-    \
-    # pip
-    PIP_DISABLE_PIP_VERSION_CHECK=on \
-    PIP_DEFAULT_TIMEOUT=100 \
-    \
-    # poetry
-    # https://python-poetry.org/docs/configuration/#using-environment-variables
-    POETRY_VERSION=1.8.2 \
-    # make poetry install to this location
-    POETRY_HOME="/opt/poetry" \
-    # make poetry create the virtual environment in the project's root
-    # it gets named `.venv`
-    POETRY_VIRTUALENVS_IN_PROJECT=true \
-    # do not ask any interactive question
-    POETRY_NO_INTERACTION=1 \
-    \
-    # paths
-    # this is where our requirements + virtual environment will live
-    PYSETUP_PATH="/opt/pysetup" \
-    VENV_PATH="/opt/pysetup/.venv"
-
-
-# prepend poetry and venv to path
-ENV PATH="$POETRY_HOME/bin:$VENV_PATH/bin:$PATH"
-

 ################################
 # BUILDER-BASE
 # Used to build deps + create our virtual environment
 ################################
-FROM python-base as builder-base
+
+# 1. use python:3.12.3-slim as the base image until https://github.com/pydantic/pydantic-core/issues/1292 gets resolved
+# 2. do not add --platform=$BUILDPLATFORM because the pydantic binaries must be resolved for the final architecture
+# Use a Python image with uv pre-installed
+FROM ghcr.io/astral-sh/uv:python3.12-bookworm-slim AS builder
+
+# Install the project into `/app`
+WORKDIR /app
+
+# Enable bytecode compilation
+ENV UV_COMPILE_BYTECODE=1
+
+# Copy from the cache instead of linking since it's a mounted volume
+ENV UV_LINK_MODE=copy
+
 RUN apt-get update \
    && apt-get install --no-install-recommends -y \
-    # deps for installing poetry
-    curl \
    # deps for building python deps
    build-essential \
    # npm
@ -61,41 +31,48 @@ RUN apt-get update \
    gcc \
    && apt-get clean \
    && rm -rf /var/lib/apt/lists/*
+# Install the project's dependencies using the lockfile and settings
+RUN --mount=type=cache,target=/root/.cache/uv \
+    --mount=type=bind,source=src/backend/base/README.md,target=src/backend/base/README.md \
+    --mount=type=bind,source=src/backend/base/uv.lock,target=src/backend/base/uv.lock \
+    --mount=type=bind,source=src/backend/base/pyproject.toml,target=src/backend/base/pyproject.toml \
+    cd src/backend/base && uv sync --frozen --no-install-project --no-dev --no-editable

-RUN --mount=type=cache,target=/root/.cache \
-    curl -sSL https://install.python-poetry.org | python3 -
+ADD ./src /app/src

-# Now we need to copy the entire project into the image
-COPY pyproject.toml poetry.lock ./
-COPY src/frontend/package.json /tmp/package.json
-RUN cd /tmp && npm install
-WORKDIR /app
-COPY src/frontend ./src/frontend
-RUN rm -rf src/frontend/node_modules
-RUN cp -a /tmp/node_modules /app/src/frontend
-COPY scripts ./scripts
-COPY Makefile ./
-COPY README.md ./
-RUN cd src/frontend && npm run build
-COPY src/backend ./src/backend
-RUN cp -r src/frontend/build src/backend/base/langflow/frontend
-RUN rm -rf src/backend/base/dist
-RUN useradd -m -u 1000 user && \
-    mkdir -p /app/langflow && \
-    chown -R user:user /app && \
-    chmod -R u+w /app/langflow
+COPY src/frontend /tmp/src/frontend
+WORKDIR /tmp/src/frontend
+RUN npm install \
+    && npm run build \
+    && cp -r build /app/src/backend/base/langflow/frontend \
+    && rm -rf /tmp/src/frontend

-# Update PATH with home/user/.local/bin
-ENV PATH="/home/user/.local/bin:${PATH}"
-RUN cd src/backend/base && $POETRY_HOME/bin/poetry build
+WORKDIR /app/src/backend/base
+RUN --mount=type=cache,target=/root/.cache/uv \
+    uv sync --frozen --no-dev --no-editable

-# Copy virtual environment and built .tar.gz from builder base
+################################
+# RUNTIME
+# Setup user, utilities and copy the virtual environment only
+################################
+FROM python:3.12.3-slim AS runtime
+
+RUN useradd user -u 1000 -g 0 --no-create-home --home-dir /app/data
+COPY --from=builder --chown=1000 /app/src/backend/base/.venv /app/src/backend/base/.venv
+
+# Place executables in the environment at the front of the path
+ENV PATH="/app/src/backend/base/.venv/bin:$PATH"
+
+LABEL org.opencontainers.image.title=langflow
+LABEL org.opencontainers.image.authors=['Langflow']
+LABEL org.opencontainers.image.licenses=MIT
+LABEL org.opencontainers.image.url=https://github.com/langflow-ai/langflow
+LABEL org.opencontainers.image.source=https://github.com/langflow-ai/langflow

 USER user
-# Install the package from the .tar.gz
-RUN python -m pip install /app/src/backend/base/dist/*.tar.gz --user
+WORKDIR /app

 ENV LANGFLOW_HOST=0.0.0.0
 ENV LANGFLOW_PORT=7860

-CMD ["python", "-m", "langflow", "run"]
+CMD ["langflow-base", "run"]
--- a/docker/dev.Dockerfile
+++ b/docker/dev.Dockerfile
@ -1,20 +1,26 @@
-FROM python:3.12-bookworm
+FROM ghcr.io/astral-sh/uv:python3.12-bookworm-slim
 ENV TZ=UTC

 WORKDIR /app

-RUN apt update -y
-RUN apt install \
+RUN apt-get update && apt-get install -y \
    build-essential \
    curl \
    npm \
-    -y
+    && apt-get clean \
+    && rm -rf /var/lib/apt/lists/*

 COPY . /app

-RUN pip install poetry
-RUN poetry config virtualenvs.create false
-RUN poetry install --no-interaction --no-ansi
+# Install dependencies using uv
+RUN --mount=type=cache,target=/root/.cache/uv \
+    --mount=type=bind,source=uv.lock,target=uv.lock \
+    --mount=type=bind,source=README.md,target=README.md \
+    --mount=type=bind,source=pyproject.toml,target=pyproject.toml \
+    --mount=type=bind,source=src/backend/base/README.md,target=src/backend/base/README.md \
+    --mount=type=bind,source=src/backend/base/uv.lock,target=src/backend/base/uv.lock \
+    --mount=type=bind,source=src/backend/base/pyproject.toml,target=src/backend/base/pyproject.toml \
+    uv sync --frozen --no-install-project --no-dev

 EXPOSE 7860
 EXPOSE 3000