bug: continue to debug cross-platform test issues (#9260)

This commit is contained in:
Eric Pinzur 2025-07-31 14:45:08 +02:00 • committed by GitHub
commit 860c41bacf
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
4 changed files with 55 additions and 64 deletions

View file

@ -25,7 +25,7 @@ gh workflow run cross-platform-test.yml \
``` ```
### 2. Test from Source ### 2. Test from Source
Builds and tests langflow from current branch source code. Builds and tests langflow from current branch source code using release-like dependency resolution (transforms workspace dependencies to published packages for testing parity).
**Via GitHub UI:** **Via GitHub UI:**
1. Go to **Actions** → **Cross-Platform Installation Test** 1. Go to **Actions** → **Cross-Platform Installation Test**
@ -34,7 +34,7 @@ Builds and tests langflow from current branch source code.
**Via CLI:** **Via CLI:**
```bash ```bash
# Test current branch # Test current branch with release-like dependencies
gh workflow run cross-platform-test.yml -f test-from-pypi=false gh workflow run cross-platform-test.yml -f test-from-pypi=false
``` ```
@ -58,8 +58,9 @@ jobs:
- **macOS**: Intel (AMD64), Apple Silicon (ARM64) - **macOS**: Intel (AMD64), Apple Silicon (ARM64)
- **Windows**: AMD64 - **Windows**: AMD64
- **Python versions**: - **Python versions**:
- **Linux & macOS**: 3.10 and 3.13 - **All platforms**: 3.10, 3.12, and 3.13
- **Windows**: 3.10 and 3.12 (3.12 used instead of 3.13 for better stability) - **Stability**: 3.10 and 3.12 are required to pass (blocking)
- **Preview**: 3.13 testing is optional (non-blocking) to monitor ecosystem readiness
## What Gets Tested ## What Gets Tested
@ -93,7 +94,7 @@ gh workflow run cross-platform-test.yml \
### Installation Methods ### Installation Methods
- **PyPI testing**: Uses `uv pip install` with official PyPI packages - **PyPI testing**: Uses `uv pip install` with official PyPI packages
- **Source testing**: Builds wheels from source, then installs locally - **Source testing**: Transforms workspace dependencies to published packages (like nightly builds), then builds wheels from source and installs locally
- **Dependencies**: Automatically installs additional packages (`openai`) for full functionality - **Dependencies**: Automatically installs additional packages (`openai`) for full functionality
### Health Checking ### Health Checking
@ -102,8 +103,8 @@ gh workflow run cross-platform-test.yml \
- **Timeout**: Configurable timeout with proper cross-platform handling - **Timeout**: Configurable timeout with proper cross-platform handling
### Platform-Specific Optimizations ### Platform-Specific Optimizations
- **Windows**: Uses Python 3.12 for better package ecosystem stability - **Stable versions**: Python 3.10 and 3.12 provide reliable package ecosystem support
- **Unix**: Uses Python 3.13 for latest language features where stable - **Preview testing**: Python 3.13 runs as non-blocking to monitor when it becomes viable
- **Virtual Environments**: Uses `uv venv --seed` for consistent pip availability - **Virtual Environments**: Uses `uv venv --seed` for consistent pip availability
### Workflow Architecture ### Workflow Architecture
@ -129,6 +130,7 @@ cross-platform-test.yml
**Manual (`workflow_dispatch`):** **Manual (`workflow_dispatch`):**
- Simple boolean toggle: "Test from PyPI" vs "Test from Source" - Simple boolean toggle: "Test from PyPI" vs "Test from Source"
- Source testing always uses release-like dependency resolution for testing parity
- User-friendly parameter names - User-friendly parameter names
- Context-specific success/failure messages - Context-specific success/failure messages
@ -162,9 +164,9 @@ test-installation:
## Known Issues ## Known Issues
### macOS AMD64 Python 3.13 Compilation Failures ### macOS Compilation Issues (Historical)
**Issue**: Nightly/release builds may fail on macOS AMD64 with Python 3.13 due to `chroma-hnswlib` compilation errors: **Issue**: Previously, nightly/release builds could fail on macOS with Python 3.13 due to `chroma-hnswlib` compilation errors:
``` ```
clang: error: unsupported argument 'native' to option '-march=' clang: error: unsupported argument 'native' to option '-march='
error: command '/usr/bin/clang++' failed with exit code 1 error: command '/usr/bin/clang++' failed with exit code 1
@ -184,10 +186,11 @@ error: command '/usr/bin/clang++' failed with exit code 1
- Uses published PyPI package with full dependency tree including `chromadb==0.5.23` - Uses published PyPI package with full dependency tree including `chromadb==0.5.23`
3. **macOS clang** doesn't support `-march=native` flag used by `chroma-hnswlib` compilation 3. **macOS clang** doesn't support `-march=native` flag used by `chroma-hnswlib` compilation
**Workarounds**: **Current Status**:
- Manual testing (source builds) works consistently - **Stable testing**: Python 3.10 and 3.12 are required to pass (blocking jobs)
- Issue only affects nightly/release automated builds - **Preview testing**: Python 3.13 runs as non-blocking to monitor ecosystem readiness
- Some runs may succeed if compatible `chroma-hnswlib` wheels are available on PyPI - **Compilation issues**: Python 3.13 may still fail due to `chroma-hnswlib` but won't block releases
- **Manual testing**: Source builds now use the same dependency transformation as nightly builds for testing parity
**Files Involved**: **Files Involved**:
- `scripts/ci/update_uv_dependency.py` - Modifies dependency resolution - `scripts/ci/update_uv_dependency.py` - Modifies dependency resolution

View file

@ -12,11 +12,6 @@ on:
required: false required: false
type: string type: string
default: "" default: ""
test-timeout:
description: "Timeout for langflow server startup test (minutes)"
required: false
type: number
default: 5
workflow_call: workflow_call:
inputs: inputs:
base-artifact-name: base-artifact-name:
@ -27,11 +22,6 @@ on:
description: "Name of the main package artifact" description: "Name of the main package artifact"
required: true required: true
type: string type: string
test-timeout:
description: "Timeout for langflow server startup test (minutes)"
required: false
type: number
default: 5
jobs: jobs:
build-if-needed: build-if-needed:
@ -46,6 +36,19 @@ jobs:
steps: steps:
- name: Checkout code - name: Checkout code
uses: actions/checkout@v4 uses: actions/checkout@v4
- name: Transform dependencies to simulate release build
if: github.event_name == 'workflow_dispatch'
run: |
echo "🔧 Transforming workspace dependencies to published packages..."
# Use latest published versions for transformation
BASE_VERSION=$(python scripts/ci/pypi_nightly_tag.py base | sed 's/^v//')
MAIN_VERSION=$(python scripts/ci/pypi_nightly_tag.py main | sed 's/^v//')
echo "Using base version: $BASE_VERSION"
echo "Using main version: $MAIN_VERSION"
# Transform dependencies like nightly builds do
python scripts/ci/update_pyproject_combined.py main "v$MAIN_VERSION" "v$BASE_VERSION"
echo "✅ Dependencies transformed to use published packages"
- name: Setup Environment - name: Setup Environment
uses: astral-sh/setup-uv@v6 uses: astral-sh/setup-uv@v6
with: with:
@ -92,28 +95,43 @@ jobs:
arch: amd64 arch: amd64
runner: ubuntu-latest runner: ubuntu-latest
python-version: "3.10" python-version: "3.10"
- os: linux
arch: amd64
runner: ubuntu-latest
python-version: "3.12"
- os: linux - os: linux
arch: amd64 arch: amd64
runner: ubuntu-latest runner: ubuntu-latest
python-version: "3.13" python-version: "3.13"
continue-on-error: true
# macOS AMD64 # macOS AMD64
- os: macos - os: macos
arch: amd64 arch: amd64
runner: macos-13 runner: macos-13
python-version: "3.10" python-version: "3.10"
- os: macos
arch: amd64
runner: macos-13
python-version: "3.12"
- os: macos - os: macos
arch: amd64 arch: amd64
runner: macos-13 runner: macos-13
python-version: "3.13" python-version: "3.13"
continue-on-error: true
# macOS ARM64 (Apple Silicon) # macOS ARM64 (Apple Silicon)
- os: macos - os: macos
arch: arm64 arch: arm64
runner: macos-latest runner: macos-latest
python-version: "3.10" python-version: "3.10"
- os: macos
arch: arm64
runner: macos-latest
python-version: "3.12"
- os: macos - os: macos
arch: arm64 arch: arm64
runner: macos-latest runner: macos-latest
python-version: "3.13" python-version: "3.13"
continue-on-error: true
# Windows AMD64 # Windows AMD64
- os: windows - os: windows
arch: amd64 arch: amd64
@ -123,6 +141,11 @@ jobs:
arch: amd64 arch: amd64
runner: windows-latest runner: windows-latest
python-version: "3.12" python-version: "3.12"
- os: windows
arch: amd64
runner: windows-latest
python-version: "3.13"
continue-on-error: true
steps: steps:
- name: Determine install method - name: Determine install method
@ -217,12 +240,6 @@ jobs:
- name: Install main package from wheel (Unix) - name: Install main package from wheel (Unix)
if: steps.install-method.outputs.method == 'wheel' && matrix.os != 'windows' if: steps.install-method.outputs.method == 'wheel' && matrix.os != 'windows'
env:
# Fix chroma-hnswlib compilation on macOS by overriding problematic compiler flags
CFLAGS: ${{ matrix.os == 'macos' && matrix.python-version == '3.13' && '-O3 -mmacosx-version-min=10.7 -march=x86-64' || '' }}
CXXFLAGS: ${{ matrix.os == 'macos' && matrix.python-version == '3.13' && '-O3 -mmacosx-version-min=10.7 -march=x86-64 -stdlib=libc++' || '' }}
# Alternative: Force use of compatible architecture flags
ARCHFLAGS: ${{ matrix.os == 'macos' && matrix.python-version == '3.13' && '-arch x86_64' || '' }}
run: | run: |
ls -la ./main-dist/ ls -la ./main-dist/
find ./main-dist -name "*.whl" -type f find ./main-dist -name "*.whl" -type f
@ -248,12 +265,6 @@ jobs:
- name: Install langflow from PyPI (Unix) - name: Install langflow from PyPI (Unix)
if: steps.install-method.outputs.method == 'pypi' && matrix.os != 'windows' if: steps.install-method.outputs.method == 'pypi' && matrix.os != 'windows'
env:
# Fix chroma-hnswlib compilation on macOS by overriding problematic compiler flags
CFLAGS: ${{ matrix.os == 'macos' && matrix.python-version == '3.13' && '-O3 -mmacosx-version-min=10.7 -march=x86-64' || '' }}
CXXFLAGS: ${{ matrix.os == 'macos' && matrix.python-version == '3.13' && '-O3 -mmacosx-version-min=10.7 -march=x86-64 -stdlib=libc++' || '' }}
# Alternative: Force use of compatible architecture flags
ARCHFLAGS: ${{ matrix.os == 'macos' && matrix.python-version == '3.13' && '-arch x86_64' || '' }}
run: | run: |
if [ -n "${{ inputs.langflow-version }}" ]; then if [ -n "${{ inputs.langflow-version }}" ]; then
uv pip install --python ./test-env/bin/python langflow==${{ inputs.langflow-version }} uv pip install --python ./test-env/bin/python langflow==${{ inputs.langflow-version }}
@ -290,16 +301,12 @@ jobs:
- name: Test server startup (Windows) - name: Test server startup (Windows)
if: matrix.os == 'windows' if: matrix.os == 'windows'
timeout-minutes: ${{ inputs.test-timeout || 5 }} timeout-minutes: 5
env:
TIMEOUT_MINUTES: ${{ inputs.test-timeout || 5 }}
run: | run: |
# Start server in background # Start server in background
$serverProcess = Start-Process -FilePath ".\test-env\Scripts\python.exe" -ArgumentList "-m", "langflow", "run", "--host", "localhost", "--port", "7860", "--backend-only" -PassThru -WindowStyle Hidden $serverProcess = Start-Process -FilePath ".\test-env\Scripts\python.exe" -ArgumentList "-m", "langflow", "run", "--host", "localhost", "--port", "7860", "--backend-only" -PassThru -WindowStyle Hidden
# Wait for server to be ready # Wait for server to be ready (GitHub Actions will timeout after 5 minutes)
$timeoutSeconds = $env:TIMEOUT_MINUTES * 60
$elapsed = 0
do { do {
try { try {
$response = Invoke-WebRequest -Uri "http://localhost:7860/health_check" -UseBasicParsing -TimeoutSec 5 $response = Invoke-WebRequest -Uri "http://localhost:7860/health_check" -UseBasicParsing -TimeoutSec 5
@ -309,14 +316,8 @@ jobs:
} }
} catch { } catch {
Start-Sleep -Seconds 5 Start-Sleep -Seconds 5
$elapsed += 5
} }
} while ($elapsed -lt $timeoutSeconds) } while ($true)
if ($elapsed -ge $timeoutSeconds) {
Write-Host "❌ Server failed to start within timeout on ${{ matrix.os }}-${{ matrix.arch }}"
exit 1
}
# Stop the server process # Stop the server process
Stop-Process -Id $serverProcess.Id -Force -ErrorAction SilentlyContinue Stop-Process -Id $serverProcess.Id -Force -ErrorAction SilentlyContinue
@ -324,32 +325,21 @@ jobs:
- name: Test server startup (Unix) - name: Test server startup (Unix)
if: matrix.os != 'windows' if: matrix.os != 'windows'
timeout-minutes: ${{ inputs.test-timeout || 5 }} timeout-minutes: 5
env:
TIMEOUT_MINUTES: ${{ inputs.test-timeout || 5 }}
run: | run: |
# Start server in background # Start server in background
./test-env/bin/python -m langflow run --host localhost --port 7860 --backend-only & ./test-env/bin/python -m langflow run --host localhost --port 7860 --backend-only &
SERVER_PID=$! SERVER_PID=$!
# Wait for server to be ready (using bash loop instead of timeout command) # Wait for server to be ready (GitHub Actions will timeout after 5 minutes)
TIMEOUT_SECONDS=$((TIMEOUT_MINUTES * 60)) while true; do
ELAPSED=0
while [ $ELAPSED -lt $TIMEOUT_SECONDS ]; do
if curl -f http://localhost:7860/health_check >/dev/null 2>&1; then if curl -f http://localhost:7860/health_check >/dev/null 2>&1; then
echo "✅ Server is ready on ${{ matrix.os }}-${{ matrix.arch }}" echo "✅ Server is ready on ${{ matrix.os }}-${{ matrix.arch }}"
break break
fi fi
sleep 5 sleep 5
ELAPSED=$((ELAPSED + 5))
done done
if [ $ELAPSED -ge $TIMEOUT_SECONDS ]; then
echo "❌ Server failed to start within timeout on ${{ matrix.os }}-${{ matrix.arch }}"
kill $SERVER_PID 2>/dev/null || true
exit 1
fi
# Clean shutdown # Clean shutdown
kill $SERVER_PID 2>/dev/null || true kill $SERVER_PID 2>/dev/null || true
sleep 5 sleep 5

View file

@ -208,7 +208,6 @@ jobs:
with: with:
base-artifact-name: "dist-base" base-artifact-name: "dist-base"
main-artifact-name: "dist-main" main-artifact-name: "dist-main"
test-timeout: 120
publish-base: publish-base:
name: Publish Langflow Base to PyPI name: Publish Langflow Base to PyPI

View file

@ -215,7 +215,6 @@ jobs:
with: with:
base-artifact-name: "dist-nightly-base" base-artifact-name: "dist-nightly-base"
main-artifact-name: "dist-nightly-main" main-artifact-name: "dist-nightly-main"
test-timeout: 120
publish-nightly-base: publish-nightly-base:
name: Publish Langflow Base Nightly to PyPI name: Publish Langflow Base Nightly to PyPI