feat: Add template tests (#9083)

* add template tests

* remove files

* adding validate flow build

* add validate endpoint and flow execution

* Update .github/workflows/template-tests.yml

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* Update src/backend/base/langflow/utils/template_validation.py

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>

* [autofix.ci] apply automated fixes

* change workflow running

* add ci

* fix test

* fix test

* delete when push

* fix: Exclude template tests from unit test bundle

Template tests are already run separately in CI via the test-templates job.
This change prevents duplicate execution and eliminates timeout failures
in the unit test suite by excluding slow template execution tests.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: Exclude template tests from unit test bundle

Template tests are already run separately in CI via the test-templates job.
This change prevents duplicate execution and eliminates timeout failures
in the unit test suite by excluding slow template execution tests.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: Exclude template tests from unit test bundle

Template tests are already run separately in CI via the test-templates job.
This change prevents duplicate execution and eliminates timeout failures
in the unit test suite by excluding slow template execution tests.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: Remove remaining merge conflict markers

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: Improve validate.py unit tests to eliminate CI failures

Fixed 4 failing tests in test_validate.py:
- test_code_with_syntax_error: Better error message handling for syntax errors
- test_raises_error_for_missing_function: Handle StopIteration along with ValueError
- test_creates_simple_class: Use optional constructor parameter to avoid TypeError
- test_handles_validation_error: Use proper ValidationError constructor from pydantic_core
- test_creates_context_with_langflow_imports: Remove invalid module patching
- test_creates_mock_classes_on_import_failure: Use proper import mocking

All 50 validate tests now pass consistently, improving CI stability.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* enhance: Add comprehensive edge case tests for template_validation.py

Added 6 additional test cases to improve coverage of template_validation.py:
- test_validate_stream_exception: Tests Graph.validate_stream() exception handling
- test_code_validation_other_exceptions: Tests TypeError/KeyError/AttributeError handling
- test_vertices_sorted_without_end_vertex_events: Tests variable usage tracking
- test_vertex_count_tracking: Tests vertex_count increment paths
- test_empty_lines_in_stream: Tests empty line handling in event streams
- test_event_stream_validation_exception: Tests exception handling in _validate_event_stream

These tests target the remaining 7 uncovered lines to maximize coverage percentage.
Total tests: 40 (all passing)

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>

.github/workflows/ci.yml

@@ -207,6 +207,35 @@ jobs:
     name: Test Docs Build
     uses: ./.github/workflows/docs_test.yml
 
+  test-templates:
+    needs: [path-filter, set-ci-condition]
+    name: Test Starter Templates
+    if: ${{ needs.path-filter.outputs.starter-projects == 'true' && needs.set-ci-condition.outputs.should-run-tests == 'true' }}
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ inputs.branch || github.ref }}
+
+      - name: Set up Python 3.12
+        uses: actions/setup-python@v5
+        with:
+          python-version: 3.12
+
+      - name: Install uv
+        uses: astral-sh/setup-uv@v3
+        with:
+          version: "latest"
+
+      - name: Install dependencies
+        run: |
+          uv sync --dev
+
+      - name: Test all starter project templates
+        run: |
+          uv run pytest src/backend/tests/unit/template/test_starter_projects.py -v
+
   # https://github.com/langchain-ai/langchain/blob/master/.github/workflows/check_diffs.yml
   ci_success:
     name: "CI Success"
@@ -217,6 +246,7 @@ jobs:
         test-frontend,
         lint-backend,
         test-docs-build,
+        test-templates,
         set-ci-condition,
         path-filter
       ]

.github/workflows/template-tests.yml

@@ -0,0 +1,40 @@
+name: Template Tests
+
+on:
+  pull_request:
+    paths:
+      - 'src/backend/base/langflow/initial_setup/starter_projects/**'
+      - 'src/backend/tests/unit/template/test_starter_projects.py'
+      - 'src/backend/base/langflow/utils/template_validation.py'
+      - '.github/workflows/template-tests.yml'
+  workflow_dispatch:
+
+permissions:
+  contents: read
+  pull-requests: read
+
+jobs:
+  test-starter-projects:
+    runs-on: ubuntu-latest
+
+    steps:
+    - name: Checkout code
+      uses: actions/checkout@v4
+
+    - name: Set up Python 3.12
+      uses: actions/setup-python@v5
+      with:
+        python-version: 3.12
+
+    - name: Install uv
+      uses: astral-sh/setup-uv@v3
+      with:
+        version: "latest"
+
+    - name: Install dependencies
+      run: |
+        uv sync --dev
+
+    - name: Test all starter project templates
+      run: |
+        uv run pytest src/backend/tests/unit/template/test_starter_projects.py -v