fix: correct loop component dependencies (#8091)

* feat: Minimal experiment with zipping pre- and post-loop lists

Update test JSON to demonstrate a simple workflow using custom components for sequence generation and zipping, with a loop component to process the data. The changes include:

- Replaced previous components with custom components
- Added a sequence maker component
- Added a zipper component
- Configured loop component to work with the new components
- Updated flow description and last tested version

* feat: Refactor Loop Test workflow with enhanced component interactions

Update LoopTest.json to demonstrate a more complex data processing workflow:
- Modify MyZipper component to return Message instead of Data
- Update Loop component's stop condition logic
- Adjust node positions and connections
- Upgrade last tested version to 1.2.0

* test: Enhance Loop Component Test with JSON Parsing and Assertion

Add more robust testing for the Loop component by:
- Parsing TextOutput event from the response
- Extracting and parsing JSON data
- Adding detailed assertions to verify loop output
- Improving test coverage for loop component interactions

* refactor: simplify LoopTest.json structure and update node definitions

- Reduced the size of LoopTest.json by removing unnecessary edges and nodes.
- Updated node definitions for `ParseData` and `MessagetoData` components to enhance clarity and maintainability.
- Adjusted connections between nodes to reflect the new structure, ensuring proper data flow.
- Improved documentation within the JSON structure for better understanding of component functionalities.

* feat: add method to retrieve incoming edge by target parameter

- Implemented `get_incoming_edge_by_target_param` method in both `Component` and `Vertex` classes to facilitate the retrieval of source vertex IDs for incoming edges targeting specific parameters.
- Enhanced performance by caching outgoing and incoming edges in the `Vertex` class to avoid redundant calculations.

* feat: add dependency update method in LoopComponent

- Introduced `update_dependency` method to manage dependencies for the next iteration in the loop.
- Refactored existing logic to ensure proper handling of current items and loop termination conditions.
- Enhanced code clarity and maintainability by restructuring the flow of data processing within the loop.

* [autofix.ci] apply automated fixes

* refactor: update message assertions in TestLoopComponentWithAPI for accuracy

* feat: enhance LoopTest.json structure and component definitions

- Expanded the LoopTest.json file to include additional nodes and edges, improving the representation of component interactions.
- Updated definitions for `MyZipper`, `LoopComponent`, `MessagetoData`, and `ChatOutput` to enhance clarity and functionality.
- Introduced new properties and methods in components to support better data handling and processing.
- Improved documentation within the JSON structure for better understanding of component functionalities and usage.

* feat: add ran_at_least_once tracking to RunnableVerticesManager

- Introduced a new set, `ran_at_least_once`, to track vertices that have been executed at least once.
- Updated serialization methods to include the new property for state management.
- Enhanced logic in `all_predecessors_are_fulfilled` to prevent infinite loops for loop vertices that have already run.

* fix: add error handling for missing vertex in Component class

* refactor: improve variable naming and enhance readability in TestLoopComponentWithAPI

* feat: track vertex execution in run_manager by adding ran_at_least_once tracking

* feat: Enhance LoopComponent with dependency management and improved item output handling

- Added a method to update dependencies for the LoopComponent to ensure proper execution order.
- Improved item output logic to handle stopping conditions more effectively and update dependencies for subsequent runs.
- Refactored the item_output method to streamline the flow of data processing and context management.

* [autofix.ci] apply automated fixes

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Edwin Jose <edwin.jose@datastax.com>
Co-authored-by: Eric Hare <ericrhare@gmail.com>

src/backend/tests/unit/components/logic/test_loop.py

@@ -1,3 +1,4 @@
+import json
 from uuid import UUID
 
 import orjson
@@ -52,15 +53,12 @@ class TestLoopComponentWithAPI(ComponentTestBaseWithClient):
 
     async def check_messages(self, flow_id):
         messages = await aget_messages(flow_id=UUID(flow_id), order="ASC")
-        assert len(messages) == 2
+        assert len(messages) == 1
         assert messages[0].session_id == flow_id
-        assert messages[0].sender == "User"
-        assert messages[0].sender_name == "User"
-        assert messages[0].text != ""
-        assert messages[1].session_id == flow_id
-        assert messages[1].sender == "Machine"
-        assert messages[1].sender_name == "AI"
-        assert len(messages[1].text) > 0
+        assert messages[0].sender == "Machine"
+        assert messages[0].sender_name == "AI"
+        assert len(messages[0].text) > 0
+        return messages
 
     async def test_build_flow_loop(self, client, json_loop_test, logged_in_headers):
         """Test building a flow with a loop component."""
@@ -77,13 +75,31 @@ class TestLoopComponentWithAPI(ComponentTestBaseWithClient):
         assert events_response.status_code == 200
 
         # Process the events stream
+        chat_output = None
+        lines = []
         async for line in events_response.aiter_lines():
             if not line:  # Skip empty lines
                 continue
+            lines.append(line)
+            if "ChatOutput" in line:
+                chat_output = json.loads(line)
             # Process events if needed
             # We could add specific assertions here for loop-related events
+        assert chat_output is not None
+        messages = await self.check_messages(flow_id)
+        ai_message = messages[0].text
+        json_data = orjson.loads(ai_message)
 
-        await self.check_messages(flow_id)
+        # Use a for loop for better debugging
+        found = []
+        json_data = [(data["text"], q_dict) for data, q_dict in json_data]
+        for text, q_dict in json_data:
+            expected_text = f"==> {q_dict['q']}"
+            assert expected_text in text, (
+                f"Found {found} until now, but expected '{expected_text}' not found in '{text}',"
+                f"and the json_data is {json_data}"
+            )
+            found.append(expected_text)
 
     async def test_run_flow_loop(self, client: AsyncClient, created_api_key, json_loop_test, logged_in_headers):
         flow_id = await self._create_flow(client, json_loop_test, logged_in_headers)