feat: adds new JSON viewer (#5407)
* feat: Add JsonEditor component for JSON editing This commit adds a new component called JsonEditor, which allows users to edit JSON data. The component uses the JSONEditor library and provides options for customizing the editor's appearance and behavior. It also includes functionality for updating the edited data and triggering a callback when the data changes. The component is initialized with an initial data object and can be updated with new data through props. The component is designed to be reusable and can be easily integrated into other parts of the application. * feat: Add JsonEditor component for JSON editing and replace JsonView The commit adds a new component called JsonEditor for editing JSON data. It replaces the previous JsonView component used for displaying JSON data. This change improves the functionality and user experience of the application. * [autofix.ci] apply automated fixes * Added json editor to package * Change jsonEditor to use new vanilla jsonEditor * Added color variables * Removed unused buttons on json editor * Removed unused dark store * Fixed state management on dictAreaModal * Change default DictComponent value to dict * removed unused checks * Changed to forward ref of json editor * removed nav bar * Fixed value not being received * Added check if value is null and handleOnNewValue on this case * Removed unused button on json editor * Adds auto focus and change mode to text * ♻️ (jsonEditor/index.tsx): remove unnecessary comments and improve code readability by removing redundant comments and empty dependency array in useEffect. * Fixed dict component test * Refactor json-input component to use VanillaJsonEditor and JsonEditor component * Remove react-json-view-lite and react18-json-view dependencies * [autofix.ci] apply automated fixes * Refactor jsonEditor component to add readOnly prop * Refactor json-output-view component to add read-only prop * Refactor switchOutputView component to add JSON output view * Refactor CSS styles for JSON editor buttons * Update package-lock.json to add new dependencies for @mapbox/node-pre-gyp and remove jsdom and its related modules * [autofix.ci] apply automated fixes * Refactor textModal component to remove unused imports and dependencies * add filter property to jsonEditor * [autofix.ci] apply automated fixes * Refactor jsonEditor component to handle transform queries and display error messages * [autofix.ci] apply automated fixes * Refactor jsonEditor component to add filter property and handle transform queries * Refactor jsonEditor component to add filter property and handle transform queries * Refactor Output class to add options property * Refactor json-output-view.tsx to update default height value * Refactor jsonEditor component to normalize transform query path * Refactor jsonEditor component to add JSONQuery support for transform queries * Add jsonquerylang library as a dependency * Refactor utils.py to add support for applying JSON filters * Refactor base.py to add apply_options method for JSON filtering * Fix jsonOutputView to set filter option to undefined when empty * Enhance apply_json_filter to support Data objects and improve query handling * Improve json filtering in Output class to return original result if filtered result is None * Refactor Component class to simplify map_outputs method and enhance output handling * move jsonquerylang to langflow-base * Refactor Component and Output classes for improved output handling and type consistency; enhance apply_json_filter to ensure proper data processing and add comprehensive unit tests. * Add filter_data method to Data class and update apply_options in Output class to utilize it; enhance apply_json_filter return type to Data. * Import apply_json_filter in Data class to enable data filtering functionality * Add validation for JSON results in JsonEditor; ensure only objects and arrays are accepted and handle serialization errors * 📝 (App.css): Add styles for jse-menu separator and cm-gutters to improve UI layout ♻️ (index.tsx): Refactor imports and code formatting for better readability and maintainability 🔧 (index.tsx): Update BaseModal component styles to improve UI layout and responsiveness * 📝 (custom.css): add extra line at the end of the file for consistency 📝 (App.css): add styling for .jse-menu .jse-button.jse-group-button class 📝 (classes.css): add extra line at the end of the file for consistency * 🐛 (data.py): fix custom_serializer to handle bytes objects by decoding them to utf-8 ♻️ (base.py): refactor options field in Output class to accept BaseModel, dict, or None for better flexibility and compatibility * Refactor apply_json_filter to directly use jsonquery on result, removing unnecessary string conversion * ✨ (base.py): Introduce OutputOptions model for output filtering and update Output class to use it * 🐛 (classes.css): Hide search box background and adjust container positioning for better layout * ✨ (jsonEditor): Enhance filtering functionality with improved state management and user feedback * Fixed package lock * [autofix.ci] apply automated fixes * Updated package lock * [autofix.ci] apply automated fixes --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> Co-authored-by: Lucas Oliveira <lucas.edu.oli@hotmail.com> Co-authored-by: cristhianzl <cristhian.lousa@gmail.com> Co-authored-by: Gabriel Luiz Freitas Almeida <gabriel@langflow.org>
This commit is contained in:
anovazzi1
GitHub
parent
aeb228ce94
commit
eec1fb23d3
25 changed files with 3171 additions and 1155 deletions
|
|
@ -155,8 +155,7 @@ class Component(CustomComponent):
|
|||
self._reset_all_output_values()
|
||||
if self.inputs is not None:
|
||||
self.map_inputs(self.inputs)
|
||||
if self.outputs is not None:
|
||||
self.map_outputs(self.outputs)
|
||||
self.map_outputs()
|
||||
|
||||
# Final setup
|
||||
self._set_output_types(list(self._outputs_map.values()))
|
||||
|
|
@ -405,7 +404,7 @@ class Component(CustomComponent):
|
|||
msg = f"Output {name} not found in {self.__class__.__name__}"
|
||||
raise ValueError(msg)
|
||||
|
||||
def map_outputs(self, outputs: list[Output]) -> None:
|
||||
def map_outputs(self) -> None:
|
||||
"""Maps the given list of outputs to the component.
|
||||
|
||||
Args:
|
||||
|
|
@ -417,6 +416,19 @@ class Component(CustomComponent):
|
|||
Returns:
|
||||
None
|
||||
"""
|
||||
# override outputs (generated from the class code) with vertex outputs
|
||||
# if they exist (generated from the frontend)
|
||||
outputs = []
|
||||
if self._vertex and self._vertex.outputs:
|
||||
for output in self._vertex.outputs:
|
||||
try:
|
||||
output_ = Output(**output)
|
||||
outputs.append(output_)
|
||||
except ValidationError as e:
|
||||
msg = f"Invalid output: {e}"
|
||||
raise ValueError(msg) from e
|
||||
else:
|
||||
outputs = self.outputs
|
||||
for output in outputs:
|
||||
if output.name is None:
|
||||
msg = "Output name cannot be None."
|
||||
|
|
@ -980,8 +992,9 @@ class Component(CustomComponent):
|
|||
and self._vertex.graph.flow_id is not None
|
||||
):
|
||||
result.set_flow_id(self._vertex.graph.flow_id)
|
||||
|
||||
result = output.apply_options(result)
|
||||
output.value = result
|
||||
|
||||
return result
|
||||
|
||||
def _build_artifact(self, result):
|
||||
|
|
|
|||
|
|
@ -235,6 +235,19 @@ class Data(BaseModel):
|
|||
def __eq__(self, /, other):
|
||||
return isinstance(other, Data) and self.data == other.data
|
||||
|
||||
def filter_data(self, filter_str: str) -> "Data":
|
||||
"""Filters the data dictionary based on the filter string.
|
||||
|
||||
Args:
|
||||
filter_str (str): The filter string to apply to the data dictionary.
|
||||
|
||||
Returns:
|
||||
Data: The filtered Data.
|
||||
"""
|
||||
from langflow.template.utils import apply_json_filter
|
||||
|
||||
return apply_json_filter(self.data, filter_str)
|
||||
|
||||
|
||||
def custom_serializer(obj):
|
||||
if isinstance(obj, datetime):
|
||||
|
|
@ -246,6 +259,8 @@ def custom_serializer(obj):
|
|||
return str(obj)
|
||||
if isinstance(obj, BaseModel):
|
||||
return obj.model_dump()
|
||||
if isinstance(obj, bytes):
|
||||
return obj.decode("utf-8", errors="replace")
|
||||
# Add more custom serialization rules as needed
|
||||
msg = f"Type {type(obj)} not serializable"
|
||||
raise TypeError(msg)
|
||||
|
|
|
|||
|
|
@ -20,6 +20,7 @@ from pydantic import (
|
|||
from langflow.field_typing import Text
|
||||
from langflow.field_typing.range_spec import RangeSpec
|
||||
from langflow.helpers.custom import format_type
|
||||
from langflow.schema.data import Data
|
||||
from langflow.type_extraction.type_extraction import post_process_type
|
||||
|
||||
|
||||
|
|
@ -172,6 +173,11 @@ class Input(BaseModel):
|
|||
return v
|
||||
|
||||
|
||||
class OutputOptions(BaseModel):
|
||||
filter: str | None = None
|
||||
"""Filter to be applied to the output data."""
|
||||
|
||||
|
||||
class Output(BaseModel):
|
||||
types: list[str] = Field(default=[])
|
||||
"""List of output types for the field."""
|
||||
|
|
@ -202,6 +208,9 @@ class Output(BaseModel):
|
|||
allows_loop: bool = Field(default=False)
|
||||
"""Specifies if the output allows looping."""
|
||||
|
||||
options: OutputOptions | None = Field(default=None)
|
||||
"""Options for the output."""
|
||||
|
||||
tool_mode: bool = Field(default=True)
|
||||
"""Specifies if the output should be used as a tool"""
|
||||
|
||||
|
|
@ -222,7 +231,6 @@ class Output(BaseModel):
|
|||
result = handler(self)
|
||||
if self.value == UNDEFINED:
|
||||
result["value"] = UNDEFINED.value
|
||||
|
||||
return result
|
||||
|
||||
@model_validator(mode="after")
|
||||
|
|
@ -234,4 +242,14 @@ class Output(BaseModel):
|
|||
raise ValueError(msg)
|
||||
if self.display_name is None:
|
||||
self.display_name = self.name
|
||||
# Convert dict options to OutputOptions model
|
||||
if isinstance(self.options, dict):
|
||||
self.options = OutputOptions(**self.options)
|
||||
return self
|
||||
|
||||
def apply_options(self, result):
|
||||
if not self.options:
|
||||
return result
|
||||
if self.options.filter and isinstance(result, Data):
|
||||
return result.filter_data(self.options.filter)
|
||||
return result
|
||||
|
|
|
|||
|
|
@ -2,6 +2,8 @@ from pathlib import Path
|
|||
|
||||
from platformdirs import user_cache_dir
|
||||
|
||||
from langflow.schema.data import Data
|
||||
|
||||
|
||||
def raw_frontend_data_is_valid(raw_frontend_data):
|
||||
"""Check if the raw frontend data is valid for processing."""
|
||||
|
|
@ -96,3 +98,65 @@ def update_frontend_node_with_template_values(frontend_node, raw_frontend_node):
|
|||
frontend_node["description"] = raw_frontend_node.get("description", frontend_node.get("description", ""))
|
||||
|
||||
return frontend_node
|
||||
|
||||
|
||||
def apply_json_filter(result, filter_) -> Data:
|
||||
"""Apply a json filter to the result.
|
||||
|
||||
Args:
|
||||
result (Data): The JSON data to filter
|
||||
filter_ (str): The filter query string in jsonquery format
|
||||
|
||||
Returns:
|
||||
Data: The filtered result
|
||||
"""
|
||||
if not filter_ or not filter_.strip():
|
||||
return result
|
||||
# if result is a Data object, get the data
|
||||
if isinstance(result, Data):
|
||||
result = result.data
|
||||
try:
|
||||
from jsonquerylang import jsonquery
|
||||
|
||||
# If query doesn't start with '.', add it to match jsonquery syntax
|
||||
return Data(data=jsonquery(result, filter_))
|
||||
|
||||
except (ImportError, ValueError, TypeError):
|
||||
# Fallback to basic path-based filtering
|
||||
# or if there's an error processing the query
|
||||
# Normalize array access notation and handle direct key access
|
||||
filter_str = filter_.strip()
|
||||
normalized_query = "." + filter_str if not filter_str.startswith(".") else filter_str
|
||||
normalized_query = normalized_query.replace("[", ".[")
|
||||
path = normalized_query.strip().split(".")
|
||||
path = [p for p in path if p]
|
||||
|
||||
current = result
|
||||
for key in path:
|
||||
if current is None:
|
||||
return None
|
||||
|
||||
# Handle array access
|
||||
if key.startswith("[") and key.endswith("]"):
|
||||
try:
|
||||
index = int(key[1:-1])
|
||||
if not isinstance(current, list) or index >= len(current):
|
||||
return None
|
||||
current = current[index]
|
||||
except (ValueError, TypeError):
|
||||
return None
|
||||
# Handle object access
|
||||
elif isinstance(current, dict):
|
||||
if key not in current:
|
||||
return None
|
||||
current = current[key]
|
||||
# Handle array operation
|
||||
elif isinstance(current, list):
|
||||
try:
|
||||
current = [item[key] for item in current if isinstance(item, dict) and key in item]
|
||||
except (TypeError, KeyError):
|
||||
return None
|
||||
else:
|
||||
return None
|
||||
|
||||
return Data(data=current)
|
||||
|
|
|
|||
|
|
@ -81,6 +81,7 @@ dependencies = [
|
|||
"mcp>=1.1.2",
|
||||
"aiosqlite>=0.20.0",
|
||||
"greenlet>=3.1.1",
|
||||
"jsonquerylang>=1.1.1",
|
||||
"sqlalchemy[aiosqlite]>=2.0.38,<3.0.0",
|
||||
]
|
||||
|
||||
|
|
|
|||
1
src/backend/tests/unit/template/utils/__init__.py
Normal file
1
src/backend/tests/unit/template/utils/__init__.py
Normal file
|
|
@ -0,0 +1 @@
|
|||
"""Utils tests package."""
|
||||
|
|
@ -0,0 +1,97 @@
|
|||
import pytest
|
||||
from hypothesis import given
|
||||
from hypothesis import strategies as st
|
||||
from langflow.schema.data import Data
|
||||
from langflow.template.utils import apply_json_filter
|
||||
|
||||
|
||||
# Helper function to create nested dictionaries
|
||||
def dict_strategy():
|
||||
return st.recursive(
|
||||
st.one_of(st.integers(), st.text(), st.floats(allow_nan=False, allow_infinity=False), st.booleans()),
|
||||
lambda children: st.lists(children, max_size=5) | st.dictionaries(st.text(), children, max_size=5),
|
||||
max_leaves=10,
|
||||
)
|
||||
|
||||
|
||||
# Test basic dictionary access
|
||||
@given(data=st.dictionaries(st.text(), st.integers()), key=st.text())
|
||||
def test_basic_dict_access(data, key):
|
||||
if key in data:
|
||||
result = apply_json_filter(data, key)
|
||||
assert result == data[key]
|
||||
else:
|
||||
result = apply_json_filter(data, key)
|
||||
assert result is None
|
||||
|
||||
|
||||
# Test array access
|
||||
@given(data=st.lists(st.integers(), min_size=1), index=st.integers())
|
||||
def test_array_access(data, index):
|
||||
filter_str = f"[{index}]"
|
||||
result = apply_json_filter(data, filter_str)
|
||||
if 0 <= index < len(data):
|
||||
assert result == data[index]
|
||||
else:
|
||||
assert result is None
|
||||
|
||||
|
||||
# Test nested object access
|
||||
@given(nested_data=dict_strategy())
|
||||
def test_nested_object_access(nested_data):
|
||||
# Wrap in Data object to test both raw and Data object inputs
|
||||
data_obj = Data(data=nested_data)
|
||||
result = apply_json_filter(data_obj, "")
|
||||
assert result == nested_data
|
||||
|
||||
|
||||
# Test edge cases
|
||||
@pytest.mark.parametrize(
|
||||
("input_data", "filter_str", "expected"),
|
||||
[
|
||||
({}, "", {}), # Empty dict, empty filter
|
||||
([], "", []), # Empty list, empty filter
|
||||
(None, "any.path", None), # None input
|
||||
({"a": 1}, None, {"a": 1}), # None filter
|
||||
({"a": 1}, " ", {"a": 1}), # Whitespace filter
|
||||
],
|
||||
)
|
||||
def test_edge_cases(input_data, filter_str, expected):
|
||||
result = apply_json_filter(input_data, filter_str)
|
||||
assert result == expected
|
||||
|
||||
|
||||
# Test complex nested access
|
||||
@given(data=st.dictionaries(keys=st.text(), values=st.dictionaries(keys=st.text(), values=st.lists(st.integers()))))
|
||||
def test_complex_nested_access(data):
|
||||
if data:
|
||||
outer_key = next(iter(data))
|
||||
if data[outer_key]:
|
||||
inner_key = next(iter(data[outer_key]))
|
||||
filter_str = f"{outer_key}.{inner_key}"
|
||||
result = apply_json_filter(data, filter_str)
|
||||
assert result == data[outer_key][inner_key]
|
||||
|
||||
|
||||
# Test array operations on objects
|
||||
@given(data=st.lists(st.dictionaries(keys=st.text(), values=st.integers(), min_size=1), min_size=1))
|
||||
def test_array_object_operations(data):
|
||||
if data and all(data):
|
||||
key = next(iter(data[0]))
|
||||
result = apply_json_filter(data, key)
|
||||
expected = [item[key] for item in data if key in item]
|
||||
assert result == expected
|
||||
|
||||
|
||||
# Test invalid inputs
|
||||
@pytest.mark.parametrize(
|
||||
("input_data", "filter_str"),
|
||||
[
|
||||
({"a": 1}, "[invalid]"), # Invalid array index
|
||||
([1, 2, 3], "nonexistent"), # Nonexistent key on array
|
||||
({"a": 1}, "..[invalid]"), # Invalid syntax
|
||||
],
|
||||
)
|
||||
def test_invalid_inputs(input_data, filter_str):
|
||||
result = apply_json_filter(input_data, filter_str)
|
||||
assert result is None or isinstance(result, dict | list | Data)
|
||||
Loading…
Add table
Add a link
Reference in a new issue