jyapayne/langflow
1
0
Fork 0
Code Issues Pull requests Projects Releases 1 Packages Wiki Activity Actions

feat: Add JSON field extraction and enhanced URL validation (#6051)

Browse source 
* URL component improvement - JSON URL

* [autofix.ci] apply automated fixes

* [autofix.ci] apply automated fixes (attempt 2/3)

* ♻️ (url.py): refactor URLComponent class to simplify data_dict creation by using dictionary unpacking instead of manual key-value pairs

* [autofix.ci] apply automated fixes

* [autofix.ci] apply automated fixes (attempt 2/3)

* 📝 (url.py): improve formatting of info string for DropdownInput in URLComponent class
♻️ (url.py): refactor ensure_url method to simplify logic and improve readability
🐛 (url.py): fix error handling in URLComponent class for invalid JSON content

* [autofix.ci] apply automated fixes

* ✨ (url.py): Add BoolInput and StrInput to support new features in URLComponent
📝 (url.py): Update description in URLComponent to provide more detailed information about its functionality
♻️ (url.py): Refactor update_build_config method in URLComponent to dynamically update fields based on selected format
🐛 (url.py): Fix ensure_url method in URLComponent to ensure valid URLs are provided and handle exceptions properly
🐛 (url.py): Fix fetch_content method in URLComponent to handle cases where no valid URLs are provided and improve error handling
🐛 (url.py): Fix fetch_content_text method in URLComponent to correctly format and clean text output based on selected format and settings
🐛 (url.py): Fix as_dataframe method in URLComponent to return fetched content as a DataFrame object

* [autofix.ci] apply automated fixes

* [autofix.ci] apply automated fixes (attempt 2/3)

* ♻️ (url.py): remove unnecessary comments and improve code readability by removing redundant comments and adjusting code structure.

* [autofix.ci] apply automated fixes

* 📝 (url.py): improve readability by splitting long description and info strings into multiple lines
🐛 (url.py): handle cases where invalid URLs or JSON URLs are provided, and provide informative error messages
🐛 (url.py): handle cases where no valid URLs are provided and raise an error with a clear message

* 🔧 (Blog Writer.json, Custom Component Maker.json, Graph Vector Store RAG.json): resolve merge conflicts in JSON files related to the 'format' field options to ensure consistency across starter projects.

* [autofix.ci] apply automated fixes

* 🐛 (url.py): fix validation of JSON content from URLs to ensure correct handling of JSON data
✨ (url.py): introduce async validation of JSON content from URLs using aiohttp to improve performance and reliability

* [autofix.ci] apply automated fixes

* [autofix.ci] apply automated fixes (attempt 2/3)

* merge fix

* ✅ (test_audio_file.wav): update test_audio_file.wav to fix binary file differences in the test asset

* 🐛 (test_url_component.py): update error message format to improve clarity and consistency

* update templates

* 🐛 (test_database.py): fix error handling in test_read_flows_components_only_paginated to properly catch and log exceptions during test execution

* 📝 (backend): Add noqa comments to files to ignore specific linting rule A005
🔧 (test_database.py): Remove duplicate import statement for sqlalchemy
♻️ (test_database.py): Refactor test_read_flows_components_only_paginated function for better readability and maintainability

* 🐛 (test_database.py): fix test_read_flows_components_only_paginated to handle exceptions and provide more context in case of failure

* 📝 (test_database.py): remove unnecessary comment to improve code readability and maintainability

* 📝 (backend): Remove unnecessary noqa comments from __init__.py files
🔧 (test_database.py): Refactor test_read_flows_components_only_paginated function for better readability and maintainability

* [autofix.ci] apply automated fixes

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
This commit is contained in:
Cristhian Zanforlin Lousa 2025-03-10 09:28:51 -03:00 • committed by GitHub
commit 47753d37d3
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
10 changed files with 638 additions and 252 deletions
Download patch file Download diff file Expand all files Collapse all files

2
src/backend/tests/unit/components/data/test_url_component.py
View file

@ -111,7 +111,7 @@ class TestURLComponent(ComponentTestBaseWithoutClient):
component.set_attributes({"urls": ["not_a_valid_url"]})
# Test that invalid URLs raise a ValueError
with pytest.raises(ValueError, match="Invalid URL: http://not_a_valid_url"):
with pytest.raises(ValueError, match="Invalid URL - http://not_a_valid_url"):
component.fetch_content()
def test_url_component_multiple_urls(self, mock_web_load):

3
src/backend/tests/unit/test_database.py
View file

@ -181,12 +181,15 @@ async def test_read_flows_components_only_paginated(client: AsyncClient, logged_
FlowCreate(name=f"Flow {i}", description="description", data={}, is_component=True)
for i in range(number_of_flows)
]
for flow in flows:
response = await client.post("api/v1/flows/", json=flow.model_dump(), headers=logged_in_headers)
assert response.status_code == 201
response = await client.get(
"api/v1/flows/", headers=logged_in_headers, params={"components_only": True, "get_all": False}
)
assert response.status_code == 200
response_json = response.json()
assert response_json["total"] == 10