jyapayne/langflow
1
0
Fork 0
Code Issues Pull requests Projects Releases 1 Packages Wiki Activity Actions

feat(component): API Request Component Overhaul (#5007)

Browse source 
* style: Add icon property to WebhookComponent class

* style: Update input components in APIRequest class

* fix pr

* [autofix.ci] apply automated fixes

* remove debug print statement from api_request component

* remove debug print statement from api_request component

* feat: add dynamic cURL mode to APIRequestComponent

- Add cURL command parsing and field population
- Implement dynamic UI updates based on cURL input
- Support JSON body and headers extraction from cURL
- Add real-time refresh for cURL mode toggle

* refactor(APIRequestComponent): improve HTTP methods and cURL handling

- Add DELETE method support
- Enhance cURL parameter handling and UI visibility
- Fix JSON decode error handling with specific exception
- Update method visibility for DELETE requests

* style: Update input components in APIRequest class

* [autofix.ci] apply automated fixes

* remove debug print statement from api_request component

* remove debug print statement from api_request component

* feat: add dynamic cURL mode to APIRequestComponent

- Add cURL command parsing and field population
- Implement dynamic UI updates based on cURL input
- Support JSON body and headers extraction from cURL
- Add real-time refresh for cURL mode toggle

* refactor(APIRequestComponent): improve HTTP methods and cURL handling

- Add DELETE method support
- Enhance cURL parameter handling and UI visibility
- Fix JSON decode error handling with specific exception
- Update method visibility for DELETE requests

* [autofix.ci] apply automated fixes

* git commit -m "ui(api-request): adjust field visibility and requirements

- Move query_params to advanced section
- Make URL field required"

* feat(api): enhance curl parsing and update unit tests

- Improve MultilineInput handling in APIRequestComponent for curl commands
- Update parse_curl unit test to match expected data structure
- Ensure consistent format for headers and body in test assertions

* [autofix.ci] apply automated fixes

* fix(api-request): improve UI/UX and fix initial field visibility

- Fix body field flickering on component load
- Enhance URL/cURL field toggle behavior

* ✅ (filterSidebar.spec.ts): add tests for clicking on edit button modal, show headers button, and closing the modal to ensure correct behavior and visibility of elements

* 📝 (dictComponent/index.tsx): update data-testid attribute to dynamically set based on editNode value for better testability
📝 (nestedComponent.spec.ts): update test data and selectors to match changes in dictComponent/index.tsx for accurate testing

* refactor(api_request): improve component UX and field handling

- Update tool_mode placement to align with main branch changes
- Remove temporary required field validation to prevent UI conflicts
- Add field clearing logic when switching between cURL and URL modes
- Update component description to be more concise

* git commit -m "fix: resolve merge conflicts with upstream in api request component

* fix: resolve merge conflicts with upstream in api request component

* Delete src/backend/tests/unit/test_data_components.py

BREAKING CHANGE: Removed test_data_components.py as it has been replaced by test_api_request_component.py

* git commit -m "test(api-request): update test_parse_curl to match TableInput format

* style(test_api-request): apply ruff formatting rules

* [autofix.ci] apply automated fixes

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Cristhian Zanforlin Lousa <cristhian.lousa@gmail.com>
Co-authored-by: Edwin Jose <edwin.jose@datastax.com>
This commit is contained in:
VICTOR CORREA GOMES 2025-01-20 13:23:59 -03:00 • committed by GitHub
commit 420cd92faa
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
5 changed files with 328 additions and 76 deletions
Download patch file Download diff file Expand all files Collapse all files

378
src/backend/base/langflow/components/data/api_request.py
View file

@ -14,70 +14,119 @@ import validators
from langflow.base.curl.parse import parse_context
from langflow.custom import Component
from langflow.io import BoolInput, DataInput, DropdownInput, IntInput, MessageTextInput, NestedDictInput, Output
from langflow.io import (
BoolInput,
DataInput,
DropdownInput,
FloatInput,
IntInput,
MessageTextInput,
MultilineInput,
Output,
StrInput,
TableInput,
)
from langflow.schema import Data
from langflow.schema.dotdict import dotdict
class APIRequestComponent(Component):
display_name = "API Request"
description = (
"This component allows you to make HTTP requests to one or more URLs. "
"You can provide headers and body as either dictionaries or Data objects. "
"Additionally, you can append query parameters to the URLs.\n\n"
"**Note:** Check advanced options for more settings."
)
description = "Make HTTP requests using URLs or cURL commands."
icon = "Globe"
name = "APIRequest"
default_keys = ["urls", "method", "query_params"]
inputs = [
MessageTextInput(
name="urls",
display_name="URLs",
list=True,
info="Enter one or more URLs, separated by commas.",
advanced=False,
tool_mode=True,
),
MessageTextInput(
MultilineInput(
name="curl",
display_name="cURL",
info="Paste a curl command to populate the fields. "
"This will fill in the dictionary fields for headers and body.",
advanced=False,
refresh_button=True,
info=(
"Paste a curl command to populate the fields. "
"This will fill in the dictionary fields for headers and body."
),
advanced=True,
real_time_refresh=True,
tool_mode=True,
),
DropdownInput(
name="method",
display_name="Method",
options=["GET", "POST", "PATCH", "PUT"],
value="GET",
info="The HTTP method to use (GET, POST, PATCH, PUT).",
options=["GET", "POST", "PATCH", "PUT", "DELETE"],
info="The HTTP method to use.",
real_time_refresh=True,
),
NestedDictInput(
name="headers",
display_name="Headers",
info="The headers to send with the request as a dictionary. This is populated when using the CURL field.",
input_types=["Data"],
),
NestedDictInput(
name="body",
display_name="Body",
info="The body to send with the request as a dictionary (for POST, PATCH, PUT). "
"This is populated when using the CURL field.",
input_types=["Data"],
BoolInput(
name="use_curl",
display_name="Use cURL",
value=False,
info="Enable cURL mode to populate fields from a cURL command.",
real_time_refresh=True,
),
DataInput(
name="query_params",
display_name="Query Parameters",
info="The query parameters to append to the URL.",
tool_mode=True,
advanced=True,
),
TableInput(
name="body",
display_name="Body",
info="The body to send with the request as a dictionary (for POST, PATCH, PUT).",
table_schema=[
{
"name": "key",
"display_name": "Key",
"type": "str",
"description": "Parameter name",
},
{
"name": "value",
"display_name": "Value",
"description": "Parameter value",
},
],
value=[],
input_types=["Data"],
advanced=True,
),
TableInput(
name="headers",
display_name="Headers",
info="The headers to send with the request as a dictionary.",
table_schema=[
{
"name": "key",
"display_name": "Header",
"type": "str",
"description": "Header name",
},
{
"name": "value",
"display_name": "Value",
"type": "str",
"description": "Header value",
},
],
value=[],
advanced=True,
input_types=["Data"],
),
IntInput(
name="timeout",
display_name="Timeout",
value=5,
info="The timeout to use for the request.",
advanced=True,
),
BoolInput(
name="follow_redirects",
@ -109,39 +158,224 @@ class APIRequestComponent(Component):
Output(display_name="Data", name="data", method="make_requests"),
]
def _parse_json_value(self, value: Any) -> Any:
"""Parse a value that might be a JSON string."""
if not isinstance(value, str):
return value
try:
parsed = json.loads(value)
except json.JSONDecodeError:
return value
else:
return parsed
def _process_body(self, body: Any) -> dict:
"""Process the body input into a valid dictionary.
Args:
body: The body to process, can be dict, str, or list
Returns:
Processed dictionary
"""
if body is None:
return {}
if isinstance(body, dict):
return self._process_dict_body(body)
if isinstance(body, str):
return self._process_string_body(body)
if isinstance(body, list):
return self._process_list_body(body)
return {}
def _process_dict_body(self, body: dict) -> dict:
"""Process dictionary body by parsing JSON values."""
return {k: self._parse_json_value(v) for k, v in body.items()}
def _process_string_body(self, body: str) -> dict:
"""Process string body by attempting JSON parse."""
try:
return self._process_body(json.loads(body))
except json.JSONDecodeError:
return {"data": body}
def _process_list_body(self, body: list) -> dict:
"""Process list body by converting to key-value dictionary."""
processed_dict = {}
try:
for item in body:
if not self._is_valid_key_value_item(item):
continue
key = item["key"]
value = self._parse_json_value(item["value"])
processed_dict[key] = value
except (KeyError, TypeError, ValueError) as e:
self.log(f"Failed to process body list: {e}")
return {} # Return empty dictionary instead of None
return processed_dict
def _is_valid_key_value_item(self, item: Any) -> bool:
"""Check if an item is a valid key-value dictionary."""
return isinstance(item, dict) and "key" in item and "value" in item
def parse_curl(self, curl: str, build_config: dotdict) -> dotdict:
"""Parse a cURL command and update build configuration.
Args:
curl: The cURL command to parse
build_config: The build configuration to update
Returns:
Updated build configuration
"""
try:
parsed = parse_context(curl)
# Update basic configuration
build_config["urls"]["value"] = [parsed.url]
build_config["method"]["value"] = parsed.method.upper()
build_config["headers"]["value"] = dict(parsed.headers)
build_config["headers"]["advanced"] = True
build_config["body"]["advanced"] = True
if parsed.data:
# Process headers
headers_list = [{"key": k, "value": v} for k, v in parsed.headers.items()]
build_config["headers"]["value"] = headers_list
if headers_list:
build_config["headers"]["advanced"] = False
# Process body data
if not parsed.data:
build_config["body"]["value"] = []
elif parsed.data:
try:
json_data = json.loads(parsed.data)
build_config["body"]["value"] = json_data
if isinstance(json_data, dict):
body_list = [
{"key": k, "value": json.dumps(v) if isinstance(v, dict | list) else str(v)}
for k, v in json_data.items()
]
build_config["body"]["value"] = body_list
build_config["body"]["advanced"] = False
else:
build_config["body"]["value"] = [{"key": "data", "value": json.dumps(json_data)}]
build_config["body"]["advanced"] = False
except json.JSONDecodeError:
self.log("Error decoding JSON data")
else:
build_config["body"]["value"] = {}
build_config["body"]["value"] = [{"key": "data", "value": parsed.data}]
build_config["body"]["advanced"] = False
except Exception as exc:
msg = f"Error parsing curl: {exc}"
self.log(msg)
raise ValueError(msg) from exc
return build_config
def update_build_config(self, build_config: dotdict, field_value: Any, field_name: str | None = None):
if field_name == "curl" and field_value:
def update_build_config(self, build_config: dotdict, field_value: Any, field_name: str | None = None) -> dotdict:
if field_name == "use_curl":
build_config = self._update_curl_mode(build_config, use_curl=field_value)
# Fields that should not be reset
preserve_fields = {"timeout", "follow_redirects", "save_to_file", "include_httpx_metadata", "use_curl"}
# Mapping between input types and their reset values
type_reset_mapping = {
TableInput: [],
BoolInput: False,
IntInput: 0,
FloatInput: 0.0,
MessageTextInput: "",
StrInput: "",
MultilineInput: "",
DropdownInput: "GET",
DataInput: {},
}
for input_field in self.inputs:
# Only reset if field is not in preserve list
if input_field.name not in preserve_fields:
reset_value = type_reset_mapping.get(type(input_field), None)
build_config[input_field.name]["value"] = reset_value
self.log(f"Reset field {input_field.name} to {reset_value}")
elif field_name == "method" and not self.use_curl:
build_config = self._update_method_fields(build_config, field_value)
elif field_name == "curl" and self.use_curl and field_value:
build_config = self.parse_curl(field_value, build_config)
return build_config
def _update_curl_mode(self, build_config: dotdict, *, use_curl: bool) -> dotdict:
always_visible = ["method", "use_curl"]
for field in self.inputs:
field_name = field.name
field_config = build_config.get(field_name)
if isinstance(field_config, dict):
if field_name in always_visible:
field_config["advanced"] = False
elif field_name == "urls":
field_config["advanced"] = use_curl
elif field_name == "curl":
field_config["advanced"] = not use_curl
field_config["real_time_refresh"] = use_curl
elif field_name in ["body", "headers"]:
field_config["advanced"] = True # Always keep body and headers in advanced when use_curl is False
else:
field_config["advanced"] = use_curl
else:
self.log(f"Expected dict for build_config[{field_name}], got {type(field_config).__name__}")
if not use_curl:
current_method = build_config.get("method", {}).get("value", "GET")
build_config = self._update_method_fields(build_config, current_method)
return build_config
def _update_method_fields(self, build_config: dotdict, method: str) -> dotdict:
common_fields = [
"urls",
"method",
"use_curl",
]
always_advanced_fields = [
"body",
"headers",
"timeout",
"follow_redirects",
"save_to_file",
"include_httpx_metadata",
]
body_fields = ["body"]
for field in self.inputs:
field_name = field.name
field_config = build_config.get(field_name)
if isinstance(field_config, dict):
if field_name in common_fields:
field_config["advanced"] = False
elif field_name in body_fields:
field_config["advanced"] = method not in ["POST", "PUT", "PATCH"]
elif field_name in always_advanced_fields:
field_config["advanced"] = True
else:
field_config["advanced"] = True
else:
self.log(f"Expected dict for build_config[{field_name}], got {type(field_config).__name__}")
return build_config
async def make_request(
self,
client: httpx.AsyncClient,
method: str,
url: str,
headers: dict | None = None,
body: dict | None = None,
body: Any = None,
timeout: int = 5,
*,
follow_redirects: bool = True,
@ -153,24 +387,15 @@ class APIRequestComponent(Component):
msg = f"Unsupported method: {method}"
raise ValueError(msg)
if isinstance(body, str) and body:
try:
body = json.loads(body)
except Exception as e:
msg = f"Error decoding JSON data: {e}"
self.log.exception(msg)
body = None
raise ValueError(msg) from e
data = body or None
redirection_history = []
# Process body using the new helper method
processed_body = self._process_body(body)
try:
response = await client.request(
method,
url,
headers=headers,
json=data,
json=processed_body,
timeout=timeout,
follow_redirects=follow_redirects,
)
@ -216,20 +441,18 @@ class APIRequestComponent(Component):
}
)
return Data(data=metadata)
# Populate result when not saving to a file
if is_binary:
result = response.content
else:
try:
result = response.json()
except Exception: # noqa: BLE001
self.log("Error decoding JSON response")
except json.JSONDecodeError:
self.log("Failed to decode JSON response")
result = response.text.encode("utf-8")
# Add result to metadata
metadata.update({"result": result})
# Add metadata to the output
if include_httpx_metadata:
metadata.update(
{
@ -271,7 +494,6 @@ class APIRequestComponent(Component):
async def make_requests(self) -> list[Data]:
method = self.method
urls = [url.strip() for url in self.urls if url.strip()]
curl = self.curl
headers = self.headers or {}
body = self.body or {}
timeout = self.timeout
@ -279,6 +501,9 @@ class APIRequestComponent(Component):
save_to_file = self.save_to_file
include_httpx_metadata = self.include_httpx_metadata
if self.use_curl and self.curl:
self._build_config = self.parse_curl(self.curl, dotdict())
invalid_urls = [url for url in urls if not validators.url(url)]
if invalid_urls:
msg = f"Invalid URLs provided: {invalid_urls}"
@ -289,14 +514,11 @@ class APIRequestComponent(Component):
else:
query_params = self.query_params.data if self.query_params else {}
if curl:
self._build_config = self.parse_curl(curl, dotdict())
# Process headers here
headers = self._process_headers(headers)
if isinstance(headers, Data):
headers = headers.data
if isinstance(body, Data):
body = body.data
# Process body
body = self._process_body(body)
bodies = [body] * len(urls)
@ -316,7 +538,7 @@ class APIRequestComponent(Component):
save_to_file=save_to_file,
include_httpx_metadata=include_httpx_metadata,
)
for u, rec in zip(urls, bodies, strict=True)
for u, rec in zip(urls, bodies, strict=False)
]
)
self.status = results
@ -335,20 +557,17 @@ class APIRequestComponent(Component):
Tuple[bool, Path | None]:
A tuple containing a boolean indicating if the content is binary and the full file path (if applicable).
"""
# Determine if the content is binary
content_type = response.headers.get("Content-Type", "")
is_binary = "application/octet-stream" in content_type or "application/binary" in content_type
if not with_file_path:
return is_binary, None
# Step 1: Set up a subdirectory for the component in the OS temp directory
component_temp_dir = Path(tempfile.gettempdir()) / self.__class__.__name__
# Create directory asynchronously
await aiofiles_os.makedirs(component_temp_dir, exist_ok=True)
# Step 2: Extract filename from Content-Disposition
filename = None
if "Content-Disposition" in response.headers:
content_disposition = response.headers["Content-Disposition"]
@ -394,3 +613,30 @@ class APIRequestComponent(Component):
def _headers_to_dict(self, headers: httpx.Headers) -> dict[str, str]:
"""Convert HTTP headers to a dictionary with lowercased keys."""
return {k.lower(): v for k, v in headers.items()}
def _process_headers(self, headers: Any) -> dict:
"""Process the headers input into a valid dictionary.
Args:
headers: The headers to process, can be dict, str, or list
Returns:
Processed dictionary
"""
if headers is None:
return {}
if isinstance(headers, dict):
return headers
if isinstance(headers, list):
processed_headers = {}
try:
for item in headers:
if not self._is_valid_key_value_item(item):
continue
key = item["key"]
value = item["value"]
processed_headers[key] = value
except (KeyError, TypeError, ValueError) as e:
self.log(f"Failed to process headers list: {e}")
return {} # Return empty dictionary instead of None
return processed_headers
return {}

4
src/backend/tests/unit/components/data/test_api_request_component.py
View file

@ -34,8 +34,8 @@ def test_parse_curl(api_request):
# Assert
assert new_build_config["method"]["value"] == "GET"
assert new_build_config["urls"]["value"] == ["https://example.com/api/test"]
assert new_build_config["headers"]["value"] == {"Content-Type": "application/json"}
assert new_build_config["body"]["value"] == {"key": "value"}
assert new_build_config["headers"]["value"] == [{"key": "Content-Type", "value": "application/json"}]
assert new_build_config["body"]["value"] == [{"key": "key", "value": "value"}]
# HTTPx Metadata testing

2
src/frontend/src/components/core/parameterRenderComponent/components/dictComponent/index.tsx
View file

@ -45,7 +45,7 @@ export default function DictComponent({
"hover:bg-mute w-full font-medium text-primary",
editNode ? "h-fit px-3 py-0.5" : "",
)}
data-testid="dict-input"
data-testid={editNode ? `edit_${id}` : `${id}`}
>
<ForwardedIconComponent
strokeWidth={ICON_STROKE_WIDTH}

4
src/frontend/tests/core/features/filterSidebar.spec.ts
View file

@ -111,6 +111,10 @@ test(
).not.toBeVisible();
await expect(page.getByTestId("logicCondition")).not.toBeVisible();
await page.getByTestId("edit-button-modal").click();
await page.getByTestId("showheaders").click();
await page.getByText("Close").last().click();
await page.getByTestId("handle-apirequest-shownode-headers-left").click();
await expect(page.getByTestId("disclosure-data")).toBeVisible();

16
src/frontend/tests/core/unit/nestedComponent.spec.ts
View file

@ -13,20 +13,22 @@ test(
});
await page.getByTestId("blank-flow").click();
await page.getByTestId("sidebar-search-input").click();
await page.getByTestId("sidebar-search-input").fill("api request");
await page.getByTestId("sidebar-search-input").fill("alter metadata");
await page.waitForSelector('[data-testid="dataAPI Request"]', {
await page.waitForSelector('[data-testid="processingAlter Metadata"]', {
timeout: 3000,
});
await page
.getByTestId("dataAPI Request")
.first()
.dragTo(page.locator('//*[@id="react-flow-id"]'));
.getByTestId("processingAlter Metadata")
.hover()
.then(async () => {
await page.getByTestId("add-component-button-alter-metadata").click();
});
await adjustScreenView(page);
await page.getByTestId("dict_nesteddict_headers").first().click();
await page.getByTestId("dict_nesteddict_metadata").first().click();
await page
.getByText("{")
.last()
@ -62,7 +64,7 @@ test(
await page.getByTestId("more-options-modal").click();
await page.getByTestId("advanced-button-modal").click();
await page.getByTestId("dict_nesteddict_edit_headers").first().click();
await page.getByTestId("edit_dict_nesteddict_edit_metadata").last().click();
expect(await page.getByText("keytest", { exact: true }).count()).toBe(1);
expect(await page.getByText("keytest1", { exact: true }).count()).toBe(1);