rename Record to Data
This commit is contained in:
parent
a562ae5b0b
commit
1d0056f4fc
142 changed files with 818 additions and 868 deletions
|
|
@ -24,15 +24,15 @@ Provide the session ID to clear its message history.
|
|||
|
||||
---
|
||||
|
||||
## Extract Key From Record
|
||||
## Extract Key From Data
|
||||
|
||||
This component extracts specified keys from a record.
|
||||
|
||||
**Parameters**
|
||||
|
||||
- **Record:**
|
||||
- **Data:**
|
||||
|
||||
- **Display Name:** Record
|
||||
- **Display Name:** Data
|
||||
- **Info:** The record from which to extract keys.
|
||||
|
||||
- **Keys:**
|
||||
|
|
@ -138,9 +138,9 @@ This component generates a notification.
|
|||
- **Display Name:** Name
|
||||
- **Info:** The notification's name.
|
||||
|
||||
- **Record:**
|
||||
- **Data:**
|
||||
|
||||
- **Display Name:** Record
|
||||
- **Display Name:** Data
|
||||
- **Info:** Optionally, a record to store in the notification.
|
||||
|
||||
- **Append:**
|
||||
|
|
|
|||
|
|
@ -13,7 +13,7 @@ This component retrieves stored chat messages based on a specific session ID.
|
|||
- **Number of messages:** Number of messages to retrieve.
|
||||
- **Session ID:** The session ID of the chat history.
|
||||
- **Order:** Choose the message order, either "Ascending" or "Descending".
|
||||
- **Record template:** (Optional) Template to convert a record to text. If left empty, the system dynamically sets it to the record's text key.
|
||||
- **Data template:** (Optional) Template to convert a record to text. If left empty, the system dynamically sets it to the record's text key.
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -124,5 +124,5 @@ Update a record with text-based key/value pairs, similar to updating a Python di
|
|||
|
||||
#### Parameters
|
||||
|
||||
- **Record:** The record to update.
|
||||
- **Data:** The record to update.
|
||||
- **New data:** The new data to update the record with.
|
||||
|
|
|
|||
|
|
@ -8,11 +8,11 @@ They also dynamically change the Playground and can be renamed to facilitate bui
|
|||
|
||||
## Inputs
|
||||
|
||||
Inputs are components used to define where data enters your flow. They can receive data from the user, a database, or any other source that can be converted to Text or Record.
|
||||
Inputs are components used to define where data enters your flow. They can receive data from the user, a database, or any other source that can be converted to Text or Data.
|
||||
|
||||
The difference between Chat Input and other Input components is the output format, the number of configurable fields, and the way they are displayed in the Playground.
|
||||
|
||||
Chat Input components can output `Text` or `Record`. When you want to pass the sender name or sender to the next component, use the `Record` output. To pass only the message, use the `Text` output, useful when saving the message to a database or memory system like Zep.
|
||||
Chat Input components can output `Text` or `Data`. When you want to pass the sender name or sender to the next component, use the `Data` output. To pass only the message, use the `Text` output, useful when saving the message to a database or memory system like Zep.
|
||||
|
||||
You can find out more about Chat Input and other Inputs [here](#chat-input).
|
||||
|
||||
|
|
@ -38,8 +38,8 @@ This component collects user input from the chat.
|
|||
|
||||
<Admonition type="note" title="Note">
|
||||
<p>
|
||||
If `As Record` is `true` and the `Message` is a `Record`, the data of the
|
||||
`Record` will be updated with the `Sender`, `Sender Name`, and `Session ID`.
|
||||
If `As Data` is `true` and the `Message` is a `Data`, the data of the `Data`
|
||||
will be updated with the `Sender`, `Sender Name`, and `Session ID`.
|
||||
</p>
|
||||
</Admonition>
|
||||
|
||||
|
|
@ -70,11 +70,11 @@ The **Text Input** component adds an **Input** field on the Playground. This ena
|
|||
**Parameters**
|
||||
|
||||
- **Value:** Specifies the text input value. This is where the user inputs text data that will be passed to the next component in the sequence. If no value is provided, it defaults to an empty string.
|
||||
- **Record Template:** Specifies how a `Record` should be converted into `Text`.
|
||||
- **Data Template:** Specifies how a `Data` should be converted into `Text`.
|
||||
|
||||
The **Record Template** field is used to specify how a `Record` should be converted into `Text`. This is particularly useful when you want to extract specific information from a `Record` and pass it as text to the next component in the sequence.
|
||||
The **Data Template** field is used to specify how a `Data` should be converted into `Text`. This is particularly useful when you want to extract specific information from a `Data` and pass it as text to the next component in the sequence.
|
||||
|
||||
For example, if you have a `Record` with the following structure:
|
||||
For example, if you have a `Data` with the following structure:
|
||||
|
||||
```json
|
||||
{
|
||||
|
|
@ -84,9 +84,9 @@ For example, if you have a `Record` with the following structure:
|
|||
}
|
||||
```
|
||||
|
||||
A template with `Name: {name}, Age: {age}` will convert the `Record` into a text string of `Name: John Doe, Age: 30`.
|
||||
A template with `Name: {name}, Age: {age}` will convert the `Data` into a text string of `Name: John Doe, Age: 30`.
|
||||
|
||||
If you pass more than one `Record`, the text will be concatenated with a new line separator.
|
||||
If you pass more than one `Data`, the text will be concatenated with a new line separator.
|
||||
|
||||
## Outputs
|
||||
|
||||
|
|
@ -112,8 +112,8 @@ This component sends a message to the chat.
|
|||
|
||||
<Admonition type="note" title="Note">
|
||||
<p>
|
||||
If `As Record` is `true` and the `Message` is a `Record`, the data in the
|
||||
`Record` is updated with the `Sender`, `Sender Name`, and `Session ID`.
|
||||
If `As Data` is `true` and the `Message` is a `Data`, the data in the `Data`
|
||||
is updated with the `Sender`, `Sender Name`, and `Session ID`.
|
||||
</p>
|
||||
</Admonition>
|
||||
|
||||
|
|
@ -154,4 +154,5 @@ The `PromptTemplate` component enables users to create prompts and define variab
|
|||
|
||||
After defining a variable in the prompt template, it acts as its own component input. See [Prompt Customization](../administration/prompt-customization) for more details.
|
||||
|
||||
- **template:** The template used to format an individual request.
|
||||
- **template:** The template used to format an individual request.import Admonition from "@theme/Admonition";
|
||||
import ZoomableImage from "/src/theme/ZoomableImage.js";
|
||||
|
|
|
|||
|
|
@ -1,14 +1,14 @@
|
|||
# Text and Record
|
||||
# Text and Data
|
||||
|
||||
In Langflow 1.0, we added two main input and output types: `Text` and `Record`.
|
||||
In Langflow 1.0, we added two main input and output types: `Text` and `Data`.
|
||||
|
||||
`Text` is a simple string input and output type, while `Record` is a structure very similar to a dictionary in Python. It is a key-value pair data structure.
|
||||
`Text` is a simple string input and output type, while `Data` is a structure very similar to a dictionary in Python. It is a key-value pair data structure.
|
||||
|
||||
We've created a few components to help you work with these types. Let's see how a few of them work.
|
||||
|
||||
## Records To Text
|
||||
|
||||
This is a component that takes in Records and outputs a `Text`. It does this using a template string and concatenating the values of the `Record`, one per line.
|
||||
This is a component that takes in Records and outputs a `Text`. It does this using a template string and concatenating the values of the `Data`, one per line.
|
||||
|
||||
If we have the following Records:
|
||||
|
||||
|
|
@ -32,13 +32,13 @@ Alice: Hello!
|
|||
John: Hi!
|
||||
```
|
||||
|
||||
## Create Record
|
||||
## Create Data
|
||||
|
||||
This component allows you to create a `Record` from a number of inputs. You can add as many key-value pairs as you want (as long as it is less than 15). Once you've picked that number you'll need to write the name of the Key and can pass `Text` values from other components to it.
|
||||
This component allows you to create a `Data` from a number of inputs. You can add as many key-value pairs as you want (as long as it is less than 15). Once you've picked that number you'll need to write the name of the Key and can pass `Text` values from other components to it.
|
||||
|
||||
## Documents To Records
|
||||
|
||||
This component takes in a LangChain `Document` and outputs a `Record`. It does this by extracting the `page_content` and the `metadata` from the `Document` and adding them to the `Record` as text and data respectively.
|
||||
This component takes in a LangChain `Document` and outputs a `Data`. It does this by extracting the `page_content` and the `metadata` from the `Document` and adding them to the `Data` as text and data respectively.
|
||||
|
||||
## Why is this useful?
|
||||
|
||||
|
|
|
|||
|
|
@ -4,14 +4,18 @@ import ZoomableImage from "/src/theme/ZoomableImage.js";
|
|||
import ReactPlayer from "react-player";
|
||||
import Admonition from "@theme/Admonition";
|
||||
|
||||
# Create Record
|
||||
# Create Data
|
||||
|
||||
In Langflow, a `Record` has a structure very similar to a Python dictionary. It is a key-value pair data structure.
|
||||
In Langflow, a `Data` has a structure very similar to a Python dictionary. It is a key-value pair data structure.
|
||||
|
||||
The **Create Record** component allows you to dynamically create a `Record` from a specified number of inputs. You can add as many key-value pairs as you want (as long as it is less than 15 😅). Once you've chosen the number of `Records`, add keys and fill up values, or pass on values from other components to the component using the input handles.
|
||||
The **Create Data** component allows you to dynamically create a `Data` from a specified number of inputs. You can add as many key-value pairs as you want (as long as it is less than 15 😅). Once you've chosen the number of `Records`, add keys and fill up values, or pass on values from other components to the component using the input handles.
|
||||
|
||||
<div
|
||||
style={{ marginBottom: "20px", display: "flex", justifyContent: "center" }}
|
||||
>
|
||||
<ReactPlayer playing controls url="/videos/create_record.mp4" />
|
||||
</div>
|
||||
import ThemedImage from "@theme/ThemedImage"; import useBaseUrl from
|
||||
"@docusaurus/useBaseUrl"; import ZoomableImage from
|
||||
"/src/theme/ZoomableImage.js"; import ReactPlayer from "react-player"; import
|
||||
Admonition from "@theme/Admonition";
|
||||
|
|
|
|||
|
|
@ -33,7 +33,7 @@ import requests
|
|||
from typing import Dict
|
||||
|
||||
from langflow import CustomComponent
|
||||
from langflow.schema import Record
|
||||
from langflow.schema import Data
|
||||
|
||||
|
||||
class NotionDatabaseProperties(CustomComponent):
|
||||
|
|
@ -61,7 +61,7 @@ class NotionDatabaseProperties(CustomComponent):
|
|||
self,
|
||||
database_id: str,
|
||||
notion_secret: str,
|
||||
) -> Record:
|
||||
) -> Data:
|
||||
url = f"https://api.notion.com/v1/databases/{database_id}"
|
||||
headers = {
|
||||
"Authorization": f"Bearer {notion_secret}",
|
||||
|
|
@ -74,7 +74,7 @@ class NotionDatabaseProperties(CustomComponent):
|
|||
data = response.json()
|
||||
properties = data.get("properties", {})
|
||||
|
||||
record = Record(text=str(response.json()), data=properties)
|
||||
record = Data(text=str(response.json()), data=properties)
|
||||
self.status = f"Retrieved {len(properties)} properties from the Notion database.\n {record.text}"
|
||||
return record
|
||||
```
|
||||
|
|
|
|||
|
|
@ -39,7 +39,7 @@ import requests
|
|||
import json
|
||||
from typing import Dict, Any, List
|
||||
from langflow.custom import CustomComponent
|
||||
from langflow.schema import Record
|
||||
from langflow.schema import Data
|
||||
|
||||
class NotionListPages(CustomComponent):
|
||||
display_name = "List Pages [Notion]"
|
||||
|
|
@ -83,7 +83,7 @@ class NotionListPages(CustomComponent):
|
|||
notion_secret: str,
|
||||
database_id: str,
|
||||
query_payload: str = "{}",
|
||||
) -> List[Record]:
|
||||
) -> List[Data]:
|
||||
try:
|
||||
query_data = json.loads(query_payload)
|
||||
filter_obj = query_data.get("filter")
|
||||
|
|
@ -127,14 +127,14 @@ class NotionListPages(CustomComponent):
|
|||
)
|
||||
|
||||
combined_text += text
|
||||
records.append(Record(text=text, data=page_data))
|
||||
records.append(Data(text=text, data=page_data))
|
||||
|
||||
self.status = combined_text.strip()
|
||||
return records
|
||||
|
||||
except Exception as e:
|
||||
self.status = f"An error occurred: {str(e)}"
|
||||
return [Record(text=self.status, data=[])]
|
||||
return [Data(text=self.status, data=[])]
|
||||
```
|
||||
|
||||
<Admonition type="info" title="Example Usage">
|
||||
|
|
|
|||
|
|
@ -30,7 +30,7 @@ import requests
|
|||
from typing import List
|
||||
|
||||
from langflow import CustomComponent
|
||||
from langflow.schema import Record
|
||||
from langflow.schema import Data
|
||||
|
||||
|
||||
class NotionUserList(CustomComponent):
|
||||
|
|
@ -52,7 +52,7 @@ class NotionUserList(CustomComponent):
|
|||
def build(
|
||||
self,
|
||||
notion_secret: str,
|
||||
) -> List[Record]:
|
||||
) -> List[Data]:
|
||||
url = "https://api.notion.com/v1/users"
|
||||
headers = {
|
||||
"Authorization": f"Bearer {notion_secret}",
|
||||
|
|
@ -84,7 +84,7 @@ class NotionUserList(CustomComponent):
|
|||
output += f"{key.replace('_', ' ').title()}: {value}\n"
|
||||
output += "________________________\n"
|
||||
|
||||
record = Record(text=output, data=record_data)
|
||||
record = Data(text=output, data=record_data)
|
||||
records.append(record)
|
||||
|
||||
self.status = "\n".join(record.text for record in records)
|
||||
|
|
|
|||
|
|
@ -36,7 +36,7 @@ import requests
|
|||
from typing import Dict, Any
|
||||
|
||||
from langflow import CustomComponent
|
||||
from langflow.schema import Record
|
||||
from langflow.schema import Data
|
||||
|
||||
|
||||
class NotionPageContent(CustomComponent):
|
||||
|
|
@ -64,7 +64,7 @@ class NotionPageContent(CustomComponent):
|
|||
self,
|
||||
page_id: str,
|
||||
notion_secret: str,
|
||||
) -> Record:
|
||||
) -> Data:
|
||||
blocks_url = f"https://api.notion.com/v1/blocks/{page_id}/children?page_size=100"
|
||||
headers = {
|
||||
"Authorization": f"Bearer {notion_secret}",
|
||||
|
|
@ -80,7 +80,7 @@ class NotionPageContent(CustomComponent):
|
|||
content = self.parse_blocks(blocks_data["results"])
|
||||
|
||||
self.status = content
|
||||
return Record(data={"content": content}, text=content)
|
||||
return Data(data={"content": content}, text=content)
|
||||
|
||||
def parse_blocks(self, blocks: list) -> str:
|
||||
content = ""
|
||||
|
|
|
|||
|
|
@ -26,7 +26,7 @@ import requests
|
|||
from typing import Dict, Any
|
||||
|
||||
from langflow import CustomComponent
|
||||
from langflow.schema import Record
|
||||
from langflow.schema import Data
|
||||
|
||||
|
||||
class NotionPageUpdate(CustomComponent):
|
||||
|
|
@ -61,7 +61,7 @@ class NotionPageUpdate(CustomComponent):
|
|||
page_id: str,
|
||||
properties: str,
|
||||
notion_secret: str,
|
||||
) -> Record:
|
||||
) -> Data:
|
||||
url = f"https://api.notion.com/v1/pages/{page_id}"
|
||||
headers = {
|
||||
"Authorization": f"Bearer {notion_secret}",
|
||||
|
|
@ -88,7 +88,7 @@ class NotionPageUpdate(CustomComponent):
|
|||
output += f"{prop_name}: {prop_value}\n"
|
||||
|
||||
self.status = output
|
||||
return Record(data=updated_page)
|
||||
return Data(data=updated_page)
|
||||
```
|
||||
|
||||
Let's break down the key parts of this component:
|
||||
|
|
@ -99,7 +99,7 @@ Let's break down the key parts of this component:
|
|||
|
||||
- The component interacts with the Notion API to update the page properties. It constructs the API URL, headers, and request data based on the provided parameters.
|
||||
|
||||
- The processed data is returned as a `Record` object, which can be connected to other components in the Langflow flow. The `Record` object contains the updated page data.
|
||||
- The processed data is returned as a `Data` object, which can be connected to other components in the Langflow flow. The `Data` object contains the updated page data.
|
||||
|
||||
- The component also stores the updated page properties in the `status` attribute for logging and debugging purposes.
|
||||
|
||||
|
|
|
|||
|
|
@ -36,7 +36,7 @@ To use the `NotionSearch` component in a Langflow flow, follow these steps:
|
|||
import requests
|
||||
from typing import Dict, Any, List
|
||||
from langflow.custom import CustomComponent
|
||||
from langflow.schema import Record
|
||||
from langflow.schema import Data
|
||||
|
||||
class NotionSearch(CustomComponent):
|
||||
display_name = "Search Notion"
|
||||
|
|
@ -88,7 +88,7 @@ class NotionSearch(CustomComponent):
|
|||
query: str = "",
|
||||
filter_value: str = "page",
|
||||
sort_direction: str = "descending",
|
||||
) -> List[Record]:
|
||||
) -> List[Data]:
|
||||
try:
|
||||
url = "https://api.notion.com/v1/search"
|
||||
headers = {
|
||||
|
|
@ -135,14 +135,14 @@ class NotionSearch(CustomComponent):
|
|||
|
||||
text += f"type: {result['object']}\nlast_edited_time: {result['last_edited_time']}\n\n"
|
||||
combined_text += text
|
||||
records.append(Record(text=text, data=result_data))
|
||||
records.append(Data(text=text, data=result_data))
|
||||
|
||||
self.status = combined_text
|
||||
return records
|
||||
|
||||
except Exception as e:
|
||||
self.status = f"An error occurred: {str(e)}"
|
||||
return [Record(text=self.status, data=[])]
|
||||
return [Data(text=self.status, data=[])]
|
||||
```
|
||||
|
||||
## Example Usage
|
||||
|
|
|
|||
|
|
@ -16,7 +16,7 @@ We have a special channel in our Discord server dedicated to Langflow 1.0 migrat
|
|||
- Continued support for LangChain and new support for multiple frameworks
|
||||
- Redesigned sidebar and customizable interaction panel
|
||||
- New Native Categories and Components
|
||||
- Improved user experience with Text and Record modes
|
||||
- Improved user experience with Text and Data modes
|
||||
- CustomComponent for all components
|
||||
- Compatibility with previous versions using Runnable Executor
|
||||
- Multiple flows in the canvas
|
||||
|
|
@ -58,11 +58,11 @@ Langflow 1.0 introduces many new native categories, including Inputs, Outputs, H
|
|||
|
||||
**Guide coming soon**
|
||||
|
||||
## New Way of Using Langflow: Text and Record (and more to come)
|
||||
## New Way of Using Langflow: Text and Data (and more to come)
|
||||
|
||||
With the introduction of Text and Record types connections between Components are more intuitive and easier to understand. This is the first step in a series of improvements to the way you interact with Langflow. Learn how to use Text, and Record and how they help you build better flows.
|
||||
With the introduction of Text and Data types connections between Components are more intuitive and easier to understand. This is the first step in a series of improvements to the way you interact with Langflow. Learn how to use Text, and Data and how they help you build better flows.
|
||||
|
||||
[Learn more about Text and Record](../components/text-and-record)
|
||||
[Learn more about Text and Data](../components/text-and-record)
|
||||
|
||||
## CustomComponent for All Components
|
||||
|
||||
|
|
|
|||
|
|
@ -4,10 +4,10 @@ from langchain.agents import AgentExecutor, BaseMultiActionAgent, BaseSingleActi
|
|||
from langchain_core.messages import BaseMessage
|
||||
from langchain_core.runnables import Runnable
|
||||
|
||||
from langflow.base.agents.utils import get_agents_list, records_to_messages
|
||||
from langflow.base.agents.utils import data_to_messages, get_agents_list
|
||||
from langflow.custom import CustomComponent
|
||||
from langflow.field_typing import Text, Tool
|
||||
from langflow.schema import Record
|
||||
from langflow.schema import Data
|
||||
|
||||
|
||||
class LCAgentComponent(CustomComponent):
|
||||
|
|
@ -49,7 +49,7 @@ class LCAgentComponent(CustomComponent):
|
|||
agent: Union[Runnable, BaseSingleActionAgent, BaseMultiActionAgent, AgentExecutor],
|
||||
inputs: str,
|
||||
tools: List[Tool],
|
||||
message_history: Optional[List[Record]] = None,
|
||||
message_history: Optional[List[Data]] = None,
|
||||
handle_parsing_errors: bool = True,
|
||||
output_key: str = "output",
|
||||
) -> Text:
|
||||
|
|
@ -64,7 +64,7 @@ class LCAgentComponent(CustomComponent):
|
|||
)
|
||||
input_dict: dict[str, str | list[BaseMessage]] = {"input": inputs}
|
||||
if message_history:
|
||||
input_dict["chat_history"] = records_to_messages(message_history)
|
||||
input_dict["chat_history"] = data_to_messages(message_history)
|
||||
result = await runnable.ainvoke(input_dict)
|
||||
self.status = result
|
||||
if output_key in result:
|
||||
|
|
|
|||
|
|
@ -13,7 +13,7 @@ from langchain_core.prompts import BasePromptTemplate, ChatPromptTemplate
|
|||
from langchain_core.tools import BaseTool
|
||||
from pydantic import BaseModel
|
||||
|
||||
from langflow.schema import Record
|
||||
from langflow.schema import Data
|
||||
|
||||
from .default_prompts import XML_AGENT_PROMPT
|
||||
|
||||
|
|
@ -34,17 +34,17 @@ class AgentSpec(BaseModel):
|
|||
hub_repo: Optional[str] = None
|
||||
|
||||
|
||||
def records_to_messages(records: List[Record]) -> List[BaseMessage]:
|
||||
def data_to_messages(data: List[Data]) -> List[BaseMessage]:
|
||||
"""
|
||||
Convert a list of records to a list of messages.
|
||||
Convert a list of data to a list of messages.
|
||||
|
||||
Args:
|
||||
records (List[Record]): The records to convert.
|
||||
data (List[Data]): The data to convert.
|
||||
|
||||
Returns:
|
||||
List[Message]: The records as messages.
|
||||
List[Message]: The data as messages.
|
||||
"""
|
||||
return [record.to_lc_message() for record in records]
|
||||
return [value.to_lc_message() for value in data]
|
||||
|
||||
|
||||
def validate_and_create_xml_agent(
|
||||
|
|
|
|||
|
|
@ -8,7 +8,7 @@ import chardet
|
|||
import orjson
|
||||
import yaml
|
||||
|
||||
from langflow.schema import Record
|
||||
from langflow.schema import Data
|
||||
|
||||
# Types of files that can be read simply by file.read()
|
||||
# and have 100% to be completely readable
|
||||
|
|
@ -82,7 +82,7 @@ def retrieve_file_paths(
|
|||
|
||||
# ! Removing unstructured dependency until
|
||||
# ! 3.12 is supported
|
||||
# def partition_file_to_record(file_path: str, silent_errors: bool) -> Optional[Record]:
|
||||
# def partition_file_to_record(file_path: str, silent_errors: bool) -> Optional[Data]:
|
||||
# # Use the partition function to load the file
|
||||
# from unstructured.partition.auto import partition # type: ignore
|
||||
|
||||
|
|
@ -93,11 +93,11 @@ def retrieve_file_paths(
|
|||
# raise ValueError(f"Error loading file {file_path}: {e}") from e
|
||||
# return None
|
||||
|
||||
# # Create a Record
|
||||
# # Create a Data
|
||||
# text = "\n\n".join([Text(el) for el in elements])
|
||||
# metadata = elements.metadata if hasattr(elements, "metadata") else {}
|
||||
# metadata["file_path"] = file_path
|
||||
# record = Record(text=text, data=metadata)
|
||||
# record = Data(text=text, data=metadata)
|
||||
# return record
|
||||
|
||||
|
||||
|
|
@ -129,7 +129,7 @@ def parse_pdf_to_text(file_path: str) -> str:
|
|||
return "\n\n".join([page.extract_text() for page in reader.pages])
|
||||
|
||||
|
||||
def parse_text_file_to_record(file_path: str, silent_errors: bool) -> Optional[Record]:
|
||||
def parse_text_file_to_record(file_path: str, silent_errors: bool) -> Optional[Data]:
|
||||
try:
|
||||
if file_path.endswith(".pdf"):
|
||||
text = parse_pdf_to_text(file_path)
|
||||
|
|
@ -156,7 +156,7 @@ def parse_text_file_to_record(file_path: str, silent_errors: bool) -> Optional[R
|
|||
raise ValueError(f"Error loading file {file_path}: {e}") from e
|
||||
return None
|
||||
|
||||
record = Record(data={"file_path": file_path, "text": text})
|
||||
record = Data(data={"file_path": file_path, "text": text})
|
||||
return record
|
||||
|
||||
|
||||
|
|
@ -167,21 +167,21 @@ def parse_text_file_to_record(file_path: str, silent_errors: bool) -> Optional[R
|
|||
# silent_errors: bool,
|
||||
# max_concurrency: int,
|
||||
# use_multithreading: bool,
|
||||
# ) -> List[Optional[Record]]:
|
||||
# ) -> List[Optional[Data]]:
|
||||
# if use_multithreading:
|
||||
# records = parallel_load_records(file_paths, silent_errors, max_concurrency)
|
||||
# data = parallel_load_data(file_paths, silent_errors, max_concurrency)
|
||||
# else:
|
||||
# records = [partition_file_to_record(file_path, silent_errors) for file_path in file_paths]
|
||||
# records = list(filter(None, records))
|
||||
# return records
|
||||
# data = [partition_file_to_record(file_path, silent_errors) for file_path in file_paths]
|
||||
# data = list(filter(None, data))
|
||||
# return data
|
||||
|
||||
|
||||
def parallel_load_records(
|
||||
def parallel_load_data(
|
||||
file_paths: List[str],
|
||||
silent_errors: bool,
|
||||
max_concurrency: int,
|
||||
load_function: Callable = parse_text_file_to_record,
|
||||
) -> List[Optional[Record]]:
|
||||
) -> List[Optional[Data]]:
|
||||
with futures.ThreadPoolExecutor(max_workers=max_concurrency) as executor:
|
||||
loaded_files = executor.map(
|
||||
lambda file_path: load_function(file_path, silent_errors),
|
||||
|
|
|
|||
|
|
@ -1,67 +1,67 @@
|
|||
from typing import List
|
||||
|
||||
from langflow.graph.schema import ResultData, RunOutputs
|
||||
from langflow.schema import Record
|
||||
from langflow.schema import Data
|
||||
|
||||
|
||||
def build_records_from_run_outputs(run_outputs: RunOutputs) -> List[Record]:
|
||||
def build_data_from_run_outputs(run_outputs: RunOutputs) -> List[Data]:
|
||||
"""
|
||||
Build a list of records from the given RunOutputs.
|
||||
Build a list of data from the given RunOutputs.
|
||||
|
||||
Args:
|
||||
run_outputs (RunOutputs): The RunOutputs object containing the output data.
|
||||
|
||||
Returns:
|
||||
List[Record]: A list of records built from the RunOutputs.
|
||||
List[Data]: A list of data built from the RunOutputs.
|
||||
|
||||
"""
|
||||
if not run_outputs:
|
||||
return []
|
||||
records = []
|
||||
data = []
|
||||
for result_data in run_outputs.outputs:
|
||||
if result_data:
|
||||
records.extend(build_records_from_result_data(result_data))
|
||||
return records
|
||||
data.extend(build_data_from_result_data(result_data))
|
||||
return data
|
||||
|
||||
|
||||
def build_records_from_result_data(result_data: ResultData, get_final_results_only: bool = True) -> List[Record]:
|
||||
def build_data_from_result_data(result_data: ResultData, get_final_results_only: bool = True) -> List[Data]:
|
||||
"""
|
||||
Build a list of records from the given ResultData.
|
||||
Build a list of data from the given ResultData.
|
||||
|
||||
Args:
|
||||
result_data (ResultData): The ResultData object containing the result data.
|
||||
get_final_results_only (bool, optional): Whether to include only final results. Defaults to True.
|
||||
|
||||
Returns:
|
||||
List[Record]: A list of records built from the ResultData.
|
||||
List[Data]: A list of data built from the ResultData.
|
||||
|
||||
"""
|
||||
messages = result_data.messages
|
||||
if not messages:
|
||||
return []
|
||||
records = []
|
||||
data = []
|
||||
for message in messages:
|
||||
message_dict = message if isinstance(message, dict) else message.model_dump()
|
||||
if get_final_results_only:
|
||||
result_data_dict = result_data.model_dump()
|
||||
results = result_data_dict.get("results", {})
|
||||
inner_result = results.get("result", {})
|
||||
record = Record(data={"result": inner_result, "message": message_dict}, text_key="result")
|
||||
records.append(record)
|
||||
return records
|
||||
record = Data(data={"result": inner_result, "message": message_dict}, text_key="result")
|
||||
data.append(record)
|
||||
return data
|
||||
|
||||
|
||||
def format_flow_output_records(records: List[Record]) -> str:
|
||||
def format_flow_output_data(data: List[Data]) -> str:
|
||||
"""
|
||||
Format the flow output records into a string.
|
||||
Format the flow output data into a string.
|
||||
|
||||
Args:
|
||||
records (List[Record]): The list of records to format.
|
||||
data (List[Data]): The list of data to format.
|
||||
|
||||
Returns:
|
||||
str: The formatted flow output records.
|
||||
str: The formatted flow output data.
|
||||
|
||||
"""
|
||||
result = "Flow run output:\n"
|
||||
results = "\n".join([record.result for record in records if record.data["message"]])
|
||||
results = "\n".join([value.result for value in data if value.data["message"]])
|
||||
return result + results
|
||||
|
|
|
|||
|
|
@ -3,7 +3,7 @@ from typing import Optional, Union
|
|||
from langflow.base.data.utils import IMG_FILE_TYPES, TEXT_FILE_TYPES
|
||||
from langflow.custom import Component
|
||||
from langflow.memory import store_message
|
||||
from langflow.schema import Record
|
||||
from langflow.schema import Data
|
||||
from langflow.schema.message import Message
|
||||
|
||||
|
||||
|
|
@ -35,9 +35,9 @@ class ChatComponent(Component):
|
|||
"advanced": True,
|
||||
},
|
||||
"record_template": {
|
||||
"display_name": "Record Template",
|
||||
"display_name": "Data Template",
|
||||
"multiline": True,
|
||||
"info": "In case of Message being a Record, this template will be used to convert it to text.",
|
||||
"info": "In case of Message being a Data, this template will be used to convert it to text.",
|
||||
"advanced": True,
|
||||
},
|
||||
"files": {
|
||||
|
|
@ -65,14 +65,14 @@ class ChatComponent(Component):
|
|||
self,
|
||||
sender: Optional[str] = "User",
|
||||
sender_name: Optional[str] = "User",
|
||||
input_value: Optional[Union[str, Record, Message]] = None,
|
||||
input_value: Optional[Union[str, Data, Message]] = None,
|
||||
files: Optional[list[str]] = None,
|
||||
session_id: Optional[str] = None,
|
||||
return_message: Optional[bool] = False,
|
||||
) -> Message:
|
||||
message: Message | None = None
|
||||
|
||||
if isinstance(input_value, Record):
|
||||
if isinstance(input_value, Data):
|
||||
# Update the data of the record
|
||||
message = Message.from_record(input_value)
|
||||
else:
|
||||
|
|
|
|||
|
|
@ -2,8 +2,8 @@ from typing import Optional
|
|||
|
||||
from langflow.custom import Component
|
||||
from langflow.field_typing import Text
|
||||
from langflow.helpers.record import records_to_text
|
||||
from langflow.schema import Record
|
||||
from langflow.helpers.record import data_to_text
|
||||
from langflow.schema import Data
|
||||
|
||||
|
||||
class TextComponent(Component):
|
||||
|
|
@ -14,13 +14,13 @@ class TextComponent(Component):
|
|||
return {
|
||||
"input_value": {
|
||||
"display_name": "Value",
|
||||
"input_types": ["Text", "Record"],
|
||||
"info": "Text or Record to be passed.",
|
||||
"input_types": ["Text", "Data"],
|
||||
"info": "Text or Data to be passed.",
|
||||
},
|
||||
"record_template": {
|
||||
"display_name": "Record Template",
|
||||
"display_name": "Data Template",
|
||||
"multiline": True,
|
||||
"info": "Template to convert Record to Text. If left empty, it will be dynamically set to the Record's text key.",
|
||||
"info": "Template to convert Data to Text. If left empty, it will be dynamically set to the Data's text key.",
|
||||
"advanced": True,
|
||||
},
|
||||
}
|
||||
|
|
@ -30,12 +30,12 @@ class TextComponent(Component):
|
|||
input_value: Optional[Text] = "",
|
||||
record_template: Optional[str] = "{text}",
|
||||
) -> Text:
|
||||
if isinstance(input_value, Record):
|
||||
if isinstance(input_value, Data):
|
||||
if record_template == "":
|
||||
# it should be dynamically set to the Record's .text_key value
|
||||
# it should be dynamically set to the Data's .text_key value
|
||||
# meaning, if text_key = "bacon", then record_template = "{bacon}"
|
||||
record_template = "{" + input_value.text_key + "}"
|
||||
input_value = records_to_text(template=record_template, records=input_value)
|
||||
input_value = data_to_text(template=record_template, data=input_value)
|
||||
self.status = input_value
|
||||
if not input_value:
|
||||
input_value = ""
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
from typing import Optional
|
||||
|
||||
from langflow.custom import CustomComponent
|
||||
from langflow.schema import Record
|
||||
from langflow.schema import Data
|
||||
|
||||
|
||||
class BaseMemoryComponent(CustomComponent):
|
||||
|
|
@ -33,14 +33,14 @@ class BaseMemoryComponent(CustomComponent):
|
|||
"advanced": True,
|
||||
},
|
||||
"record_template": {
|
||||
"display_name": "Record Template",
|
||||
"display_name": "Data Template",
|
||||
"multiline": True,
|
||||
"info": "Template to convert Record to Text. If left empty, it will be dynamically set to the Record's text key.",
|
||||
"info": "Template to convert Data to Text. If left empty, it will be dynamically set to the Data's text key.",
|
||||
"advanced": True,
|
||||
},
|
||||
}
|
||||
|
||||
def get_messages(self, **kwargs) -> list[Record]:
|
||||
def get_messages(self, **kwargs) -> list[Data]:
|
||||
raise NotImplementedError
|
||||
|
||||
def add_message(
|
||||
|
|
|
|||
|
|
@ -2,16 +2,16 @@ from copy import deepcopy
|
|||
|
||||
from langchain_core.documents import Document
|
||||
|
||||
from langflow.schema import Record
|
||||
from langflow.schema import Data
|
||||
from langflow.schema.message import Message
|
||||
|
||||
|
||||
def record_to_string(record: Record) -> str:
|
||||
def record_to_string(record: Data) -> str:
|
||||
"""
|
||||
Convert a record to a string.
|
||||
|
||||
Args:
|
||||
record (Record): The record to convert.
|
||||
record (Data): The record to convert.
|
||||
|
||||
Returns:
|
||||
str: The record as a string.
|
||||
|
|
@ -32,18 +32,18 @@ def dict_values_to_string(d: dict) -> dict:
|
|||
# Do something similar to the above
|
||||
d_copy = deepcopy(d)
|
||||
for key, value in d_copy.items():
|
||||
# it could be a list of records or documents or strings
|
||||
# it could be a list of data or documents or strings
|
||||
if isinstance(value, list):
|
||||
for i, item in enumerate(value):
|
||||
if isinstance(item, Message):
|
||||
d_copy[key][i] = item.text
|
||||
elif isinstance(item, Record):
|
||||
elif isinstance(item, Data):
|
||||
d_copy[key][i] = record_to_string(item)
|
||||
elif isinstance(item, Document):
|
||||
d_copy[key][i] = document_to_string(item)
|
||||
elif isinstance(value, Message):
|
||||
d_copy[key] = value.text
|
||||
elif isinstance(value, Record):
|
||||
elif isinstance(value, Data):
|
||||
d_copy[key] = record_to_string(value)
|
||||
elif isinstance(value, Document):
|
||||
d_copy[key] = document_to_string(value)
|
||||
|
|
|
|||
Some files were not shown because too many files have changed in this diff Show more
Loading…
Reference in a new issue