Creating BaseFileComponent, integrating with API Request "to file" feature (#4731)

* adding ability for APIRequest to retry and save to a file

* [autofix.ci] apply automated fixes

* adding ability for APIRequest to retry and save to a file

* [autofix.ci] apply automated fixes

* initial refactor of FileComponent to handle Data input

* shifting potentially common logic into BaseFileComponent

* improving readability and fixing problems

* [autofix.ci] apply automated fixes

* addressing linting

* [autofix.ci] apply automated fixes

* linting part 2

* [autofix.ci] apply automated fixes

* linting part 3

* preserve input fields on data objects

* [autofix.ci] apply automated fixes

* ensuring processed data is linked to correct file data object

* [autofix.ci] apply automated fixes

* addressing linting

* [autofix.ci] apply automated fixes

* fixing edge case

* allowing specific failure of missing file without forcing silent_errors

* Base_file mypy errors

* [autofix.ci] apply automated fixes

* Update base_file.py

* Update base_file.py

* Fix starter templates

* Update test_vector_store_rag.py

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Eric Hare <ericrhare@gmail.com>
This commit is contained in:
Phil Miesle 2024-11-26 04:36:17 +00:00 • committed by GitHub
commit c6f27b5af8
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
8 changed files with 863 additions and 250 deletions

View file

@ -24,7 +24,7 @@ def ingestion_graph():
file_component.set(path="test.txt")
file_component.set_on_output(name="data", value=Data(text="This is a test file."), cache=True)
text_splitter = SplitTextComponent(_id="text-splitter-123")
text_splitter.set(data_inputs=file_component.load_file)
text_splitter.set(data_inputs=file_component.load_files)
openai_embeddings = OpenAIEmbeddingsComponent(_id="openai-embeddings-123")
openai_embeddings.set(
openai_api_key="sk-123", openai_api_base="https://api.openai.com/v1", openai_api_type="openai"