feat: add Spider Web Scraper & Crawler (#2439)

* spider files

* rebuild required

* add spider-client here

* Feat: Spider Web Crawler & Scraper

* Feat: spider integration

* new input not working

* [autofix.ci] apply automated fixes

* [autofix.ci] apply automated fixes (attempt 2/3)

* fix: add outputs and configure build method

* style: run ruff

* Refactor SpiderTool to use 'crawl' instead of 'build' for generating Markdown content

* chore: add type ignore

* chore: new lock

* chore: Update mem0ai dependency to version 0.0.5

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Gabriel Luiz Freitas Almeida <gabriel@langflow.org>
This commit is contained in:
William Espegren 2024-08-08 14:53:05 +02:00 • committed by GitHub
commit 7a36cc9ebf
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
12 changed files with 2580 additions and 68 deletions

View file

@ -106,6 +106,7 @@ langchain-google-community = "1.0.7"
wolframalpha = "^5.1.3"
astra-assistants = "^2.0.15"
composio-langchain = "^0.3.28"
spider-client = "^0.0.27"
[tool.poetry.group.dev.dependencies]