The knowledge base loader node loads data (files or scraped content from URLs) into an existing knowledge base.

​
Node Inputs

  1. Knowledge Base: Select the knowledge base you would like to query.
    • Type: KnowledgeBase

You can choose an existing Knowledge Base in two ways:

  1. Select: You can select from the dropdown a Knowledge Base in created in VectorShift platform.
  2. Variable: You can pass in a variable containing reference to the Knowledge Base you want to select.

If Data Type is File:

  1. Documents to load: The file to be added to the selected Knowledge Base. Note: to convert text to file, use the Text to File node.
    • Type: List<File>

If Data Type is URL:

  1. URL to load: The raw URL link (e.g., https://vectorshift.ai/)
    • Type: Text

​
Node Parameters

If Data Type is URL:

  1. Loader Type: The type of URL loader you want to use. The available loaders are URL, Wikipedia Query, Youtube URL, Arxiv Query and Repository URL. The default value is URL.
  2. Recursive: Scrape sub-pages of the provided link.
  3. Re-scrape Frequency: The frequency that the URL will be re-scraped. The available options are Never, Hourly, Daily, Weekly and Monthly. The default value is Never.

​
Node Outputs

The knowledge base loader node does not have any node outputs.

​
Example

The below example shows a pipeline that loads content from vectorshift.ai into a knowledge base.

  • Text Node: Contains the URL “vectorshift.ai”
  • Knowledge Base Loader Node:
    • Knowledge Base: Select the knowledge base you want to load the data into from the “Knowledge Base” drop down
    • Data Type: URL
    • URL to load: {{text_0.text}}