Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.vectorshift.ai/llms.txt

Use this file to discover all available pages before exploring further.

What this builds. A pipeline that mixes batch and non-batch nodes — a shared system-prompt text node runs once, while two chained add_batch(...) LLM nodes fan out across each input independently. You’ll end up with. A saved Batch Config Example pipeline whose batch_refine node consumes the vectorised output of batch_llm. No output nodes are wired because vec<T> isn’t yet a supported OutputNode type. 
import vectorshift
from vectorshift.pipeline import Pipeline
from vectorshift.request import VectorshiftApiError

PIPELINE_NAME = "Batch Config Example"

try:
    pipeline = Pipeline.fetch(name=PIPELINE_NAME)
    print(f"Pipeline fetched: id={pipeline.id}, branch_id={pipeline.branch_id}")
except Exception as e:
    print(f"Error fetching pipeline: {e}")
    pipeline = Pipeline.new(name=PIPELINE_NAME)
    print(f"Pipeline created: id={pipeline.id}, branch_id={pipeline.branch_id}")

try:

    # --- Input: a list of strings ---
    input_node = pipeline.add(name="queries", id="queries").input(
        input_type="string",
    )

    # --- Non-batch node: shared system prompt (runs once) ---
    system_prompt = pipeline.add(name="system_prompt", id="system_prompt").text(
        text="You are a helpful assistant. Summarize the following query in one sentence.",
    )

    # --- Batch LLM: processes each input independently ---
    # add_batch automatically sets execution_mode="batch".
    # Vectorizable inputs (prompt, system) accept lists, and
    # vectorizable outputs (response, tokens_used) produce lists.
    batch_llm = pipeline.add_batch(name="batch_llm", id="batch_llm").llm(
        provider="openai",
        model="gpt-4o",
        stream=False,
        prompt=input_node.text,
        system=[system_prompt.text],
    )

    # --- Second batch node chained from the first ---
    batch_llm_2 = pipeline.add_batch(name="batch_refine", id="batch_refine").llm(
        provider="openai",
        model="gpt-4o",
        stream=False,
        prompt=batch_llm.response,
        system=["Rewrite the following to be more concise."],
    )

    # ======================================================================
    # Batch nodes produce vec<T> outputs which are not natively supported
    # by the OutputNode, so no output nodes are wired here.
    # ======================================================================

    pipeline.save()
    print(f"Pipeline saved:  id={pipeline.id}, branch_id={pipeline.branch_id}")

except VectorshiftApiError as e:
    print(f"API error: {e.status_code} {e.method} {e.endpoint}")
    print(f"Message: {e.error_message}")

​
Expected output

Pipeline created: id=..., branch_id=...
Pipeline saved:  id=..., branch_id=...
add_batch(...) is identical to add(...) except it sets execution_mode="batch" and treats vectorisable inputs as lists.

​
See also

Sub-pipelines (list mode)

Fan out a sub-pipeline across a list of inputs.

Intermediate results

Inspect every node’s output as it streams.

Pipeline reference

Full method surface.