Image to Text

Leverage the Multimodal image processing capabilities of OpenAI's GPT4 Vision model

The ImageToText node has system and prompt inputs similar to the OpenAI LLM and an additional image input. The ImageToText node can accept images loaded from files as input as well as images generated by an Image Generation node. You can also load PDFs to be processed as images.

The following example pipeline generates an image using DALLE-3 and then uses the vision model to generate a description.

PDF file inputs will be converted into images. For example we can create a pipeline to analyze the pdf of a research paper

Within the "System" prompt, provide instructions on how the model should behave. Within the "Prompt", you have the option to pass additional information to the "Prompt", such as a user message from a input node.

Applications

The ImageToText node is useful for the following applications

• image classification

• pdf processing

• table extraction from images

For information on prompting the model and its limitations see the OpenAI documentation

Usage

The OpenAI vision model is billed based on input and output token usage similar to the LLM nodes. In addition each image input adds an additional cost in terms of input tokens.