Execute custom code in your pipelines
Transformations allow you to write custom Python code that executes during your pipelines. You can define new and edit existing Transformations by navigating to the Transformations tab. You can access Transformations by using the Transformations node in the Home tab of the Pipeline builder.
To create a new Transformation, you can take the following steps:
Enter the parameters for your transformation, including the Name, Function Name, Description, Inputs, and Outputs. Function names, input names, and output names should be valid Python function and variable names, respectively.
In this example, first we will create a transformation that formats the date and time in a human-readable manner. Create a transformation with the following parameters:
format_date_time
Formats the date and time in a human-readable manner
date_time
formatted_date_time
In the code, the datetime
module is used to format the date and time.
Note: The imports take place in the function definition.
Also, it is a good practice to test the transformation in the editor before deploying the changes. To test the transformation, you can use the Test Code
button in the top right corner of the editor accordion.
In this example, we will use the transformation in a pipeline.
{{time_0.output}}
Demo date time transformation
{{transformation_0.formatted_date_time}}
In this example, we will create a transformation that converts the content of a file to base64 and outputs the first 10 characters. Create a transformation with the following parameters:
file_to_base64
Converts the content of a file to base64 and outputs the first 10 characters
input_file
base64_content
In the code, the base64
module is used to convert the content of a file to base64.
Also, it is a good practice to test the transformation in the editor before deploying the changes. To test the transformation, you can use the Test Code
button in the top right corner of the editor accordion.
In this example, we will use the transformation in a pipeline.
{{input_0.file}}
Demo File Transformation
{{transformation_0.base64_content}}
Execute custom code in your pipelines
Transformations allow you to write custom Python code that executes during your pipelines. You can define new and edit existing Transformations by navigating to the Transformations tab. You can access Transformations by using the Transformations node in the Home tab of the Pipeline builder.
To create a new Transformation, you can take the following steps:
Enter the parameters for your transformation, including the Name, Function Name, Description, Inputs, and Outputs. Function names, input names, and output names should be valid Python function and variable names, respectively.
In this example, first we will create a transformation that formats the date and time in a human-readable manner. Create a transformation with the following parameters:
format_date_time
Formats the date and time in a human-readable manner
date_time
formatted_date_time
In the code, the datetime
module is used to format the date and time.
Note: The imports take place in the function definition.
Also, it is a good practice to test the transformation in the editor before deploying the changes. To test the transformation, you can use the Test Code
button in the top right corner of the editor accordion.
In this example, we will use the transformation in a pipeline.
{{time_0.output}}
Demo date time transformation
{{transformation_0.formatted_date_time}}
In this example, we will create a transformation that converts the content of a file to base64 and outputs the first 10 characters. Create a transformation with the following parameters:
file_to_base64
Converts the content of a file to base64 and outputs the first 10 characters
input_file
base64_content
In the code, the base64
module is used to convert the content of a file to base64.
Also, it is a good practice to test the transformation in the editor before deploying the changes. To test the transformation, you can use the Test Code
button in the top right corner of the editor accordion.
In this example, we will use the transformation in a pipeline.
{{input_0.file}}
Demo File Transformation
{{transformation_0.base64_content}}