What are Data Flows

Data flows in Data Integration define the flow of data between source and target data assets, and any operations on that data. Data engineers and ETL developers can then analyze or gather insights and use the data to make business decisions.

To design a data flow in Data Integration, you use components called operators to build the flow of data from source to target, and apply transformations on attributes to aggregate, cleanse, and shape the data. You can also use parameters to externalize and override values so that different configurations of sources, targets, and transformations can be reused within the same data flow.

You can use Data Integration built-in functions with operators in the data flow to build expressions that process data or extract data values from arguments passed to the functions. You can also create custom (user-defined) functions for use in expressions.

Create data flows from a project or folder details page. After building a data flow, create an integration task that's used to publish and run the data flow.