Publishing a Task to OCI Data Flow

Before you publish a task to an application in OCI Data Flow, ensure that you have an Object Storage bucket and the relevant permissions and IAM policies.

See Required Setup and Policies.

Note

Only integration and data loader tasks can be published to OCI Data Flow.

1. On the Projects or Folders list page, select the project or folder in which tasks are saved. If you need help finding the list page of projects or folders, see Listing Projects or Listing Folders.
2. On the project or folder details page, select Tasks.
3. In the tasks list, select Publish to OCI Data Flow from the Actions menu () for the task that you want to publish.
  
  Only integration and data loader tasks can be published to OCI Data Flow.
4. On the Publish to OCI Data Flow page, complete the Application information section as follows:
  
  Select the compartment in which to create the Data Flow application.
  
  Enter a name and description (optional) for the Data Flow application.
5. Complete the Resource configuration section as follows:
  
  Select a shape for the Spark driver host.
  
  Select a shape for the Spark executor host.
  
  Enter the number of executors to launch when the Data Flow application is run.
6. Complete the Object Storage file configuration section as follows:
  
  Select the Object Storage data asset to use.
  
  Select a connection to the data asset that you have selected.
  
  Select the compartment that has the Object Storage bucket you want to use.
  
  Select the bucket to upload the JAR to.
7. (Optional) Select Show advanced options and select Secure access to private subnet. Then select a private endpoint in OCI Data Flow to use for this application.
8. Select Validate task to check the configuration for any errors that might cause the publish to fail.
9. If validation is successful, select Publish.
  A notification message appears with a View publish status link. Select the link to monitor the publish status in the Oracle Cloud Infrastructure Data Flow publish history panel. See Viewing the OCI Data Flow Publish History for a Task.
Use the oci data-integration external-application create command and required parameters to publish a task to an application in OCI Data Flow:
oci data-integration external-application create [OPTIONS]
For a complete list of flags and variable options for CLI commands, see the Command Line Reference.
Run the CreateExternalPublication operation to publish a task to an application in OCI Data Flow.

Oracle Cloud Infrastructure Documentation

Publishing a Task to OCI Data Flow