oracle.oci.oci_data_science_pipeline – Manage a Pipeline resource in Oracle Cloud Infrastructure

Note

This plugin is part of the oracle.oci collection (version 5.5.0).

You might already have this collection installed if you are using the ansible package. It is not included in ansible-core. To check whether it is installed, run ansible-galaxy collection list.

To install it, use: ansible-galaxy collection install oracle.oci.

To use it in a playbook, specify: oracle.oci.oci_data_science_pipeline.

New in version 2.9.0: of oracle.oci

Synopsis

  • This module allows the user to create, update and delete a Pipeline resource in Oracle Cloud Infrastructure

  • For state=present, creates a new Pipeline.

  • This resource has the following action operations in the oracle.oci.oci_data_science_pipeline_actions module: change_compartment.

Requirements

The below requirements are needed on the host that executes this module.

Parameters

Parameter Choices/Defaults Comments
api_user
string
The OCID of the user, on whose behalf, OCI APIs are invoked. If not set, then the value of the OCI_USER_ID environment variable, if any, is used. This option is required if the user is not specified through a configuration file (See config_file_location). To get the user's OCID, please refer https://docs.us-phoenix-1.oraclecloud.com/Content/API/Concepts/apisigningkey.htm.
api_user_fingerprint
string
Fingerprint for the key pair being used. If not set, then the value of the OCI_USER_FINGERPRINT environment variable, if any, is used. This option is required if the key fingerprint is not specified through a configuration file (See config_file_location). To get the key pair's fingerprint value please refer https://docs.us-phoenix-1.oraclecloud.com/Content/API/Concepts/apisigningkey.htm.
api_user_key_file
string
Full path and filename of the private key (in PEM format). If not set, then the value of the OCI_USER_KEY_FILE variable, if any, is used. This option is required if the private key is not specified through a configuration file (See config_file_location). If the key is encrypted with a pass-phrase, the api_user_key_pass_phrase option must also be provided.
api_user_key_pass_phrase
string
Passphrase used by the key referenced in api_user_key_file, if it is encrypted. If not set, then the value of the OCI_USER_KEY_PASS_PHRASE variable, if any, is used. This option is required if the key passphrase is not specified through a configuration file (See config_file_location).
auth_purpose
string
    Choices:
  • service_principal
The auth purpose which can be used in conjunction with 'auth_type=instance_principal'. The default auth_purpose for instance_principal is None.
auth_type
string
    Choices:
  • api_key ←
  • instance_principal
  • instance_obo_user
  • resource_principal
  • security_token
The type of authentication to use for making API requests. By default auth_type="api_key" based authentication is performed and the API key (see api_user_key_file) in your config file will be used. If this 'auth_type' module option is not specified, the value of the OCI_ANSIBLE_AUTH_TYPE, if any, is used. Use auth_type="instance_principal" to use instance principal based authentication when running ansible playbooks within an OCI compute instance.
cert_bundle
string
The full path to a CA certificate bundle to be used for SSL verification. This will override the default CA certificate bundle. If not set, then the value of the OCI_ANSIBLE_CERT_BUNDLE variable, if any, is used.
compartment_id
string
The OCID of the compartment where you want to create the pipeline.
Required for create using state=present.
Required for update when environment variable OCI_USE_NAME_AS_IDENTIFIER is set.
Required for delete when environment variable OCI_USE_NAME_AS_IDENTIFIER is set.
config_file_location
string
Path to configuration file. If not set then the value of the OCI_CONFIG_FILE environment variable, if any, is used. Otherwise, defaults to ~/.oci/config.
config_profile_name
string
The profile to load from the config file referenced by config_file_location. If not set, then the value of the OCI_CONFIG_PROFILE environment variable, if any, is used. Otherwise, defaults to the "DEFAULT" profile in config_file_location.
configuration_details
dictionary
This parameter is updatable.
command_line_arguments
string
The command line arguments to set for steps in the pipeline.
environment_variables
dictionary
Environment variables to set for steps in the pipeline.
maximum_runtime_in_minutes
integer
A time bound for the execution of the entire Pipeline. Timer starts when the Pipeline Run is in progress.
type
string / required
    Choices:
  • DEFAULT
The type of pipeline.
defined_tags
dictionary
Defined tags for this resource. Each key is predefined and scoped to a namespace. See Resource Tags. Example: `{"Operations": {"CostCenter": "42"}}`
This parameter is updatable.
delete_related_job_runs
boolean
    Choices:
  • no
  • yes
A boolean value to specify whether to delete related jobRuns or not.
delete_related_pipeline_runs
boolean
    Choices:
  • no
  • yes
A boolean value to specify whether to delete related PipelineRuns or not.
description
string
A short description of the pipeline.
This parameter is updatable.
display_name
string
A user-friendly display name for the resource.
Required for create, update, delete when environment variable OCI_USE_NAME_AS_IDENTIFIER is set.
This parameter is updatable when OCI_USE_NAME_AS_IDENTIFIER is not set.

aliases: name
force_create
boolean
    Choices:
  • no ←
  • yes
Whether to attempt non-idempotent creation of a resource. By default, create resource is an idempotent operation, and doesn't create the resource if it already exists. Setting this option to true, forcefully creates a copy of the resource, even if it already exists.This option is mutually exclusive with key_by.
freeform_tags
dictionary
Free-form tags for this resource. Each tag is a simple key-value pair with no predefined name, type, or namespace. See Resource Tags. Example: `{"Department": "Finance"}`
This parameter is updatable.
infrastructure_configuration_details
dictionary
block_storage_size_in_gbs
integer / required
The size of the block storage volume to attach to the instance.
shape_config_details
dictionary
memory_in_gbs
float
A pipeline step run instance of type VM.Standard.E3.Flex allows memory to be specified. This specifies the size of the memory in GBs.
ocpus
float
A pipeline step run instance of type VM.Standard.E3.Flex allows the ocpu count to be specified.
shape_name
string / required
The shape used to launch the instance for all step runs in the pipeline.
key_by
list / elements=string
The list of attributes of this resource which should be used to uniquely identify an instance of the resource. By default, all the attributes of a resource are used to uniquely identify a resource.
log_configuration_details
dictionary
This parameter is updatable.
enable_auto_log_creation
boolean
    Choices:
  • no
  • yes
If automatic on-behalf-of log object creation is enabled for pipeline runs.
enable_logging
boolean
    Choices:
  • no
  • yes
If customer logging is enabled for pipeline.
log_group_id
string
The OCID of the log group.
log_id
string
The OCID of the log.
pipeline_id
string
The OCID of the pipeline.
Required for update using state=present when environment variable OCI_USE_NAME_AS_IDENTIFIER is not set.
Required for delete using state=absent when environment variable OCI_USE_NAME_AS_IDENTIFIER is not set.

aliases: id
project_id
string
The OCID of the project to associate the pipeline with.
Required for create using state=present.
realm_specific_endpoint_template_enabled
boolean
    Choices:
  • no
  • yes
Enable/Disable realm specific endpoint template for service client. By Default, realm specific endpoint template is disabled. If not set, then the value of the OCI_REALM_SPECIFIC_SERVICE_ENDPOINT_TEMPLATE_ENABLED variable, if any, is used.
region
string
The Oracle Cloud Infrastructure region to use for all OCI API requests. If not set, then the value of the OCI_REGION variable, if any, is used. This option is required if the region is not specified through a configuration file (See config_file_location). Please refer to https://docs.us-phoenix-1.oraclecloud.com/Content/General/Concepts/regions.htm for more information on OCI regions.
state
string
    Choices:
  • present ←
  • absent
The state of the Pipeline.
Use state=present to create or update a Pipeline.
Use state=absent to delete a Pipeline.
step_details
list / elements=dictionary
Array of step details for each step.
Required for create using state=present.
This parameter is updatable.
depends_on
list / elements=string
The list of step names this current step depends on for execution.
description
string
A short description of the step.
This parameter is updatable.
is_artifact_uploaded
boolean
    Choices:
  • no
  • yes
A flag to indicate whether the artifact has been uploaded for this step or not.
Applicable when step_type is 'CUSTOM_SCRIPT'
job_id
string
The OCID of the job to be used as a step.
Required when step_type is 'ML_JOB'
step_configuration_details
dictionary
command_line_arguments
string
The command line arguments to set for step.
Applicable when step_type is 'ML_JOB'
environment_variables
dictionary
Environment variables to set for step.
Applicable when step_type is 'ML_JOB'
maximum_runtime_in_minutes
integer
A time bound for the execution of the step.
Applicable when step_type is 'ML_JOB'
step_infrastructure_configuration_details
dictionary
Applicable when step_type is 'CUSTOM_SCRIPT'
block_storage_size_in_gbs
integer / required
The size of the block storage volume to attach to the instance.
Required when step_type is 'CUSTOM_SCRIPT'
shape_config_details
dictionary
Applicable when step_type is 'CUSTOM_SCRIPT'
memory_in_gbs
float
A pipeline step run instance of type VM.Standard.E3.Flex allows memory to be specified. This specifies the size of the memory in GBs.
Applicable when step_type is 'CUSTOM_SCRIPT'
ocpus
float
A pipeline step run instance of type VM.Standard.E3.Flex allows the ocpu count to be specified.
Applicable when step_type is 'CUSTOM_SCRIPT'
shape_name
string / required
The shape used to launch the instance for all step runs in the pipeline.
Required when step_type is 'CUSTOM_SCRIPT'
step_name
string / required
The name of the step. It must be unique within the pipeline. This is used to create the pipeline DAG.
This parameter is updatable.
step_type
string / required
    Choices:
  • ML_JOB
  • CUSTOM_SCRIPT
The type of step.
This parameter is updatable.
tenancy
string
OCID of your tenancy. If not set, then the value of the OCI_TENANCY variable, if any, is used. This option is required if the tenancy OCID is not specified through a configuration file (See config_file_location). To get the tenancy OCID, please refer https://docs.us-phoenix-1.oraclecloud.com/Content/API/Concepts/apisigningkey.htm
wait
boolean
    Choices:
  • no
  • yes ←
Whether to wait for create or delete operation to complete.
wait_timeout
integer
Time, in seconds, to wait when wait=yes. Defaults to 1200 for most of the services but some services might have a longer wait timeout.

Examples

- name: Create pipeline
  oci_data_science_pipeline:
    # required
    project_id: "ocid1.project.oc1..xxxxxxEXAMPLExxxxxx"
    compartment_id: "ocid1.compartment.oc1..xxxxxxEXAMPLExxxxxx"
    step_details:
    - # required
      job_id: "ocid1.job.oc1..xxxxxxEXAMPLExxxxxx"
      step_type: ML_JOB
      step_name: step_name_example

      # optional
      depends_on: [ "depends_on_example" ]
      description: description_example
      step_configuration_details:
        # optional
        maximum_runtime_in_minutes: 56
        environment_variables: null
        command_line_arguments: command_line_arguments_example

    # optional
    infrastructure_configuration_details:
      # required
      shape_name: shape_name_example
      block_storage_size_in_gbs: 56

      # optional
      shape_config_details:
        # optional
        ocpus: 3.4
        memory_in_gbs: 3.4
    display_name: display_name_example
    description: description_example
    configuration_details:
      # required
      type: DEFAULT

      # optional
      maximum_runtime_in_minutes: 56
      environment_variables: null
      command_line_arguments: command_line_arguments_example
    log_configuration_details:
      # optional
      enable_logging: true
      enable_auto_log_creation: true
      log_group_id: "ocid1.loggroup.oc1..xxxxxxEXAMPLExxxxxx"
      log_id: "ocid1.log.oc1..xxxxxxEXAMPLExxxxxx"
    freeform_tags: {'Department': 'Finance'}
    defined_tags: {'Operations': {'CostCenter': 'US'}}

- name: Update pipeline
  oci_data_science_pipeline:
    # required
    pipeline_id: "ocid1.pipeline.oc1..xxxxxxEXAMPLExxxxxx"

    # optional
    display_name: display_name_example
    description: description_example
    configuration_details:
      # required
      type: DEFAULT

      # optional
      maximum_runtime_in_minutes: 56
      environment_variables: null
      command_line_arguments: command_line_arguments_example
    log_configuration_details:
      # optional
      enable_logging: true
      enable_auto_log_creation: true
      log_group_id: "ocid1.loggroup.oc1..xxxxxxEXAMPLExxxxxx"
      log_id: "ocid1.log.oc1..xxxxxxEXAMPLExxxxxx"
    step_details:
    - # required
      job_id: "ocid1.job.oc1..xxxxxxEXAMPLExxxxxx"
      step_type: ML_JOB
      step_name: step_name_example

      # optional
      depends_on: [ "depends_on_example" ]
      description: description_example
      step_configuration_details:
        # optional
        maximum_runtime_in_minutes: 56
        environment_variables: null
        command_line_arguments: command_line_arguments_example
    freeform_tags: {'Department': 'Finance'}
    defined_tags: {'Operations': {'CostCenter': 'US'}}

- name: Update pipeline using name (when environment variable OCI_USE_NAME_AS_IDENTIFIER is set)
  oci_data_science_pipeline:
    # required
    compartment_id: "ocid1.compartment.oc1..xxxxxxEXAMPLExxxxxx"
    display_name: display_name_example

    # optional
    description: description_example
    configuration_details:
      # required
      type: DEFAULT

      # optional
      maximum_runtime_in_minutes: 56
      environment_variables: null
      command_line_arguments: command_line_arguments_example
    log_configuration_details:
      # optional
      enable_logging: true
      enable_auto_log_creation: true
      log_group_id: "ocid1.loggroup.oc1..xxxxxxEXAMPLExxxxxx"
      log_id: "ocid1.log.oc1..xxxxxxEXAMPLExxxxxx"
    step_details:
    - # required
      job_id: "ocid1.job.oc1..xxxxxxEXAMPLExxxxxx"
      step_type: ML_JOB
      step_name: step_name_example

      # optional
      depends_on: [ "depends_on_example" ]
      description: description_example
      step_configuration_details:
        # optional
        maximum_runtime_in_minutes: 56
        environment_variables: null
        command_line_arguments: command_line_arguments_example
    freeform_tags: {'Department': 'Finance'}
    defined_tags: {'Operations': {'CostCenter': 'US'}}

- name: Delete pipeline
  oci_data_science_pipeline:
    # required
    pipeline_id: "ocid1.pipeline.oc1..xxxxxxEXAMPLExxxxxx"
    state: absent

    # optional
    delete_related_pipeline_runs: true
    delete_related_job_runs: true

- name: Delete pipeline using name (when environment variable OCI_USE_NAME_AS_IDENTIFIER is set)
  oci_data_science_pipeline:
    # required
    compartment_id: "ocid1.compartment.oc1..xxxxxxEXAMPLExxxxxx"
    display_name: display_name_example
    state: absent

Return Values

Common return values are documented here, the following are the fields unique to this module:

Key Returned Description
pipeline
complex
on success
Details of the Pipeline resource acted upon by the current operation

Sample:
{'compartment_id': 'ocid1.compartment.oc1..xxxxxxEXAMPLExxxxxx', 'configuration_details': {'command_line_arguments': 'command_line_arguments_example', 'environment_variables': {}, 'maximum_runtime_in_minutes': 56, 'type': 'DEFAULT'}, 'created_by': 'created_by_example', 'defined_tags': {'Operations': {'CostCenter': 'US'}}, 'description': 'description_example', 'display_name': 'display_name_example', 'freeform_tags': {'Department': 'Finance'}, 'id': 'ocid1.resource.oc1..xxxxxxEXAMPLExxxxxx', 'infrastructure_configuration_details': {'block_storage_size_in_gbs': 56, 'shape_config_details': {'memory_in_gbs': 3.4, 'ocpus': 3.4}, 'shape_name': 'shape_name_example'}, 'lifecycle_details': 'lifecycle_details_example', 'lifecycle_state': 'CREATING', 'log_configuration_details': {'enable_auto_log_creation': True, 'enable_logging': True, 'log_group_id': 'ocid1.loggroup.oc1..xxxxxxEXAMPLExxxxxx', 'log_id': 'ocid1.log.oc1..xxxxxxEXAMPLExxxxxx'}, 'project_id': 'ocid1.project.oc1..xxxxxxEXAMPLExxxxxx', 'step_details': [{'depends_on': [], 'description': 'description_example', 'is_artifact_uploaded': True, 'job_id': 'ocid1.job.oc1..xxxxxxEXAMPLExxxxxx', 'step_configuration_details': {'command_line_arguments': 'command_line_arguments_example', 'environment_variables': {}, 'maximum_runtime_in_minutes': 56}, 'step_infrastructure_configuration_details': {'block_storage_size_in_gbs': 56, 'shape_config_details': {'memory_in_gbs': 3.4, 'ocpus': 3.4}, 'shape_name': 'shape_name_example'}, 'step_name': 'step_name_example', 'step_type': 'ML_JOB'}], 'system_tags': {}, 'time_created': '2013-10-20T19:20:30+01:00', 'time_updated': '2013-10-20T19:20:30+01:00'}
 
compartment_id
string
on success
The OCID of the compartment where you want to create the pipeline.

Sample:
ocid1.compartment.oc1..xxxxxxEXAMPLExxxxxx
 
configuration_details
complex
on success

   
command_line_arguments
string
on success
The command line arguments to set for steps in the pipeline.

Sample:
command_line_arguments_example
   
environment_variables
dictionary
on success
Environment variables to set for steps in the pipeline.
   
maximum_runtime_in_minutes
integer
on success
A time bound for the execution of the entire Pipeline. Timer starts when the Pipeline Run is in progress.

Sample:
56
   
type
string
on success
The type of pipeline.

Sample:
DEFAULT
 
created_by
string
on success
The OCID of the user who created the pipeline.

Sample:
created_by_example
 
defined_tags
dictionary
on success
Defined tags for this resource. Each key is predefined and scoped to a namespace. See Resource Tags. Example: `{"Operations": {"CostCenter": "42"}}`

Sample:
{'Operations': {'CostCenter': 'US'}}
 
description
string
on success
A short description of the pipeline.

Sample:
description_example
 
display_name
string
on success
A user-friendly display name for the resource.

Sample:
display_name_example
 
freeform_tags
dictionary
on success
Free-form tags for this resource. Each tag is a simple key-value pair with no predefined name, type, or namespace. See Resource Tags. Example: `{"Department": "Finance"}`

Sample:
{'Department': 'Finance'}
 
id
string
on success
The OCID of the pipeline.

Sample:
ocid1.resource.oc1..xxxxxxEXAMPLExxxxxx
 
infrastructure_configuration_details
complex
on success

   
block_storage_size_in_gbs
integer
on success
The size of the block storage volume to attach to the instance.

Sample:
56
   
shape_config_details
complex
on success

     
memory_in_gbs
float
on success
A pipeline step run instance of type VM.Standard.E3.Flex allows memory to be specified. This specifies the size of the memory in GBs.

Sample:
3.4
     
ocpus
float
on success
A pipeline step run instance of type VM.Standard.E3.Flex allows the ocpu count to be specified.

Sample:
3.4
   
shape_name
string
on success
The shape used to launch the instance for all step runs in the pipeline.

Sample:
shape_name_example
 
lifecycle_details
string
on success
A message describing the current state in more detail. For example, can be used to provide actionable information for a resource in 'Failed' state.

Sample:
lifecycle_details_example
 
lifecycle_state
string
on success
The current state of the pipeline.

Sample:
CREATING
 
log_configuration_details
complex
on success

   
enable_auto_log_creation
boolean
on success
If automatic on-behalf-of log object creation is enabled for pipeline runs.

Sample:
True
   
enable_logging
boolean
on success
If customer logging is enabled for pipeline.

Sample:
True
   
log_group_id
string
on success
The OCID of the log group.

Sample:
ocid1.loggroup.oc1..xxxxxxEXAMPLExxxxxx
   
log_id
string
on success
The OCID of the log.

Sample:
ocid1.log.oc1..xxxxxxEXAMPLExxxxxx
 
project_id
string
on success
The OCID of the project to associate the pipeline with.

Sample:
ocid1.project.oc1..xxxxxxEXAMPLExxxxxx
 
step_details
complex
on success
Array of step details for each step.
   
depends_on
list / elements=string
on success
The list of step names this current step depends on for execution.
   
description
string
on success
A short description of the step.

Sample:
description_example
   
is_artifact_uploaded
boolean
on success
A flag to indicate whether the artifact has been uploaded for this step or not.

Sample:
True
   
job_id
string
on success
The OCID of the job to be used as a step.

Sample:
ocid1.job.oc1..xxxxxxEXAMPLExxxxxx
   
step_configuration_details
complex
on success

     
command_line_arguments
string
on success
The command line arguments to set for step.

Sample:
command_line_arguments_example
     
environment_variables
dictionary
on success
Environment variables to set for step.
     
maximum_runtime_in_minutes
integer
on success
A time bound for the execution of the step.

Sample:
56
   
step_infrastructure_configuration_details
complex
on success

     
block_storage_size_in_gbs
integer
on success
The size of the block storage volume to attach to the instance.

Sample:
56
     
shape_config_details
complex
on success

       
memory_in_gbs
float
on success
A pipeline step run instance of type VM.Standard.E3.Flex allows memory to be specified. This specifies the size of the memory in GBs.

Sample:
3.4
       
ocpus
float
on success
A pipeline step run instance of type VM.Standard.E3.Flex allows the ocpu count to be specified.

Sample:
3.4
     
shape_name
string
on success
The shape used to launch the instance for all step runs in the pipeline.

Sample:
shape_name_example
   
step_name
string
on success
The name of the step. It must be unique within the pipeline. This is used to create the pipeline DAG.

Sample:
step_name_example
   
step_type
string
on success
The type of step.

Sample:
ML_JOB
 
system_tags
dictionary
on success
Usage of system tag keys. These predefined keys are scoped to namespaces. Example: `{"orcl-cloud": {"free-tier-retained": "true"}}`
 
time_created
string
on success
The date and time the resource was created in the timestamp format defined by RFC3339. Example: 2020-08-06T21:10:29.41Z

Sample:
2013-10-20T19:20:30+01:00
 
time_updated
string
on success
The date and time the resource was updated in the timestamp format defined by RFC3339. Example: 2020-08-06T21:10:29.41Z

Sample:
2013-10-20T19:20:30+01:00


Authors

  • Oracle (@oracle)