Oracle Cloud Infrastructure Documentation

Set Up Resource Discovery

Set up resource discovery to generate Terraform files for existing resources in your compartment.

Use the OCI resource discovery feature to:

  • Create state files for existing resources in the Console, and then add those resources to a Terraform setup.
  • Duplicate your existing infrastructure in a new tenancy or region.
  • Detect state drift for updated resources.

In this tutorial, you set up Oracle Cloud Infrastructure Terraform provider's resource discovery feature in your local environment. To confirm your setup, you run resource discovery to fetch information from your tenancy and create a script for it.

Key tasks include how to:

  • Create RSA keys.
  • Install Terraform OCI provider binaries.
  • Set up Terraform OCI provider API authentication variables.
  • Authenticate your OCI provider CLI commands.
  • Create a script in your environment, regarding the availability domains in your tenancy, through the resource discovery feature.
A diagram of a user connected from a local environment to an Oracle Cloud Infrastructure tenancy. The local environment is Linux and has Terraform OCI Provider binaries installed. There is an arrow from the local environment connected to Terraform Registry in the cloud. There is a second arrow from the local environment sending a message to the user's Oracle Cloud Infrastructure tenancy labeled Authenticate?. The third arrow is from the tenancy to the local environment labeled Fetch Data. These arrows suggest that the user has set up their OCI provider variables to be authenticated by their tenancy. The user can then fetch information from the tenancy, by using Terraform OCI provider commands and Terraform Registry. In this example, the tenancy displays three availability domains and that is the information that Resource Discovery is fetching to create Terraform scripts in the user's environment.

For more information, see Use Cases and Benefits.

Before You Begin

To successfully perform this tutorial, you must have the following:

MacOS or Linux
Windows 10

1. Prepare

Prepare your environment for authenticating and running resource discovery commands. Also, gather the information your account needs to authenticate your commands.

Create RSA Keys

You create RSA keys for API signing in to your Oracle Cloud Infrastructure account.

Note

If you're using Cloud Shell or Resource Manager, skip creating the RSA keys. You're already authenticated when you sign in to the OCI Console.
  1. Open a terminal window.
  2. Under your home directory, make an .oci directory. 
    mkdir <your-home-directory>/.oci

    Example for Oracle Linux:

    mkdir /home/opc/.oci
    Note

    If you're using Windows Subsystem for Linux (WSL), create the /.oci directory directly in the Linux environment. If you create the /.oci directory in a /mnt folder (Windows file system), you're required to use the chmod command to change permissions for the WSL configuration files.
  3. Generate a 2048-bit private key in a PEM format: 
    openssl genrsa -out <your-home-directory>/.oci/<your-rsa-key-name>.pem 2048
  4. Change permissions, so only you can read and write to the private key file: 
    chmod 600 <your-home-directory>/.oci/<your-rsa-key-name>.pem
  5. Generate the public key: 
    openssl rsa -pubout -in <your-home-directory>/.oci/<your-rsa-key-name>.pem -out $HOME/.oci/<your-rsa-key-name>_public.pem
  6. Copy the public key.
    In the terminal, enter:
    cat <your-home-directory>/.oci/<your-rsa-key-name>_public.pem

    Example (excerpt):

    -----BEGIN PUBLIC KEY-----
MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAoTFqF...
...
-----END PUBLIC KEY——
  7. Add the public key to your user account.
    1. Sign in to the Oracle Cloud Console.
    2. In the navigation bar, select the Profile menu and then select User settings or My profile, depending on the option that you see.
    3. Select API keys.
    4. Select Add API key.
    5. Select Paste a public key.
    6. Paste the value from the previous step, including the lines with BEGIN PUBLIC KEY and END PUBLIC KEY.
    7. Select Add.

      The Configuration file preview dialog box opens. Example:

      [DEFAULT]
user=ocid1.user.oc1..exampleid
fingerprint=exampleid
tenancy=ocid1.tenancy.oc1..exampleid
region=us-ashburn-1
key_file=<path to your private keyfile> # TODO
    8. Select Copy, then paste into your notepad.

      The configuration file preview includes information you'll need later, such as tenancy and user OCIDs, fingerprint, and region.

You have now set up the RSA keys to connect to your OCI account.

Reference
How to Generate an API Signing Key
Add List Policy

If your username is in the Administrators group, then skip this section. Otherwise, ask your administrator to add the following policy to your tenancy:

allow group <a-group-that-your-username-belongs-to> to read all-resources in tenancy

With this privilege, you can list all the resources in your tenancy.

Steps to Add the Policy
  1. Sign in to the Oracle Cloud Console.
  2. In the navigation bar, select the Profile menu and then select User settings or My profile, depending on the option that you see.
  3. Select Groups or My groups, depending on the option that you see.
  4. In a notepad, copy the name of a group that your username belongs to.
  5. Open the navigation menu  and select Identity & Security. Under Identity, select Policies.
  6. Select the compartment: <your-tenancy>(root)
  7. Select Create Policy.
  8. On the Create Policy page, enter the following values:
    • Name: list-resources
    • Description: Allow the group <a-group-that-your-username-belongs-to> to list the resources in this tenancy.
    • Compartment: <your-tenancy>(root)
  9. For Policy Builder, select Show manual editor.
  10. Paste in the following policy:
    allow group <a-group-that-your-username-belongs-to> to read all-resources in tenancy
  11. Select Create.

Reference: Common Policies

Gather Required Information

Prepare the information you need to authenticate your Terraform OCI Provider commands and copy them into your notepad.

Note

If you're using Cloud Shell, you only need to find the <tenancy-ocid> from the following step.
  1. Collect the following credential information from the Oracle Cloud Console.
    • Tenancy OCID: <tenancy-ocid>
      • In the navigation bar, select the Profile menu and then select Tenancy: <your_tenancy_name>.
    • User OCID: <user-ocid>
      • In the navigation bar, select the Profile menu and then select User settings or My profile, depending on the option that you see.
      • Copy OCID.
    • Fingerprint: <fingerprint>
      • In the navigation bar, select the Profile menu and then select User settings or My profile, depending on the option that you see.
      • Select API Keys.
      • Copy the fingerprint associated with the RSA public key you made in the Create RSA Keys section. The format is: xx:xx:xx...xx.
    • Region: <region-identifier>
  2. Collect the following information from your environment.
    • Private Key Path: <rsa-private-key-path>
      • Path to the RSA private key you made at Create RSA Keys. Example: $HOME/.oci/<your-rsa-key-name>.pem.
Install Terraform OCI Provider
Note

If you're using Cloud Shell, skip this section.

Install the latest version of Terraform OCI Provider binaries v4.2+:

  1. From a browser, go to HashiCorp Releases.
  2. Select the latest link.
  3. Find the link for your environment and then copy the link address. Example for Linux 64 bit: 
    https://releases.hashicorp.com/terraform-provider-oci/4.59.0/terraform-provider-oci_4.59.0_linux_amd64.zip
  4. In your environment, create a temp directory and change to that directory: 
    mkdir temp
    cd temp
  5. Download the Terraform zip file. Example: 
    wget https://releases.hashicorp.com/terraform-provider-oci/4.59.0/terraform-provider-oci_4.59.0_linux_amd64.zip
  6. Unzip the file. Example: 
    unzip terraform-provider-oci_4.59.0_linux_amd64.zip
  7. Move the unzipped folder to /usr/local/bin. Example: 
    sudo mv terraform-provider-oci_v4.59.0_x4 /usr/local/bin
    Note

    Use sudo for the /usr/local/bin directory if it doesn't let you run the commands.
  8. Use a symbolic link to shorten the terraform-provider-oci_<version> command: 
    sudo ln -s <source-executable> <shorter-alias>
    cd /usr/local/bin/
    sudo ln -s terraform-provider-oci_<version> tf-oci

    For this tutorial, use tf-oci for the <shorter-alias>, instead of the <source-executable>.

  9. Go back to your home directory: 
    cd
  10. Check the Terraform OCI provider version:
    Note

    On MacOS, you need to create a security exception for the executable.
    tf-oci

    Example output: [INFO] terraform-provider-oci 4.59.0.

2. Create an Authentication Script

Create a shell script to assign authentication information to OCI provider authentication variables. Your Oracle Cloud Infrastructure account authenticates your OCI provider commands through the values assigned to these parameters.

Add Authentication
Note

If you're using Cloud Shell or Resource Manager, you don't need to add authentication. Proceed to section 3. Discover a Resource.
  1. Create an executable script file, somewhere in your PATH, and name it provider-oci.sh. For example, if ~/bin is in your PATH, the steps are as follows: 
    cd ~/bin
    touch provider-oci.sh
    chmod 755 provider-oci.sh
    ls -l

    You get something like this:

    -rwxr-xr-x 1 <your-username> <your-username> 0 <date> provider-oci.sh
  2. Add the following code to provider-oci.sh.

    Replace the fields with brackets with the information you collected at Gather Required Information.

    #!/bin/bash
export TF_VAR_tenancy_ocid="<tenancy-ocid>"
export TF_VAR_user_ocid="<user-ocid>"
export TF_VAR_fingerprint="<fingerprint>"
export TF_VAR_private_key_path="<rsa-private-key-path>"
export TF_VAR_region="<region-identifier>"
  3. Save the provider-oci.sh file.
Explanation
  • Go to Authentication (Resource Discovery).
  • Set up the following environment variables for API Key based authentication:
    • TF_VAR_tenancy_ocid
    • TF_VAR_user_ocid
    • TF_VAR_fingerprint
    • TF_VAR_private_key_path
    • TF_VAR_region
Export Environment Variables

In this section, you run the provider-oci.sh file to add the OCI provider authentication variables to your environment variables.

  1. From your $HOME/bin directory, run the provider-oci.sh file. 
    source provider-oci.sh
  2. Confirm that the OCI provider authentication variables are added to your environment variables. 
    env | sort

    The variables are displayed in alphabetical order. Example:

    ...
TF_VAR_fingerprint=<fingerprint>
TF_VAR_private_key_path=<rsa-private-key-path>
TF_VAR_region=<region-identifier>
TF_VAR_tenancy_ocid=<tenancy-ocid>
TF_VAR_user_ocid=<user-ocid>
...
  3. To load the environment variables, when a new shell starts, append the source ~/bin/provider-oci.sh command to .bashrc. 
    source ~/bin/provider-oci.sh

3. Discover a Resource

Discover the availability domains in your tenancy.

Run Resource Discovery

Every tenancy has a list of availability domains. By discovering the availability domains in your region, you confirm that:

  • Your Oracle Cloud Infrastructure account can authenticate your Oracle Cloud Infrastructure provider commands.
  • You can get information from your account with the resource discovery feature.
  1. In your $HOME directory, create a directory called resource-discovery.
    cd
    mkdir resource-discovery
  2. If you're using Cloud Shell, find the installed OCI Terraform Provider file in the /usr/bin directory.
    ls /usr/bin

    The file name is:

    terraform-provider-oci_<version>

    Sample file name:

    terraform-provider-oci_v5.2.1
  3. Run the following command:
    • In Cloud Shell:
      terraform-provider-oci_<version> -command=export -compartment_id=<tenancy-ocid> -services=availability_domain -output_path=$HOME/resource-discovery
    • On your compute instance or in your local environment:
      tf-oci -command=export -compartment_id=<tenancy-ocid> -services=availability_domain -output_path=$HOME/resource-discovery
    Important

    • Replace <tenancy-ocid> with the information from Gather Required Information.
    • If you're using Cloud Shell, replace terraform-provider-oci_<version> with the file name that you found in step 2.

    Sample output:

    ...
INFO <date> [INFO] ===> Generating resource 'oci_identity_availability_domain.export_QnsC-US-ASHBURN-AD-1'
INFO <date> [INFO] ===> Generating resource 'oci_identity_availability_domain.export_QnsC-US-ASHBURN-AD-2'
INFO <date> [INFO] ===> Generating resource 'oci_identity_availability_domain.export_QnsC-US-ASHBURN-AD-3'
...
INFO <date> Found 6 'availability_domain' resources. Generated under '/<home-directory>/resource-discovery/availability_domain.tf'.
INFO <date> === COMPLETED ===
    Note

    • The tenancy OCID is the compartment OCID for the root compartment. Providing a specific <compartment-ocid> or your <tenancy-ocid> outputs the same availability domains.
    • To discover identity resources, you don't need to mention a compartment OCID. In the previous example, you get the same result if you remove the compartment_id from the command. The compartment_id is there for you to learn the syntax for other services.
Troubleshooting
  • error message: Failed to query available provider packages:
    • If you're on a VPN, check your proxy settings.
  • 401 errors - (Service error:NotAuthenticated):
    • You have an incorrect value for one of the following:
      • tenancy OCID
      • user OCID
      • fingerprint
      • RSA private key (the path or the key)
  • no such host:
    • You have an incorrect value for the following:
      • region identifier
Explanation
  • Go to Usage (Resource Discovery).

    Two command options are listed.

  • Select the command option that uses compartment_id as a base command:
    terraform-provider-oci -command=export -compartment_id= -output_path=
  • Update the base command:
    • Instead of terraform-provider-oci, use your symbolic link command:

      tf-oci

    • Review the Parameter Description section to add proper values to the parameters in the base command:
      • Use the export command to perform resource discovery:

        -command=export

      • Add the OCID of the compartment that you're discovering resources in:

        -compartment_id=<tenancy-ocid>

      • Create and then specify a directory for the discovered resources:

        Example: -output_path=$HOME/resource-discovery

  • Search for the phrase availability_domain and observe the following information:
    Resources that are dependent on availability domains will be generated under availability_domain.tf file. These include:

    oci_core_boot_volume
    oci_file_storage_file_system
    oci_file_storage_mount_target
    oci_file_storage_snapshot
  • For services, use:
    • availability_domain (Even though it's not a service, it works with service.)
    • Example: -services=availability_domain
    • You don't need quotation marks around the service names.
  • Construct the command:

    Example:

    tf-oci -command=export -compartment_id=<tenancy-ocid> -services=availability_domain -output_path=$HOME/resource-discovery
    Note

    The resource discovery command doesn't create a directory for the discovered resources. Create a directory and specify the path in your command.
Review Discovered Resources
  1. From your $HOME directory, change to the resource-discovery directory. 
    cd resource-discovery
  2. View the contents of the directory. 
    ls

    Sample output:

    availability_domain.tf  provider.tf  vars.tf
  3. Review the Terraform script for availability domains: 
    cat availability_domain.tf

    Sample output:

    ## This configuration was generated by terraform-provider-oci

data oci_identity_availability_domain export_QnsC-US-ASHBURN-AD-1 {
  compartment_id = var.compartment_ocid
  ad_number      = "1"
}
data oci_identity_availability_domain export_QnsC-US-ASHBURN-AD-2 {
  compartment_id = var.compartment_ocid
  ad_number      = "2"
}
data oci_identity_availability_domain export_QnsC-US-ASHBURN-AD-3 {
  compartment_id = var.compartment_ocid
  ad_number      = "3"
}
resource oci_file_storage_file_system export_JennFS ... <some info>
resource oci_file_storage_snapshot export_Snapshot-1 ... <some info>
resource oci_file_storage_mount_target export_MountTarget-xxx ... <some info>
    Note

    The availability_domain.tf Terraform script uses a variable called compartment_ocid. Find the value of this variable, in vars.tf.
  4. Review the variables: 
    cat vars.tf

    Sample output:

    variable compartment_ocid { default = "<tenancy-ocid>" }
variable region { default = "us-ashburn-1" }
  5. Review the provider script information: 
    cat provider.tf

    Sample output:

    provider oci {
        region = var.region
}
    Note

    • The provider.tf denotes that you are using the Terraform OCI provider with provider oci { }.
    • The provider.tf file does not include your authentication information, because you provide authentication information through your environment variables.

Congratulations! Your Oracle Cloud Infrastructure account can now authenticate your Terraform OCI provider commands. And your environment is ready to run the resource discovery commands.

References: