Scenario: Sending Metrics to Object Storage

Learn how to send metrics to a bucket in Object Storage using Connector Hub.

Goal

The goal of this scenario is to archive metrics data for compute instances and block volumes to an Object Storage bucket. In this scenario, the resources are in two compartments, so you'll select metrics in both compartments.

Setting Up This Scenario

Setting up this scenario involves creating a connector to move compute instance and block volume metrics (oci_computeagent and oci_blockstore metric namespaces) to a bucket. All metrics of these namespaces are included.

You can complete these tasks in the Oracle Cloud Infrastructure Console, CLI, or API.

    1. On the Connectors list page, select Create connector. If you need help finding the list page, see Listing Connectors.
    2. On the Create connector page, enter a user-friendly name for the new connector and an optional description. Avoid entering confidential information.
      Example: "Metrics to Bucket"
    3. For Resource compartment, select the compartment where you want to store the new connector.
    4. Under Configure connector, select your source and target services to move log data to a metric:
      • Source: Monitoring
      • Target: Object Storage
    5. Under Configure source, select the compute instance and block volume metrics in each compartment:
      1. Metrics compartment: The first compartment containing the compute instances and block volumes you want metrics for.

      2. Namespaces: Select oci_computeagent and oci_blockstore.

      3. Select + Another compartment to add the second compartment.

      4. Metrics compartment: The second compartment containing the compute instances and block volumes you want metrics for.

      5. Namespaces: Select oci_computeagent and oci_blockstore.

    6. Skip the optional task (Configure task).
    7. Under Configure target, select the bucket you want to move the metrics to:
      • Select the Compartment that contains the bucket.
      • Select the Bucket you want.
    8. Accept default policies: Select the Create link provided for each default policy.
    9. Select Create.

    The creation process begins, and its progress is displayed. On completion, the connector's details page opens. The new connector immediately begins moving metrics to the selected bucket. The metrics are contained in objects within the bucket.

    1. Define the Monitoring source for the connector: Save a file named source.json.

      Example source.json file
      {
        "kind": "monitoring",
        "monitoringSources": [
          {
            "compartmentId": "<first_compartment_OCID>",
            "namespaceDetails": {
              "kind": "selected",
              "namespaces": [
                {
                  "namespace": "oci_computeagent",
                  "metrics": {
                    "kind": "all"
                  }
                },
                {
                  "namespace": "oci_blockstore",
                  "metrics": {
                    "kind": "all"
                  }
                }
              ]
            }
          },
          {
            "compartmentId": "<second_compartment_OCID>",
            "namespaceDetails": {
              "kind": "selected",
              "namespaces": [
                {
                  "namespace": "oci_computeagent",
                  "metrics": {
                    "kind": "all"
                  }
                },
                {
                  "namespace": "oci_blockstore",
                  "metrics": {
                    "kind": "all"
                  }
                }
              ]
            }
          }
        ]
      }
    2. Define the Object Storage target for the connector: Save a file named target.json.

      Example target.json file
      {
        "kind": "objectStorage",
        "bucketName": "<bucket_name>",
        "objectNamePrefix": "<prefix>"
      }
    3. Create the connector using the oci sch service-connector create command:

      Example command for creating a connector
      oci sch service-connector create
       --compartment-id <compartment_OCID>
       --display-name "Metrics to Bucket"
       --source file://source.json
       --target file://target.json

      For a complete list of parameters and values for CLI commands, see the CLI Command Reference.

      For help with troubleshooting, see Troubleshooting Connectors and Troubleshooting Notifications.

    For a complete list of flags and variable options for CLI commands, see the Command Line Reference.

  • Use the CreateServiceConnector operation to create the connector.

    Example CreateServiceConnector request
    POST /20200909/serviceConnectors
    Host: service-connector-hub.us-phoenix-1.oraclecloud.com
    <authorization and other headers>
    {
      "compartmentId": "<compartment_OCID>",
      "displayName": "Metrics to Bucket",
      "source": {
        "kind": "monitoring",
        "monitoringSources": [
          {
            "compartmentId": "<first_compartment_OCID>",
            "namespaceDetails": {
              "kind": "selected",
              "namespaces": [
                {
                  "namespace": "oci_computeagent",
                  "metrics": {
                    "kind": "all"
                  }
                },
                {
                  "namespace": "oci_blockstore",
                  "metrics": {
                    "kind": "all"
                  }
                }
              ]
            }
          },
          {
            "compartmentId": "<second_compartment_OCID>",
            "namespaceDetails": {
              "kind": "selected",
              "namespaces": [
                {
                  "namespace": "oci_computeagent",
                  "metrics": {
                    "kind": "all"
                  }
                },
                {
                  "namespace": "oci_blockstore",
                  "metrics": {
                    "kind": "all"
                  }
                }
              ]
            }
          }
        ]
      },
      "target": {
        "kind": "objectStorage",
        "bucketName": "<bucket_name>",
        "objectNamePrefix": "<prefix>"
      }
    }

What to Do Next

To view objects in the Console, select the name of the bucket shown on the connector's details page. (You might need to scroll down to see the bucket name.)

For information about downloading objects, see Downloading an Object Storage Object.

If no metrics exist in the bucket, then see troubleshooting information at No Data is Being Moved. For general help with troubleshooting, see Troubleshooting Connectors.