Oracle Cloud Infrastructure Documentation

Troubleshooting and Known Issues for Exadata Services

Use the information in this section to resolve common errors and provisioning issues in your Oracle Database@Azure environments.

The issues covered in this guide do not cover general issues related to Oracle Database@Azure configuration, settings, and account setup. For more information on those topics, see Oracle Database@Azure Overview.

Troubleshooting Topics

Known Issues

Terminations and Microsoft Azure Locks for Exadata Services

Oracle advises removal of all Microsoft Azure locks on Oracle Database@Azure resources before terminating an Exadata Services resource.

For example, if you created a Microsoft Azure private endpoint, remove that resource first. If you have a policy that prevents the deletion of locked resources, the Oracle Database@Azure workflow will fail because Oracle Database@Azure cannot delete the lock.

Private DNS Fully-Qualified Domain Name (FQDN) Can't Contain More Than Four (4) Labels for an Oracle Exadata VM Cluster

When creating an Oracle Exadata VM Cluster, if you select a private DNS zone whose FQDN has more than 4 labels (delimited by a period '.'), you get the following error: 

Error returned by CreateCloudVmCluster operation in Database service. (400, InvalidParameter, false) domain name cannot contain more than 4 labels
The workaround is to rename the private DNS that caused the error, or select a different private DNS whose FQDN has four (4) or less labels.

Not Authorized Error When Private DNS With No Tags is Used for Exadata Services

When creating an Oracle Exadata VM Cluster, if you select a private DNS zone created without any tags, the default OCI tag oracle-tags is automatically generated for the VM cluster. If the tag namespace isn't authorized in the OCI tenancy, this might cause an error. 

404 NotAuthorizedOrNotFound
The workaround is to add the following policies to the OCI tenancy:
Allow any user to use tag-namespaces in tenancy where request.principal.type = ‘multicloudlink’
Allow any user to manage tag-defaults in tenancy where request.principal.type = ‘multicloudlink’

IP Address Requirement Differences for Exadata Services

There are IP address requirement differences between Oracle Database@Azure and Oracle Cloud Infrastructure (OCI).

In the Requirements for IP Address Space documentation, the following changes for Oracle Database@Azure must be considered.
  • Oracle Database@Azure only supports Exadata X9M. All other shapes are unsupported.
  • Oracle Database@Azure reserves 13 IP addresses for the client subnet versus 3 for OCI requirements.

Maximum Transmission Unit (MTU) Differences for Exadata Services

Maximum transmission unit (MTU) is the largest frame size (in bytes) that can be sent. It is a configurable setting.

  • Azure default MTU size = 1500 bytes
  • OCI default MTU size = 9000 bytes

Due to the differences in default MTU sizes, Path MTU Discovery (PMTUD) is required to determine the proper MTU between Azure resources and OCI services. For PMTUD to work, customers must allow Internet Control Message Protocol (ICMP) Type 3 code 4 on the entire network stack. Turning off ICMP completely can result in connectivity issues.

For more information, see the following:

Private DNS Zone Limitation for Exadata Services

When provisioning Exadata Services, a private DNS zone can only select zones with four labels or less.

For example, a.b.c.d is allowed, while a.b.c.d.e is not allowed.

Automatic Network Ingress Configuration for Exadata Services

You can connect a Microsoft Azure VM to an Oracle Exadata VM Cluster if both are in the same virtual network (VNet). The functionality is automatic and requires no additional changes to network security group (NSG) rules. If you need to connect an Microsoft Azure VM from a different VNet than the one where the Oracle Exadata VM Cluster was created, an additional step to configure NSG traffic rules to allow the other VNet's traffic to flow properly.

As an example, if you have two (2) VNets (A and B) with VNet A serving the Microsoft Azure VM and VNet B serving the Oracle Exadata VM Cluster, you need to add VNet A's CIDR address to the NSG route table in OCI.

Table 1-2 Default Client NSG Rules

Direction Source or Destination Protocol Details Description

Direction: Egress

Stateless: No

Destination Type: CIDR

Destination: 0.0.0.0/0

All Protocols Allow: All traffic for all ports Default NSG egress rule

Direction: Ingress

Stateless: No

Source Type: CIDR

Source: Microsoft Azure VNet CIDR

TCP

Source Port Range: All

Destination Port Range: All

Allow: TCP traffic for ports: All

Ingress all TCP from Microsoft Azure VNet.

Direction: Ingress

Stateless: No

Source Type: CIDR

Source: Microsoft AzureVNet CIDR

ICMP

Type: All

Code: All

Allow: ICMP traffic for: All

Ingress all ICMP from Microsoft Azure VNet.

Table 1-3 Default Backup NSG Rules

Direction Source or Destination Protocol Details Description

Direction: Egress

Stateless: No

Destination Type: Service

Destination: OCI IAD object storage

TCP

Source Port Range: All

Destination Port Range: 443

Allow: TCP traffic for ports: 443 HTTPS

Allows access to object storage.

Direction: Ingress

Stateless: No

Source Type: CIDR

Source: 0.0.0.0/0

ICMP

Type: 3

Code: 4

Allow: ICMP traffic for: 3, 4 Destination Unreachable: Fragmentation Needed and Don't Fragment was Set

Allows Path MTU Discovery fragmentation messages.

Oracle Exadata VM Cluster Provisioning Fails with Authorization Error

Provisioning an Oracle Exadata VM Cluster fails with an authorization error as follows: 

Authorization Failed
The client <client_name> with object id <object_id> does not have authorization to perform action 'Oracle.Database/location/operationStatuses/read' over scope <scope_details> or scope is invalid. If access was recently granted, please refresh your credentials.
The failure occurs because the user performing the action doesn't have Azure permissions for the Microsoft.BareMetal/BareMetalConnections resource.
Workaround
  1. Ensure that no Azure policy assigned to the user or the subscription is preventing the user from performing the action. If the user has specific permissions assigned to them directly, add the following resources to the authorized list.
    • Microsoft.BareMetal/BareMetalConnections
    • Microsoft.Network/privateDnsZones
  2. Delete the failed Exadata VM Cluster from the Azure UI.
  3. Wait 30 minutes to ensure that the Exadata VM Cluster is fully terminated in both Azure and OCI. This wait period ensures that all dependent resources are also deleted.
  4. Provision your new Exadata VM Cluster.

Cannot Delete Exadata Infrastructure

Azure has a targeted timeout of up to one (1) hour for the deletion of an Exadata VM Cluster. It may happen that this time is not enough for all the dependent resources in Exadata Infrastructure to be totally deleted, in particular network resources.

Azure may change the Exadata VM Cluster status to TERMINATED, but in OCI the dependent resources may be still being removed. In this situation, If the customer tries to delete the Exadata VM Infrastructure AND the deletion of the Exa VM Cluster is not finished on the OCI side, the following error message will appear:
Cannot delete Exadata infrastructure. All associated VM clusters must be deleted before you delete the Exadata infrastructure. (Code: 400)
The workaround is to wait until all the Exadata VM Clusters associated with the Exadata Infrastructure have been successfully terminated in OCI.

"Scaling In Progress" Status Doesn't Display for Pluggable Database (PDB)

Pluggable databases (PDBs) don't display a "Scaling in progress" status while a scaling operation is in progress.

Instead, the PDB status goes from "Accepted" to "Updating". No workaround available.

Oracle Exadata VM Cluster Provisioning Fails Because Number of Available IPs Reported by OCI and Azure Doesn't Match

Azure reports the wrong number of available IPs in the subnet, causing Oracle Exadata VM Cluster provisioning to fail. This causes the following error: 

Error returned by CreateCloudVmCluster operation in Database service.(400, InvalidParameter, false) Cidr block of the subnet must have at least 11 ip addresses available.
Verify the correct number of available IP addresses in the subnet using the OCI console. For more information, see Listing Private IP Addresses. The workaround is to reconfigure the subnet according to the prerequisites.

Metrics Reporting Losses in Microsoft Azure

If any exporter sends a metric that is older than 20 minutes, the agents drops it and customers lose that data.

This 20 minute time-out is a limitation of Microsoft Azure, and applies to any exporter. For example, here are some scenarios where metrics may be lost.
  • Certificates expired. Exporter cannot establish connection with agent and a backlog of metrics builds. After rotating to new certificates, the metrics are stale and because they are more than 20 minutes old, the agent drops the metrics and customers will see these metrics.
  • Backlog: When the exporter, for any reason, develops a backlog that results in the exporter taking longer than usual to process the metrics, metric will be lost if they are received by the agent more than 20 minutes old.
These are not an exhaustive list of examples, but are intended to provide context to metrics reporting loss.