Oracle Cloud Infrastructure Documentation

Adding Cable Connections for Peering

A peer connection between Private Cloud Appliance racks requires additional physical connections. Dedicated cabling must be installed between the spine switches and the data center network.

Racks with factory-installed software version 3.0.2-b1261765 or later already have all the necessary internal network interfaces and connections. Only external cabling for the peer connection is required. The ZFS Storage Appliances use the same physical connections for their data replication.

In existing installations where first-generation disaster recovery is configured, an active replication network between the ZFS Storage Appliances is in place. When upgrading or patching to the latest appliance software, the existing replication network remains active. The new physical connections from the spine switches are used for peering traffic only.

Data Center Cabling for Rack Peering

Direct peering between racks requires dedicated cabling for each participating system. The additional connections between the spine switches and the data center network are the physical basis on which the network tunnels of the peer connection are configured.

For the purpose of peering, port 6 on each spine switch must be connected to the data center network. To provide the required connection speed of 10 or 25 Gbps, a 4-way breakout cable is attached to spine port 6. From the breakout cable, 1 transceiver is connected to the data center network. Cabling must be identical for both spine switches.

Internal Cabling for Rack Peering

Appliance rack configurations shipped from the factory before the release of the native DR service do not have the required internal cabling to enable replication through the peer connection tunnels. They lack these crucial components:

  • PCIe 25GbE network interface card in some models of the ZFS Storage Appliance Controllers

  • Ethernet cabling between ZFS Storage Appliance Controllers and spine switches (port 27)

These components can be added to existing installations, so their hardware configuration is equivalent to racks with factory-installed software version 3.0.2-b1261765 or later. Contact Oracle for assistance and additional information.

Backward Compatibility

The native DR service supports both cabling layouts:

  • Peering Topology: combined peer connection and storage replication network through the spine switches

  • Compatibility Topology: peer connection and physically separated direct replication link between the ZFS Storage Appliances

The compatibility topology provides different options for existing installations after upgrading or patching to software version 3.0.2-b1261765 or later. If you have a first-generation DR setup, you can choose to continue with this configuration, on condition that you do not establish a peer connection at the appliance level. However, Oracle recommends migrating the existing configuration to the native DR service, in accordance with your infrastructure design and maintenance schedule. Data center cabling for the peer connection must be added, but you can continue to use the existing storage replication connection. For more information, see Migrating to the Native Disaster Recovery Service.