Install OCI Service Operator for Kubernetes for Service Mesh

The OCI Service Operator for Kubernetes makes it easy to create, manage, and connect to OCI resources from a Kubernetes environment. Kubernetes users can simply install OCI Service Operator for Kubernetes and perform actions on OCI resources using the Kubernetes API. The OCI Service Operator for Kubernetes removes the need to use the OCI CLI or other OCI developer tools to interact with a service API.

OCI Service Operator for Kubernetes is based on the Operator Framework, an open source toolkit used to manage Operators. It uses the controller-runtime library, which provides high-level APIs and abstractions to write operational logic and also provides tools for scaffolding and code generation for Operators.

For more information on the Operator SDK, see the Operator Framework site.

Install the Operator SDK and OLM

Install the Operator SDK on your local machine using the following steps.

Install the Operator SDK
Go to the Operator SDK installation page and follow the installation instructions for your operating system.
Install the Operator Lifecycle Manager (OLM). The OLM helps users install, update, and manage the lifecycle of Kubernetes native applications (Operators) and their associated services running in clusters. Use the following command to install:
```
operator-sdk olm install --version 0.20.0
```
Note

Local access to your Kubernetes cluster must be set up on your machine before you can perform this step.

Verify your installation of OLM by first checking for all the necessary Customer Resource Definitions (CRDs) in the cluster. Run the following command.

operator-sdk olm status

The command produces output similar to the following:

INFO[0003] Fetching CRDs for version "v0.20.0"          
INFO[0003] Fetching resources for resolved version "v0.20.0" 
INFO[0021] Successfully got OLM status for version "v0.20.0" 

NAME                                            NAMESPACE    KIND                        STATUS
operatorgroups.operators.coreos.com                          CustomResourceDefinition    Installed
operatorconditions.operators.coreos.com                      CustomResourceDefinition    Installed
olmconfigs.operators.coreos.com                              CustomResourceDefinition    Installed
installplans.operators.coreos.com                            CustomResourceDefinition    Installed
clusterserviceversions.operators.coreos.com                  CustomResourceDefinition    Installed
olm-operator-binding-olm                                     ClusterRoleBinding          Installed
operatorhubio-catalog                           olm          CatalogSource               Installed
olm-operators                                   olm          OperatorGroup               Installed
aggregate-olm-view                                           ClusterRole                 Installed
catalog-operator                                olm          Deployment                  Installed
cluster                                                      OLMConfig                   Installed
operators.operators.coreos.com                               CustomResourceDefinition    Installed
olm-operator                                    olm          Deployment                  Installed
subscriptions.operators.coreos.com                           CustomResourceDefinition    Installed
aggregate-olm-edit                                           ClusterRole                 Installed
olm                                                          Namespace                   Installed
global-operators                                operators    OperatorGroup               Installed
operators                                                    Namespace                   Installed
packageserver                                   olm          ClusterServiceVersion       Installed
olm-operator-serviceaccount                     olm          ServiceAccount              Installed
catalogsources.operators.coreos.com                          CustomResourceDefinition    Installed
system:controller:operator-lifecycle-manager                 ClusterRole                 Installed

Deploy OCI Service Operator for Kubernetes

The OCI Service Operator for Kubernetes OLM bundle contains all the required details like CRDs, RBACs, and ConfigMaps. Deployment installs the OCI Service Operator for Kubernetes in the Kubernetes cluster. First, create a namespace for your operator.

kubectl create ns oci-service-operator-system

Note

If deployed in your application namespace, the operator functions normally. This option is an alternative to creating an operator namespace.

Note

For disconnected realms, build your own OSOK Operator by following the steps from https://github.com/oracle/oci-service-operator/blob/main/BUILD_OSOK_OPERATOR.md. Replace the Bundle in the following command with the one that was built manually to install OSOK Operator.

Install the OCI Service Operator for Kubernetes Operator in the Kubernetes cluster in your namespace (oci-service-operator-system) using the following command.

operator-sdk run bundle iad.ocir.io/oracle/oci-service-operator-bundle:X.X.X -n oci-service-operator-system --timeout 5m

Note

Replace X.X.X with current version of the OCI Service Operator for Kubernetes. To get the current version, go to the GitHub release site at: https://github.com/oracle/oci-service-operator/releases

Note

Users must be logged into the Oracle Registry at iad.ocir.io in Docker to run the command. To ensure you are logged in, see Pulling Images Using the Docker CLI.

The command produces output similar to the following:

INFO[0036] Successfully created registry pod: iad-ocir-io-oracle-oci-service-operator-bundle-X-X-X 
INFO[0036] Created CatalogSource: oci-service-operator-catalog 
INFO[0037] OperatorGroup "operator-sdk-og" created      
INFO[0037] Created Subscription: oci-service-operator-vX-X-X-sub 
INFO[0040] Approved InstallPlan install-tzk5f for the Subscription: oci-service-operator-vX-X-X-sub 
INFO[0040] Waiting for ClusterServiceVersion "oci-service-operator-system/oci-service-operator.vX.X.X" to reach 'Succeeded' phase 
INFO[0040]   Waiting for ClusterServiceVersion "oci-service-operator-system/oci-service-operator.vX.X.X" to appear 
INFO[0048]   Found ClusterServiceVersion "oci-service-operator-system/oci-service-operator.vX.X.X" phase: Pending 
INFO[0049]   Found ClusterServiceVersion "oci-service-operator-system/oci-service-operator.vX.X.X" phase: InstallReady 
INFO[0053]   Found ClusterServiceVersion "oci-service-operator-system/oci-service-operator.vX.X.X" phase: Installing 
INFO[0066]   Found ClusterServiceVersion "oci-service-operator-system/oci-service-operator.vX.X.X" phase: Succeeded 
INFO[0067] OLM has successfully installed "oci-service-operator.vX.X.X"

To verify the installation, perform the following steps.

Check for custom definitions with kubectl.

$ kubectl get crds | grep "servicemesh.oci.oracle.com" | awk -F. '{print $1}'
accesspolicies
ingressgatewaydeployments
ingressgatewayroutetables
ingressgateways
meshes
virtualdeploymentbindings
virtualdeployments
virtualserviceroutetables
virtualservices

Verify the successful installation of webhooks corresponding to all the Service Mesh resources with kubectl.

First, check validating webhooks.

$ kubectl get validatingwebhookconfiguration | grep "servicemesh.oci.oracle.cloud.com" | awk -F. '{print $1}'
ap-validator
ig-validator
igd-validator
igrt-validator
mesh-validator
vd-validator
vdb-validator
vs-validator
vsrt-validator

Next, check mutating webhooks.

$ kubectl get mutatingwebhookconfiguration | grep "servicemesh.oci.oracle.com" | awk -F. '{print $1}'
proxy-injector

For more information on troubleshooting, see: Troubleshooting Service Mesh.

Install Metrics Server

If you are going to use Ingress Gateway, install metrics server. The Kubernetes Horizontal Pod Autoscalar uses the metrics server to scale the number of pods in the ingress gateway based on CPU usage. The metrics server is installed using the following command:

kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/high-availability.yaml

Bring your own Certificate for Service Operator (Optional)

As an alternative, bring your own certificate (BYOC) to secure communications. Kubernetes provides a certificates.k8s.io API, which lets you provision TLS certificates signed by a Certificate Authority (CA) that you control. The CA and certificates are used by your workloads to establish trust. To establish trust between OCI Service Operator for Kubernetes and Kubernetes, use certificates.k8s.io to create the CA and certificates. Patch the oci-service-operator-controller-manager-service-cert secret to use the new CA and certificates.

For more information, see: Manage TLS Certificates in a Cluster

To generate the CA and CertificateBundle use tools like cfssl(Cloudfare SSL) or OpenSSL. To use a user created CA, install the default configuration and patch the kubernetes.io/tls generated by OLM for OCI Service Operator for Kubernetes. The following steps demonstrate the configuration steps using CFSSL.

Note

In the following steps, replace {operators-namespace} with the namespace your operator runs in.

Create a Certificate Signing Request(CSR) and private key with cfssl following steps:
Note

For more information on these steps, see: Create a certificate signing request.
```
cat <<EOF | cfssl genkey - | cfssljson -bare server
{
  "hosts": [
    "oci-service-operator-webhook-service.{operators-namespace}.svc.cluster.local",
    "oci-service-operator-controller-manager-service.{operators-namespace}.svc.cluster.local"
  ],
  "CN": "kube.operator.cert",
  "key": {
    "algo": "ecdsa",
    "size": 256
  }
}
EOF
```
This command generates two files. The server.csr file contains the PEM encoded CSR. The server-key.pem file contains the PEM encoded key to the certificate that is yet to be created.
Create CSR manifest and send it to Kubernetes following section.
Note

For more information on these steps, see: Create a CertificateSigningRequest object to send to the Kubernetes API.
```
cat <<EOF | kubectl apply -f -
apiVersion: certificates.k8s.io/v1
kind: CertificateSigningRequest
metadata:
  name: oci-service-operator-controller.{operators-namespace}
spec:
  request: $(cat server.csr | base64 | tr -d '\n')
  signerName: oci.com
  usages:
  - digital signature
  - key encipherment
  - server auth
EOF
```
After running this command, the CSR created is in a pending state.
```
kubectl describe csr {csr-name}
```
Approve the CSR.
```
kubectl certificate approve {csr-name}
```
Validate the state after approving. Confirm that the value is: CONDITION-Approved.

Sign the CSR.

Note

For more information on these steps, see: Sign the CertificateSigningRequest.

Create a certificate authority key file (ca-key.pem) and certificate (ca.pem).

cat <<EOF | cfssl gencert -initca - | cfssljson -bare ca
{
  "CN": "My Example Signer",
  "key": {
    "algo": "rsa",
    "size": 2048
  }
}
EOF

Issue a certificate by creating server signing request. Create the server-signing-config.json file.

{
    "signing": {
        "default": {
            "usages": [
                "digital signature",
                "key encipherment",
                "server auth"
            ],
            "expiry": "876000h",
            "ca_constraint": {
                "is_ca": false
            }
        }
    }
}

Use the server-signing-config.json signing configuration, the certificate authority key file, and certificate file to sign the certificate request:

kubectl get csr oci-service-operator-controller.{operators-namespace} -o jsonpath='{.spec.request}' | \
  base64 --decode | \
  cfssl sign -ca ca.pem -ca-key ca-key.pem -config server-signing-config.json - | \
  cfssljson -bare ca-signed-server

Upload signed certificate.

kubectl get csr oci-service-operator-controller.{operators-namespace} -o json | \
  jq '.status.certificate = "'$(base64 ca-signed-server.pem | tr -d '\n')'"' | \
  kubectl replace --raw /apis/certificates.k8s.io/v1/certificatesigningrequests/oci-service-operator-controller.{operators-namespace}/status -f -

After applying the preceding change, validate the CSR condition is "Approved, Issued."

Download the certificate.

kubectl get csr {csr-name} -o jsonpath='{.status.certificate}' \
    | base64 --decode > server.crt

Running preceding command creates the server.crt file.

Encode and patch server.crt from step (3)(d), the ca-key.pem file from step (3)(a), and the server-key.pem file from step (1).

kubectl patch secret oci-service-operator-controller-manager-service-cert --namespace {operators-namespace} --type merge --patch }}{{'{"data":  "olmCAKey": "'$(base64 ca.pem | tr -d }}{{'\n')'", "tls.key": "'$(base64 server-key.pem | tr -d }}{{'\n')'", "tls.crt": "'$(base64 ca-signed-server.pem | tr -d }}{{'\n')'"}'

Validate oci-service-operator-controller-manager-service-cert contains the latest values.
With the preceding steps complete, the resources are created and working. However, if you get an error: x509: certificate signed by unknown authority, review the following steps to troubleshoot.
1. Validate if the CSR has a condition of "Approved, Issued." Otherwise, start again from step (3)(c).
2. Validate if base64 decoded olmCAKey matches the server.crt from step (3)(d). Otherwise, update the olmCAKey with the server.crt file.
3. Validate if base64 decoded tls.key matches the server-key.pem file from step (1). Otherwise, update the tls.key with the server-key.pem file.
4. Validate if base64 decoded tls.crt matches the ca-key.pem file from step (3)(a). Otherwise, update tls.crt with the ca-key.pem file.

Set up Resources Required for Service Mesh

Set up Steps

To start using Service Mesh, you have a few requirements to fulfill before you can create your own meshes.

If you are not an administrator, ask your administrator to give you access to resources of three services: Service Mesh, Vault, and Certificates. Your administrator can give you access to these services with Identity and Access Management (IAM) policies.

To prepare your tenancy to use Service Mesh, perform the following steps with your OCI account.

Create a vault to store the encryption key for your certificate authority. See Managing Vaults for details.
Add a Hardware Security Module (HSM) encryption key for your vault. See Managing Keys for details. For your certificate authority key, you can use a Rivest-Shamir-Adleman (RSA) 2,048-bit key or 4,096-bit key.
Create a certificate authority. See Managing Certificate Authorities for details.
Note

The Maximum Validity Duration for Certificates (Days) setting on the certificate authority must be at least 45 days.
Create a TLS certificate so clients can make TLS connections to your application. This certificate keeps HTTP requests to ingress gateways TLS encrypted. See Managing Certificates for details.

More Information

For more information on IAM policies related to Service Mesh, see:

Note

If you need to remove Service Mesh support from your application, see the Unmeshify Your Application (Optional) section in Service Mesh Overview Tutorial.

Next: Set up Policies required for Service Mesh

Oracle Cloud Infrastructure Documentation

Install OCI Service Operator for Kubernetes for Service Mesh

Install the Operator SDK and OLM 🔗

Deploy OCI Service Operator for Kubernetes 🔗

Install Metrics Server 🔗