Authorizing Pods to Access Non-OCI Resources Using OpenID Connect (OIDC) Discovery
Find out about using OpenID Connect (OIDC) Discovery to authenticate application pods running on clusters you create with Container Engine for Kubernetes (OKE), so that the pods can call service APIs on external cloud providers.
You might want application pods running on a Kubernetes cluster you've created with Kubernetes Engine to communicate with cloud service APIs hosted on external cloud providers (such as GCP, AWS, and Azure). To ensure the security of the resources hosted by an external cloud provider, the application running on the cluster must authenticate and manage identity. However, managing credentials directly can be a cumbersome process for applications, and key rotation is a manual process with considerable overhead.
OpenID Connect (OIDC) is an industry standard to make such integrations more straightforward. OpenID Connect is an identity layer built on top of OAuth 2.0. OpenID Connect supports a discovery protocol (often referred to as "OIDC Discovery") that uses an OpenID Provider Configuration document (known as the "discovery document") to authenticate applications.
Kubernetes Engine provides support for OIDC Discovery, enabling you to build applications that interact with other cloud services without the need to hard code and manually rotate API authentication keys. You can optionally enable OIDC Discovery when you create or update an enhanced cluster with Kubernetes Engine.
At a high level, when you enable OIDC Discovery for a cluster, the application's service account token is authenticated and (if valid) exchanged for an access token. The access token is then used to authenticate the application with the API on the external cloud provider.
For more information, see Service account issuer discovery in the Kubernetes documentation.
OIDC Discovery in more detail
Kubernetes gives a service account to every application pod running on a cluster, and creates a service account token for each service account. Kubernetes manages the lifecycle of service account tokens. The service account token is a bearer token that the application attaches to API requests, and is structured as a JSON Web Token (JWT).
When you enable OIDC Discovery for a cluster, Kubernetes Engine exposes an OIDC Issuer, and adds the URL of the OIDC Issuer to the service account token as the value of the iss claim.
The location of an OIDC discovery document is relative to the OIDC Issuer's URL, and is a publicly accessible and unauthenticated well-known endpoint. The OIDC discovery document includes the publicly accessible and unauthenticated location of a JSON Web Key Set (JWKS) that contains the public keys to verify the service account token. The external cloud provider's identity provider uses the OIDC discovery document to locate the JWKS, and uses the JWKS to verify the service account token.
For additional security, the cloud service API called by an application pod might require a particular audience to be present in the service account token that Kubernetes creates. If the cloud service API does require a particular audience, you can specify that audience by setting the audience property of a projected volume in the pod manifest. The audience you specify is included as the value of the aud claim in the service account token, and can then be verified by the external cloud provider. You can also specify a validity period for the service account token by setting the value of the expirationSeconds property of the projected volume in the pod manifest. For more information see ServiceAccount token volume projection in the Kubernetes documentation.
If the external cloud provider's identity provider successfully verifies the service account token using the JWKS, the identity provider exchanges the service account token for an access token. The access token is then used to authenticate the application, and authorize the application to use the API on the external cloud provider.
OIDC Discovery token exchange process
- An OIDC Issuer. The OIDC issuer is exposed over HTTPS with a publicly trusted TLS/SSL certificate.
- A JSON Web Key Set (JWKS), containing the public keys used to verify the service account token. The JWKS is stored in a publicly accessible and unauthenticated bucket in the Object Storage service. See JWKS Example.
- An OIDC discovery document (in JSON format) that includes the location of the JWKS. The discovery document is stored in a publicly accessible and unauthenticated bucket in the Object Storage service. See Discovery Document Example.
When defining an application pod that sends requests to an external API, if the external cloud provider's identity provider expects a specific audience in the service account token, specify a projected serviceAccountToken in the pod manifest and set its audience property accordingly. See Pod Manifest Example.
The following is an overview of the token exchange process:
-
When you create the application pod on the cluster, Kubernetes gives the pod a service account token. The service account token includes the URL of the Kubernetes Engine OIDC Issuer as the value of the
issclaim in the JWT. If you specified an audience in the pod manifest, the audience is included in the JWT as the value of theaudclaim. See Service Account Token Example. - The application pod makes a request to an API on an external cloud provider and presents the encoded service account token to the external cloud provider in the Authorization header of the API request.
- The external cloud provider's identity provider decodes the service account token, extracts the URL of the Kubernetes Engine OIDC Issuer from the
issclaim, appends/.well-known/openid-configurationto the URL as the location of the OIDC discovery document, and sends a request to that URL to obtain the discovery document. - The Kubernetes Engine OIDC Issuer returns the OIDC discovery document, which contains the URL of the location of the JWKS in the
jwks_urifield. See Discovery Document Example. - The external cloud provider's identity provider extracts the URL from the OIDC discovery document and sends a request to that URL to get the JWKS. See JWKS Example.
- The external cloud provider's identity provider uses the JWKS to validate the service account token originally presented by the application pod.
- Assuming the service account token is valid, the external cloud provider's identity provider returns an access token to the application pod.
- The application pod presents the access token to the API on the external cloud provider to prove its identity, and successfully makes API requests.
The token exchange process is shown in the following diagram:
Notes on OIDC Discovery
Note the following points when using OIDC Discovery:
- OIDC Discovery is supported on clusters running Kubernetes version 1.21 or later.
- OIDC Discovery is only supported on VCN-native clusters (clusters with Kubernetes API endpoints in a subnet in your own VCN). See Migrating to VCN-Native Clusters.
- OIDC Discovery is supported on managed nodes, virtual nodes, and self-managed nodes.
- OIDC Discovery is supported on enhanced clusters (but not on basic clusters).
Example Pod Manifest, Service Account Token, Discovery Document, and JWKS for OIDC Discovery
Pod Manifest Example
An example manifest for an application pod calling a cloud service API. In this example the external cloud provider's identity provider expects the service account token to specify an audience named vault:
apiVersion: v1
kind: Pod
metadata:
name: nginx
spec:
containers:
- image: nginx
name: nginx
volumeMounts:
- mountPath: /var/run/secrets/tokens
name: vault-token
serviceAccountName: build-robot
volumes:
- name: vault-token
projected:
sources:
- serviceAccountToken:
path: vault-token
expirationSeconds: 7200
audience: vaultService Account Token Example
An example service account token that Kubernetes might create for the pod in Pod Manifest Example:
{
"aud": [
"vault"
],
"exp": 1731613413,
"iat": 1700077413,
"iss": "https://objectstorage.us-ashburn-1.oci.customer-oci.com/n/okecustprod/b/oidc/o/{discoveryUUID}",
"kubernetes.io": {
"namespace": "kube-system",
"node": {
"name": "127.0.0.1",
"uid": "58456cb0-dd00-45ed-b797-5578fdceaced"
},
"pod": {
"name": "build-robot-69cbfb9798-jv9gn",
"uid": "778a530c-b3f4-47c0-9cd5-ab018fb64f33"
},
"serviceaccount": {
"name": "build-robot",
"uid": "a087d5a0-e1dd-43ec-93ac-f13d89cd13af"
},
"warnafter": 1700081020
},
"nbf": 1700077413,
"sub": "system:serviceaccount:kube-system:build-robot"
}Discovery Document Example
An example discovery document that Kubernetes Engine might issue, including the location of the JWKS specified by the jwks_uri field:
{
"issuer": "https://objectstorage.us-ashburn-1.oci.customer-oci.com/n/okecustprod/b/oidc/o/{discoveryUUID}",
"jwks_uri": "https://objectstorage.us-ashburn-1.oci.customer-oci.com/n/okecustprod/b/oidc/o/{discoveryUUID}/jwks",
"response_types_supported": [
"id_token"
],
"subject_types_supported": [
"public"
],
"id_token_signing_alg_values_supported": [
"RS256"
]
}JWKS Example
An example JWKS to authenticate the service account token:
{
"keys": [
{
"kty": "RSA",
"kid": "42148Kf",
"use": "sig",
"alg": "RS256",
"n": "iGaLqP6y-SJCCBq5Hv6pGDbG_SQ______asdf3sC",
"e": "AQAB"
},
{
"kty": "RSA",
"kid": "bEaunmA",
"use": "sig",
"alg": "RS256",
"n": "BISvILNyn-lUu4goZSXBD9ackM9______RpUlq2w",
"e": "AQAB"
}
]
}Enabling a Cluster for OIDC Discovery
To enable application pods running on a cluster you create with Kubernetes Engine to authenticate using OIDC Discovery when accessing APIs hosted on an external cloud provider, you must set the cluster's Enable OIDC Discovery property.
Using the CLI to Enable a Cluster for OIDC Discovery
For information about using the CLI, see Command Line Interface (CLI). For a complete list of flags and options available for CLI commands, see the Command Line Reference.
Creating A Cluster and Enabling OIDC Discovery
To use the CLI to create an enhanced cluster that uses OIDC Discovery to authenticate application pods when accessing APIs hosted on an external cloud provider, include the --open-id-connect-discovery-enabled parameter in the oci ce cluster create command.
For example:
oci ce cluster create \
--compartment-id ocid1.compartment.oc1..aaaaaaaa______n5q \
--name sales \
--vcn-id ocid1.vcn.oc1.phx.aaaaaaaa______lhq \
--type ENHANCED_CLUSTER \
--kubernetes-version v1.25.4 \
--service-lb-subnet-ids "[\"ocid1.subnet.oc1.phx.aaaaaaaa______g7q"]" \
--endpoint-subnet-id ocid1.subnet.oc1.phx.aaaaaaaa______sna \
--endpoint-public-ip-enabled true \
--endpoint-nsg-ids "[\"ocid1.networksecuritygroup.oc1.phx.aaaaaaaa______5qq\"]" \
--cluster-pod-network-options '[{"cniType":"OCI_VCN_IP_NATIVE"}]' \
--open-id-connect-discovery-enabled trueEditing A Cluster To Enable OIDC Discovery
To use the CLI to update an enhanced cluster to use OIDC Discovery to authenticate application pods when accessing APIs hosted on an external cloud provider, set the isOpenIdConnectDiscoveryEnabled attribute to true:
- In a suitable editor, create a JSON file with a name of your choice (these instructions assume the file is called
cluster-enable-oidc.json) containing the following:{ "options": { "openIdConnectDiscovery": { "isOpenIdConnectDiscoveryEnabled": true } } } - Save and close the
cluster-enable-oidc.jsonfile. -
Update the cluster by entering:
oci ce cluster update --cluster-id <cluster-ocid> --from-json file://<path-to-file>where:
--cluster-id <cluster-ocid>is the OCID of the cluster in which you want to enable OIDC Discovery.--from-json file://<path-to-file>specifies the location of the file to use when updating the cluster.
For example:
oci ce cluster update --cluster-id ocid1.cluster.oc1.iad.aaaaaaaaaf______jrd --from-json file://./cluster-enable-oidc.json
Using the API to Enable a Cluster for OIDC Discovery
Run the CreateCluster operation to create a cluster, or the UpdateCluster operation to edit a cluster, and specify true as the value of the isOpenIdConnectDiscoveryEnabled