Guide to the Secure Configuration of Red Hat OpenShift Container Platform 4

with profile Australian Cyber Security Centre (ACSC) Essential Eight
This profile contains configuration checks for Red Hat OpenShift Container Platform that align to the Australian Cyber Security Centre (ACSC) Essential Eight. A copy of the Essential Eight in Linux Environments guide can be found at the ACSC website: https://www.cyber.gov.au/acsc/view-all-content/publications/hardening-linux-workstations-and-servers
This guide presents a catalog of security-relevant configuration settings for Red Hat OpenShift Container Platform 4. It is a rendering of content structured in the eXtensible Configuration Checklist Description Format (XCCDF) in order to support security automation. The SCAP content is is available in the scap-security-guide package which is developed at https://www.open-scap.org/security-policies/scap-security-guide.

Providing system administrators with such guidance informs them how to securely configure systems under their control in a variety of network roles. Policy makers and baseline creators can use this catalog of settings, with its associated references to higher-level security control catalogs, in order to assist them in security baseline creation. This guide is a catalog, not a checklist, and satisfaction of every item is not likely to be possible or sensible in many operational scenarios. However, the XCCDF format enables granular selection and adjustment of settings, and their association with OVAL and OCIL content provides an automated checking capability. Transformations of this document, and its associated automated checking content, are capable of providing baselines that meet a diverse set of policy objectives. Some example XCCDF Profiles, which are selections of items that form checklists and can be used as baselines, are available with this guide. They can be processed, in an automated fashion, with tools that support the Security Content Automation Protocol (SCAP). The NIST National Checklist Program (NCP), which provides required settings for the United States Government, is one example of a baseline created from this guidance.
Do not attempt to implement any of the settings in this guide without first testing them in a non-operational environment. The creators of this guidance assume no responsibility whatsoever for its use by other parties, and makes no guarantees, expressed or implied, about its quality, reliability, or any other characteristic.

Profile Information

Profile TitleAustralian Cyber Security Centre (ACSC) Essential Eight
Profile IDxccdf_org.ssgproject.content_profile_e8

CPE Platforms

  • cpe:/a:redhat:openshift_container_platform_node_on_ovn:4
  • cpe:/a:redhat:openshift_container_platform_node_on_sdn:4
  • cpe:/o:redhat:openshift_container_platform_node:4
  • cpe:/a:redhat:openshift_container_platform_on_aws:4
  • cpe:/a:redhat:openshift_container_platform_on_azure:4
  • cpe:/a:redhat:openshift_container_platform_on_gcp:4
  • cpe:/a:redhat:openshift_container_platform_on_ovn:4
  • cpe:/a:redhat:openshift_container_platform_on_sdn:4
  • cpe:/a:redhat:openshift_container_platform:4.10
  • cpe:/a:redhat:openshift_container_platform:4.11
  • cpe:/a:redhat:openshift_container_platform:4.12
  • cpe:/a:redhat:openshift_container_platform:4.13
  • cpe:/a:redhat:openshift_container_platform:4.14
  • cpe:/a:redhat:openshift_container_platform:4.15
  • cpe:/a:redhat:openshift_container_platform:4.16
  • cpe:/a:redhat:openshift_container_platform:4.17
  • cpe:/a:redhat:openshift_container_platform:4.18
  • cpe:/a:redhat:openshift_container_platform:4.6
  • cpe:/a:redhat:openshift_container_platform:4.7
  • cpe:/a:redhat:openshift_container_platform:4.8
  • cpe:/a:redhat:openshift_container_platform:4.9
  • cpe:/a:redhat:openshift_container_platform:4.1

Revision History

Current version: 0.1.68

  • draft (as of 2023-06-15)

Table of Contents

  1. Kubernetes Settings
    1. OpenShift Kube API Server
    2. Authentication
    3. Kubernetes - General Security Practices
    4. Role-based Access Control
    5. Kubernetes - Registry Security Practices
    6. Security Context Constraints (SCC)

Checklist

Group   Guide to the Secure Configuration of Red Hat OpenShift Container Platform 4   Group contains 7 groups and 14 rules
Group   Kubernetes Settings   Group contains 6 groups and 14 rules
[ref]   Each section of this configuration guide includes information about the configuration of a Kubernetes cluster and a set of recommendations for hardening the configuration. For each hardening recommendation, information on how to implement the control and/or how to verify or audit the control is provided. In some cases, remediation information is also provided. Some of the settings in the hardening guide are in place by default. The audit information for these settings is provided in order to verify that the cluster administrator has not made changes that would be less secure. A small number of items require configuration. Finally, there are some recommendations that require decisions by the system operator, such as audit log size, retention, and related settings.
Group   OpenShift Kube API Server   Group contains 2 rules
[ref]   This section contains recommendations for kube-apiserver configuration.

Rule   Configure the Encryption Provider Cipher   [ref]

When you enable etcd encryption, the following OpenShift API server and Kubernetes API server resources are encrypted:

  • Secrets
  • ConfigMaps
  • Routes
  • OAuth access tokens
  • OAuth authorize tokens

When you enable etcd encryption, encryption keys are created. These keys are rotated on a weekly basis. You must have these keys in order to restore from an etcd backup.

To ensure the correct cipher, set the encryption type aescbc in the apiserver object which configures the API server itself.

spec:
  encryption:
    type: aescbc

For more information, follow the relevant documentation.

Warning:  This rule's check operates on the cluster configuration dump. Therefore, you need to use a tool that can query the OCP API, retrieve the following:
  • {{if ne .hypershift_cluster "None"}}/apis/hypershift.openshift.io/v1beta1/namespaces/clusters/hostedclusters/{{.hypershift_cluster}}{{else}}/apis/config.openshift.io/v1/apiservers/cluster{{end}} API endpoint, filter with with the jq utility using the following filter {{if ne .hypershift_cluster "None"}}[.spec.secretEncryption.type]{{else}}[.spec.encryption.type]{{end}} and persist it to the local /kubernetes-api-resources/apis/config.openshift.io/v1/apiservers/cluster#a1d4b20a86b76e7e2d634dbeff420b1a80df6800836dad1b552314d1b24a18cb file.
Rationale:
etcd is a highly available key-value store used by OpenShift deployments for persistent storage of all REST API objects. These objects are sensitive in nature and should be encrypted at rest to avoid any disclosures. Where etcd encryption is used, it is important to ensure that the appropriate set of encryption providers is used. Currently, aescbc is the only type supported by OCP.
Severity: 
medium
Rule ID:xccdf_org.ssgproject.content_rule_api_server_encryption_provider_cipher
Identifiers and References

Identifiers:  CCE-83585-0

References:  CIP-003-8 R4.2, SC-28, SC-28(1), Req-2.2, SRG-APP-000429-CTR-001060, 1.2.33, 1.2.34


---
apiVersion: config.openshift.io/v1
kind: APIServer
metadata:
  name: cluster
spec:
  encryption:
    type: aescbc

Rule   Use Strong Cryptographic Ciphers on the API Server   [ref]

To ensure that the API Server is configured to only use strong cryptographic ciphers, verify the openshift-kube-apiserver configmap contains the following set of ciphers, with no additions:
"servingInfo":{
  ...
  "cipherSuites": [
    "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256",
    "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",
    "TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384",
    "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384",
    "TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256",
    "TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256",
  ],
  ...
Warning:  This rule's check operates on the cluster configuration dump. Therefore, you need to use a tool that can query the OCP API, retrieve the following:
  • {{if ne .hypershift_cluster "None"}}/api/v1/namespaces/{{.hypershift_namespace_prefix}}-{{.hypershift_cluster}}/configmaps/kas-config{{else}}/api/v1/namespaces/openshift-kube-apiserver/configmaps/config{{end}} API endpoint, filter with with the jq utility using the following filter {{if ne .hypershift_cluster "None"}}.data."config.json" | fromjson{{else}}.data."config.yaml" | fromjson{{end}} and persist it to the local /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430 file.
Warning:  Once configured, API Server clients that cannot support modern cryptographic ciphers will not be able to make connections to the API server.
Rationale:
TLS ciphers have had a number of known vulnerabilities and weaknesses, which can reduce the protection provided. By default, OpenShift supports a number of TLS ciphersuites including some that have security concerns, weakening the protection provided.
Severity: 
medium
Rule ID:xccdf_org.ssgproject.content_rule_api_server_tls_cipher_suites
Identifiers and References

References:  CM-6, Req-2.2, Req-2.2.3, Req-2.3, SRG-APP-000516-CTR-001325, 1.2.35

Group   Authentication   Group contains 1 rule
[ref]   In cloud workloads, there are many ways to create and configure to multiple authentication services. Some of these authentication methods by not be secure or common methodologies, or they may not be secure by default. This section introduces mechanisms for configuring authentication systems Kubernetes.

Rule   Do Not Use htpasswd-based IdP   [ref]

For users to interact with OpenShift Container Platform, they must first authenticate to the cluster. The authentication layer identifies the user associated with requests to the OpenShift Container Platform API. The authorization layer then uses information about the requesting user to determine if the request is allowed. Understanding authentication | Authentication | OpenShift Container Platform

The OpenShift Container Platform includes a built-in OAuth server for token-based authentication. Developers and administrators obtain OAuth access tokens to authenticate themselves to the API. It is recommended for an administrator to configure OAuth to specify an identity provider after the cluster is installed. User access to the cluster is managed through the identity provider. Understanding identity provider configuration | Authentication | OpenShift Container Platform

However, not all Identity Providers supported by OpenShift provide the same level of capabilities. As an example, the htpasswd Identity Provider only checks the username and password match and provides no means of 2FA, account lockout or notification mechanism. This rule therefore only allows a subset of identity providers.

Warning:  This rule's check operates on the cluster configuration dump. Therefore, you need to use a tool that can query the OCP API, retrieve the following:
  • {{if ne .hypershift_cluster "None"}}/apis/hypershift.openshift.io/v1beta1/namespaces/clusters/hostedclusters/{{.hypershift_cluster}}{{else}}/apis/config.openshift.io/v1/oauths/cluster{{end}} API endpoint, filter with with the jq utility using the following filter {{if ne .hypershift_cluster "None"}}.spec.configuration.oauth{{else}}.spec{{end}} and persist it to the local /kubernetes-api-resources/apis/config.openshift.io/v1/oauths/cluster#489c53adb0325a207f2120d4dee0ef775dad56dceaa74bafc10bf32c1da46e9e file.
Rationale:

With any authentication mechanism the ability to revoke credentials if they are compromised or no longer required, is a key control. Kubernetes client certificate authentication does not allow for this due to a lack of support for certificate revocation.

OpenShift's built-in OAuth server allows credential revocation by relying on the Identity provider, as well as giving the administrators the ability to revoke any tokens given to a specific user.

In addition, using an external Identity provider allows for setting up notifications on account creation or deletion, multi-factor authentication, disabling inactive accounts or other features required by different compliance standards.

Severity: 
medium
Rule ID:xccdf_org.ssgproject.content_rule_ocp_idp_no_htpasswd
Identifiers and References

Identifiers:  CCE-84209-6

References:  CIP-003-8 R5.1.1, CIP-003-8 R5.3, CIP-004-6 R2.2.2, CIP-004-6 R2.2.3, CIP-004-6 R2.3, CIP-007-3 R.1.3, CIP-007-3 R5, CIP-007-3 R5.1, CIP-007-3 R5.1.1, CIP-007-3 R5.1.2, CIP-007-3 R5.1.3, CIP-007-3 R5.2, CIP-007-3 R5.2.1, CIP-007-3 R5.2.3, CIP-007-3 R5.3.1, CIP-007-3 R5.3.2, CIP-007-3 R5.3.3, AC-2(1), AC-2(2), AC-2(3), AC-2(4), AC-2(7), AC-2(8), AC-7, IA-2, IA-2(1), IA-2(2), IA-2(3), IA-2(5), IA-2(8), IA-2(9), IA-2(12), IA-5(4), SC-12(1), SRG-APP-000023-CTR-000055

Group   Kubernetes - General Security Practices   Group contains 2 rules
[ref]   Contains evaluations for general security practices for operating a Kubernetes environment.

Rule   This is a helper rule to fetch the required api resource for detecting HyperShift OCP version   [ref]

no description
Warning:  This rule's check operates on the cluster configuration dump. Therefore, you need to use a tool that can query the OCP API, retrieve the following:
  • /apis/hypershift.openshift.io/v1beta1/namespaces/clusters/hostedclusters/{{.hypershift_cluster}} API endpoint, filter with with the jq utility using the following filter [.status.version.history[].version] and persist it to the local /kubernetes-api-resources/hypershift/version file. This rule will be a hidden rule true
Rationale:
no rationale
Severity: 
medium
Rule ID:xccdf_org.ssgproject.content_rule_version_detect_in_hypershift
Identifiers and References

Rule   This is a helper rule to fetch the required api resource for detecting OCP version   [ref]

no description
Warning:  This rule's check operates on the cluster configuration dump. Therefore, you need to use a tool that can query the OCP API, retrieve the following:
  • {{.ocp_version_api_path}} API endpoint, filter with with the jq utility using the following filter {{.ocp_version_yaml_path}} and persist it to the local /kubernetes-api-resources/ocp/version file. This rule will be a hidden rule true
Rationale:
no rationale
Severity: 
medium
Rule ID:xccdf_org.ssgproject.content_rule_version_detect_in_ocp
Identifiers and References
Group   Role-based Access Control   Group contains 3 rules
[ref]   Role-based access control (RBAC) objects determine whether a user is allowed to perform a given action within a project. Cluster administrators can use the cluster roles and bindings to control who has various access levels to the OpenShift Container Platform platform itself and all projects. Developers can use local roles and bindings to control who has access to their projects. Note that authorization is a separate step from authentication, which is more about determining the identity of who is taking the action.

Rule   Ensure that the cluster-admin role is only used where required   [ref]

The RBAC role cluster-admin provides wide-ranging powers over the environment and should be used only where and when needed.
Rationale:
Kubernetes provides a set of default roles where RBAC is used. Some of these roles such as cluster-admin provide wide-ranging privileges which should only be applied where absolutely necessary. Roles such as cluster-admin allow super-user access to perform any action on any resource. When used in a ClusterRoleBinding, it gives full control over every resource in the cluster and in all namespaces. When used in a RoleBinding, it gives full control over every resource in the rolebinding's namespace, including the namespace itself.
Severity: 
medium
Rule ID:xccdf_org.ssgproject.content_rule_rbac_limit_cluster_admin
Identifiers and References

References:  CIP-003-8 R6, CIP-004-6 R3, CIP-007-3 R6.1, CM-6, CM-6(1), CM-8(3), Req-2.2, Req-7.1.2, Req-10.5.1, SRG-APP-000516-CTR-001325, 5.1.1

Rule   Minimize Access to Pod Creation   [ref]

The ability to create pods in a namespace can provide a number of opportunities for privilege escalation. Where applicable, remove create access to pod objects in the cluster.
Rationale:
The ability to create pods in a cluster opens up the cluster for privilege escalation.
Severity: 
medium
Rule ID:xccdf_org.ssgproject.content_rule_rbac_pod_creation_access
Identifiers and References

References:  CIP-003-8 R6, CIP-004-6 R3, CIP-007-3 R6.1, CM-6, CM-6(1), Req-2.2, SRG-APP-000516-CTR-001325, 5.1.4

Rule   Minimize Wildcard Usage in Cluster and Local Roles   [ref]

Kubernetes Cluster and Local Roles provide access to resources based on sets of objects and actions that can be taken on those objects. It is possible to set either of these using a wildcard * which matches all items. This violates the principle of least privilege and leaves a cluster in a more vulnerable state to privilege abuse.
Rationale:
The principle of least privilege recommends that users are provided only the access required for their role and nothing more. The use of wildcard rights grants is likely to provide excessive rights to the Kubernetes API.
Severity: 
medium
Rule ID:xccdf_org.ssgproject.content_rule_rbac_wildcard_use
Identifiers and References

References:  CIP-003-8 R6, CIP-004-6 R3, CIP-007-3 R6.1, CM-6, CM-6(1), Req-2.2, SRG-APP-000516-CTR-001325, 5.1.3

Group   Kubernetes - Registry Security Practices   Group contains 2 rules
[ref]   Contains evaluations for Kubernetes registry security practices, and cluster-wide registry configuration.

Rule   Allowed registries are configured   [ref]

The configuration registrySources.allowedRegistries determines the permitted registries that the OpenShift container runtime can access for builds and pods. This configuration setting ensures that all registries other than those specified are blocked. You can set the allowed repositories by applying the following manifest using
oc patch
, e.g. if you save the following snippet to
/tmp/allowed-registries-patch.yaml
spec:
  registrySources:
    allowedRegistries:
    - my-trusted-registry.internal.example.com
you would call
oc patch image.config.openshift.io cluster --patch="$(cat /tmp/allowed-registries-patch.yaml)" --type=merge
Warning:  This rule's check operates on the cluster configuration dump. Therefore, you need to use a tool that can query the OCP API, retrieve the /apis/config.openshift.io/v1/images/cluster API endpoint to the local /kubernetes-api-resources/apis/config.openshift.io/v1/images/cluster file.
Rationale:
Allowed registries should be configured to restrict the registries that the OpenShift container runtime can access, and all other registries should be blocked.
Severity: 
medium
Rule ID:xccdf_org.ssgproject.content_rule_ocp_allowed_registries
Identifiers and References

References:  CM-5(3), CM-7(2), CM-7(5), CM-11, SRG-APP-000456-CTR-001125

Rule   Allowed registries for import are configured   [ref]

The configuration allowedRegistriesForImport limits the container image registries from which normal users may import images. This is important to control, as a user who can stand up a malicious registry can then import content which claims to include the SHAs of legitimate content layers. You can set the allowed repositories for import by applying the following manifest using
oc patch
, e.g. if you save the following snippet to
/tmp/allowed-import-registries-patch.yaml
spec:
  allowedRegistriesForImport:
  - domainName: my-trusted-registry.internal.example.com
    insecure: false
you would call
oc patch image.config.openshift.io cluster --patch="$(cat /tmp/allowed-import-registries-patch.yaml)" --type=merge
Warning:  This rule's check operates on the cluster configuration dump. Therefore, you need to use a tool that can query the OCP API, retrieve the /apis/config.openshift.io/v1/images/cluster API endpoint to the local /kubernetes-api-resources/apis/config.openshift.io/v1/images/cluster file.
Rationale:
Allowed registries for import should be specified to limit the registries from which users may import images.
Severity: 
medium
Rule ID:xccdf_org.ssgproject.content_rule_ocp_allowed_registries_for_import
Identifiers and References

References:  CM-5(3), CM-7(2), CM-7(5), CM-11, SRG-APP-000456-CTR-001125

Group   Security Context Constraints (SCC)   Group contains 4 rules
[ref]   Similar to the way that RBAC resources control user access, administrators can use Security Context Constraints (SCCs) to control permissions for pods. These permissions include actions that a pod, a collection of containers, can perform and what resources it can access. You can use SCCs to define a set of conditions that a pod must run with in order to be accepted into the system.

Rule   Limit Container Capabilities   [ref]

Containers should not enable more capabilities than needed as this opens the door for malicious use. To enable only the required capabilities, the appropriate Security Context Constraints (SCCs) should set capabilities as a list in allowedCapabilities.
Warning:  This rule's check operates on the cluster configuration dump. Therefore, you need to use a tool that can query the OCP API, retrieve the following:
  • /apis/security.openshift.io/v1/securitycontextconstraints API endpoint, filter with with the jq utility using the following filter [.items[] | select(.metadata.name | test("{{.var_sccs_with_allowed_capabilities_regex}}"; "") | not) | select(.allowedCapabilities != null) | .metadata.name] and persist it to the local /kubernetes-api-resources/apis/security.openshift.io/v1/securitycontextconstraints#821f2c3e5c4100d5609ac6070d8a51075295651614a05c65a5f744ba751c15b0 file.
Rationale:
By default, containers run with a default set of capabilities as assigned by the Container Runtime which can include dangerous or highly privileged capabilities. Capabilities should be dropped unless absolutely critical for the container to run software as added capabilities that are not required allow for malicious containers or attackers.
Severity: 
medium
Rule ID:xccdf_org.ssgproject.content_rule_scc_limit_container_allowed_capabilities
Identifiers and References

References:  CIP-003-8 R6, CIP-004-6 R3, CIP-007-3 R6.1, CM-6, CM-6(1), Req-2.2, SRG-APP-000516-CTR-001325, 5.2.8

Rule   Limit Containers Ability to Escalate Privileges   [ref]

Containers should be limited to only the privileges required to run and should not be allowed to escalate their privileges. To prevent containers from escalating privileges, the appropriate Security Context Constraints (SCCs) should set allowPrivilegeEscalation to false.
Rationale:
Privileged containers have access to more of the Linux Kernel capabilities and devices. If a privileged container were compromised, an attacker would have full access to the container and host.
Severity: 
medium
Rule ID:xccdf_org.ssgproject.content_rule_scc_limit_privilege_escalation
Identifiers and References

Identifiers:  CCE-83447-3

References:  CIP-003-8 R6, CIP-004-6 R3, CIP-007-3 R6.1, CM-6, CM-6(1), Req-2.2, SRG-APP-000516-CTR-001325, 5.2.5

Rule   Limit Privileged Container Use   [ref]

Containers should be limited to only the privileges required to run. To prevent containers from running as privileged containers, the appropriate Security Context Constraints (SCCs) should set allowPrivilegedContainer to false.
Rationale:
Privileged containers have access to all Linux Kernel capabilities and devices. If a privileged container were compromised, an attacker would have full access to the container and host.
Severity: 
medium
Rule ID:xccdf_org.ssgproject.content_rule_scc_limit_privileged_containers
Identifiers and References

References:  CIP-003-8 R6, CIP-004-6 R3, CIP-007-3 R6.1, CM-6, CM-6(1), Req-2.2, SRG-APP-000342-CTR-000775, SRG-APP-000142-CTR-000330, 5.2.1

Rule   Limit Container Running As Root User   [ref]

Containers should run as a random non-privileged user. To prevent containers from running as root user, the appropriate Security Context Constraints (SCCs) should set .runAsUser.type to MustRunAsRange.
Rationale:
It is strongly recommended that containers running on OpenShift should support running as any arbitrary UID. OpenShift will then assign a random, non-privileged UID to the running container instance. This avoids the risk from containers running with specific uids that could map to host service accounts, or an even greater risk of running as root level service. OpenShift uses the default security context constraints (SCC), restricted, to prevent containers from running as root or other privileged user ids. Pods may be configured to use an scc policy that allows the container to run as a specific uid, including root(0) when approved. Only a cluster administrator may grant the change of an scc policy.
Severity: 
medium
Rule ID:xccdf_org.ssgproject.content_rule_scc_limit_root_containers
Identifiers and References

References:  CIP-003-8 R6, CIP-004-6 R3, CIP-007-3 R6.1, CM-6, CM-6(1), Req-2.2, SRG-APP-000342-CTR-000775, 5.2.6

Red Hat and Red Hat Enterprise Linux are either registered trademarks or trademarks of Red Hat, Inc. in the United States and other countries. All other names are registered trademarks or trademarks of their respective companies.