Guide to the Secure Configuration of Red Hat OpenShift Container Platform 4

with profile NIST 800-53 Moderate-Impact Baseline for Red Hat OpenShift
This compliance profile reflects the core set of Moderate-Impact Baseline configuration settings for deployment of Red Hat OpenShift Container Platform into U.S. Defense, Intelligence, and Civilian agencies. Development partners and sponsors include the U.S. National Institute of Standards and Technology (NIST), U.S. Department of Defense, the National Security Agency, and Red Hat. This baseline implements configuration requirements from the following sources: - NIST 800-53 control selections for Moderate-Impact systems (NIST 800-53) For any differing configuration requirements, e.g. password lengths, the stricter security setting was chosen. Security Requirement Traceability Guides (RTMs) and sample System Security Configuration Guides are provided via the scap-security-guide-docs package. This profile reflects U.S. Government consensus content and is developed through the ComplianceAsCode initiative, championed by the National Security Agency. Except for differences in formatting to accommodate publishing processes, this profile mirrors ComplianceAsCode content as minor divergences, such as bugfixes, work through the consensus and release processes.
This guide presents a catalog of security-relevant configuration settings for Red Hat OpenShift Container Platform 4. It is a rendering of content structured in the eXtensible Configuration Checklist Description Format (XCCDF) in order to support security automation. The SCAP content is is available in the scap-security-guide package which is developed at https://www.open-scap.org/security-policies/scap-security-guide.

Providing system administrators with such guidance informs them how to securely configure systems under their control in a variety of network roles. Policy makers and baseline creators can use this catalog of settings, with its associated references to higher-level security control catalogs, in order to assist them in security baseline creation. This guide is a catalog, not a checklist, and satisfaction of every item is not likely to be possible or sensible in many operational scenarios. However, the XCCDF format enables granular selection and adjustment of settings, and their association with OVAL and OCIL content provides an automated checking capability. Transformations of this document, and its associated automated checking content, are capable of providing baselines that meet a diverse set of policy objectives. Some example XCCDF Profiles, which are selections of items that form checklists and can be used as baselines, are available with this guide. They can be processed, in an automated fashion, with tools that support the Security Content Automation Protocol (SCAP). The NIST National Checklist Program (NCP), which provides required settings for the United States Government, is one example of a baseline created from this guidance.
Do not attempt to implement any of the settings in this guide without first testing them in a non-operational environment. The creators of this guidance assume no responsibility whatsoever for its use by other parties, and makes no guarantees, expressed or implied, about its quality, reliability, or any other characteristic.

Profile Information

Profile TitleNIST 800-53 Moderate-Impact Baseline for Red Hat OpenShift
Profile IDxccdf_org.ssgproject.content_profile_moderate

CPE Platforms

  • cpe:/a:redhat:openshift_container_platform:4.1
  • cpe:/o:redhat:openshift_container_platform_node:4

Revision History

Current version: 0.1.55

  • draft (as of 2021-03-19)

Table of Contents

  1. OpenShift Settings
    1. OpenShift Kube API Server
    2. Kubernetes Kubelet Settings
    3. OpenShift Classification Banner
    4. OpenShift - Registry Security Practices
    5. OpenShift etcd Settings
    6. Authentication

Checklist

Group   Guide to the Secure Configuration of Red Hat OpenShift Container Platform 4   Group contains 7 groups and 15 rules
Group   OpenShift Settings   Group contains 6 groups and 15 rules
[ref]   Each section of this configuration guide includes information about the default configuration of an OpenShift cluster and a set of recommendations for hardening the configuration. For each hardening recommendation, information on how to implement the control and/or how to verify or audit the control is provided. In some cases, remediation information is also provided. Many of the settings in the hardening guide are in place by default. The audit information for these settings is provided in order to verify that the cluster admininstrator has not made changes that would be less secure than the OpenShift defaults. A small number of items require configuration. Finally, there are some recommendations that require decisions by the system operator, such as audit log size, retention, and related settings.
Group   OpenShift Kube API Server   Group contains 7 rules
[ref]   This section contains recommendations for kube-apiserver configuration.

Rule   Configure the Encryption Provider Cipher   [ref]

To ensure the correct cipher, set the encryption type aescbc in the apiserver object which configures the API server itself.
spec:
  encryption:
    type: aescbc
For more information, follow the relevant documentation.
Warning:  This rule's check operates on the cluster configuration dump. Therefore, you need to use a tool that can query the OCP API, retreive the /apis/config.openshift.io/v1/apiservers/cluster API endpoint to the local /kubernetes-api-resources/apis/config.openshift.io/v1/apiservers/cluster file.
Rationale:
aescbc is currently the strongest encryption provider, it should be preferred over other providers.
Severity: 
medium
Rule ID:xccdf_org.ssgproject.content_rule_api_server_encryption_provider_cipher
Identifiers and References

References:  1.2.34, SC-28(1)


apiVersion: config.openshift.io/v1
kind: APIServer
metadata:
  name: cluster
spec:
  encryption:
    type: aescbc

Rule   Ensure that Audit Log Forwarding Is Enabled   [ref]

OpenShift audit works at the API server level, logging all requests coming to the server. Audit is on by default and the best practice is to ship audit logs off the cluster for retention. The cluster-logging-operator is able to do this with the
ClusterLogForwarders
resource. The forementioned resource can be configured to logs to different third party systems. For more information on this, please reference the official documentation: https://docs.openshift.com/container-platform/4.6/logging/cluster-logging-external.html
Warning:  This rule's check operates on the cluster configuration dump. Therefore, you need to use a tool that can query the OCP API, retreive the /apis/logging.openshift.io/v1/namespaces/openshift-logging/clusterlogforwarders/instance API endpoint to the local /kubernetes-api-resources/apis/logging.openshift.io/v1/namespaces/openshift-logging/clusterlogforwarders/instance file.
Rationale:
Retaining logs ensures the ability to go back in time to investigate or correlate any events. Offloading audit logs from the cluster ensures that an attacker that has access to the cluster will not be able to tamper with the logs because of the logs being stored off-site.
Severity: 
medium
Rule ID:xccdf_org.ssgproject.content_rule_audit_log_forwarding_enabled
Identifiers and References

Identifiers:  CCE-84076-9

References:  1.2.23, AU-9(2)

Rule   Configure the Certificate Key for the API Server   [ref]

To ensure the API Server utilizes its own TLS certificates, the tls-private-key-file must be configured. Verify that the apiServerArguments section has the tls-private-key-file configured in the config configmap in the openshift-kube-apiserver namespace similar to:
"apiServerArguments":{
...
"tls-private-key-file": [
  "/etc/kubernetes/static-pod-certs/secrets/service-network-serving-certkey/tls.key"
],
...
}
Warning:  This rule's check operates on the cluster configuration dump. Therefore, you need to use a tool that can query the OCP API, retreive the /api/v1/namespaces/openshift-kube-apiserver/configmaps/config API endpoint to the local /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config file.
Rationale:
API Server communication contains sensitive parameters that should remain encrypted in transit. Configure the API Server to serve only HTTPS traffic.
Severity: 
medium
Rule ID:xccdf_org.ssgproject.content_rule_api_server_tls_private_key
Identifiers and References

Identifiers:  CCE-84282-3

References:  1.2.30, SC-8(1), SC-8(2)

Rule   Configure the Certificate for the API Server   [ref]

To ensure the API Server utilizes its own TLS certificates, the tls-cert-file must be configured. Verify that the apiServerArguments section has the tls-cert-file configured in the config configmap in the openshift-kube-apiserver namespace similar to:
"apiServerArguments":{
...
"tls-cert-file": [
  "/etc/kubernetes/static-pod-certs/secrets/service-network-serving-certkey/tls.crt"
],
...
}
Warning:  This rule's check operates on the cluster configuration dump. Therefore, you need to use a tool that can query the OCP API, retreive the /api/v1/namespaces/openshift-kube-apiserver/configmaps/config API endpoint to the local /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config file.
Rationale:
API Server communication contains sensitive parameters that should remain encrypted in transit. Configure the API Server to serve only HTTPS traffic.
Severity: 
medium
Rule ID:xccdf_org.ssgproject.content_rule_api_server_tls_cert
Identifiers and References

Identifiers:  CCE-83779-9

References:  1.2.30, SC-8(1), SC-8(2)

Rule   Configure the Encryption Provider   [ref]

To encrypt the etcd key-value store, set the encryption type aescbc in the apiserver object which configures the API server itself.
spec:
  encryption:
    type: aescbc
For more information, follow the relevant documentation.
Warning:  This rule's check operates on the cluster configuration dump. Therefore, you need to use a tool that can query the OCP API, retreive the /apis/config.openshift.io/v1/apiservers/cluster API endpoint to the local /kubernetes-api-resources/apis/config.openshift.io/v1/apiservers/cluster file.
Rationale:
etcd is a highly available key-value store used by OpenShift deployments for persistent storage of all REST API objects. These objects are sensitive in nature and should be encrypted at rest to avoid any disclosures.
Severity: 
medium
Rule ID:xccdf_org.ssgproject.content_rule_api_server_encryption_provider_config
Identifiers and References

Identifiers:  CCE-83585-0

References:  1.2.33, SC-28(1)


apiVersion: config.openshift.io/v1
kind: APIServer
metadata:
  name: cluster
spec:
  encryption:
    type: aescbc

Rule   Configure the etcd Certificate Authority for the API Server   [ref]

To ensure etcd is configured to make use of TLS encryption for client connections, follow the OpenShift documentation and setup the TLS connection between the API Server and etcd. Then, verify that apiServerArguments has the etcd-cafile configured in the openshift-kube-apiserver config configmap to something similar to:
"apiServerArguments": {
  ...
    "etcd-cafile": [
        "/etc/kubernetes/static-pod-resources/configmaps/etcd-serving-ca/ca-bundle.crt"
    ],
  ...
Warning:  This rule's check operates on the cluster configuration dump. Therefore, you need to use a tool that can query the OCP API, retreive the /api/v1/namespaces/openshift-kube-apiserver/configmaps/config API endpoint to the local /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config file.
Rationale:
etcd is a highly-available key-value store used by OpenShift deployments for persistent storage of all REST API objects. These objects are sensitive in nature and should be protected by client authentication. This requires the API Server to identify itself to the etcd server using a SSL Certificate Authority file.
Severity: 
medium
Rule ID:xccdf_org.ssgproject.content_rule_api_server_etcd_ca
Identifiers and References

Identifiers:  CCE-84216-1

References:  1.2.32, SC-8(1), SC-8(2)

Rule   Configure the Client Certificate Authority for the API Server   [ref]

Certificates must be provided to fully setup TLS client certificate authentication. To ensure the API Server utilizes its own TLS certificates, the clientCA must be configured. Verify that servingInfo has the clientCA configured in the openshift-kube-apiserver config configmap to something similar to:
"apiServerArguments": {
  ...
    "client-ca-file": [
      "/etc/kubernetes/static-pod-certs/configmaps/client-ca/ca-bundle.crt"
    ],
  ...
Warning:  This rule's check operates on the cluster configuration dump. Therefore, you need to use a tool that can query the OCP API, retreive the /api/v1/namespaces/openshift-kube-apiserver/configmaps/config API endpoint to the local /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config file.
Rationale:
API Server communication contains sensitive parameters that should remain encrypted in transit. Configure the API Server to serve only HTTPS traffic. If -clientCA is set, any request presenting a client certificate signed by one of the authorities in the client-ca-file is authenticated with an identity corresponding to the CommonName of the client certificate.
Severity: 
medium
Rule ID:xccdf_org.ssgproject.content_rule_api_server_client_ca
Identifiers and References

Identifiers:  CCE-84284-9

References:  1.2.31, SC-8(1), SC-8(2)

Group   Kubernetes Kubelet Settings   Group contains 2 rules
[ref]   The Kubernetes Kubelet is an agent that runs on each node in the cluster. It makes sure that containers are running in a pod. The kubelet takes a set of PodSpecs that are provided through various mechanisms and ensures that the containers described in those PodSpecs are running and healthy. The kubelet doesn’t manage containers which were not created by Kubernetes.

Rule   Ensure That The kubelet Client Certificate Is Correctly Set   [ref]

To ensure the kubelet TLS client certificate is configured, edit the kubelet configuration file /etc/kubernetes/kubelet.conf and configure the kubelet certificate file.
tlsCertFile: /path/to/TLS/cert.key
Warning:  This rule's check operates on the cluster configuration dump. Therefore, you need to use a tool that can query the OCP API, retreive the /api/v1/namespaces/openshift-kube-apiserver/configmaps/config API endpoint to the local /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config file.
Rationale:
Without cryptographic integrity protections, information can be altered by unauthorized users without detection.
Severity: 
medium
Rule ID:xccdf_org.ssgproject.content_rule_kubelet_configure_tls_cert
Identifiers and References

Identifiers:  CCE-83396-2

References:  4.2.10, SC-8(1), SC-8(2)

Rule   Ensure That The kubelet Server Key Is Correctly Set   [ref]

To ensure the kubelet TLS private server key certificate is configured, edit the kubelet configuration file /etc/kubernetes/kubelet.conf and configure the kubelet private key file.
tlsPrivateKeyFile: /path/to/TLS/private.key
Warning:  This rule's check operates on the cluster configuration dump. Therefore, you need to use a tool that can query the OCP API, retreive the /api/v1/namespaces/openshift-kube-apiserver/configmaps/config API endpoint to the local /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config file.
Rationale:
Without cryptographic integrity protections, information can be altered by unauthorized users without detection.
Severity: 
medium
Rule ID:xccdf_org.ssgproject.content_rule_kubelet_configure_tls_key
Identifiers and References

References:  4.2.10, SC-8(1), SC-8(2)

Group   OpenShift Classification Banner   Group contains 1 rule
[ref]   OpenShift Console Notifications enable creating custom Classification Banners

Rule   Enable Classification Banner on OpenShift Console   [ref]

A classification banner can be configured by creating a Console Notification CRD on OpenShift
Warning:  This rule's check operates on the cluster configuration dump. Therefore, you need to use a tool that can query the OCP API, retreive the /apis/console.openshift.io/v1/consolenotifications/classification-banner API endpoint to the local /kubernetes-api-resources/apis/console.openshift.io/v1/consolenotifications/classification-banner file.
Rationale:
Displays to users organization-defined system use notification message or banner before granting access to the system that provides privacy and security notices consistent with applicable federal laws
Severity: 
medium
Rule ID:xccdf_org.ssgproject.content_rule_classification_banner
Identifiers and References

References:  AC-8

Group   OpenShift - Registry Security Practices   Group contains 2 rules
[ref]   Contains evaluations for OpenShift registry security practices, and cluster-wide registry configuration.

Rule   Allowed registries for import are configured   [ref]

The configuration allowedRegistriesForImport limits the container image registries from which normal users may import images. This is important to control, as a user who can stand up a malicious registry can then import content which claims to include the SHAs of legimitate content layers. You can set the allowed repositories for import by applying the following manifest using
oc patch
, e.g. if you save the following snippet to
/tmp/allowed-import-registries-patch.yaml
spec:
  allowedRegistriesForImport:
  - domainName: my-trusted-registry.internal.example.com
    insecure: false
you would call
oc patch image.config.openshift.io cluster --patch="$(cat /tmp/allowed-import-registries-patch.yaml)" --type=merge
Warning:  This rule's check operates on the cluster configuration dump. Therefore, you need to use a tool that can query the OCP API, retreive the /apis/config.openshift.io/v1/images/cluster API endpoint to the local /kubernetes-api-resources/apis/config.openshift.io/v1/images/cluster file.
Rationale:
Allowed registries for import should be specified to limit the registries from which users may import images.
Severity: 
medium
Rule ID:xccdf_org.ssgproject.content_rule_ocp_allowed_registries_for_import
Identifiers and References

Rule   Allowed registries are configured   [ref]

The configuration registrySources.allowedRegistries determines the permitted registries that the OpenShift container runtime can access for builds and pods. This configuration setting ensures that all registries other than those specified are blocked. You can set the allowed repositories by applying the following manifest using
oc patch
, e.g. if you save the following snippet to
/tmp/allowed-registries-patch.yaml
spec:
  registrySources:
    allowedRegistries:
    - my-trusted-registry.internal.example.com
you would call
oc patch image.config.openshift.io cluster --patch="$(cat /tmp/allowed-registries-patch.yaml)" --type=merge
Warning:  This rule's check operates on the cluster configuration dump. Therefore, you need to use a tool that can query the OCP API, retreive the /apis/config.openshift.io/v1/images/cluster API endpoint to the local /kubernetes-api-resources/apis/config.openshift.io/v1/images/cluster file.
Rationale:
Allowed registries should be configured to restrict the registries that the OpenShift container runtime can access, and all other registries should be blocked.
Severity: 
medium
Rule ID:xccdf_org.ssgproject.content_rule_ocp_allowed_registries
Identifiers and References
Group   OpenShift etcd Settings   Group contains 2 rules
[ref]   Contains rules that check correct OpenShift etcd settings.

Rule   Ensure That The etcd Peer Key File Is Correctly Set   [ref]

To ensure the etcd service is serving TLS to peers, make sure the etcd-pod* ConfigMaps in the openshift-etcd namespace contain the following argument for the etcd binary in the etcd pod:
oc get -nopenshift-etcd cm etcd-pod -oyaml | grep "\-\-peer-key-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-peer/etcd-peer-NODE_NAME.key"
Warning:  This rule's check operates on the cluster configuration dump. Therefore, you need to use a tool that can query the OCP API, retreive the /api/v1/namespaces/openshift-etcd/configmaps/etcd-pod API endpoint to the local /kubernetes-api-resources/api/v1/namespaces/openshift-etcd/configmaps/etcd-pod file.
Rationale:
Without cryptographic integrity protections, information can be altered by unauthorized users without detection.
Severity: 
medium
Rule ID:xccdf_org.ssgproject.content_rule_etcd_peer_key_file
Identifiers and References

Identifiers:  CCE-83711-2

References:  2.4, SC-8(1), SC-8(2)

Rule   Ensure That The etcd Peer Client Certificate Is Correctly Set   [ref]

To ensure the etcd service is serving TLS to peers, make sure the etcd-pod* ConfigMaps in the openshift-etcd namespace contain the following argument for the etcd binary in the etcd pod:
--peer-cert-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-peer/etcd-peer-NODE_NAME.crt
Warning:  This rule's check operates on the cluster configuration dump. Therefore, you need to use a tool that can query the OCP API, retreive the /api/v1/namespaces/openshift-etcd/configmaps/etcd-pod API endpoint to the local /kubernetes-api-resources/api/v1/namespaces/openshift-etcd/configmaps/etcd-pod file.
Rationale:
Without cryptographic integrity protections, information can be altered by unauthorized users without detection.
Severity: 
medium
Rule ID:xccdf_org.ssgproject.content_rule_etcd_peer_cert_file
Identifiers and References

Identifiers:  CCE-83847-4

References:  2.4, SC-8(1), SC-8(2)

Group   Authentication   Group contains 1 rule
[ref]   In cloud workloads, there are many ways to create and configure to multiple authentication services. Some of these authentication methods by not be secure or common methodologies, or they may not be secure by default. This section introduces mechanisms for configuring authentication systems to OpenShift.

Rule   Do Not Use htpasswd-based Authentication   [ref]

TBD
Warning:  This rule's check operates on the cluster configuration dump. Therefore, you need to use a tool that can query the OCP API, retreive the /apis/config.openshift.io/v1/oauths/cluster API endpoint to the local /kubernetes-api-resources/apis/config.openshift.io/v1/oauths/cluster file.
Rationale:
TBD
Severity: 
medium
Rule ID:xccdf_org.ssgproject.content_rule_ocp_idp_no_htpasswd
Identifiers and References

Identifiers:  CCE-84209-6

Red Hat and Red Hat Enterprise Linux are either registered trademarks or trademarks of Red Hat, Inc. in the United States and other countries. All other names are registered trademarks or trademarks of their respective companies.