Guide to the Secure Configuration of Red Hat OpenShift Container Platform 3

with profile Open Computing Information Security Profile for OpenShift Node
This baseline was inspired by the Center for Internet Security (CIS) Kubernetes Benchmark, v1.2.0 - 01-31-2017. For the ComplianceAsCode project to remain in compliance with CIS' terms and conditions, specifically Restrictions(8), note there is no representation or claim that the OpenCIS profile will ensure a system is in compliance or consistency with the CIS baseline.
This guide presents a catalog of security-relevant configuration settings for Red Hat OpenShift Container Platform 3. It is a rendering of content structured in the eXtensible Configuration Checklist Description Format (XCCDF) in order to support security automation. The SCAP content is is available in the scap-security-guide package which is developed at https://www.open-scap.org/security-policies/scap-security-guide.

Providing system administrators with such guidance informs them how to securely configure systems under their control in a variety of network roles. Policy makers and baseline creators can use this catalog of settings, with its associated references to higher-level security control catalogs, in order to assist them in security baseline creation. This guide is a catalog, not a checklist, and satisfaction of every item is not likely to be possible or sensible in many operational scenarios. However, the XCCDF format enables granular selection and adjustment of settings, and their association with OVAL and OCIL content provides an automated checking capability. Transformations of this document, and its associated automated checking content, are capable of providing baselines that meet a diverse set of policy objectives. Some example XCCDF Profiles, which are selections of items that form checklists and can be used as baselines, are available with this guide. They can be processed, in an automated fashion, with tools that support the Security Content Automation Protocol (SCAP). The NIST National Checklist Program (NCP), which provides required settings for the United States Government, is one example of a baseline created from this guidance.
Do not attempt to implement any of the settings in this guide without first testing them in a non-operational environment. The creators of this guidance assume no responsibility whatsoever for its use by other parties, and makes no guarantees, expressed or implied, about its quality, reliability, or any other characteristic.

Profile Information

Profile TitleOpen Computing Information Security Profile for OpenShift Node
Profile IDxccdf_org.ssgproject.content_profile_opencis-node

CPE Platforms

  • cpe:/a:redhat:openshift_container_platform:3.10
  • cpe:/a:redhat:openshift_container_platform:3.11

Revision History

Current version: 0.1.44

  • draft (as of 2019-05-03)

Table of Contents

  1. OpenShift Settings
    1. Kubernetes Kubelet Settings
    2. Permissions

Checklist

Group   Guide to the Secure Configuration of Red Hat OpenShift Container Platform 3   Group contains 4 groups and 20 rules
Group   OpenShift Settings   Group contains 3 groups and 20 rules

[ref]   Each section of this configuration guide includes information about the default configuration of an OpenShift cluster and a set of recommendations for hardening the configuration. For each hardening recommendation, information on how to implement the control and/or how to verify or audit the control is provided. In some cases, remediation information is also provided. Many of the settings in the hardening guide are in place by default. The audit information for these settings is provided in order to verify that the cluster admininstrator has not made changes that would be less secure than the OpenShift defaults. A small number of items require configuration. Finally, there are some recommendations that require decisions by the system operator, such as audit log size, retention, and related settings.

Group   Kubernetes Kubelet Settings   Group contains 11 rules

[ref]   The Kubernetes Kubelet is an agent that runs on each node in the cluster. It makes sure that containers are running in a pod. The kubelet takes a set of PodSpecs that are provided through various mechanisms and ensures that the containers described in those PodSpecs are running and healthy. The kubelet doesn’t manage containers which were not created by Kubernetes.

Rule   kubelet - Configure the Client CA Certificate   [ref]

By default, the kubelet is not configured with a CA certificate which can subject the kubelet to man-in-the-middle attacks. To configure a client CA certificate, edit the kubelet configuration file /etc/origin/node/node-config.yaml on the kubelet node(s) and set the below parameter:

  servingInfo:
  clientCA: client-ca.crt

Rationale:

Not having a CA certificate for the kubelet will subject the kubelet to possible man-in-the-middle attacks especially on unsafe or untrusted networks. Certificate validation for the kubelet allows the API server to validate the kubelet's identity.

Severity: 
medium
Identifiers and References

Identifiers:  CCE-80594-5

References:  2.1.4

Rule   kubelet - Disable the Read-Only Port   [ref]

To disable the read-only port, edit the kubelet configuration file /etc/origin/node/node-config.yaml on the kubelet node(s) and set the below parameter:

kubeletArguments:
  read-only-port:
  - '0'

Rationale:

OpenShift disables the read-only port (10255) on all nodes by setting the read-only-port kubelet flag to 0. This ensures only authenticated connections are able to receive information about the OpenShift system.

Severity: 
medium
Identifiers and References

Identifiers:  CCE-80601-8

References:  2.1.5

Rule   kubelet - Allow Automatic Firewall Configuration   [ref]

The kubelet has the ability to automatically configure the firewall to allow the containers required ports and connections to networking resources and destinations parameters potentially creating a security incident. To allow the kubelet to modify the firewall, edit the kubelet configuration file /etc/origin/node/node-config.yaml on the kubelet node(s) and set the below parameter:

kubeletArguments:
  make-iptables-util-chains:
  - 'true'

Warning:  IT IS NOT RECOMMENDED FOR ANY REASON to manually configure firewall ports for running pods and containers as this not only can create conflicts with firewall rules but can also introduce inconsistences into the allowed/disabled ports on the firewall.
Rationale:

The kubelet should automatically configure the firewall settings to allow access and networking traffic through. This ensures that when a pod or container is running that the correct ports are configured as well as removing the ports when a pod or container is no longer in existence.

Severity: 
medium
Identifiers and References

Identifiers:  CCE-80604-2

References:  2.1.8

Rule   kubelet - Enable Server Certificate Rotation   [ref]

To enable the kubelet to rotate server certificates, edit the kubelet configuration file /etc/origin/node/node-config.yaml on the kubelet node(s) and ensure RotateKubeletServerCertificate is included in the feature-gates arguments:

kubeletArguments:
  feature-gates:
    - RotateKubeletClientCertificate=true,RotateKubeletServerCertificate=true

Rationale:

Allowing the kubelet to auto-update the certificates ensure that there is no downtime in certificate renewal as well as ensures confidentiality and integrity.

Severity: 
medium
Identifiers and References

Identifiers:  CCE-80606-7

References:  1.3.7

Rule   Ensure That The kubelet Client Certificate Is Correctly Set   [ref]

To ensure the kubelet TLS client certificate is configured, edit the kubelet configuration file /etc/origin/node/node-config.yaml and configure the cert-dir path for the kubelet certificates. For example:

  cert-dir:
  - /etc/origin/node/certificates
A corresponding certificate should exist in the cert-dir. For example:
/etc/origin/node/certificates/kubelet-client-current.pem

Rationale:

Without cryptographic integrity protections, information can be altered by unauthorized users without detection.

Severity: 
medium
Identifiers and References

Identifiers:  CCE-80596-0

References:  2.1.12

Rule   kubelet - Do Not Disable Streaming Timeouts   [ref]

Timouts for streaming connections should not be disabled as they help to prevent denial-of-service attacks. To configure streaming connection timeouts, edit the kubelet configuration file /etc/origin/node/node-config.yaml on the kubelet node(s) and set the below parameter:

kubeletArguments:
  streaming-connection-timeout:
  - '4h'

Rationale:

Ensuring connections have timeouts helps to protect against denial-of-service attacks as well as disconnect inactive connections. In addition, setting connections timeouts helps to prevent from running out of ephemeral ports.

Severity: 
medium
Identifiers and References

Identifiers:  CCE-80607-5

References:  2.1.6

Rule   kubelet - Do Not Limit Event Creation   [ref]

All events should be captured and not restricted as this helps in reconstucting the chain-of-events. To prevent log creation limiting, edit the kubelet configuration file /etc/origin/node/node-config.yaml on the kubelet node(s) and set the below parameter:

kubeletArguments:
  event-qps:
  - '0'

Rationale:

All events should be captured and not restricted as this helps in reconstucting the chain-of-events.

Severity: 
medium
Identifiers and References

Identifiers:  CCE-80595-2

References:  2.1.11

Rule   Ensure That The kubelet Server Key Is Correctly Set   [ref]

To ensure the kubelet TLS server key certificate is configured, edit the kubelet configuration file /etc/origin/node/node-config.yaml and configure the cert-dir path for the kubelet certificates. For example:

  cert-dir:
  - /etc/origin/node/certificates
A corresponding certificate should exist in the cert-dir. For example:
/etc/origin/node/certificates/kubelet-server-current.pem

Rationale:

Without cryptographic integrity protections, information can be altered by unauthorized users without detection.

Severity: 
medium
Identifiers and References

Identifiers:  CCE-80597-8

References:  2.1.12

Rule   kubelet - Disable Hostname Override   [ref]

To prevent the hostname from being overrided, edit the kubelet configuration file /etc/origin/node/node-config.yaml on the kubelet node(s) and remove the hostname-override option if it exists.

Rationale:

Allowing hostnames to be overrided creates issues around resolving nodes in addition to TLS configuration, certificate validation, and log correlation and validation.

Severity: 
medium
Identifiers and References

Identifiers:  CCE-80600-0

References:  2.1.10

Rule   kubelet - Enable Client Certificate Rotation   [ref]

To enable the kubelet to rotate client certificates, edit the kubelet configuration file /etc/origin/node/node-config.yaml on the kubelet node(s) and set the below parameter:

kubeletArguments:
  feature-gates:
    - RotateKubeletClientCertificate=true

Rationale:

Allowing the kubelet to auto-update the certificates ensure that there is no downtime in certificate renewal as well as ensures confidentiality and integrity.

Severity: 
medium
Identifiers and References

Identifiers:  CCE-80603-4

References:  2.1.14

Rule   kubelet - Disable cAdvisor Port   [ref]

The cAdvisor port should be disabled as it does not require any authentication to connect to the port. To disable the cAdvisor port, edit the kubelet configuration file /etc/origin/node/node-config.yaml on the kubelet node(s) and set the below parameter:

kubeletArguments:
  cadvisor-port:
  - '0'

Rationale:

Any form of authentication to ports anonymously should be disabled. An attacker could connect to the port and gain cluster information anonymously.

Severity: 
high
Identifiers and References

Identifiers:  CCE-80599-4

References:  2.1.13

Group   Permissions   Group contains 1 group and 9 rules

[ref]   Traditional security relies heavily on file and directory permissions to prevent unauthorized users from reading or modifying files to which they should not have access.

Group   Verify Permissions on Important Files and Directories   Group contains 9 rules

[ref]   Permissions for many files on a system must be set restrictively to ensure sensitive information is properly protected. This section discusses important permission restrictions which can be verified to ensure that no harmful discrepancies have arisen.

Rule   Verify User Who Owns The OpenShift Node Configuration File   [ref]

To properly set the owner of /etc/origin/node/node-config.yaml, run the command:

$ sudo chown root /etc/origin/node/node-config.yaml 

Rationale:

The /etc/origin/node/node-config.yaml file contains information about the configuration of the OpenShift node that is configured on the system. Protection of this file is critical for OpenShift security.

Severity: 
medium
Identifiers and References

Identifiers:  CCE-80629-9

References:  2.2.2

Remediation Shell script:   (show)

Complexity:low
Disruption:low
Strategy:configure

chown 0 /etc/origin/node/node-config.yaml
Remediation Ansible snippet:   (show)

Complexity:low
Disruption:low
Strategy:configure
- name: Test for existence /etc/origin/node/node-config.yaml
  stat:
    path: /etc/origin/node/node-config.yaml
  register: file_exists
  tags:
    - file_owner_node_config
    - medium_severity
    - configure_strategy
    - low_complexity
    - low_disruption
    - no_reboot_needed
    - CCE-80629-9

- name: Ensure owner 0 on /etc/origin/node/node-config.yaml
  file:
    path: /etc/origin/node/node-config.yaml
    owner: 0
  when: file_exists.stat is defined and file_exists.stat.exists
  tags:
    - file_owner_node_config
    - medium_severity
    - configure_strategy
    - low_complexity
    - low_disruption
    - no_reboot_needed
    - CCE-80629-9

Rule   Verify Permissions on the OpenShift Node Service File   [ref]

To properly set the permissions of /etc/systemd/system/atomic-openshift-node.service, run the command:

$ sudo chmod 0644 /etc/systemd/system/atomic-openshift-node.service

Rationale:

If the /etc/systemd/system/atomic-openshift-node.service file is writable by a group-owner or the world the risk of its compromise is increased. The file contains the service configuration of the OpenShift node service that is configured on the system. Protection of this file is critical for OpenShift security.

Severity: 
medium
Identifiers and References

Identifiers:  CCE-80643-0

References:  2.2.3

Remediation Shell script:   (show)

Complexity:low
Disruption:low
Strategy:configure

chmod 0644 /etc/systemd/system/atomic-openshift-node.service
Remediation Ansible snippet:   (show)

Complexity:low
Disruption:low
Strategy:configure
- name: Test for existence /etc/systemd/system/atomic-openshift-node.service
  stat:
    path: /etc/systemd/system/atomic-openshift-node.service
  register: file_exists
  tags:
    - file_permissions_openshift_node_service
    - medium_severity
    - configure_strategy
    - low_complexity
    - low_disruption
    - no_reboot_needed
    - CCE-80643-0

- name: Ensure permission 0644 on /etc/systemd/system/atomic-openshift-node.service
  file:
    path: /etc/systemd/system/atomic-openshift-node.service
    mode: 420
  when: file_exists.stat is defined and file_exists.stat.exists
  tags:
    - file_permissions_openshift_node_service
    - medium_severity
    - configure_strategy
    - low_complexity
    - low_disruption
    - no_reboot_needed
    - CCE-80643-0

Rule   Verify Group Who Owns The OpenShift Node Configuration File   [ref]

To properly set the group owner of /etc/origin/node/node-config.yaml, run the command:

$ sudo chgrp root /etc/origin/node/node-config.yaml

Rationale:

The /etc/origin/node/node-config.yaml file contains information about the configuration of the OpenShift node that is configured on the system. Protection of this file is critical for OpenShift security.

Severity: 
medium
Identifiers and References

Identifiers:  CCE-80617-4

References:  2.2.2

Remediation Shell script:   (show)

Complexity:low
Disruption:low
Strategy:configure

chgrp 0 /etc/origin/node/node-config.yaml
Remediation Ansible snippet:   (show)

Complexity:low
Disruption:low
Strategy:configure
- name: Test for existence /etc/origin/node/node-config.yaml
  stat:
    path: /etc/origin/node/node-config.yaml
  register: file_exists
  tags:
    - file_groupowner_node_config
    - medium_severity
    - configure_strategy
    - low_complexity
    - low_disruption
    - no_reboot_needed
    - CCE-80617-4

- name: Ensure group owner 0 on /etc/origin/node/node-config.yaml
  file:
    path: /etc/origin/node/node-config.yaml
    group: 0
  when: file_exists.stat is defined and file_exists.stat.exists
  tags:
    - file_groupowner_node_config
    - medium_severity
    - configure_strategy
    - low_complexity
    - low_disruption
    - no_reboot_needed
    - CCE-80617-4

Rule   Verify Group Who Owns OpenShift Node Certificate File   [ref]

To properly set the group owner of /etc/origin/node/client-ca.crt, run the command:

$ sudo chgrp root /etc/origin/node/client-ca.crt

Rationale:

The /etc/origin/node/client-ca.crt file contains the certificate authority certificate for an OpenShift node that is configured on the system. Protection of this file is critical for OpenShift security.

Severity: 
medium
Identifiers and References

Identifiers:  CCE-80619-0

References:  2.2.8

Remediation Shell script:   (show)

Complexity:low
Disruption:low
Strategy:configure

chgrp 0 /etc/origin/node/client-ca.crt
Remediation Ansible snippet:   (show)

Complexity:low
Disruption:low
Strategy:configure
- name: Test for existence /etc/origin/node/client-ca.crt
  stat:
    path: /etc/origin/node/client-ca.crt
  register: file_exists
  tags:
    - file_groupowner_openshift_node_client_crt
    - medium_severity
    - configure_strategy
    - low_complexity
    - low_disruption
    - no_reboot_needed
    - CCE-80619-0

- name: Ensure group owner 0 on /etc/origin/node/client-ca.crt
  file:
    path: /etc/origin/node/client-ca.crt
    group: 0
  when: file_exists.stat is defined and file_exists.stat.exists
  tags:
    - file_groupowner_openshift_node_client_crt
    - medium_severity
    - configure_strategy
    - low_complexity
    - low_disruption
    - no_reboot_needed
    - CCE-80619-0

Rule   Verify User Who Owns OpenShift Node Certificate File   [ref]

To properly set the owner of /etc/origin/node/client-ca.crt, run the command:

$ sudo chown root /etc/origin/node/client-ca.crt 

Rationale:

The /etc/origin/node/client-ca.crt file contains the certificate authority certificate for an OpenShift node that is configured on the system. Protection of this file is critical for OpenShift security.

Severity: 
medium
Identifiers and References

Identifiers:  CCE-80631-5

References:  2.2.8

Remediation Shell script:   (show)

Complexity:low
Disruption:low
Strategy:configure

chown 0 /etc/origin/node/client-ca.crt
Remediation Ansible snippet:   (show)

Complexity:low
Disruption:low
Strategy:configure
- name: Test for existence /etc/origin/node/client-ca.crt
  stat:
    path: /etc/origin/node/client-ca.crt
  register: file_exists
  tags:
    - file_owner_openshift_node_client_crt
    - medium_severity
    - configure_strategy
    - low_complexity
    - low_disruption
    - no_reboot_needed
    - CCE-80631-5

- name: Ensure owner 0 on /etc/origin/node/client-ca.crt
  file:
    path: /etc/origin/node/client-ca.crt
    owner: 0
  when: file_exists.stat is defined and file_exists.stat.exists
  tags:
    - file_owner_openshift_node_client_crt
    - medium_severity
    - configure_strategy
    - low_complexity
    - low_disruption
    - no_reboot_needed
    - CCE-80631-5

Rule   Verify Group Who Owns The OpenShift Node Service File   [ref]

To properly set the group owner of /etc/systemd/system/atomic-openshift-node.service, run the command:

$ sudo chgrp root /etc/systemd/system/atomic-openshift-node.service

Rationale:

The /etc/systemd/system/atomic-openshift-node.service file contains information about the configuration of the OpenShift node service that is configured on the system. Protection of this file is critical for OpenShift security.

Severity: 
medium
Identifiers and References

Identifiers:  CCE-80620-8

References:  2.2.4

Remediation Shell script:   (show)

Complexity:low
Disruption:low
Strategy:configure

chgrp 0 /etc/systemd/system/atomic-openshift-node.service
Remediation Ansible snippet:   (show)

Complexity:low
Disruption:low
Strategy:configure
- name: Test for existence /etc/systemd/system/atomic-openshift-node.service
  stat:
    path: /etc/systemd/system/atomic-openshift-node.service
  register: file_exists
  tags:
    - file_groupowner_openshift_node_service
    - medium_severity
    - configure_strategy
    - low_complexity
    - low_disruption
    - no_reboot_needed
    - CCE-80620-8

- name: Ensure group owner 0 on /etc/systemd/system/atomic-openshift-node.service
  file:
    path: /etc/systemd/system/atomic-openshift-node.service
    group: 0
  when: file_exists.stat is defined and file_exists.stat.exists
  tags:
    - file_groupowner_openshift_node_service
    - medium_severity
    - configure_strategy
    - low_complexity
    - low_disruption
    - no_reboot_needed
    - CCE-80620-8

Rule   Verify Permissions on the OpenShift Node Configuration File   [ref]

To properly set the permissions of /etc/origin/node/node-config.yaml, run the command:

$ sudo chmod 0600 /etc/origin/node/node-config.yaml

Rationale:

If the /etc/origin/node/node-config.yaml file is writable by a group-owner or the world the risk of its compromise is increased. The file contains the configuration of an OpenShift node that is configured on the system. Protection of this file is critical for OpenShift security.

Severity: 
medium
Identifiers and References

Identifiers:  CCE-80640-6

References:  2.2.1

Remediation Shell script:   (show)

Complexity:low
Disruption:low
Strategy:configure

chmod 0600 /etc/origin/node/node-config.yaml
Remediation Ansible snippet:   (show)

Complexity:low
Disruption:low
Strategy:configure
- name: Test for existence /etc/origin/node/node-config.yaml
  stat:
    path: /etc/origin/node/node-config.yaml
  register: file_exists
  tags:
    - file_permissions_node_config
    - medium_severity
    - configure_strategy
    - low_complexity
    - low_disruption
    - no_reboot_needed
    - CCE-80640-6

- name: Ensure permission 0600 on /etc/origin/node/node-config.yaml
  file:
    path: /etc/origin/node/node-config.yaml
    mode: 384
  when: file_exists.stat is defined and file_exists.stat.exists
  tags:
    - file_permissions_node_config
    - medium_severity
    - configure_strategy
    - low_complexity
    - low_disruption
    - no_reboot_needed
    - CCE-80640-6

Rule   Verify Permissions on OpenShift Node Certificate File   [ref]

To properly set the permissions of /etc/origin/node/client-ca.crt, run the command:

$ sudo chmod 0644 /etc/origin/node/client-ca.crt

Rationale:

If the /etc/origin/node/client-ca.crt file is writable by a group-owner or the world the risk of its compromise is increased. The file contains the certificate authority certificate for an OpenShift node that is configured on the system. Protection of this file is critical for OpenShift security.

Severity: 
medium
Identifiers and References

Identifiers:  CCE-80642-2

References:  2.2.

Remediation Shell script:   (show)

Complexity:low
Disruption:low
Strategy:configure

chmod 0600 /etc/origin/node/client-ca.crt
Remediation Ansible snippet:   (show)

Complexity:low
Disruption:low
Strategy:configure
- name: Test for existence /etc/origin/node/client-ca.crt
  stat:
    path: /etc/origin/node/client-ca.crt
  register: file_exists
  tags:
    - file_permissions_openshift_node_client_crt
    - medium_severity
    - configure_strategy
    - low_complexity
    - low_disruption
    - no_reboot_needed
    - CCE-80642-2

- name: Ensure permission 0600 on /etc/origin/node/client-ca.crt
  file:
    path: /etc/origin/node/client-ca.crt
    mode: 384
  when: file_exists.stat is defined and file_exists.stat.exists
  tags:
    - file_permissions_openshift_node_client_crt
    - medium_severity
    - configure_strategy
    - low_complexity
    - low_disruption
    - no_reboot_needed
    - CCE-80642-2

Rule   Verify User Who Owns The OpenShift Node Service File   [ref]

To properly set the owner of /etc/systemd/system/atomic-openshift-node.service, run the command:

$ sudo chown root /etc/systemd/system/atomic-openshift-node.service 

Rationale:

The /etc/systemd/system/atomic-openshift-node.service file contains information about the configuration of the OpenShift node service that is configured on the system. Protection of this file is critical for OpenShift security.

Severity: 
medium
Identifiers and References

Identifiers:  CCE-80632-3

References:  2.2.4

Remediation Shell script:   (show)

Complexity:low
Disruption:low
Strategy:configure

chown 0 /etc/systemd/system/atomic-openshift-node.service
Remediation Ansible snippet:   (show)

Complexity:low
Disruption:low
Strategy:configure
- name: Test for existence /etc/systemd/system/atomic-openshift-node.service
  stat:
    path: /etc/systemd/system/atomic-openshift-node.service
  register: file_exists
  tags:
    - file_owner_openshift_node_service
    - medium_severity
    - configure_strategy
    - low_complexity
    - low_disruption
    - no_reboot_needed
    - CCE-80632-3

- name: Ensure owner 0 on /etc/systemd/system/atomic-openshift-node.service
  file:
    path: /etc/systemd/system/atomic-openshift-node.service
    owner: 0
  when: file_exists.stat is defined and file_exists.stat.exists
  tags:
    - file_owner_openshift_node_service
    - medium_severity
    - configure_strategy
    - low_complexity
    - low_disruption
    - no_reboot_needed
    - CCE-80632-3
Red Hat and Red Hat Enterprise Linux are either registered trademarks or trademarks of Red Hat, Inc. in the United States and other countries. All other names are registered trademarks or trademarks of their respective companies.