Guide to the Secure Configuration of Red Hat OpenShift Container Platform 3

with profile Open Computing Information Security Profile for OpenShift Node
This baseline was inspired by the Center for Internet Security (CIS) Kubernetes Benchmark, v1.2.0 - 01-31-2017. For the ComplianceAsCode project to remain in compliance with CIS' terms and conditions, specifically Restrictions(8), note there is no representation or claim that the OpenCIS profile will ensure a system is in compliance or consistency with the CIS baseline.
This guide presents a catalog of security-relevant configuration settings for Red Hat OpenShift Container Platform 3. It is a rendering of content structured in the eXtensible Configuration Checklist Description Format (XCCDF) in order to support security automation. The SCAP content is is available in the scap-security-guide package which is developed at https://www.open-scap.org/security-policies/scap-security-guide.

Providing system administrators with such guidance informs them how to securely configure systems under their control in a variety of network roles. Policy makers and baseline creators can use this catalog of settings, with its associated references to higher-level security control catalogs, in order to assist them in security baseline creation. This guide is a catalog, not a checklist, and satisfaction of every item is not likely to be possible or sensible in many operational scenarios. However, the XCCDF format enables granular selection and adjustment of settings, and their association with OVAL and OCIL content provides an automated checking capability. Transformations of this document, and its associated automated checking content, are capable of providing baselines that meet a diverse set of policy objectives. Some example XCCDF Profiles, which are selections of items that form checklists and can be used as baselines, are available with this guide. They can be processed, in an automated fashion, with tools that support the Security Content Automation Protocol (SCAP). The NIST National Checklist Program (NCP), which provides required settings for the United States Government, is one example of a baseline created from this guidance.
Do not attempt to implement any of the settings in this guide without first testing them in a non-operational environment. The creators of this guidance assume no responsibility whatsoever for its use by other parties, and makes no guarantees, expressed or implied, about its quality, reliability, or any other characteristic.
Profile TitleOpen Computing Information Security Profile for OpenShift Node
Profile IDxccdf_org.ssgproject.content_profile_opencis-ocp-node

Revision History

Current version: 0.1.42

  • draft (as of 2018-12-11)

Platforms

  • cpe:/a:redhat:openshift_container_platform:3.10
  • cpe:/a:redhat:openshift_container_platform:3.11

Table of Contents

  1. OpenShift Settings
    1. Kubernetes Kubelet Settings
    2. Permissions

Checklist

contains 24 rules

OpenShift Settings   [ref]group

Contains rules that check correct OpenShift settings.

contains 24 rules

Kubernetes Kubelet Settings   [ref]group

The Kubernetes Kubelet is an agent that runs on each node in the cluster. It makes sure that containers are running in a pod. The kubelet takes a set of PodSpecs that are provided through various mechanisms and ensures that the containers described in those PodSpecs are running and healthy. The kubelet doesn’t manage containers which were not created by Kubernetes.

contains 15 rules

kubelet - Configure the Client CA Certificate   [ref]rule

By default, the kubelet is not configured with a CA certificate which can subject the kubelet to man-in-the-middle attacks. To configure a client CA certificate, edit the kubelet configuration file /etc/origin/node/node-config.yaml on the kubelet node(s) and set the below parameter:

  servingInfo:
  clientCA: client-ca.crt

Rationale:

Not having a CA certificate for the kubelet will subject the kubelet to possible man-in-the-middle attacks especially on unsafe or untrusted networks. Certificate validation for the kubelet allows the API server to validate the kubelet's identity.

Severity:  medium

References:  2.1.4

kubelet - Disable the Read-Only Port   [ref]rule

To disable the read-only port, edit the kubelet configuration file /etc/origin/node/node-config.yaml on the kubelet node(s) and set the below parameter:

kubeletArguments:
  read-only-port:
  - '0'

Rationale:

Not having a CA certificate for the kubelet will subject the kubelet to possible man-in-the-middle attacks especially on unsafe or untrusted networks. Certificate validation for the kubelet allows the API server to validate the kubelet's identity.

Severity:  medium

References:  2.1.5

kubelet - Prevent Kernel Settings Modification   [ref]rule

By default, the kubelet has the ability to configure the kernel's settings and parameters potentially creating a security incident. To prevent the kubelet from changing kernel parameters, edit the kubelet configuration file /etc/origin/node/node-config.yaml on the kubelet node(s) and set the below parameter:

kubeletArguments:
  protect-kernel-defaults:
  - 'true'

Rationale:

Preventing the kubelet from modifying kernel parameters or tunables will ensure that system stays in its existing tuned and hardened state. Allowing kubelets to configure the kernel settings not only generates actionable audit records but also could open the system up to new vectors of attack.

Severity:  medium

References:  2.1.7

kubelet - Allow Automatic Firewall Configuration   [ref]rule

The kubelet has the ability to automatically configure the firewall to allow the containers required ports and connections to networking resources and destinations parameters potentially creating a security incident. To allow the kubelet to modify the firewall, edit the kubelet configuration file /etc/origin/node/node-config.yaml on the kubelet node(s) and set the below parameter:

kubeletArguments:
  make-iptables-util-chains:
  - 'true'

Warning:  -| IT IS NOT RECOMMENDED FOR ANY REASON to manually configure firewall ports for running pods and containers as this not only can create conflicts with firewall rules but can also introduce inconsistences into the allowed/disabled ports on the firewall.
Rationale:

The kubelet should automatically configure the firewall settings to allow access and networking traffic through. This ensures that when a pod or container is running that the correct ports are configured as well as removing the ports when a pod or container is no longer in existence.

Severity:  medium

References:  2.1.8

kubelet - Disable Anonymous Access   [ref]rule

By default, anonymous access to the kubelet is enabled. This configuration check ensures that anonymous requests to the kubelet are disabled. Edit the kubelet configuration file /etc/origin/node/node-config.yaml on the kubelet node(s) and set the below parameter:

kubeletArguments:
  anonymous-auth:
  - 'false'

Rationale:

When enabled, requests that are not rejected by other configured authentication methods are treated as anonymous requests. OpenShift Operators should rely on authentication to authorize access and disallow anonymous requests.

Severity:  medium

References:  2.1.2

kubelet - Enable Server Certificate Rotation   [ref]rule

To enable the kubelet to rotate server certificates, edit the kubelet configuration file /etc/origin/node/node-config.yaml on the kubelet node(s) and ensure RotateKubeletServerCertificate is included in the feature-gates arguments:

kubeletArguments:
  feature-gates:
    - RotateKubeletClientCertificate=true,RotateKubeletServerCertificate=true

Rationale:

Allowing the kubelet to auto-update the certificates ensure that there is no downtime in certificate renewal as well as ensures confidentiality and integrity.

Severity:  medium

References:  2.1.15

Ensure That The kubelet Client Certificate Is Correctly Set   [ref]rule

To ensure the kubelet TLS client certificate is configured, edit the kubelet configuration file /etc/origin/node/node-config.yaml and configure the cert-dir path for the kubelet certificates. For example:

  cert-dir:
  - /etc/origin/node/certificates
A corresponding certificate should exist in the cert-dir. For example:
/etc/origin/node/certificates/kubelet-client-current.pem

Rationale:

Without cryptographic integrity protections, information can be altered by unauthorized users without detection.

Severity:  medium

References:  2.1.12

kubelet - Disable Privileged Containers   [ref]rule

To disable privileged containers, edit the kubelet configuration file /etc/origin/node/node-config.yaml on the kubelet node(s) and set the below parameter:

kubeletArguments:
  allow-privileged:
  - 'false'

Rationale:

When enabled, privileged containers have unrestricted access and should be disabled. A privileged continer will run with security mechanisms disabled which will put the system at risk.

Severity:  medium

References:  2.1.1

kubelet - Do Not Disable Streaming Timeouts   [ref]rule

Timouts for streaming connections should not be disabled as they help to prevent denial-of-service attacks. To configure streaming connection timeouts, edit the kubelet configuration file /etc/origin/node/node-config.yaml on the kubelet node(s) and set the below parameter:

kubeletArguments:
  streaming-connection-timeout:
  - '4h'

Rationale:

Ensuring connections have timeouts helps to protect against denial-of-service attacks as well as disconnect inactive connections. In addition, setting connections timeouts helps to prevent from running out of ephemeral ports.

Severity:  medium

References:  2.1.6

kubelet - Ensure AlwaysAllow Is Not Set   [ref]rule

By default, the kubelet allows all types of authentication requests including anonymous requests. The AlwaysAll option should not be used as it will allow all types of authentication requests. To require authorization requests, edit the kubelet configuration file /etc/origin/node/node-config.yaml on the kubelet node(s) and set the below parameter:

kubeletArguments:
  authorization-mode:
  - 'Webhook'

Rationale:

When enabled, all type of authentication requests including anonymous requests are enabled regardless of what is configured on the API server. OpenShift Operators should rely on authentication to authorize access and disallow anonymous requests.

Severity:  medium

References:  2.1.3

kubelet - Do Not Limit Event Creation   [ref]rule

All events should be captured and not restricted as this helps in reconstucting the chain-of-events. To prevent log creation limiting, edit the kubelet configuration file /etc/origin/node/node-config.yaml on the kubelet node(s) and set the below parameter:

kubeletArguments:
  event-qps:
  - '0'

Rationale:

All events should be captured and not restricted as this helps in reconstucting the chain-of-events.

Severity:  medium

References:  2.1.11

Ensure That The kubelet Server Key Is Correctly Set   [ref]rule

To ensure the kubelet TLS server key certificate is configured, edit the kubelet configuration file /etc/origin/node/node-config.yaml and configure the cert-dir path for the kubelet certificates. For example:

  cert-dir:
  - /etc/origin/node/certificates
A corresponding certificate should exist in the cert-dir. For example:
/etc/origin/node/certificates/kubelet-server-current.pem

Rationale:

Without cryptographic integrity protections, information can be altered by unauthorized users without detection.

Severity:  medium

References:  2.1.12

kubelet - Disable Hostname Override   [ref]rule

To prevent the hostname from being overrided, edit the kubelet configuration file /etc/origin/node/node-config.yaml on the kubelet node(s) and remove the hostname-override option if it exists.

Rationale:

Allowing hostnames to be overrided creates issues around resolving nodes in addition to TLS configuration, certificate validation, and log correlation and validation.

Severity:  medium

References:  2.1.10

kubelet - Enable Client Certificate Rotation   [ref]rule

To enable the kubelet to rotate client certificates, edit the kubelet configuration file /etc/origin/node/node-config.yaml on the kubelet node(s) and set the below parameter:

kubeletArguments:
  feature-gates:
    - RotateKubeletClientCertificate=true

Rationale:

Allowing the kubelet to auto-update the certificates ensure that there is no downtime in certificate renewal as well as ensures confidentiality and integrity.

Severity:  medium

References:  2.1.14

kubelet - Disable cAdvisor Port   [ref]rule

The cAdvisor port should be disabled as it does not require any authentication to connect to the port. To disable the cAdvisor port, edit the kubelet configuration file /etc/origin/node/node-config.yaml on the kubelet node(s) and set the below parameter:

kubeletArguments:
  cadvisor-port:
  - '0'

Rationale:

Any form of authentication to ports anonymously should be disabled. An attacker could connect to the port and gain cluster information anonymously.

Severity:  high

References:  2.1.13

Permissions   [ref]group

Traditional security relies heavily on file and directory permissions to prevent unauthorized users from reading or modifying files to which they should not have access.

contains 9 rules

Verify Permissions on Important Files and Directories   [ref]group

Permissions for many files on a system must be set restrictively to ensure sensitive information is properly protected. This section discusses important permission restrictions which can be verified to ensure that no harmful discrepancies have arisen.

contains 9 rules

Verify User Who Owns The OpenShift Node Configuration File   [ref]rule

To properly set the owner of /etc/origin/node/node-config.yaml, run the command:

$ sudo chown root /etc/origin/node/node-config.yaml 

Rationale:

The /etc/origin/node/node-config.yaml file contains information about the configuration of the OpenShift node that is configured on the system. Protection of this file is critical for OpenShift security.

Severity:  medium

References:  2.2.2

Remediation Shell script:   (show)

Complexity:low
Disruption:low
Strategy:configure

chown 0 /etc/origin/node/node-config.yaml
Remediation Ansible snippet:   (show)

Complexity:low
Disruption:low
Strategy:configure
- name: Test for existence /etc/origin/node/node-config.yaml
  stat:
    path: /etc/origin/node/node-config.yaml
  register: file_exists

- name: Ensure owner 0 on /etc/origin/node/node-config.yaml
  file:
    path: /etc/origin/node/node-config.yaml
    owner: 0
  when: file_exists.stat.exists and True
  tags:
    - file_owner_node_config
    - medium_severity
    - configure_strategy
    - low_complexity
    - low_disruption

Verify Permissions on the OpenShift Node Service File   [ref]rule

To properly set the permissions of /etc/systemd/system/atomic-openshift-node.service, run the command:

$ sudo chmod 0644 /etc/systemd/system/atomic-openshift-node.service

Rationale:

If the /etc/systemd/system/atomic-openshift-node.service file is writable by a group-owner or the world the risk of its compromise is increased. The file contains the service configuration of the OpenShift node service that is configured on the system. Protection of this file is critical for OpenShift security.

Severity:  medium

References:  2.2.3

Remediation Shell script:   (show)

Complexity:low
Disruption:low
Strategy:configure

chmod 0644 /etc/systemd/system/atomic-openshift-node.service
Remediation Ansible snippet:   (show)

Complexity:low
Disruption:low
Strategy:configure
- name: Ensure permission 0644 on /etc/systemd/system/atomic-openshift-node.service
  file:
    path: /etc/systemd/system/atomic-openshift-node.service
    mode: 0644
  tags:
    - file_permissions_openshift_node_service
    - medium_severity
    - configure_strategy
    - low_complexity
    - low_disruption
  

Verify Group Who Owns The OpenShift Node Configuration File   [ref]rule

To properly set the group owner of /etc/origin/node/node-config.yaml, run the command:

$ sudo chgrp root /etc/origin/node/node-config.yaml

Rationale:

The /etc/origin/node/node-config.yaml file contains information about the configuration of the OpenShift node that is configured on the system. Protection of this file is critical for OpenShift security.

Severity:  medium

References:  2.2.2

Remediation Shell script:   (show)

Complexity:low
Disruption:low
Strategy:configure

chgrp 0 /etc/origin/node/node-config.yaml
Remediation Ansible snippet:   (show)

Complexity:low
Disruption:low
Strategy:configure
- name: Test for existence /etc/origin/node/node-config.yaml
  stat:
    path: /etc/origin/node/node-config.yaml
  register: file_exists

- name: Ensure group owner 0 on /etc/origin/node/node-config.yaml
  file:
    path: /etc/origin/node/node-config.yaml
    group: 0
  when: file_exists.stat.exists and True
  tags:
    - file_groupowner_node_config
    - medium_severity
    - configure_strategy
    - low_complexity
    - low_disruption

Verify Group Who Owns OpenShift Node Certificate File   [ref]rule

To properly set the group owner of /etc/origin/node/client-ca.crt, run the command:

$ sudo chgrp root /etc/origin/node/client-ca.crt

Rationale:

The /etc/origin/node/client-ca.crt file contains the certificate authority certificate for an OpenShift node that is configured on the system. Protection of this file is critical for OpenShift security.

Severity:  medium

References:  2.2.8

Remediation Shell script:   (show)

Complexity:low
Disruption:low
Strategy:configure

chgrp 0 /etc/origin/node/client-ca.crt
Remediation Ansible snippet:   (show)

Complexity:low
Disruption:low
Strategy:configure
- name: Test for existence /etc/origin/node/client-ca.crt
  stat:
    path: /etc/origin/node/client-ca.crt
  register: file_exists

- name: Ensure group owner 0 on /etc/origin/node/client-ca.crt
  file:
    path: /etc/origin/node/client-ca.crt
    group: 0
  when: file_exists.stat.exists and True
  tags:
    - file_groupowner_openshift_node_client_crt
    - medium_severity
    - configure_strategy
    - low_complexity
    - low_disruption

Verify User Who Owns OpenShift Node Certificate File   [ref]rule

To properly set the owner of /etc/origin/node/client-ca.crt, run the command:

$ sudo chown root /etc/origin/node/client-ca.crt 

Rationale:

The /etc/origin/node/client-ca.crt file contains the certificate authority certificate for an OpenShift node that is configured on the system. Protection of this file is critical for OpenShift security.

Severity:  medium

References:  2.2.8

Remediation Shell script:   (show)

Complexity:low
Disruption:low
Strategy:configure

chown 0 /etc/origin/node/client-ca.crt
Remediation Ansible snippet:   (show)

Complexity:low
Disruption:low
Strategy:configure
- name: Test for existence /etc/origin/node/client-ca.crt
  stat:
    path: /etc/origin/node/client-ca.crt
  register: file_exists

- name: Ensure owner 0 on /etc/origin/node/client-ca.crt
  file:
    path: /etc/origin/node/client-ca.crt
    owner: 0
  when: file_exists.stat.exists and True
  tags:
    - file_owner_openshift_node_client_crt
    - medium_severity
    - configure_strategy
    - low_complexity
    - low_disruption

Verify Group Who Owns The OpenShift Node Service File   [ref]rule

To properly set the group owner of /etc/systemd/system/atomic-openshift-node.service, run the command:

$ sudo chgrp root /etc/systemd/system/atomic-openshift-node.service

Rationale:

The /etc/systemd/system/atomic-openshift-node.service file contains information about the configuration of the OpenShift node service that is configured on the system. Protection of this file is critical for OpenShift security.

Severity:  medium

References:  2.2.4

Remediation Shell script:   (show)

Complexity:low
Disruption:low
Strategy:configure

chgrp 0 /etc/systemd/system/atomic-openshift-node.service
Remediation Ansible snippet:   (show)

Complexity:low
Disruption:low
Strategy:configure
- name: Test for existence /etc/systemd/system/atomic-openshift-node.service
  stat:
    path: /etc/systemd/system/atomic-openshift-node.service
  register: file_exists

- name: Ensure group owner 0 on /etc/systemd/system/atomic-openshift-node.service
  file:
    path: /etc/systemd/system/atomic-openshift-node.service
    group: 0
  when: file_exists.stat.exists and True
  tags:
    - file_groupowner_openshift_node_service
    - medium_severity
    - configure_strategy
    - low_complexity
    - low_disruption

Verify Permissions on the OpenShift Node Configuration File   [ref]rule

To properly set the permissions of /etc/origin/node/node-config.yaml, run the command:

$ sudo chmod 0600 /etc/origin/node/node-config.yaml

Rationale:

If the /etc/origin/node/node-config.yaml file is writable by a group-owner or the world the risk of its compromise is increased. The file contains the configuration of an OpenShift node that is configured on the system. Protection of this file is critical for OpenShift security.

Severity:  medium

References:  2.2.1

Remediation Shell script:   (show)

Complexity:low
Disruption:low
Strategy:configure

chmod 0600 /etc/origin/node/node-config.yaml
Remediation Ansible snippet:   (show)

Complexity:low
Disruption:low
Strategy:configure
- name: Ensure permission 0600 on /etc/origin/node/node-config.yaml
  file:
    path: /etc/origin/node/node-config.yaml
    mode: 0600
  tags:
    - file_permissions_node_config
    - medium_severity
    - configure_strategy
    - low_complexity
    - low_disruption
  

Verify Permissions on OpenShift Node Certificate File   [ref]rule

To properly set the permissions of /etc/origin/node/client-ca.crt, run the command:

$ sudo chmod 0644 /etc/origin/node/client-ca.crt

Rationale:

If the /etc/origin/node/client-ca.crt file is writable by a group-owner or the world the risk of its compromise is increased. The file contains the certificate authority certificate for an OpenShift node that is configured on the system. Protection of this file is critical for OpenShift security.

Severity:  medium

References:  2.2.

Remediation Shell script:   (show)

Complexity:low
Disruption:low
Strategy:configure

chmod 0600 /etc/origin/node/client-ca.crt
Remediation Ansible snippet:   (show)

Complexity:low
Disruption:low
Strategy:configure
- name: Ensure permission 0600 on /etc/origin/node/client-ca.crt
  file:
    path: /etc/origin/node/client-ca.crt
    mode: 0600
  tags:
    - file_permissions_openshift_node_client_crt
    - medium_severity
    - configure_strategy
    - low_complexity
    - low_disruption
  

Verify User Who Owns The OpenShift Node Service File   [ref]rule

To properly set the owner of /etc/systemd/system/atomic-openshift-node.service, run the command:

$ sudo chown root /etc/systemd/system/atomic-openshift-node.service 

Rationale:

The /etc/systemd/system/atomic-openshift-node.service file contains information about the configuration of the OpenShift node service that is configured on the system. Protection of this file is critical for OpenShift security.

Severity:  medium

References:  2.2.4

Remediation Shell script:   (show)

Complexity:low
Disruption:low
Strategy:configure

chown 0 /etc/systemd/system/atomic-openshift-node.service
Remediation Ansible snippet:   (show)

Complexity:low
Disruption:low
Strategy:configure
- name: Test for existence /etc/systemd/system/atomic-openshift-node.service
  stat:
    path: /etc/systemd/system/atomic-openshift-node.service
  register: file_exists

- name: Ensure owner 0 on /etc/systemd/system/atomic-openshift-node.service
  file:
    path: /etc/systemd/system/atomic-openshift-node.service
    owner: 0
  when: file_exists.stat.exists and True
  tags:
    - file_owner_openshift_node_service
    - medium_severity
    - configure_strategy
    - low_complexity
    - low_disruption
Red Hat and Red Hat Enterprise Linux are either registered trademarks or trademarks of Red Hat, Inc. in the United States and other countries. All other names are registered trademarks or trademarks of their respective companies.