Guide to the Secure Configuration of Red Hat OpenShift Container Platform 4

with profile Open Computing Information Security Profile for OpenShift Master Node
This baseline was inspired by the Center for Internet Security (CIS) Kubernetes Benchmark, v1.5.0 - 10-14-2019. For the ComplianceAsCode project to remain in compliance with CIS' terms and conditions, specifically Restrictions(8), note there is no representation or claim that the OpenCIS profile will ensure a system is in compliance or consistency with the CIS baseline.
This guide presents a catalog of security-relevant configuration settings for Red Hat OpenShift Container Platform 4. It is a rendering of content structured in the eXtensible Configuration Checklist Description Format (XCCDF) in order to support security automation. The SCAP content is is available in the scap-security-guide package which is developed at https://www.open-scap.org/security-policies/scap-security-guide.

Providing system administrators with such guidance informs them how to securely configure systems under their control in a variety of network roles. Policy makers and baseline creators can use this catalog of settings, with its associated references to higher-level security control catalogs, in order to assist them in security baseline creation. This guide is a catalog, not a checklist, and satisfaction of every item is not likely to be possible or sensible in many operational scenarios. However, the XCCDF format enables granular selection and adjustment of settings, and their association with OVAL and OCIL content provides an automated checking capability. Transformations of this document, and its associated automated checking content, are capable of providing baselines that meet a diverse set of policy objectives. Some example XCCDF Profiles, which are selections of items that form checklists and can be used as baselines, are available with this guide. They can be processed, in an automated fashion, with tools that support the Security Content Automation Protocol (SCAP). The NIST National Checklist Program (NCP), which provides required settings for the United States Government, is one example of a baseline created from this guidance.
Do not attempt to implement any of the settings in this guide without first testing them in a non-operational environment. The creators of this guidance assume no responsibility whatsoever for its use by other parties, and makes no guarantees, expressed or implied, about its quality, reliability, or any other characteristic.

Profile Information

Profile TitleOpen Computing Information Security Profile for OpenShift Master Node
Profile IDxccdf_org.ssgproject.content_profile_opencis-master

CPE Platforms

  • cpe:/a:redhat:openshift_container_platform:4.1

Revision History

Current version: 0.1.48

  • draft (as of 2020-01-15)

Table of Contents

  1. OpenShift Settings
    1. OpenShift etcd Settings
    2. OpenShift API Server

Checklist

Group   Guide to the Secure Configuration of Red Hat OpenShift Container Platform 4   Group contains 3 groups and 33 rules
Group   OpenShift Settings   Group contains 2 groups and 33 rules

[ref]   Each section of this configuration guide includes information about the default configuration of an OpenShift cluster and a set of recommendations for hardening the configuration. For each hardening recommendation, information on how to implement the control and/or how to verify or audit the control is provided. In some cases, remediation information is also provided. Many of the settings in the hardening guide are in place by default. The audit information for these settings is provided in order to verify that the cluster admininstrator has not made changes that would be less secure than the OpenShift defaults. A small number of items require configuration. Finally, there are some recommendations that require decisions by the system operator, such as audit log size, retention, and related settings.

Group   OpenShift etcd Settings   Group contains 10 rules

[ref]   Contains rules that check correct OpenShift etcd settings.

Rule   Disable etcd Self-Signed Certificates   [ref]

To ensure the etcd service is not using self-signed certificates, edit the etcd configuration file /etc/etcd/etcd.conf from the master node and set ETCD_AUTO_TLS to false:

ETCD_AUTO_TLS=false

Rationale:

Without cryptographic integrity protections, information can be altered by unauthorized users without detection. Using self-signed certificates ensures that the certificates are never validated against a certificate authority and could lead to compromised and invalidated data.

Severity: 
medium
Identifiers and References

References:  2.3

Remediation Shell script:   (show)

# Function to replace configuration setting in config file or add the configuration setting if
# it does not exist.
#
# Expects arguments:
#
# config_file:		Configuration file that will be modified
# key:			Configuration option to change
# value:		Value of the configuration option to change
# cce:			The CCE identifier or '@CCENUM@' if no CCE identifier exists
# format:		The printf-like format string that will be given stripped key and value as arguments,
#			so e.g. '%s=%s' will result in key=value subsitution (i.e. without spaces around =)
#
# Optional arugments:
#
# format:		Optional argument to specify the format of how key/value should be
# 			modified/appended in the configuration file. The default is key = value.
#
# Example Call(s):
#
#     With default format of 'key = value':
#     replace_or_append '/etc/sysctl.conf' '^kernel.randomize_va_space' '2' '@CCENUM@'
#
#     With custom key/value format:
#     replace_or_append '/etc/sysconfig/selinux' '^SELINUX=' 'disabled' '@CCENUM@' '%s=%s'
#
#     With a variable:
#     replace_or_append '/etc/sysconfig/selinux' '^SELINUX=' $var_selinux_state '@CCENUM@' '%s=%s'
#
function replace_or_append {
  local default_format='%s = %s' case_insensitive_mode=yes sed_case_insensitive_option='' grep_case_insensitive_option=''
  local config_file=$1
  local key=$2
  local value=$3
  local cce=$4
  local format=$5

  if [ "$case_insensitive_mode" = yes ]; then
    sed_case_insensitive_option="i"
    grep_case_insensitive_option="-i"
  fi
  [ -n "$format" ] || format="$default_format"
  # Check sanity of the input
  [ $# -ge "3" ] || { echo "Usage: replace_or_append <config_file_location> <key_to_search> <new_value> [<CCE number or literal '@CCENUM@' if unknown>] [printf-like format, default is '$default_format']" >&2; exit 1; }

  # Test if the config_file is a symbolic link. If so, use --follow-symlinks with sed.
  # Otherwise, regular sed command will do.
  sed_command=('sed' '-i')
  if test -L "$config_file"; then
    sed_command+=('--follow-symlinks')
  fi

  # Test that the cce arg is not empty or does not equal @CCENUM@.
  # If @CCENUM@ exists, it means that there is no CCE assigned.
  if [ -n "$cce" ] && [ "$cce" != '@CCENUM@' ]; then
    cce="${cce}"
  else
    cce="CCE"
  fi

  # Strip any search characters in the key arg so that the key can be replaced without
  # adding any search characters to the config file.
  stripped_key=$(sed 's/[\^=\$,;+]*//g' <<< "$key")

  # shellcheck disable=SC2059
  printf -v formatted_output "$format" "$stripped_key" "$value"

  # If the key exists, change it. Otherwise, add it to the config_file.
  # We search for the key string followed by a word boundary (matched by \>),
  # so if we search for 'setting', 'setting2' won't match.
  if LC_ALL=C grep -q -m 1 $grep_case_insensitive_option -e "${key}\\>" "$config_file"; then
    "${sed_command[@]}" "s/${key}\\>.*/$formatted_output/g$sed_case_insensitive_option" "$config_file"
  else
    # \n is precaution for case where file ends without trailing newline
    printf '\n# Per %s: Set %s in %s\n' "$cce" "$formatted_output" "$config_file" >> "$config_file"
    printf '%s\n' "$formatted_output" >> "$config_file"
  fi
}
replace_or_append '/etc/etcd/etcd.conf' '^ETCD_AUTO_TLS=' 'false' '' '%s=%s'
Remediation Ansible snippet:   (show)

Complexity:low
Disruption:low
Strategy:restrict
- name: '@RULE_TITLE@'
  lineinfile:
    path: /etc/etcd/etcd.conf
    regexp: ^ETCD_AUTO_TLS=
    line: ETCD_AUTO_TLS=false
    create: true
  tags:
    - etcd_auto_tls
    - medium_severity
    - restrict_strategy
    - low_complexity
    - low_disruption
    - no_reboot_needed

Rule   Enable The Client Certificate Authentication   [ref]

To ensure the etcd service is serving TLS to clients, edit the etcd configuration file /etc/etcd/etcd.conf on the master node and set ETCD_CLIENT_CERT_AUTH to true.

ETCD_CLIENT_CERT_AUTH=true

Rationale:

Without cryptographic integrity protections, information can be altered by unauthorized users without detection.

Severity: 
medium
Identifiers and References

References:  2.2

Remediation Shell script:   (show)

# Function to replace configuration setting in config file or add the configuration setting if
# it does not exist.
#
# Expects arguments:
#
# config_file:		Configuration file that will be modified
# key:			Configuration option to change
# value:		Value of the configuration option to change
# cce:			The CCE identifier or '@CCENUM@' if no CCE identifier exists
# format:		The printf-like format string that will be given stripped key and value as arguments,
#			so e.g. '%s=%s' will result in key=value subsitution (i.e. without spaces around =)
#
# Optional arugments:
#
# format:		Optional argument to specify the format of how key/value should be
# 			modified/appended in the configuration file. The default is key = value.
#
# Example Call(s):
#
#     With default format of 'key = value':
#     replace_or_append '/etc/sysctl.conf' '^kernel.randomize_va_space' '2' '@CCENUM@'
#
#     With custom key/value format:
#     replace_or_append '/etc/sysconfig/selinux' '^SELINUX=' 'disabled' '@CCENUM@' '%s=%s'
#
#     With a variable:
#     replace_or_append '/etc/sysconfig/selinux' '^SELINUX=' $var_selinux_state '@CCENUM@' '%s=%s'
#
function replace_or_append {
  local default_format='%s = %s' case_insensitive_mode=yes sed_case_insensitive_option='' grep_case_insensitive_option=''
  local config_file=$1
  local key=$2
  local value=$3
  local cce=$4
  local format=$5

  if [ "$case_insensitive_mode" = yes ]; then
    sed_case_insensitive_option="i"
    grep_case_insensitive_option="-i"
  fi
  [ -n "$format" ] || format="$default_format"
  # Check sanity of the input
  [ $# -ge "3" ] || { echo "Usage: replace_or_append <config_file_location> <key_to_search> <new_value> [<CCE number or literal '@CCENUM@' if unknown>] [printf-like format, default is '$default_format']" >&2; exit 1; }

  # Test if the config_file is a symbolic link. If so, use --follow-symlinks with sed.
  # Otherwise, regular sed command will do.
  sed_command=('sed' '-i')
  if test -L "$config_file"; then
    sed_command+=('--follow-symlinks')
  fi

  # Test that the cce arg is not empty or does not equal @CCENUM@.
  # If @CCENUM@ exists, it means that there is no CCE assigned.
  if [ -n "$cce" ] && [ "$cce" != '@CCENUM@' ]; then
    cce="${cce}"
  else
    cce="CCE"
  fi

  # Strip any search characters in the key arg so that the key can be replaced without
  # adding any search characters to the config file.
  stripped_key=$(sed 's/[\^=\$,;+]*//g' <<< "$key")

  # shellcheck disable=SC2059
  printf -v formatted_output "$format" "$stripped_key" "$value"

  # If the key exists, change it. Otherwise, add it to the config_file.
  # We search for the key string followed by a word boundary (matched by \>),
  # so if we search for 'setting', 'setting2' won't match.
  if LC_ALL=C grep -q -m 1 $grep_case_insensitive_option -e "${key}\\>" "$config_file"; then
    "${sed_command[@]}" "s/${key}\\>.*/$formatted_output/g$sed_case_insensitive_option" "$config_file"
  else
    # \n is precaution for case where file ends without trailing newline
    printf '\n# Per %s: Set %s in %s\n' "$cce" "$formatted_output" "$config_file" >> "$config_file"
    printf '%s\n' "$formatted_output" >> "$config_file"
  fi
}
replace_or_append '/etc/etcd/etcd.conf' '^ETCD_CLIENT_CERT_AUTH=' 'true' '' '%s=%s'
Remediation Ansible snippet:   (show)

Complexity:low
Disruption:low
Strategy:restrict
- name: '@RULE_TITLE@'
  lineinfile:
    path: /etc/etcd/etcd.conf
    regexp: ^ETCD_CLIENT_CERT_AUTH=
    line: ETCD_CLIENT_CERT_AUTH=true
    create: true
  tags:
    - etcd_client_cert_auth
    - medium_severity
    - restrict_strategy
    - low_complexity
    - low_disruption
    - no_reboot_needed

Rule   Ensure That The etcd Peer Key File Is Correctly Set   [ref]

To ensure the etcd service is serving TLS to clients, edit the etcd configuration file /etc/etcd/etcd.conf on the master on the master and adding a key file to ETCD_PEER_KEY_FILE:

ETCD_PEER_KEY_FILE=/etc/ssl/etcd/system:etcd-peer:etcd_dns_name.key

Rationale:

Without cryptographic integrity protections, information can be altered by unauthorized users without detection.

Severity: 
medium
Identifiers and References

References:  2.4

Remediation Shell script:   (show)

# Function to replace configuration setting in config file or add the configuration setting if
# it does not exist.
#
# Expects arguments:
#
# config_file:		Configuration file that will be modified
# key:			Configuration option to change
# value:		Value of the configuration option to change
# cce:			The CCE identifier or '@CCENUM@' if no CCE identifier exists
# format:		The printf-like format string that will be given stripped key and value as arguments,
#			so e.g. '%s=%s' will result in key=value subsitution (i.e. without spaces around =)
#
# Optional arugments:
#
# format:		Optional argument to specify the format of how key/value should be
# 			modified/appended in the configuration file. The default is key = value.
#
# Example Call(s):
#
#     With default format of 'key = value':
#     replace_or_append '/etc/sysctl.conf' '^kernel.randomize_va_space' '2' '@CCENUM@'
#
#     With custom key/value format:
#     replace_or_append '/etc/sysconfig/selinux' '^SELINUX=' 'disabled' '@CCENUM@' '%s=%s'
#
#     With a variable:
#     replace_or_append '/etc/sysconfig/selinux' '^SELINUX=' $var_selinux_state '@CCENUM@' '%s=%s'
#
function replace_or_append {
  local default_format='%s = %s' case_insensitive_mode=yes sed_case_insensitive_option='' grep_case_insensitive_option=''
  local config_file=$1
  local key=$2
  local value=$3
  local cce=$4
  local format=$5

  if [ "$case_insensitive_mode" = yes ]; then
    sed_case_insensitive_option="i"
    grep_case_insensitive_option="-i"
  fi
  [ -n "$format" ] || format="$default_format"
  # Check sanity of the input
  [ $# -ge "3" ] || { echo "Usage: replace_or_append <config_file_location> <key_to_search> <new_value> [<CCE number or literal '@CCENUM@' if unknown>] [printf-like format, default is '$default_format']" >&2; exit 1; }

  # Test if the config_file is a symbolic link. If so, use --follow-symlinks with sed.
  # Otherwise, regular sed command will do.
  sed_command=('sed' '-i')
  if test -L "$config_file"; then
    sed_command+=('--follow-symlinks')
  fi

  # Test that the cce arg is not empty or does not equal @CCENUM@.
  # If @CCENUM@ exists, it means that there is no CCE assigned.
  if [ -n "$cce" ] && [ "$cce" != '@CCENUM@' ]; then
    cce="${cce}"
  else
    cce="CCE"
  fi

  # Strip any search characters in the key arg so that the key can be replaced without
  # adding any search characters to the config file.
  stripped_key=$(sed 's/[\^=\$,;+]*//g' <<< "$key")

  # shellcheck disable=SC2059
  printf -v formatted_output "$format" "$stripped_key" "$value"

  # If the key exists, change it. Otherwise, add it to the config_file.
  # We search for the key string followed by a word boundary (matched by \>),
  # so if we search for 'setting', 'setting2' won't match.
  if LC_ALL=C grep -q -m 1 $grep_case_insensitive_option -e "${key}\\>" "$config_file"; then
    "${sed_command[@]}" "s/${key}\\>.*/$formatted_output/g$sed_case_insensitive_option" "$config_file"
  else
    # \n is precaution for case where file ends without trailing newline
    printf '\n# Per %s: Set %s in %s\n' "$cce" "$formatted_output" "$config_file" >> "$config_file"
    printf '%s\n' "$formatted_output" >> "$config_file"
  fi
}
replace_or_append '/etc/etcd/etcd.conf' '^ETCD_PEER_KEY_FILE=' /etc/etcd/peer.key '' '%s=%s'
Remediation Ansible snippet:   (show)

Complexity:low
Disruption:low
Strategy:restrict
- name: '@RULE_TITLE@'
  lineinfile:
    path: /etc/etcd/etcd.conf
    regexp: ^ETCD_PEER_KEY_FILE
    line: ETCD_PEER_KEY_FILE=/etc/etcd/peer.key
    create: true
  tags:
    - etcd_peer_key_file
    - medium_severity
    - restrict_strategy
    - low_complexity
    - low_disruption
    - no_reboot_needed

Rule   Ensure That The etcd Peer Client Certificate Is Correctly Set   [ref]

To ensure the etcd service is serving TLS to clients, edit the etcd configuration file /etc/etcd/etcd.conf on the master and adding a certificate to ETCD_PEER_CERT_FILE:

ETCD_PEER_CERT_FILE=/etc/ssl/etcd/system:etcd-peer:etcd_dns_name.crt

Rationale:

Without cryptographic integrity protections, information can be altered by unauthorized users without detection.

Severity: 
medium
Identifiers and References

References:  2.4

Remediation Shell script:   (show)

# Function to replace configuration setting in config file or add the configuration setting if
# it does not exist.
#
# Expects arguments:
#
# config_file:		Configuration file that will be modified
# key:			Configuration option to change
# value:		Value of the configuration option to change
# cce:			The CCE identifier or '@CCENUM@' if no CCE identifier exists
# format:		The printf-like format string that will be given stripped key and value as arguments,
#			so e.g. '%s=%s' will result in key=value subsitution (i.e. without spaces around =)
#
# Optional arugments:
#
# format:		Optional argument to specify the format of how key/value should be
# 			modified/appended in the configuration file. The default is key = value.
#
# Example Call(s):
#
#     With default format of 'key = value':
#     replace_or_append '/etc/sysctl.conf' '^kernel.randomize_va_space' '2' '@CCENUM@'
#
#     With custom key/value format:
#     replace_or_append '/etc/sysconfig/selinux' '^SELINUX=' 'disabled' '@CCENUM@' '%s=%s'
#
#     With a variable:
#     replace_or_append '/etc/sysconfig/selinux' '^SELINUX=' $var_selinux_state '@CCENUM@' '%s=%s'
#
function replace_or_append {
  local default_format='%s = %s' case_insensitive_mode=yes sed_case_insensitive_option='' grep_case_insensitive_option=''
  local config_file=$1
  local key=$2
  local value=$3
  local cce=$4
  local format=$5

  if [ "$case_insensitive_mode" = yes ]; then
    sed_case_insensitive_option="i"
    grep_case_insensitive_option="-i"
  fi
  [ -n "$format" ] || format="$default_format"
  # Check sanity of the input
  [ $# -ge "3" ] || { echo "Usage: replace_or_append <config_file_location> <key_to_search> <new_value> [<CCE number or literal '@CCENUM@' if unknown>] [printf-like format, default is '$default_format']" >&2; exit 1; }

  # Test if the config_file is a symbolic link. If so, use --follow-symlinks with sed.
  # Otherwise, regular sed command will do.
  sed_command=('sed' '-i')
  if test -L "$config_file"; then
    sed_command+=('--follow-symlinks')
  fi

  # Test that the cce arg is not empty or does not equal @CCENUM@.
  # If @CCENUM@ exists, it means that there is no CCE assigned.
  if [ -n "$cce" ] && [ "$cce" != '@CCENUM@' ]; then
    cce="${cce}"
  else
    cce="CCE"
  fi

  # Strip any search characters in the key arg so that the key can be replaced without
  # adding any search characters to the config file.
  stripped_key=$(sed 's/[\^=\$,;+]*//g' <<< "$key")

  # shellcheck disable=SC2059
  printf -v formatted_output "$format" "$stripped_key" "$value"

  # If the key exists, change it. Otherwise, add it to the config_file.
  # We search for the key string followed by a word boundary (matched by \>),
  # so if we search for 'setting', 'setting2' won't match.
  if LC_ALL=C grep -q -m 1 $grep_case_insensitive_option -e "${key}\\>" "$config_file"; then
    "${sed_command[@]}" "s/${key}\\>.*/$formatted_output/g$sed_case_insensitive_option" "$config_file"
  else
    # \n is precaution for case where file ends without trailing newline
    printf '\n# Per %s: Set %s in %s\n' "$cce" "$formatted_output" "$config_file" >> "$config_file"
    printf '%s\n' "$formatted_output" >> "$config_file"
  fi
}
replace_or_append '/etc/etcd/etcd.conf' '^ETCD_PEER_CERT_FILE=' /etc/etcd/peer.crt '' '%s=%s'
Remediation Ansible snippet:   (show)

Complexity:low
Disruption:low
Strategy:restrict
- name: '@RULE_TITLE@'
  lineinfile:
    path: /etc/etcd/etcd.conf
    regexp: ^ETCD_PEER_CERT_FILE
    line: ETCD_PEER_CERT_FILE=/etc/etcd/peer.crt
    create: true
  tags:
    - etcd_peer_cert_file
    - medium_severity
    - restrict_strategy
    - low_complexity
    - low_disruption
    - no_reboot_needed

Rule   Disable etcd Peer Self-Signed Certificates   [ref]

To ensure the etcd service is not using self-signed certificates, edit the etcd configuration file /etc/etcd/etcd.conf from the master node and set ETCD_PEER_AUTO_TLS to false:

ETCD_PEER_AUTO_TLS=false

Rationale:

Without cryptographic integrity protections, information can be altered by unauthorized users without detection. Using self-signed certificates ensures that the certificates are never validated against a certificate authority and could lead to compromised and invalidated data.

Severity: 
medium
Identifiers and References

References:  2.6

Remediation Shell script:   (show)

# Function to replace configuration setting in config file or add the configuration setting if
# it does not exist.
#
# Expects arguments:
#
# config_file:		Configuration file that will be modified
# key:			Configuration option to change
# value:		Value of the configuration option to change
# cce:			The CCE identifier or '@CCENUM@' if no CCE identifier exists
# format:		The printf-like format string that will be given stripped key and value as arguments,
#			so e.g. '%s=%s' will result in key=value subsitution (i.e. without spaces around =)
#
# Optional arugments:
#
# format:		Optional argument to specify the format of how key/value should be
# 			modified/appended in the configuration file. The default is key = value.
#
# Example Call(s):
#
#     With default format of 'key = value':
#     replace_or_append '/etc/sysctl.conf' '^kernel.randomize_va_space' '2' '@CCENUM@'
#
#     With custom key/value format:
#     replace_or_append '/etc/sysconfig/selinux' '^SELINUX=' 'disabled' '@CCENUM@' '%s=%s'
#
#     With a variable:
#     replace_or_append '/etc/sysconfig/selinux' '^SELINUX=' $var_selinux_state '@CCENUM@' '%s=%s'
#
function replace_or_append {
  local default_format='%s = %s' case_insensitive_mode=yes sed_case_insensitive_option='' grep_case_insensitive_option=''
  local config_file=$1
  local key=$2
  local value=$3
  local cce=$4
  local format=$5

  if [ "$case_insensitive_mode" = yes ]; then
    sed_case_insensitive_option="i"
    grep_case_insensitive_option="-i"
  fi
  [ -n "$format" ] || format="$default_format"
  # Check sanity of the input
  [ $# -ge "3" ] || { echo "Usage: replace_or_append <config_file_location> <key_to_search> <new_value> [<CCE number or literal '@CCENUM@' if unknown>] [printf-like format, default is '$default_format']" >&2; exit 1; }

  # Test if the config_file is a symbolic link. If so, use --follow-symlinks with sed.
  # Otherwise, regular sed command will do.
  sed_command=('sed' '-i')
  if test -L "$config_file"; then
    sed_command+=('--follow-symlinks')
  fi

  # Test that the cce arg is not empty or does not equal @CCENUM@.
  # If @CCENUM@ exists, it means that there is no CCE assigned.
  if [ -n "$cce" ] && [ "$cce" != '@CCENUM@' ]; then
    cce="${cce}"
  else
    cce="CCE"
  fi

  # Strip any search characters in the key arg so that the key can be replaced without
  # adding any search characters to the config file.
  stripped_key=$(sed 's/[\^=\$,;+]*//g' <<< "$key")

  # shellcheck disable=SC2059
  printf -v formatted_output "$format" "$stripped_key" "$value"

  # If the key exists, change it. Otherwise, add it to the config_file.
  # We search for the key string followed by a word boundary (matched by \>),
  # so if we search for 'setting', 'setting2' won't match.
  if LC_ALL=C grep -q -m 1 $grep_case_insensitive_option -e "${key}\\>" "$config_file"; then
    "${sed_command[@]}" "s/${key}\\>.*/$formatted_output/g$sed_case_insensitive_option" "$config_file"
  else
    # \n is precaution for case where file ends without trailing newline
    printf '\n# Per %s: Set %s in %s\n' "$cce" "$formatted_output" "$config_file" >> "$config_file"
    printf '%s\n' "$formatted_output" >> "$config_file"
  fi
}
replace_or_append '/etc/etcd/etcd.conf' '^ETCD_PEER_AUTO_TLS=' 'false' '' '%s=%s'
Remediation Ansible snippet:   (show)

Complexity:low
Disruption:low
Strategy:restrict
- name: '@RULE_TITLE@'
  lineinfile:
    path: /etc/etcd/etcd.conf
    regexp: ^ETCD_PEER_AUTO_TLS=
    line: ETCD_PEER_AUTO_TLS=false
    create: true
  tags:
    - etcd_peer_auto_tls
    - medium_severity
    - restrict_strategy
    - low_complexity
    - low_disruption
    - no_reboot_needed

Rule   Configure A Unique CA Certificate for etcd   [ref]

A unique and different CA certificate should be created for etcd. To ensure the etcd service is using a unique certificate, , set ETCD_TRUSTED_CA_FILE to /etc/etcd/ca.crt in /etc/etcd/etcd.conf on the master node that does NOT match the OpenShift CA certificate:

ETCD_TRUSTED_CA_FILE=/etc/ssl/etcd/ca.crt

Rationale:

A unique CA certificate that is utilized by etcd and is different from OpenShift ensures that the etcd data is still protected in the event that the OpenShift certificate is compromised.

Severity: 
medium
Identifiers and References

References:  2.7

Remediation Shell script:   (show)

# Function to replace configuration setting in config file or add the configuration setting if
# it does not exist.
#
# Expects arguments:
#
# config_file:		Configuration file that will be modified
# key:			Configuration option to change
# value:		Value of the configuration option to change
# cce:			The CCE identifier or '@CCENUM@' if no CCE identifier exists
# format:		The printf-like format string that will be given stripped key and value as arguments,
#			so e.g. '%s=%s' will result in key=value subsitution (i.e. without spaces around =)
#
# Optional arugments:
#
# format:		Optional argument to specify the format of how key/value should be
# 			modified/appended in the configuration file. The default is key = value.
#
# Example Call(s):
#
#     With default format of 'key = value':
#     replace_or_append '/etc/sysctl.conf' '^kernel.randomize_va_space' '2' '@CCENUM@'
#
#     With custom key/value format:
#     replace_or_append '/etc/sysconfig/selinux' '^SELINUX=' 'disabled' '@CCENUM@' '%s=%s'
#
#     With a variable:
#     replace_or_append '/etc/sysconfig/selinux' '^SELINUX=' $var_selinux_state '@CCENUM@' '%s=%s'
#
function replace_or_append {
  local default_format='%s = %s' case_insensitive_mode=yes sed_case_insensitive_option='' grep_case_insensitive_option=''
  local config_file=$1
  local key=$2
  local value=$3
  local cce=$4
  local format=$5

  if [ "$case_insensitive_mode" = yes ]; then
    sed_case_insensitive_option="i"
    grep_case_insensitive_option="-i"
  fi
  [ -n "$format" ] || format="$default_format"
  # Check sanity of the input
  [ $# -ge "3" ] || { echo "Usage: replace_or_append <config_file_location> <key_to_search> <new_value> [<CCE number or literal '@CCENUM@' if unknown>] [printf-like format, default is '$default_format']" >&2; exit 1; }

  # Test if the config_file is a symbolic link. If so, use --follow-symlinks with sed.
  # Otherwise, regular sed command will do.
  sed_command=('sed' '-i')
  if test -L "$config_file"; then
    sed_command+=('--follow-symlinks')
  fi

  # Test that the cce arg is not empty or does not equal @CCENUM@.
  # If @CCENUM@ exists, it means that there is no CCE assigned.
  if [ -n "$cce" ] && [ "$cce" != '@CCENUM@' ]; then
    cce="${cce}"
  else
    cce="CCE"
  fi

  # Strip any search characters in the key arg so that the key can be replaced without
  # adding any search characters to the config file.
  stripped_key=$(sed 's/[\^=\$,;+]*//g' <<< "$key")

  # shellcheck disable=SC2059
  printf -v formatted_output "$format" "$stripped_key" "$value"

  # If the key exists, change it. Otherwise, add it to the config_file.
  # We search for the key string followed by a word boundary (matched by \>),
  # so if we search for 'setting', 'setting2' won't match.
  if LC_ALL=C grep -q -m 1 $grep_case_insensitive_option -e "${key}\\>" "$config_file"; then
    "${sed_command[@]}" "s/${key}\\>.*/$formatted_output/g$sed_case_insensitive_option" "$config_file"
  else
    # \n is precaution for case where file ends without trailing newline
    printf '\n# Per %s: Set %s in %s\n' "$cce" "$formatted_output" "$config_file" >> "$config_file"
    printf '%s\n' "$formatted_output" >> "$config_file"
  fi
}
replace_or_append '/etc/etcd/etcd.conf' '^ETCD_TRUSTED_CA_FILE=' '/etc/etcd/ca.crt' '' '%s=%s'
Remediation Ansible snippet:   (show)

Complexity:low
Disruption:low
Strategy:restrict
- name: '@RULE_TITLE@'
  lineinfile:
    path: /etc/etcd/etcd.conf
    regexp: ^ETCD_TRUSTED_CA_FILE=
    line: ETCD_TRUSTED_CA_FILE=/etc/etcd/ca.crt
    create: true
  tags:
    - etcd_unique_ca
    - medium_severity
    - restrict_strategy
    - low_complexity
    - low_disruption
    - no_reboot_needed

Rule   Enable The Peer Client Certificate Authentication   [ref]

To ensure the etcd service is serving TLS to clients, edit the etcd configuration file /etc/etcd/etcd.conf on the master node and set ETCD_PEER_CLIENT_CERT_AUTH to true.

ETCD_PEER_CLIENT_CERT_AUTH=true

Rationale:

Without cryptographic integrity protections, information can be altered by unauthorized users without detection.

Severity: 
medium
Identifiers and References

References:  2.5

Remediation Shell script:   (show)

# Function to replace configuration setting in config file or add the configuration setting if
# it does not exist.
#
# Expects arguments:
#
# config_file:		Configuration file that will be modified
# key:			Configuration option to change
# value:		Value of the configuration option to change
# cce:			The CCE identifier or '@CCENUM@' if no CCE identifier exists
# format:		The printf-like format string that will be given stripped key and value as arguments,
#			so e.g. '%s=%s' will result in key=value subsitution (i.e. without spaces around =)
#
# Optional arugments:
#
# format:		Optional argument to specify the format of how key/value should be
# 			modified/appended in the configuration file. The default is key = value.
#
# Example Call(s):
#
#     With default format of 'key = value':
#     replace_or_append '/etc/sysctl.conf' '^kernel.randomize_va_space' '2' '@CCENUM@'
#
#     With custom key/value format:
#     replace_or_append '/etc/sysconfig/selinux' '^SELINUX=' 'disabled' '@CCENUM@' '%s=%s'
#
#     With a variable:
#     replace_or_append '/etc/sysconfig/selinux' '^SELINUX=' $var_selinux_state '@CCENUM@' '%s=%s'
#
function replace_or_append {
  local default_format='%s = %s' case_insensitive_mode=yes sed_case_insensitive_option='' grep_case_insensitive_option=''
  local config_file=$1
  local key=$2
  local value=$3
  local cce=$4
  local format=$5

  if [ "$case_insensitive_mode" = yes ]; then
    sed_case_insensitive_option="i"
    grep_case_insensitive_option="-i"
  fi
  [ -n "$format" ] || format="$default_format"
  # Check sanity of the input
  [ $# -ge "3" ] || { echo "Usage: replace_or_append <config_file_location> <key_to_search> <new_value> [<CCE number or literal '@CCENUM@' if unknown>] [printf-like format, default is '$default_format']" >&2; exit 1; }

  # Test if the config_file is a symbolic link. If so, use --follow-symlinks with sed.
  # Otherwise, regular sed command will do.
  sed_command=('sed' '-i')
  if test -L "$config_file"; then
    sed_command+=('--follow-symlinks')
  fi

  # Test that the cce arg is not empty or does not equal @CCENUM@.
  # If @CCENUM@ exists, it means that there is no CCE assigned.
  if [ -n "$cce" ] && [ "$cce" != '@CCENUM@' ]; then
    cce="${cce}"
  else
    cce="CCE"
  fi

  # Strip any search characters in the key arg so that the key can be replaced without
  # adding any search characters to the config file.
  stripped_key=$(sed 's/[\^=\$,;+]*//g' <<< "$key")

  # shellcheck disable=SC2059
  printf -v formatted_output "$format" "$stripped_key" "$value"

  # If the key exists, change it. Otherwise, add it to the config_file.
  # We search for the key string followed by a word boundary (matched by \>),
  # so if we search for 'setting', 'setting2' won't match.
  if LC_ALL=C grep -q -m 1 $grep_case_insensitive_option -e "${key}\\>" "$config_file"; then
    "${sed_command[@]}" "s/${key}\\>.*/$formatted_output/g$sed_case_insensitive_option" "$config_file"
  else
    # \n is precaution for case where file ends without trailing newline
    printf '\n# Per %s: Set %s in %s\n' "$cce" "$formatted_output" "$config_file" >> "$config_file"
    printf '%s\n' "$formatted_output" >> "$config_file"
  fi
}
replace_or_append '/etc/etcd/etcd.conf' '^ETCD_PEER_CLIENT_CERT_AUTH=' 'true' '' '%s=%s'
Remediation Ansible snippet:   (show)

Complexity:low
Disruption:low
Strategy:restrict
- name: '@RULE_TITLE@'
  lineinfile:
    path: /etc/etcd/etcd.conf
    regexp: ^ETCD_PEER_CLIENT_CERT_AUTH=
    line: ETCD_PEER_CLIENT_CERT_AUTH=true
    create: true
  tags:
    - etcd_peer_client_cert_auth
    - medium_severity
    - restrict_strategy
    - low_complexity
    - low_disruption
    - no_reboot_needed

Rule   Ensure That The etcd Client Certificate Is Correctly Set   [ref]

To ensure the etcd service is serving TLS to clients, edit the etcd configuration file /etc/etcd/etcd.conf on the master and adding a certificate to ETCD_CERT_FILE:

ETCD_CERT_FILE=/etc/ssl/etcd/system:etcd-server:etcd_dns_name.crt

Rationale:

Without cryptographic integrity protections, information can be altered by unauthorized users without detection.

Severity: 
medium
Identifiers and References

References:  2.1

Remediation Shell script:   (show)

# Function to replace configuration setting in config file or add the configuration setting if
# it does not exist.
#
# Expects arguments:
#
# config_file:		Configuration file that will be modified
# key:			Configuration option to change
# value:		Value of the configuration option to change
# cce:			The CCE identifier or '@CCENUM@' if no CCE identifier exists
# format:		The printf-like format string that will be given stripped key and value as arguments,
#			so e.g. '%s=%s' will result in key=value subsitution (i.e. without spaces around =)
#
# Optional arugments:
#
# format:		Optional argument to specify the format of how key/value should be
# 			modified/appended in the configuration file. The default is key = value.
#
# Example Call(s):
#
#     With default format of 'key = value':
#     replace_or_append '/etc/sysctl.conf' '^kernel.randomize_va_space' '2' '@CCENUM@'
#
#     With custom key/value format:
#     replace_or_append '/etc/sysconfig/selinux' '^SELINUX=' 'disabled' '@CCENUM@' '%s=%s'
#
#     With a variable:
#     replace_or_append '/etc/sysconfig/selinux' '^SELINUX=' $var_selinux_state '@CCENUM@' '%s=%s'
#
function replace_or_append {
  local default_format='%s = %s' case_insensitive_mode=yes sed_case_insensitive_option='' grep_case_insensitive_option=''
  local config_file=$1
  local key=$2
  local value=$3
  local cce=$4
  local format=$5

  if [ "$case_insensitive_mode" = yes ]; then
    sed_case_insensitive_option="i"
    grep_case_insensitive_option="-i"
  fi
  [ -n "$format" ] || format="$default_format"
  # Check sanity of the input
  [ $# -ge "3" ] || { echo "Usage: replace_or_append <config_file_location> <key_to_search> <new_value> [<CCE number or literal '@CCENUM@' if unknown>] [printf-like format, default is '$default_format']" >&2; exit 1; }

  # Test if the config_file is a symbolic link. If so, use --follow-symlinks with sed.
  # Otherwise, regular sed command will do.
  sed_command=('sed' '-i')
  if test -L "$config_file"; then
    sed_command+=('--follow-symlinks')
  fi

  # Test that the cce arg is not empty or does not equal @CCENUM@.
  # If @CCENUM@ exists, it means that there is no CCE assigned.
  if [ -n "$cce" ] && [ "$cce" != '@CCENUM@' ]; then
    cce="${cce}"
  else
    cce="CCE"
  fi

  # Strip any search characters in the key arg so that the key can be replaced without
  # adding any search characters to the config file.
  stripped_key=$(sed 's/[\^=\$,;+]*//g' <<< "$key")

  # shellcheck disable=SC2059
  printf -v formatted_output "$format" "$stripped_key" "$value"

  # If the key exists, change it. Otherwise, add it to the config_file.
  # We search for the key string followed by a word boundary (matched by \>),
  # so if we search for 'setting', 'setting2' won't match.
  if LC_ALL=C grep -q -m 1 $grep_case_insensitive_option -e "${key}\\>" "$config_file"; then
    "${sed_command[@]}" "s/${key}\\>.*/$formatted_output/g$sed_case_insensitive_option" "$config_file"
  else
    # \n is precaution for case where file ends without trailing newline
    printf '\n# Per %s: Set %s in %s\n' "$cce" "$formatted_output" "$config_file" >> "$config_file"
    printf '%s\n' "$formatted_output" >> "$config_file"
  fi
}
replace_or_append '/etc/etcd/etcd.conf' '^ETCD_CERT_FILE=' /etc/etcd/server.crt '' '%s=%s'
Remediation Ansible snippet:   (show)

Complexity:low
Disruption:low
Strategy:restrict
- name: '@RULE_TITLE@'
  lineinfile:
    path: /etc/etcd/etcd.conf
    regexp: ^ETCD_CERT_FILE
    line: ETCD_CERT_FILE=/etc/etcd/server.crt
    create: true
  tags:
    - etcd_cert_file
    - medium_severity
    - restrict_strategy
    - low_complexity
    - low_disruption
    - no_reboot_needed

Rule   Configure etcd Log Storage   [ref]

To ensure the etcd service is storing logs separate from data, set ETCD_WAL_DIR to /var/lib/etcd/member/wal in /etc/etcd/etcd.conf on the master node:

ETCD_WAL_DIR=/var/lib/etcd/member/wal

Rationale:

etcd log files should be stored in a separate location from the etcd data. This not only ensures data integrity but also helps to prevent IO degradation.

Severity: 
medium
Identifiers and References
Remediation Shell script:   (show)

# Function to replace configuration setting in config file or add the configuration setting if
# it does not exist.
#
# Expects arguments:
#
# config_file:		Configuration file that will be modified
# key:			Configuration option to change
# value:		Value of the configuration option to change
# cce:			The CCE identifier or '@CCENUM@' if no CCE identifier exists
# format:		The printf-like format string that will be given stripped key and value as arguments,
#			so e.g. '%s=%s' will result in key=value subsitution (i.e. without spaces around =)
#
# Optional arugments:
#
# format:		Optional argument to specify the format of how key/value should be
# 			modified/appended in the configuration file. The default is key = value.
#
# Example Call(s):
#
#     With default format of 'key = value':
#     replace_or_append '/etc/sysctl.conf' '^kernel.randomize_va_space' '2' '@CCENUM@'
#
#     With custom key/value format:
#     replace_or_append '/etc/sysconfig/selinux' '^SELINUX=' 'disabled' '@CCENUM@' '%s=%s'
#
#     With a variable:
#     replace_or_append '/etc/sysconfig/selinux' '^SELINUX=' $var_selinux_state '@CCENUM@' '%s=%s'
#
function replace_or_append {
  local default_format='%s = %s' case_insensitive_mode=yes sed_case_insensitive_option='' grep_case_insensitive_option=''
  local config_file=$1
  local key=$2
  local value=$3
  local cce=$4
  local format=$5

  if [ "$case_insensitive_mode" = yes ]; then
    sed_case_insensitive_option="i"
    grep_case_insensitive_option="-i"
  fi
  [ -n "$format" ] || format="$default_format"
  # Check sanity of the input
  [ $# -ge "3" ] || { echo "Usage: replace_or_append <config_file_location> <key_to_search> <new_value> [<CCE number or literal '@CCENUM@' if unknown>] [printf-like format, default is '$default_format']" >&2; exit 1; }

  # Test if the config_file is a symbolic link. If so, use --follow-symlinks with sed.
  # Otherwise, regular sed command will do.
  sed_command=('sed' '-i')
  if test -L "$config_file"; then
    sed_command+=('--follow-symlinks')
  fi

  # Test that the cce arg is not empty or does not equal @CCENUM@.
  # If @CCENUM@ exists, it means that there is no CCE assigned.
  if [ -n "$cce" ] && [ "$cce" != '@CCENUM@' ]; then
    cce="${cce}"
  else
    cce="CCE"
  fi

  # Strip any search characters in the key arg so that the key can be replaced without
  # adding any search characters to the config file.
  stripped_key=$(sed 's/[\^=\$,;+]*//g' <<< "$key")

  # shellcheck disable=SC2059
  printf -v formatted_output "$format" "$stripped_key" "$value"

  # If the key exists, change it. Otherwise, add it to the config_file.
  # We search for the key string followed by a word boundary (matched by \>),
  # so if we search for 'setting', 'setting2' won't match.
  if LC_ALL=C grep -q -m 1 $grep_case_insensitive_option -e "${key}\\>" "$config_file"; then
    "${sed_command[@]}" "s/${key}\\>.*/$formatted_output/g$sed_case_insensitive_option" "$config_file"
  else
    # \n is precaution for case where file ends without trailing newline
    printf '\n# Per %s: Set %s in %s\n' "$cce" "$formatted_output" "$config_file" >> "$config_file"
    printf '%s\n' "$formatted_output" >> "$config_file"
  fi
}
replace_or_append '/etc/etcd/etcd.conf' '^ETCD_WAL_DIR=' '/var/lib/etcd/member/wal' '' '%s=%s'
Remediation Ansible snippet:   (show)

Complexity:low
Disruption:low
Strategy:restrict
- name: '@RULE_TITLE@'
  lineinfile:
    path: /etc/etcd/etcd.conf
    regexp: ^ETCD_WAL_DIR=
    line: ETCD_WAL_DIR=/var/lib/etcd/member/wal
    create: true
  tags:
    - etcd_wal_dir
    - medium_severity
    - restrict_strategy
    - low_complexity
    - low_disruption
    - no_reboot_needed

Rule   Ensure That The etcd Key File Is Correctly Set   [ref]

To ensure the etcd service is serving TLS to clients, edit the etcd configuration file /etc/etcd/etcd.conf on the master and adding a key file to ETCD_KEY_FILE:

ETCD_CERT_FILE=/etc/ssl/etcd/system:etcd-server:etcd_dns_name.key

Rationale:

Without cryptographic integrity protections, information can be altered by unauthorized users without detection.

Severity: 
medium
Identifiers and References

References:  2.1

Remediation Shell script:   (show)

# Function to replace configuration setting in config file or add the configuration setting if
# it does not exist.
#
# Expects arguments:
#
# config_file:		Configuration file that will be modified
# key:			Configuration option to change
# value:		Value of the configuration option to change
# cce:			The CCE identifier or '@CCENUM@' if no CCE identifier exists
# format:		The printf-like format string that will be given stripped key and value as arguments,
#			so e.g. '%s=%s' will result in key=value subsitution (i.e. without spaces around =)
#
# Optional arugments:
#
# format:		Optional argument to specify the format of how key/value should be
# 			modified/appended in the configuration file. The default is key = value.
#
# Example Call(s):
#
#     With default format of 'key = value':
#     replace_or_append '/etc/sysctl.conf' '^kernel.randomize_va_space' '2' '@CCENUM@'
#
#     With custom key/value format:
#     replace_or_append '/etc/sysconfig/selinux' '^SELINUX=' 'disabled' '@CCENUM@' '%s=%s'
#
#     With a variable:
#     replace_or_append '/etc/sysconfig/selinux' '^SELINUX=' $var_selinux_state '@CCENUM@' '%s=%s'
#
function replace_or_append {
  local default_format='%s = %s' case_insensitive_mode=yes sed_case_insensitive_option='' grep_case_insensitive_option=''
  local config_file=$1
  local key=$2
  local value=$3
  local cce=$4
  local format=$5

  if [ "$case_insensitive_mode" = yes ]; then
    sed_case_insensitive_option="i"
    grep_case_insensitive_option="-i"
  fi
  [ -n "$format" ] || format="$default_format"
  # Check sanity of the input
  [ $# -ge "3" ] || { echo "Usage: replace_or_append <config_file_location> <key_to_search> <new_value> [<CCE number or literal '@CCENUM@' if unknown>] [printf-like format, default is '$default_format']" >&2; exit 1; }

  # Test if the config_file is a symbolic link. If so, use --follow-symlinks with sed.
  # Otherwise, regular sed command will do.
  sed_command=('sed' '-i')
  if test -L "$config_file"; then
    sed_command+=('--follow-symlinks')
  fi

  # Test that the cce arg is not empty or does not equal @CCENUM@.
  # If @CCENUM@ exists, it means that there is no CCE assigned.
  if [ -n "$cce" ] && [ "$cce" != '@CCENUM@' ]; then
    cce="${cce}"
  else
    cce="CCE"
  fi

  # Strip any search characters in the key arg so that the key can be replaced without
  # adding any search characters to the config file.
  stripped_key=$(sed 's/[\^=\$,;+]*//g' <<< "$key")

  # shellcheck disable=SC2059
  printf -v formatted_output "$format" "$stripped_key" "$value"

  # If the key exists, change it. Otherwise, add it to the config_file.
  # We search for the key string followed by a word boundary (matched by \>),
  # so if we search for 'setting', 'setting2' won't match.
  if LC_ALL=C grep -q -m 1 $grep_case_insensitive_option -e "${key}\\>" "$config_file"; then
    "${sed_command[@]}" "s/${key}\\>.*/$formatted_output/g$sed_case_insensitive_option" "$config_file"
  else
    # \n is precaution for case where file ends without trailing newline
    printf '\n# Per %s: Set %s in %s\n' "$cce" "$formatted_output" "$config_file" >> "$config_file"
    printf '%s\n' "$formatted_output" >> "$config_file"
  fi
}
replace_or_append '/etc/etcd/etcd.conf' '^ETCD_KEY_FILE=' /etc/etcd/server.key '' '%s=%s'
Remediation Ansible snippet:   (show)

Complexity:low
Disruption:low
Strategy:restrict
- name: '@RULE_TITLE@'
  lineinfile:
    path: /etc/etcd/etcd.conf
    regexp: ^ETCD_KEY_FILE=
    line: ETCD_KEY_FILE=/etc/etcd/server.key
    create: true
  tags:
    - etcd_key_file
    - medium_severity
    - restrict_strategy
    - low_complexity
    - low_disruption
    - no_reboot_needed
Group   OpenShift API Server   Group contains 23 rules

[ref]   This section contains recommendations for kube-apiserver configuration.

Rule   Prevent Insecure Port Access   [ref]

By default, traffic for the OpenShift API server is served over HTTPS with authentication and authorization, and the secure API endpoint is bound to 0.0.0.0:8443. To ensure that the insecure port configuration has not been enabled, the insecure-port setting should not exist in /etc/kubernetes/static-pod-resources/configmaps/config/config.yaml on the master node(s).

Rationale:

Configuring the API Server on an insecure port would allow unauthenticated and unencrypted access to your master node(s). It is assumed firewall rules will be configured to ensure this port is not reachable from outside the cluster, however as a defense in depth measure, OpenShift should not be configured to use insecure ports.

Severity: 
medium
Identifiers and References

References:  1.2.19

Rule   Configure the Certificate Key for the API Server   [ref]

To ensure the API Server utilizes its own TLS certificates, the keyFile must be configured. To, verify that servingInfo has the keyFile configured in the API Server pod specification file /etc/kubernetes/static-pod-resources/configmaps/config/config.yaml on the master node(s) to something similar to:

"servingInfo":{
  ...
  "keyFile":"/etc/kubernetes/static-pod-certs/secrets/service-network-serving-certkey/tls.key",
  ...

Rationale:

API Server communication contains sensitive parameters that should remain encrypted in transit. Configure the API Server to serve only HTTPS traffic.

Severity: 
medium
Identifiers and References

References:  1.2.30

Rule   Configure the Certificate for the API Server   [ref]

To ensure the API Server utilizes its own TLS certificates, the certFile must be configured. Verify that servingInfo has the certFile configured in the API Server pod specification file /etc/kubernetes/static-pod-resources/configmaps/config/config.yaml on the master node(s) to something similar to:

"servingInfo":{
  ...
  "certFile":"/etc/kubernetes/static-pod-certs/secrets/service-network-serving-certkey/tls.crt",
  ...

Rationale:

API Server communication contains sensitive parameters that should remain encrypted in transit. Configure the API Server to serve only HTTPS traffic.

Severity: 
medium
Identifiers and References

References:  1.2.30

Rule   Configure the Client Certificate Authority for the API Server   [ref]

Certificates must be provided to fully setup TLS client certificate authentication. To ensure the API Server utilizes its own TLS certificates, the clientCA must be configured. Verify that servingInfo has the clientCA configured in the API Server pod specification file /etc/kubernetes/static-pod-resources/configmaps/config/config.yaml on the master node(s) to something similar to:

"servingInfo":{
  ...
  "clientCA":"/etc/kubernetes/static-pod-certs/configmaps/client-ca/ca-bundle.crt",
  ...

Rationale:

API Server communication contains sensitive parameters that should remain encrypted in transit. Configure the API Server to serve only HTTPS traffic. If -clientCA is set, any request presenting a client certificate signed by one of the authorities in the client-ca-file is authenticated with an identity corresponding to the CommonName of the client certificate.

Severity: 
medium
Identifiers and References

References:  1.2.31

Rule   Configure the Audit Log Path   [ref]

To enable auditing on the OpenShift API Server, the audit log path must be set. Edit the API Server pod specification file /etc/kubernetes/static-pod-resources/configmaps/config/config.yaml on the master node(s) and set the audit-log-path to a suitable path and file where you would like audit logs to be written. For example:

"auditConfig":{
  ...
  "auditFilePath":"/var/log/kube-apiserver/audit.log",
  ...
 

Rationale:

Auditing of the API Server is not enabled by default. Auditing the API Server provides a security-relevant chronological set of records documenting the sequence of activities that have affected the system by users, administrators, or other system components.

Severity: 
high
Identifiers and References

References:  1.2.22

Rule   Ensure API Server authorization-mode is set to Webhook   [ref]

By default, unauthenticated/unauthorized users have no access to OpenShift nodes and the API Server authorization-mode is set to Webhook. To ensure that the API server requires authorization for API requests, validate that authorization-mode is configured to Webhook in /etc/kubernetes/static-pod-resources/configmaps/config/config.yaml:

"apiServerArguments":{
  ...
  "authorization-mode":[
    "Webhook"
  ],
  ...

Warning:  If authorization-mode is not configured to Webhook, the node systemd service (atomic-openshift-node) will not start.
Rationale:

Ensuring authorization-mode is set to Webhook helps enforce that unauthenticated/unauthorized users have no access to OpenShift nodes.

Severity: 
medium
Identifiers and References

References:  1.2.7, 1.2.8, 1.2.9

Rule   Configure the Audit Log Maximum Retention Days (maximumFileRetentionDays)   [ref]

To configure audit log retention, edit the API Server pod specification file /etc/kubernetes/static-pod-resources/configmaps/config/config.yaml on the master node(s) and set the maximumFileRetentionDays parameter to 30 or as appropriate for the number of days:

"auditConfig":{
  ...
  "maximumFileRetentionDays":30,
  ...
 

Rationale:

Retaining audit logs for a specified period of time allows OpenShift Operators to retroactively review and correlate events, such as for investigative purposes.

Severity: 
medium
Identifiers and References

References:  1.2.23

Rule   Configure the kubelet Certificate File for the API Server   [ref]

To enable certificate based kubelet authentication, follow the OpenShift documentation and setup the TLS connection between the API Server and kubelets. Then, verify that kubeletClientInfo has the certFile configured in the API Server pod specification file /etc/kubernetes/static-pod-resources/configmaps/config/config.yaml on the master node(s) to something similar to:

"kubeletClientInfo":{
  ...
  "certFile":"/etc/kubernetes/static-pod-resources/secrets/kubelet-client/tls.crt",
  ...

Rationale:

By default the API Server does not authenticate itself to the kublet's HTTPS endpoints. Requests from the API Server are treated anonymously. Configuring certificate-based kubelet authentication ensures that the API Server authenticates itself to kubelets when submitting requests.

Severity: 
high
Identifiers and References

References:  1.2.5

Rule   Disable Anonymous Authentication to the API Server   [ref]

By default, anonymous access to the OpenShift API is enabled. This configuration check ensures that anonymous requests to the OpenShift API server are disabled. Edit the API server pod specification file /etc/kubernetes/static-pod-resources/configmaps/config/config.yaml on the master node(s) and set the below parameter:

"apiServerArguments":{
  ...
  "anonymous-auth":[
    "false"
  ],
  ...

Rationale:

When enabled, requests that are not rejected by other configured authentication methods are treated as anonymous requests. These requests are then served by the API server. OpenShift Operators should rely on authentication to authorize access and disallow anonymous requests.

Severity: 
medium
Identifiers and References

References:  1.2.1

Rule   Configure the etcd Certificate Authority for the API Server   [ref]

To ensure etcd is configured to make use of TLS encryption for client connections, follow the OpenShift documentation and setup the TLS connection between the API Server and etcd. Then, verify that storageConfig has the ca configured in the API Server pod specification file /etc/kubernetes/static-pod-resources/configmaps/config/config.yaml on the master node(s) to something similar to:

"storageConfig":{
  ...
  "ca":"/etc/kubernetes/static-pod-resources/configmaps/etcd-serving-ca/ca-bundle.crt",
  ...

Rationale:

etcd is a highly-available key-value store used by OpenShift deployments for persistent storage of all REST API objects. These objects are sensitive in nature and should be protected by client authentication. This requires the API Server to identify itself to the etcd server using a SSL Certificate Authority file.

Severity: 
medium
Identifiers and References

References:  1.2.32

Rule   Configure the Maximum Retained Audit Logs   [ref]

To configure how many rotations of audit logs are retained, edit the API Server pod specification file /etc/kubernetes/static-pod-resources/configmaps/config/config.yaml on the master node(s) and set the maximumRetainedFiles parameter to 10 or to an organizationally appropriate value:

"auditConfig":{
  ...
  "maximumRetainedFiles":10,
  ...
 

Rationale:

OpenShift automatically rotates the log files. Retaining old log files ensures OpenShift Operators will have sufficient log data available for carrying out any investigation or correlation. For example, if the audit log size is set to 100 MB and the number of retained log files is set to 10, OpenShift Operators would have approximately 1 GB of log data to use during analysis.

Severity: 
low
Identifiers and References

References:  1.2.24

Rule   Configure Maximum Audit Log Size   [ref]

To rotate audit logs upon reaching a maximum size, edit the API Server pod specification file /etc/kubernetes/static-pod-resources/configmaps/config/config.yaml on the master node(s) and set the maximumFileSizeMegabytes parameter to an appropriate size in MB. For example, to set it to 100 MB:

"auditConfig":{
  ...
  "maximumFileSizeMegabytes":100,
  ...
 

Rationale:

OpenShift automatically rotates log files. Retaining old log files ensures that OpenShift Operators have sufficient log data available for carrying out any investigation or correlation. If you have set file size of 100 MB and the number of old log files to keep as 10, there would be approximately 1 GB of log data available for use in analysis.

Severity: 
medium
Identifiers and References

References:  1.2.25

Rule   Disable basic-auth-file for the API Server   [ref]

Basic Authentication should not be used. Follow the OpenShift documentation and configure alternate mechanisms for authentication. Then, edit API server pod specification file /etc/kubernetes/static-pod-resources/configmaps/config/config.yaml on the master node and remove the basic-auth-file parameter.

"apiServerArguments":{
  ...
  "basic-auth-file":[
    "/path/to/any/file"
  ],
  ...
Alternate authentication mechanisms such as tokens and certificates will need to be used. Username and password for basic authentication will be disabled.

Rationale:

Basic authentication uses plaintext credentials for authentication. Currently the basic authentication credentials last indefinitely, and the password cannot be changed without restarting the API Server. The Basic Authentication is currently supported for convenience and is not intended for production workloads.

Severity: 
medium
Identifiers and References

References:  1.2.2

Rule   Configure the kubelet Certificate Key for the API Server   [ref]

To enable certificate based kubelet authentication, follow the OpenShift documentation and setup the TLS connection between the API Server and kubelets. Then, verify that kubeletClientInfo has the keyFile configured in the API Server pod specification file /etc/kubernetes/static-pod-resources/configmaps/config/config.yaml on the master node(s) to something similar to:

"storageConfig":{
  ...
  "keyFile":"/etc/kubernetes/static-pod-resources/secrets/kubelet-client/tls.key",
  ...

Rationale:

By default the API Server does not authenticate itself to the kubelet's HTTPS endpoints. Requests from the API Server are treated anonymously. Configuring certificate-based kubelet authentication ensures that the API Server authenticates itself to kubelets when submitting requests.

Severity: 
high
Identifiers and References

References:  1.2.5

Rule   Configure the etcd Certificate Key for the API Server   [ref]

To ensure etcd is configured to make use of TLS encryption for client communications, follow the OpenShift documentation and setup the TLS connection between the API Server and etcd. Then, verify that storageConfig has the keyFile configured in the API Server pod specification file /etc/kubernetes/static-pod-resources/configmaps/config/config.yaml on the master node(s) to something similar to:

"storageConfig":{
  ...
  "keyFile":"/etc/kubernetes/static-pod-resources/secrets/etcd-client/tls.key",
  ...

Rationale:

etcd is a highly-available key-value store used by OpenShift deployments for persistent storage of all REST API objects. These objects are sensitive in nature and should be protected by client authentication. This requires the API Server to identify itself to the etcd server using a client certificate and key.

Severity: 
medium
Identifiers and References

References:  1.2.29

Rule   Disable Token-based Authentication   [ref]

To ensure OpenShift does not accept token-based authentication, follow the OpenShift documentation and configure alternate mechanisms for authentication. Then, edit the API Server pod specification file /etc/origin/master/master-config.yaml on the master node(s) and remove the token-auth-file setting.

"apiServerArguments":{
  ...
  "token-auth-file":[
    "/path/to/file"
  ],
  ...

Rationale:

The token-based authentication utilizes static tokens to authenticate requests to the API Server. The tokens are stored in clear-text in a file on the API Server, and cannot be revoked or rotated without restarting the API Server.

Severity: 
high
Identifiers and References

References:  1.2.3

Rule   Configure the API Server Request Timeout   [ref]

All components that use the API should connect via the secured port, authenticate themselves, and be authorized to use the API. To ensure such a configuration, edit the API Server pod specification file /etc/kubernetes/static-pod-resources/configmaps/config/config.yaml on the master node(s) and set the request-timeout to 300:

"apiServerArguments":{
  ...
  "request-timeout":[
    "300"
  ],
  ...

Rationale:

Setting global request timout allows extending the API Server request timeout limit to a duration appropriate to the user's connection speed. By default, it is set to 60 seconds which might be problematic on slower connections making cluster resources inaccessible once the data volume for requests exceeds what can be transmitted in 60 seconds. But, setting this timeout limit to be too large can exhaust the API Server resources making it prone to Denial-of-Service attack. It is recommended to set this limit as appropriate and change the default limit of 60 seconds only if needed.

Severity: 
medium
Identifiers and References

References:  1.2.26

Rule   Enable kubelet HTTPS connections to the API Server   [ref]

HTTPS should be used for connections between the API Server and Kubelets. Edit the API Server pod specification file /etc/kubernetes/static-pod-resources/configmaps/config/config.yaml on the master node(s) and remove the kubelet-https parameter. This will ensure communications are encrypted using TLS (the default setting).

"apiServerArguments":{
  ...
  "kubelet-https":[
    "false"
  ],
  ...

Rationale:

Connections from the API Server to Kubelets could potentially carry sensitive data such as secrets and keys. It is important to use in-transit encryption for any communication between the API Server and the Kubelets.

Severity: 
medium
Identifiers and References

References:  1.2.4

Rule   Configure the etcd Certificate for the API Server   [ref]

To ensure etcd is configured to make use of TLS encryption for client communications, follow the OpenShift documentation and setup the TLS connection between the API Server and etcd. Then, verify that storageConfig has the certFile configured in the API Server pod specification file /etc/kubernetes/static-pod-resources/configmaps/config/config.yaml on the master node(s) to something similar to:

"storageConfig":{
  ...
  "certFile":"/etc/kubernetes/static-pod-resources/secrets/etcd-client/tls.crt",
  ...

Rationale:

etcd is a highly-available key-value store used by OpenShift deployments for persistent storage of all REST API objects. These objects are sensitive in nature and should be protected by client authentication. This requires the API Server to identify itself to the etcd server using a client certificate and key.

Severity: 
medium
Identifiers and References

References:  1.2.29

Rule   Disable Use of the Insecure Bind Address   [ref]

OpenShift should not bind to non-loopback insecure addresses. Edit the API Server pod specification file /etc/kubernetes/static-pod-resources/configmaps/config/config.yaml on the master node(s) and remove the insecure-bind-address parameter.

"apiServerArguments":{
  ...
  "insecure-bind-address":[
    "127.0.0.1"
  ],
  ...

Rationale:

If the API Server is bound to an insecure address the installation would be susceptible to unauthented and unencrypted access to the master node(s). The API Server does not perform authentication checking for insecure binds and the traffic is generally not encrypted.

Severity: 
medium
Identifiers and References

References:  1.2.18

Rule   Configure the kubelet Certificate Authority for the API Server   [ref]

To ensure OpenShift verifies kubelet certificates before establishing connections, follow the OpenShift documentation and setup the TLS connection between the API Server and kubelets. Then, verify that kubeletClientInfo has the ca configured in the API Server pod specification file /etc/kubernetes/static-pod-resources/configmaps/config/config.yaml on the master node(s) to something similar to:

"kubeletClientInfo":{
  ...
  "ca":"/etc/kubernetes/static-pod-resources/configmaps/kubelet-serving-ca/ca-bundle.crt",
  ...

Rationale:

Connections from the API Server to the kubelet are used for fetching logs for pods, attaching (through kubectl) to running pods, and using the kubelet port-forwarding functionality. These connections terminate at the kubelet HTTPS endpoint. By default, the API Server does not verify the kubelet serving certificate, which makes the connection subject to man-in-the-middle attacks, and unsafe to run over untrusted and/or public networks.

Severity: 
high
Identifiers and References

References:  1.2.6

Rule   Enable the Secure Port for the API Server   [ref]

To ensure traffic is served over HTTPS, edit the API Server pod specification file /etc/kubernetes/static-pod-resources/configmaps/config/config.yaml on the master node(s) and either remove the secure-port or set it to a different (non-zero) desired port. in /etc/kubernetes/static-pod-resources/configmaps/config/config.yaml:

"apiServerArguments":{
  ...
  "secure-port":[
    "8443"
  ],
  ...

Rationale:

The secure port is used to serve HTTPS with authentication and authorization. If secure-port is disabled, as indicated with a value of 0, HTTPS traffic will not be served and all traffic will be unencrypted.

Severity: 
medium
Identifiers and References

References:  1.2.20

Red Hat and Red Hat Enterprise Linux are either registered trademarks or trademarks of Red Hat, Inc. in the United States and other countries. All other names are registered trademarks or trademarks of their respective companies.