Customizing Inventory Files for Your Cluster

Ansible inventory files describe the details about the hosts in your cluster, as well as the cluster configuration details for your OpenShift Origin installation. The OpenShift Origin installation playbooks read your inventory file to know where and how to install OpenShift Origin across your set of hosts.

See Ansible documentation for details on the format of an inventory file, including basics on YAML syntax.

When you install the openshift-ansible-utils RPM package as described in Host Preparation, Ansible dependencies create a file at the default location of /etc/ansible/hosts. However, the file is simply the default Ansible example and has no variables related specifically to OpenShift Origin configuration. To successfully install OpenShift Origin, you must replace the default contents of the file with your own desired configuration per your cluster topography and requirements.

The following sections describe commonly used variables to set in your inventory file during cluster installation. Many of the Ansible variables described are optional. Accepting the default values for required variables should suffice for development environments, but for production environments, it is recommended you read through and become familiar with the various options available.

You can review Example Inventory Files for various examples to use as a starting point for your cluster installation.

Images require a version number policy in order to maintain updates. See the Image Version Tag Policy section in the Architecture Guide for more information.

Configuring Cluster Variables

To assign environment variables during the Ansible install that apply more globally to your OpenShift Origin cluster overall, indicate the desired variables in the /etc/ansible/hosts file on separate, single lines within the [OSEv3:vars] section. For example:

[OSEv3:vars]

openshift_master_identity_providers=[{'name': 'htpasswd_auth',
'login': 'true', 'challenge': 'true',
'kind': 'HTPasswdPasswordIdentityProvider',}]

openshift_master_default_subdomain=apps.test.example.com

If a parameter value in the Ansible inventory file contains special characters, such as #, { or }, you must double-escape the value (that is enclose the value in both single and double quotation marks). For example, to use mypasswordwith###hashsigns as a value for the variable openshift_cloudprovider_openstack_password, declare it as openshift_cloudprovider_openstack_password='"mypasswordwith###hashsigns"' in the Ansible host inventory file.

The following tables describe variables for use with the Ansible installer that can be assigned cluster-wide:

Table 1. General Cluster Variables
Variable Purpose

ansible_ssh_user

This variable sets the SSH user for the installer to use and defaults to root. This user should allow SSH-based authentication without requiring a password. If using SSH key-based authentication, then the key should be managed by an SSH agent.

ansible_become

If ansible_ssh_user is not root, this variable must be set to true and the user must be configured for passwordless sudo.

debug_level

This variable sets which INFO messages are logged to the systemd-journald.service. Set one of the following:

  • 0 to log errors and warnings only

  • 2 to log normal information (This is the default level.)

  • 4 to log debugging-level information

  • 6 to log API-level debugging information (request / response)

  • 8 to log body-level API debugging information

For more information on debug log levels, see Configuring Logging Levels.

containerized

If set to true, containerized OpenShift Origin services are run on all target master and node hosts in the cluster instead of installed using RPM packages. If set to false or unset, the default RPM method is used. RHEL Atomic Host requires the containerized method, and is automatically selected for you based on the detection of the /run/ostree-booted file. See Installing on Containerized Hosts for more details.

openshift_clock_enabled

Whether to enable Network Time Protocol (NTP) on cluster nodes. true by default.

To prevent masters and nodes in the cluster from going out of sync, do not change the default value of this parameter.

openshift_master_admission_plugin_config

This variable sets the parameter and arbitrary JSON values as per the requirement in your inventory hosts file.

openshift_master_audit_config

This variable enables API service auditing. See Audit Configuration for more information.

openshift_master_cluster_hostname

This variable overrides the host name for the cluster, which defaults to the host name of the master.

openshift_master_cluster_public_hostname

This variable overrides the public host name for the cluster, which defaults to the host name of the master.

openshift_master_cluster_method

Optional. This variable defines the HA method when deploying multiple masters. Supports the native method. See Multiple Masters for more information.

openshift_rolling_restart_mode

This variable enables rolling restarts of HA masters (i.e., masters are taken down one at a time) when upgrading/automated_upgrades.adoc#install-config-upgrading-automated-upgrades[running the upgrade playbook directly]. It defaults to services, which allows rolling restarts of services on the masters. It can instead be set to system, which enables rolling, full system restarts.

openshift_master_identity_providers

This variable sets the identity provider. The default value is Deny All. If you use a supported identity provider, configure OpenShift Origin to use it.

openshift_master_named_certificates

These variables are used to configure custom certificates which are deployed as part of the installation. See Configuring Custom Certificates for more information.

openshift_master_overwrite_named_certificates

openshift_hosted_router_certificate

Provide the location of the custom certificates for the hosted router.

openshift_hosted_registry_cert_expire_days

Validity of the auto-generated registry certificate in days. Defaults to 730 (2 years).

openshift_ca_cert_expire_days

Validity of the auto-generated CA certificate in days. Defaults to 1825 (5 years).

openshift_node_cert_expire_days

Validity of the auto-generated node certificate in days. Defaults to 730 (2 years).

openshift_master_cert_expire_days

Validity of the auto-generated master certificate in days. Defaults to 730 (2 years).

etcd_ca_default_days

Validity of the auto-generated external etcd certificates in days. Controls validity for etcd CA, peer, server and client certificates. Defaults to 1825 (5 years).

os_firewall_use_firewalld

Set to true to use firewalld instead of the default iptables. Not available on RHEL Atomic Host. See the Configuring the Firewall section for more information.

openshift_master_session_name

These variables override defaults for session options in the OAuth configuration. See Configuring Session Options for more information.

openshift_master_session_max_seconds

openshift_master_session_auth_secrets

openshift_master_session_encryption_secrets

openshift_set_node_ip

This variable configures nodeIP in the node configuration. This variable is needed in cases where it is desired for node traffic to go over an interface other than the default network interface. The host variable openshift_ip can also be configured on each node to set a specific IP that might not be the IP of the default route.

openshift_master_image_policy_config

Sets imagePolicyConfig in the master configuration. See Image Configuration for details.

openshift_router_selector

Default node selector for automatically deploying router pods. See Configuring Node Host Labels for details.

openshift_registry_selector

Default node selector for automatically deploying registry pods. See Configuring Node Host Labels for details.

openshift_template_service_broker_namespaces

This variable enables the template service broker by specifying one or more namespaces whose templates will be served by the broker.

osm_default_node_selector

This variable overrides the node selector that projects will use by default when placing pods, which is defined by the projectConfig.defaultNodeSelector field in the master configuration file. Starting in OpenShift Origin 3.9, this defaults to node-role.kubernetes.io/compute=true if undefined.

openshift_docker_additional_registries

OpenShift Origin adds the specified additional registry or registries to the docker configuration. These are the registries to search. If the registry requires access to a port other than 80, include the port number required in the form of <address>:<port>.

For example:

openshift_docker_additional_registries=example.com:443

If you need to configure your cluster to use an alternate registry, set oreg_url rather than rely on openshift_docker_additional_registries.

openshift_docker_insecure_registries

OpenShift Origin adds the specified additional insecure registry or registries to the docker configuration. For any of these registries, secure sockets layer (SSL) is not verified. Can be set to the host name or IP address of the host. 0.0.0.0/0 is not a valid setting for the IP address.

openshift_docker_blocked_registries

OpenShift Origin adds the specified blocked registry or registries to the docker configuration. Block the listed registries. Setting this to all blocks everything not in the other variables.

openshift_metrics_hawkular_hostname

This variable sets the host name for integration with the metrics console by overriding metricsPublicURL in the master configuration for cluster metrics. If you alter this variable, ensure the host name is accessible via your router. See Configuring Cluster Metrics for details.

openshift_clusterid

This variable is a cluster identifier unique to the AWS Availability Zone. Using this avoids potential issues in Amazon Web Service (AWS) with multiple zones or multiple clusters. See Labeling Clusters for AWS for details.

openshift_image_tag

Use this variable to specify a container image tag to install or configure.

openshift_pkg_version

Use this variable to specify an RPM version to install or configure.

If you modify the openshift_image_tag or the openshift_pkg_version variables after the cluster is set up, then an upgrade can be triggered, resulting in downtime.

  • If openshift_image_tag is set, its value is used for all hosts in containerized environments, even those that have another version installed. If

  • openshift_pkg_version is set, its value is used for all hosts in RPM-based environments, even those that have another version installed.

Table 2. Networking Variables
Variable Purpose

openshift_master_default_subdomain

This variable overrides the default subdomain to use for exposed routes.

os_sdn_network_plugin_name

This variable configures which OpenShift SDN plug-in to use for the pod network, which defaults to redhat/openshift-ovs-subnet for the standard SDN plug-in. Set the variable to redhat/openshift-ovs-multitenant to use the multitenant SDN plug-in.

osm_cluster_network_cidr

This variable overrides the SDN cluster network CIDR block. This is the network from which pod IPs are assigned. This network block should be a private block and must not conflict with existing network blocks in your infrastructure to which pods, nodes, or the master may require access. Defaults to 10.128.0.0/14 and cannot be arbitrarily re-configured after deployment, although certain changes to it can be made in the SDN master configuration.

openshift_portal_net

This variable configures the subnet in which services will be created within the OpenShift Origin SDN. This network block should be private and must not conflict with any existing network blocks in your infrastructure to which pods, nodes, or the master may require access to, or the installation will fail. Defaults to 172.30.0.0/16, and cannot be re-configured after deployment. If changing from the default, avoid 172.17.0.0/16, which the docker0 network bridge uses by default, or modify the docker0 network.

osm_host_subnet_length

This variable specifies the size of the per host subnet allocated for pod IPs by OpenShift Origin SDN. Defaults to 9 which means that a subnet of size /23 is allocated to each host; for example, given the default 10.128.0.0/14 cluster network, this will allocate 10.128.0.0/23, 10.128.2.0/23, 10.128.4.0/23, and so on. This cannot be re-configured after deployment.

openshift_node_proxy_mode

This variable specifies the service proxy mode to use: either iptables for the default, pure-iptables implementation, or userspace for the user space proxy.

openshift_use_flannel

This variable enables flannel as an alternative networking layer instead of the default SDN. If enabling flannel, disable the default SDN with the openshift_use_openshift_sdn variable. For more information, see Using Flannel.

openshift_use_openshift_sdn

Set to false to disable the OpenShift SDN plug-in.

Configuring Deployment Type

Various defaults used throughout the playbooks and roles used by the installer are based on the deployment type configuration (usually defined in an Ansible inventory file).

Ensure the openshift_deployment_type parameter in your inventory file’s [OSEv3:vars] section is set to origin to install the OpenShift Origin variant:

[OSEv3:vars]
openshift_deployment_type=origin

Configuring Host Variables

To assign environment variables to hosts during the Ansible installation, indicate the desired variables in the /etc/ansible/hosts file after the host entry in the [masters] or [nodes] sections. For example:

[masters]
ec2-52-6-179-239.compute-1.amazonaws.com openshift_public_hostname=ose3-master.public.example.com

The following table describes variables for use with the Ansible installer that can be assigned to individual host entries:

Table 3. Host Variables
Variable Purpose

openshift_hostname

This variable overrides the internal cluster host name for the system. Use this when the system’s default IP address does not resolve to the system host name.

openshift_public_hostname

This variable overrides the system’s public host name. Use this for cloud installations, or for hosts on networks using a network address translation (NAT).

openshift_ip

This variable overrides the cluster internal IP address for the system. Use this when using an interface that is not configured with the default route.openshift_ip can be used for etcd.

openshift_public_ip

This variable overrides the system’s public IP address. Use this for cloud installations, or for hosts on networks using a network address translation (NAT).

containerized

If set to true, containerized OpenShift Origin services are run on the target master and node hosts instead of installed using RPM packages. If set to false or unset, the default RPM method is used. RHEL Atomic Host requires the containerized method, and is automatically selected for you based on the detection of the /run/ostree-booted file. See Installing on Containerized Hosts for more details.

openshift_node_labels

This variable adds labels to nodes during installation. See Configuring Node Host Labels for more details.

openshift_node_kubelet_args

This variable is used to configure kubeletArguments on nodes, such as arguments used in container and image garbage collection, and to specify resources per node. kubeletArguments are key value pairs that are passed directly to the Kubelet that match the Kubelet’s command line arguments. kubeletArguments are not migrated or validated and may become invalid if used. These values override other settings in node configuration which may cause invalid configurations. Example usage: {'image-gc-high-threshold': ['90'],'image-gc-low-threshold': ['80']}.

openshift_docker_options

This variable configures additional docker options within /etc/sysconfig/docker, such as options used in Managing Container Logs. It is recommended to use json-file.

The following example shows the configuration of Docker to use the json-file log driver, where Docker will rotate between three 1 MB log files:

"--log-driver json-file --log-opt max-size=1M --log-opt max-file=3"

openshift_schedulable

This variable configures whether the host is marked as a schedulable node, meaning that it is available for placement of new pods. See Configuring Schedulability on Masters.

openshift_node_problem_detector_install

This variable is used to activate the Node Problem Detector. If set to false, the default, the Node Problem Detector is not installed or started.

Configuring Master API Port

To configure the default ports used by the master API, configure the following variables in the /etc/ansible/hosts file:

Table 4. Master API Port
Variable Purpose

openshift_master_api_port

This variable sets the port number to access the OpenShift Origin API.

For example:

openshift_master_api_port=3443

The web console port setting (openshift_master_console_port) must match the API server port (openshift_master_api_port).

Configuring Cluster Pre-install Checks

Pre-install checks are a set of diagnostic tasks that run as part of the openshift_health_checker Ansible role. They run prior to an Ansible installation of OpenShift Origin, ensure that required inventory values are set, and identify potential issues on a host that can prevent or interfere with a successful installation.

The following table describes available pre-install checks that will run before every Ansible installation of OpenShift Origin:

Table 5. Pre-install Checks
Check Name Purpose

memory_availability

This check ensures that a host has the recommended amount of memory for the specific deployment of OpenShift Origin. Default values have been derived from the latest installation documentation. A user-defined value for minimum memory requirements may be set by setting the openshift_check_min_host_memory_gb cluster variable in your inventory file.

disk_availability

This check only runs on etcd, master, and node hosts. It ensures that the mount path for an OpenShift Origin installation has sufficient disk space remaining. Recommended disk values are taken from the latest installation documentation. A user-defined value for minimum disk space requirements may be set by setting openshift_check_min_host_disk_gb cluster variable in your inventory file.

docker_storage

Only runs on hosts that depend on the docker daemon (nodes and containerized installations). Checks that docker's total usage does not exceed a user-defined limit. If no user-defined limit is set, docker's maximum usage threshold defaults to 90% of the total size available. The threshold limit for total percent usage can be set with a variable in your inventory file: max_thinpool_data_usage_percent=90. A user-defined limit for maximum thinpool usage may be set by setting the max_thinpool_data_usage_percent cluster variable in your inventory file.

docker_storage_driver

Ensures that the docker daemon is using a storage driver supported by OpenShift Origin. If the devicemapper storage driver is being used, the check additionally ensures that a loopback device is not being used. For more information, see Docker’s Use the Device Mapper Storage Driver guide.

docker_image_availability

Attempts to ensure that images required by an OpenShift Origin installation are available either locally or in at least one of the configured container image registries on the host machine.

package_version

Runs on yum-based systems determining if multiple releases of a required OpenShift Origin package are available. Having multiple releases of a package available during an enterprise installation of OpenShift suggests that there are multiple yum repositories enabled for different releases, which may lead to installation problems. This check is skipped if the openshift_release variable is not defined in the inventory file.

package_availability

Runs prior to non-containerized installations of OpenShift Origin. Ensures that RPM packages required for the current installation are available.

package_update

Checks whether a yum update or package installation will succeed, without actually performing it or running yum on the host.

To disable specific pre-install checks, include the variable openshift_disable_check with a comma-delimited list of check names in your inventory file. For example:

openshift_disable_check=memory_availability,disk_availability

A similar set of health checks meant to run for diagnostics on existing clusters can be found in Ansible-based Health Checks. Another set of checks for checking certificate expiration can be found in Redeploying Certificates.

Configuring System Containers

System containers provide a way to containerize services that need to run before the docker daemon is running. They are Docker-formatted containers that use:

System containers are therefore stored and run outside of the traditional docker service. For more details on system container technology, see Running System Containers in the Red Hat Enterprise Linux Atomic Host: Managing Containers documentation.

You can configure your OpenShift Origin installation to run certain components as system containers instead of their RPM or standard containerized methods. Currently, the docker and etcd components can be run as system containers in OpenShift Origin.

System containers are currently OS-specific because they require specific versions of atomic and systemd. For example, different system containers are created for RHEL, Fedora, or CentOS. Ensure that the system containers you are using match the OS of the host they will run on.

Running etcd as a System Container

When using the RPM-based installation method for OpenShift Origin, etcd is installed using RPM packages on any RHEL systems. When using the containerized installation method, the rhel7/etcd image is used instead for RHEL or RHEL Atomic Hosts.

However, you can configure your OpenShift Origin installation to alternatively run etcd as a system container. Whereas the standard containerized method uses a systemd service named etcd_container, the system container method uses the service name etcd, same as the RPM-based method. The data directory for etcd using this method is /var/lib/etcd.

To run etcd as a system container, set the following cluster variable in your inventory file in the [OSEv3:vars] section:

openshift_use_etcd_system_container=True

Configuring a Registry Location

If you are using an image registry other than the default at registry.access.redhat.com, specify the desired registry within the /etc/ansible/hosts file.

oreg_url={registry}/openshift3/ose-${component}:${version}
openshift_examples_modify_imagestreams=true
Table 6. Registry Variables
Variable Purpose

oreg_url

Set to the alternate image location. Necessary if you are not using the default registry at registry.access.redhat.com. The default component inherits the image prefix and version from the oreg_url value.

openshift_examples_modify_imagestreams

Set to true if pointing to a registry other than the default. Modifies the image stream location to the value of oreg_url.

For example:

oreg_url=example.com/openshift3/ose-${component}:${version}
openshift_examples_modify_imagestreams=true

Configuring a Registry Route

To allow users to push and pull images to the internal Docker registry from outside of the OpenShift Origin cluster, configure the registry route in the /etc/ansible/hosts file. By default, the registry route is docker-registry-default.router.default.svc.cluster.local.

Table 7. Registry Route Variables
Variable Purpose

openshift_hosted_registry_routehost

Set to the value of the desired registry route. The route contains either a name that resolves to an infrastructure node where a router manages communication or the subdomain that you set as the default application subdomain wildcard value. For example, if you set the openshift_master_default_subdomain parameter to apps.example.com and *.apps.example.com resolves to infrastructure nodes or a load balancer, you might use registry.apps.example.com as the registry route.

openshift_hosted_registry_routecertificates

Set the paths to the registry certificates. If you do not provide values for the certificate locations, certificates are generated. You can define locations for the following certificates:

  • certfile

  • keyfile

  • cafile

openshift_hosted_registry_routetermination

Set to one of the following values:

  • Set to reencrypt to terminate encryption at the edge router and re-encrypt it with a new certificate supplied by the destination.

  • Set to passthrough to terminate encryption at the destination. The destination is responsible for decrypting traffic.

For example:

openshift_hosted_registry_routehost=<path>
openshift_hosted_registry_routetermination=reencrypt
openshift_hosted_registry_routecertificates= "{'certfile': '<path>/org-cert.pem', 'keyfile': '<path>/org-privkey.pem', 'cafile': '<path>/org-chain.pem'}"

Configuring the Registry Console

If you are using a Cockpit registry console image other than the default or require a specific version of the console, specify the desired registry within the /etc/ansible/hosts file:

openshift_cockpit_deployer_prefix=<registry_name>/<namespace>/
openshift_cockpit_deployer_version=<cockpit_image_tag>
Table 8. Registry Variables
Variable Purpose

openshift_cockpit_deployer_prefix

Specify the URL and path to the directory where the image is located. The default value is kubernetes, so the value is cockpit/kubernetes:latest. This differs from several other images that have the origin- prefix.

openshift_cockpit_deployer_version

Specify the Cockpit image version.

For example: If your image is at registry.example.com/openshift3/registry-console and you require version 3.10.1, enter:

openshift_cockpit_deployer_prefix='registry.example.com/openshift3/'
openshift_cockpit_deployer_version='3.10.1'

Configuring GlusterFS Persistent Storage

GlusterFS can be configured to provide persistent storage and dynamic provisioning for OpenShift Origin. It can be used both containerized within OpenShift Origin (Containerized GlusterFS) and non-containerized on its own nodes (External GlusterFS).

Additional information and examples, including the ones below, can be found at Persistent Storage Using GlusterFS.

Configuring Containerized GlusterFS

See Containerized GlusterFS Considerations for specific host preparations and prerequisites.

  1. In your inventory file, add glusterfs in the [OSEv3:children] section to enable the [glusterfs] group:

    [OSEv3:children]
    masters
    nodes
    glusterfs
  2. Add a [glusterfs] section with entries for each storage node that will host the GlusterFS storage. For each node, set glusterfs_devices to a list of raw block devices that will be completely managed as part of a GlusterFS cluster. There must be at least one device listed. Each device must be bare, with no partitions or LVM PVs. Specifying the variable takes the form:

    <hostname_or_ip> glusterfs_devices='[ "</path/to/device1/>", "</path/to/device2>", ... ]'

    For example:

    [glusterfs]
    node11.example.com glusterfs_devices='[ "/dev/xvdc", "/dev/xvdd" ]'
    node12.example.com glusterfs_devices='[ "/dev/xvdc", "/dev/xvdd" ]'
    node13.example.com glusterfs_devices='[ "/dev/xvdc", "/dev/xvdd" ]'
  3. Add the hosts listed under [glusterfs] to the [nodes] group:

    [nodes]
    ...
    node11.example.com openshift_schedulable=True
    node12.example.com openshift_schedulable=True
    node13.example.com openshift_schedulable=True

Configuring External GlusterFS

  1. In your inventory file, add glusterfs in the [OSEv3:children] section to enable the [glusterfs] group:

    [OSEv3:children]
    masters
    nodes
    glusterfs
  2. Include the following variables in the [OSEv3:vars] section, adjusting them as needed for your configuration:

    [OSEv3:vars]
    ...
    openshift_storage_glusterfs_is_native=false
    openshift_storage_glusterfs_storageclass=true
    openshift_storage_glusterfs_heketi_is_native=true
    openshift_storage_glusterfs_heketi_executor=ssh
    openshift_storage_glusterfs_heketi_ssh_port=22
    openshift_storage_glusterfs_heketi_ssh_user=root
    openshift_storage_glusterfs_heketi_ssh_sudo=false
    openshift_storage_glusterfs_heketi_ssh_keyfile="/root/.ssh/id_rsa"
  3. Add a [glusterfs] section with entries for each storage node that will host the GlusterFS storage. For each node, set glusterfs_devices to a list of raw block devices that will be completely managed as part of a GlusterFS cluster. There must be at least one device listed. Each device must be bare, with no partitions or LVM PVs. Also, set glusterfs_ip to the IP address of the node. Specifying the variable takes the form:

    <hostname_or_ip> glusterfs_ip=<ip_address> glusterfs_devices='[ "</path/to/device1/>", "</path/to/device2>", ... ]'

    For example:

    [glusterfs]
    gluster1.example.com glusterfs_ip=192.168.10.11 glusterfs_devices='[ "/dev/xvdc", "/dev/xvdd" ]'
    gluster2.example.com glusterfs_ip=192.168.10.12 glusterfs_devices='[ "/dev/xvdc", "/dev/xvdd" ]'
    gluster3.example.com glusterfs_ip=192.168.10.13 glusterfs_devices='[ "/dev/xvdc", "/dev/xvdd" ]'

Configuring an OpenShift Container Registry

An integrated OpenShift Container Registry can be deployed using the installer.

Configuring Registry Storage

If no registry storage options are used, the default OpenShift Container Registry is ephemeral and all data will be lost when the pod no longer exists. There are several options for enabling registry storage when using the advanced installer:

Option A: NFS Host Group

The use of NFS for registry storage is not recommended in OpenShift Origin.

When the following variables are set, an NFS volume is created during cluster installation with the path <nfs_directory>/<volume_name> on the host within the [nfs] host group. For example, the volume path using these options would be /exports/registry:

[OSEv3:vars]

openshift_hosted_registry_storage_kind=nfs
openshift_hosted_registry_storage_access_modes=['ReadWriteMany']
openshift_hosted_registry_storage_nfs_directory=/exports
openshift_hosted_registry_storage_nfs_options='*(rw,root_squash)'
openshift_hosted_registry_storage_volume_name=registry
openshift_hosted_registry_storage_volume_size=10Gi

Option B: External NFS Host

The use of NFS for registry storage is not recommended in OpenShift Origin.

To use an external NFS volume, one must already exist with a path of <nfs_directory>/<volume_name> on the storage host. The remote volume path using the following options would be nfs.example.com:/exports/registry.

[OSEv3:vars]

openshift_hosted_registry_storage_kind=nfs
openshift_hosted_registry_storage_access_modes=['ReadWriteMany']
openshift_hosted_registry_storage_host=nfs.example.com
openshift_hosted_registry_storage_nfs_directory=/exports
openshift_hosted_registry_storage_volume_name=registry
openshift_hosted_registry_storage_volume_size=10Gi

Upgrading or Installing OpenShift Origin with NFS

The use of NFS for the core OpenShift Origin components is not recommended, as NFS (and the NFS Protocol) does not provide the proper consistency needed for the applications that make up the OpenShift Origin infrastructure.

As a result, the installer and update playbooks require an option to enable the use of NFS with core infrastructure components.

# Enable unsupported configurations, things that will yield a partially
# functioning cluster but would not be supported for production use
#openshift_enable_unsupported_configurations=false

If you see the following messages when upgrading or installing your cluster, then an additional step is required.

TASK [Run variable sanity checks] **********************************************
fatal: [host.example.com]: FAILED! => {"failed": true, "msg": "last_checked_host: host.example.com, last_checked_var: openshift_hosted_registry_storage_kind;nfs is an unsupported type for openshift_hosted_registry_storage_kind. openshift_enable_unsupported_configurations=True mustbe specified to continue with this configuration."}

In your Ansible inventory file, specify the following parameter:

[OSEv3:vars]
openshift_enable_unsupported_configurations=True

Option C: OpenStack Platform

An OpenStack storage configuration must already exist.

[OSEv3:vars]

openshift_hosted_registry_storage_kind=openstack
openshift_hosted_registry_storage_access_modes=['ReadWriteOnce']
openshift_hosted_registry_storage_openstack_filesystem=ext4
openshift_hosted_registry_storage_openstack_volumeID=3a650b4f-c8c5-4e0a-8ca5-eaee11f16c57
openshift_hosted_registry_storage_volume_size=10Gi

Option D: AWS or Another S3 Storage Solution

The simple storage solution (S3) bucket must already exist.

[OSEv3:vars]

#openshift_hosted_registry_storage_kind=object
#openshift_hosted_registry_storage_provider=s3
#openshift_hosted_registry_storage_s3_accesskey=access_key_id
#openshift_hosted_registry_storage_s3_secretkey=secret_access_key
#openshift_hosted_registry_storage_s3_bucket=bucket_name
#openshift_hosted_registry_storage_s3_region=bucket_region
#openshift_hosted_registry_storage_s3_chunksize=26214400
#openshift_hosted_registry_storage_s3_rootdirectory=/registry
#openshift_hosted_registry_pullthrough=true
#openshift_hosted_registry_acceptschema2=true
#openshift_hosted_registry_enforcequota=true

If you are using a different S3 service, such as Minio or ExoScale, also add the region endpoint parameter:

openshift_hosted_registry_storage_s3_regionendpoint=https://myendpoint.example.com/

Option E: Containerized GlusterFS

Similar to configuring Containerized GlusterFS, GlusterFS can be configured to provide storage for an OpenShift Container Registry during the initial installation of the cluster to offer redundant and reliable storage for the registry.

See Containerized GlusterFS Considerations for specific host preparations and prerequisites.

  1. In your inventory file, set the following variable under [OSEv3:vars]:

    [OSEv3:vars]
    ...
    openshift_hosted_registry_storage_kind=glusterfs
  2. Add glusterfs_registry in the [OSEv3:children] section to enable the [glusterfs_registry] group:

    [OSEv3:children]
    masters
    nodes
    glusterfs_registry
  3. Add a [glusterfs_registry] section with entries for each storage node that will host the GlusterFS storage. For each node, set glusterfs_devices to a list of raw block devices that will be completely managed as part of a GlusterFS cluster. There must be at least one device listed. Each device must be bare, with no partitions or LVM PVs. Specifying the variable takes the form:

    <hostname_or_ip> glusterfs_devices='[ "</path/to/device1/>", "</path/to/device2>", ... ]'

    For example:

    [glusterfs_registry]
    node11.example.com glusterfs_devices='[ "/dev/xvdc", "/dev/xvdd" ]'
    node12.example.com glusterfs_devices='[ "/dev/xvdc", "/dev/xvdd" ]'
    node13.example.com glusterfs_devices='[ "/dev/xvdc", "/dev/xvdd" ]'
  4. Add the hosts listed under [glusterfs_registry] to the [nodes] group:

    [nodes]
    ...
    node11.example.com openshift_schedulable=True
    node12.example.com openshift_schedulable=True
    node13.example.com openshift_schedulable=True

Option F: Google Cloud Storage (GCS) bucket on Google Compute Engine (GCE)

A GCS bucket must already exist.

[OSEv3:vars]

openshift_hosted_registry_storage_provider=gcs
openshift_hosted_registry_storage_gcs_bucket=bucket01
openshift_hosted_registry_storage_gcs_keyfile=test.key
openshift_hosted_registry_storage_gcs_rootdirectory=/registry

Option G: vSphere Volume with vSphere Cloud Provider (VCP)

The vSphere Cloud Provider must be configured with a datastore accessible by the OpenShift Origin nodes.

When using vSphere volume for the registry, you must set the storage access mode to ReadWriteOnce and the replica count to 1:

[OSEv3:vars]

openshift_hosted_registry_storage_kind=vsphere
openshift_hosted_registry_storage_access_modes=['ReadWriteOnce']
openshift_hosted_registry_storage_annotations=['volume.beta.kubernetes.io/storage-provisioner: kubernetes.io/vsphere-volume']
openshift_hosted_registry_replicas=1

Configuring Global Proxy Options

If your hosts require use of a HTTP or HTTPS proxy in order to connect to external hosts, there are many components that must be configured to use the proxy, including masters, Docker, and builds. Node services only connect to the master API requiring no external access and therefore do not need to be configured to use a proxy.

In order to simplify this configuration, the following Ansible variables can be specified at a cluster or host level to apply these settings uniformly across your environment.

See install_config/build_defaults_overrides.adoc#install-config-build-defaults-overrides[Configuring Global Build Defaults and Overrides] for more information on how the proxy environment is defined for builds.

Table 9. Cluster Proxy Variables
Variable Purpose

openshift_http_proxy

This variable specifies the HTTP_PROXY environment variable for masters and the Docker daemon.

openshift_https_proxy

This variable specifices the HTTPS_PROXY environment variable for masters and the Docker daemon.

openshift_no_proxy

This variable is used to set the NO_PROXY environment variable for masters and the Docker daemon. Provide a comma-separated list of host names, domain names, or wildcard host names that do not use the defined proxy. By default, this list is augmented with the list of all defined OpenShift Origin host names.

openshift_generate_no_proxy_hosts

This boolean variable specifies whether or not the names of all defined OpenShift hosts and *.cluster.local should be automatically appended to the NO_PROXY list. Defaults to true; set it to false to override this option.

openshift_builddefaults_http_proxy

This variable defines the HTTP_PROXY environment variable inserted into builds using the BuildDefaults admission controller. If openshift_http_proxy is set, this variable will inherit that value; you only need to set this if you want your builds to use a different value.

openshift_builddefaults_https_proxy

This variable defines the HTTPS_PROXY environment variable inserted into builds using the BuildDefaults admission controller. If openshift_https_proxy is set, this variable will inherit that value; you only need to set this if you want your builds to use a different value.

openshift_builddefaults_no_proxy

This variable defines the NO_PROXY environment variable inserted into builds using the BuildDefaults admission controller. If openshift_no_proxy is set, this variable will inherit that value; you only need to set this if you want your builds to use a different value.

openshift_builddefaults_git_http_proxy

This variable defines the HTTP proxy used by git clone operations during a build, defined using the BuildDefaults admission controller. If openshift_builddefaults_http_proxy is set, this variable will inherit that value; you only need to set this if you want your git clone operations to use a different value.

openshift_builddefaults_git_https_proxy

This variable defines the HTTPS proxy used by git clone operations during a build, defined using the BuildDefaults admission controller. If openshift_builddefaults_https_proxy is set, this variable will inherit that value; you only need to set this if you want your git clone operations to use a different value.

If any of:

  • openshift_no_proxy

  • openshift_https_proxy

  • openshift_http_proxy

are set, then all cluster hosts will have an automatically generated NO_PROXY environment variable injected into several service configuration scripts. The default .svc domain and your cluster’s dns_domain (typically .cluster.local) will also be added.

Setting openshift_generate_no_proxy_hosts to false in your inventory will not disable the automatic addition of the .svc domain and the cluster domain. These are required and added automatically if any of the above listed proxy parameters are set.

Configuring the Firewall

If you are changing the default firewall, ensure that each host in your cluster is using the same firewall type to prevent inconsistencies.

While iptables is the default firewall, firewalld is recommended for new installations.

OpenShift Origin uses iptables as the default firewall, but you can configure your cluster to use firewalld during the install process.

Because iptables is the default firewall, OpenShift Origin is designed to have it configured automatically. However, iptables rules can break OpenShift Origin if not configured correctly. The advantages of firewalld include allowing multiple objects to safely share the firewall rules.

To use firewalld as the firewall for an OpenShift Origin installation, add the os_firewall_use_firewalld variable to the list of configuration variables in the Ansible host file at install:

[OSEv3:vars]
os_firewall_use_firewalld=True

Setting this variable to true opens the required ports and adds rules to the default zone, ensuring that firewalld is configured correctly.

Using the firewalld default configuration comes with limited configuration options, and cannot be overridden. For example, while you can set up a storage network with interfaces in multiple zones, the interface that nodes communicate on must be in the default zone.

Configuring Schedulability on Masters

Any hosts you designate as masters during the installation process should also be configured as nodes so that the masters are configured as part of the OpenShift SDN. You must do so by adding entries for these hosts to the [nodes] section:

[nodes]
master.example.com

In previous versions of OpenShift Origin, master hosts were marked as unschedulable nodes by default by the installer, meaning that new pods could not be placed on the hosts. Starting with OpenShift Origin 3.9, however, masters are marked schedulable automatically during installation. This change is mainly so that the web console, which used to run as part of the master itself, can instead be run as a pod deployed to the master.

If you want to change the schedulability of a host post-installation, see Marking Nodes as Unschedulable or Schedulable.

Configuring Node Host Labels

You can assign labels to node hosts during the Ansible install by configuring the /etc/ansible/hosts file. Labels are useful for determining the placement of pods onto nodes using the scheduler. Other than region=infra (referred to as dedicated infrastructure nodes and discussed further in Configuring Dedicated Infrastructure Nodes), the actual label names and values are arbitrary and can be assigned however you see fit per your cluster’s requirements.

To assign labels to a node host during an Ansible install, use the openshift_node_labels variable with the desired labels added to the desired node host entry in the [nodes] section. In the following example, labels are set for a region called primary and a zone called east:

[nodes]
node1.example.com openshift_node_labels="{'region': 'primary', 'zone': 'east'}"

Starting in OpenShift Origin 3.9, masters are now marked as schedulable nodes by default. As a result, the default node selector (defined in the master configuration file’s projectConfig.defaultNodeSelector field to determine which node that projects will use by default when placing pods, and previously left blank by default) is now set by default during cluster installations. It is set to node-role.kubernetes.io/compute=true unless overridden using the osm_default_node_selector Ansible variable.

In addition, whether osm_default_node_selector is set or not, the following automatic labeling occurs for hosts defined in your inventory file during installation:

  • non-master, non-dedicated infrastructure nodes hosts (for example, the node1.example.com host shown above) are labeled with node-role.kubernetes.io/compute=true

  • master nodes are labeled node-role.kubernetes.io/master=true

This ensures that the default node selector has available nodes to choose from when determining pod placement.

If you accept the default node selector of node-role.kubernetes.io/compute=true during installation, ensure that you do not only have dedicated infrastructure nodes as the non-master nodes defined in your cluster. In that scenario, application pods would fail to deploy because no nodes with the node-role.kubernetes.io/compute=true label would be available to match the default node selector when scheduling pods for projects.

See Setting the Cluster-wide Default Node Selector for steps on adjusting this setting post-installation if needed.

Configuring Dedicated Infrastructure Nodes

It is recommended for production environments that you maintain dedicated infrastructure nodes where the registry and router pods can run separately from pods used for user applications.

The openshift_router_selector and openshift_registry_selector Ansible settings determine the label selectors used when placing registry and router pods. They are set to region=infra by default:

# default selectors for router and registry services
# openshift_router_selector='region=infra'
# openshift_registry_selector='region=infra'

The registry and router are only able to run on node hosts with the region=infra label, which are then considered dedicated infrastructure nodes. Ensure that at least one node host in your OpenShift Origin environment has the region=infra label. For example:

[nodes]
infra-node1.example.com openshift_node_labels="{'region': 'infra','zone': 'default'}"

If there is not a node in the [nodes] section that matches the selector settings, the default router and registry will be deployed as failed with Pending status.

If you do not intend to use OpenShift Origin to manage the registry and router, configure the following Ansible settings:

openshift_hosted_manage_registry=false
openshift_hosted_manage_router=false

If you are using an image registry other than the default registry.access.redhat.com, you need to specify the desired registry in the /etc/ansible/hosts file.

As described in Configuring Schedulability on Masters, master hosts are marked schedulable by default. If you label a master host with region=infra and have no other dedicated infrastructure nodes, the master hosts must also be marked as schedulable. Otherwise, the registry and router pods cannot be placed anywhere:

[nodes]
master.example.com openshift_node_labels="{'region': 'infra','zone': 'default'}" openshift_schedulable=true

Configuring Session Options

Session options in the OAuth configuration are configurable in the inventory file. By default, Ansible populates a sessionSecretsFile with generated authentication and encryption secrets so that sessions generated by one master can be decoded by the others. The default location is /etc/origin/master/session-secrets.yaml, and this file will only be re-created if deleted on all masters.

You can set the session name and maximum number of seconds with openshift_master_session_name and openshift_master_session_max_seconds:

openshift_master_session_name=ssn
openshift_master_session_max_seconds=3600

If provided, openshift_master_session_auth_secrets and openshift_master_encryption_secrets must be equal length.

For openshift_master_session_auth_secrets, used to authenticate sessions using HMAC, it is recommended to use secrets with 32 or 64 bytes:

openshift_master_session_auth_secrets=['DONT+USE+THIS+SECRET+b4NV+pmZNSO']

For openshift_master_encryption_secrets, used to encrypt sessions, secrets must be 16, 24, or 32 characters long, to select AES-128, AES-192, or AES-256:

openshift_master_session_encryption_secrets=['DONT+USE+THIS+SECRET+b4NV+pmZNSO']

Configuring Custom Certificates

Custom serving certificates for the public host names of the OpenShift Origin API and web console can be deployed during cluster installation and are configurable in the inventory file.

Custom certificates should only be configured for the host name associated with the publicMasterURL which can be set using openshift_master_cluster_public_hostname. Using a custom serving certificate for the host name associated with the masterURL (openshift_master_cluster_hostname) will result in TLS errors as infrastructure components will attempt to contact the master API using the internal masterURL host.

Certificate and key file paths can be configured using the openshift_master_named_certificates cluster variable:

openshift_master_named_certificates=[{"certfile": "/path/to/custom1.crt", "keyfile": "/path/to/custom1.key", "cafile": "/path/to/custom-ca1.crt"}]

File paths must be local to the system where Ansible will be run. Certificates are copied to master hosts and are deployed within the /etc/origin/master/named_certificates/ directory.

Ansible detects a certificate’s Common Name and Subject Alternative Names. Detected names can be overridden by providing the "names" key when setting openshift_master_named_certificates:

openshift_master_named_certificates=[{"certfile": "/path/to/custom1.crt", "keyfile": "/path/to/custom1.key", "names": ["public-master-host.com"], "cafile": "/path/to/custom-ca1.crt"}]

Certificates configured using openshift_master_named_certificates are cached on masters, meaning that each additional Ansible run with a different set of certificates results in all previously deployed certificates remaining in place on master hosts and within the master configuration file.

If you would like openshift_master_named_certificates to be overwritten with the provided value (or no value), specify the openshift_master_overwrite_named_certificates cluster variable:

openshift_master_overwrite_named_certificates=true

For a more complete example, consider the following cluster variables in an inventory file:

openshift_master_cluster_method=native
openshift_master_cluster_hostname=lb-internal.openshift.com
openshift_master_cluster_public_hostname=custom.openshift.com

To overwrite the certificates on a subsequent Ansible run, you could set the following:

openshift_master_named_certificates=[{"certfile": "/root/STAR.openshift.com.crt", "keyfile": "/root/STAR.openshift.com.key", "names": ["custom.openshift.com"]}]
openshift_master_overwrite_named_certificates=true

Configuring Certificate Validity

By default, the certificates used to govern the etcd, master, and kubelet expire after two to five years. The validity (length in days until they expire) for the auto-generated registry, CA, node, and master certificates can be configured during installation using the following variables (default values shown):

[OSEv3:vars]

openshift_hosted_registry_cert_expire_days=730
openshift_ca_cert_expire_days=1825
openshift_node_cert_expire_days=730
openshift_master_cert_expire_days=730
etcd_ca_default_days=1825

These values are also used when redeploying certificates via Ansible post-installation.

Configuring Cluster Metrics

Cluster metrics are not set to automatically deploy. Set the following to enable cluster metrics during cluster installation:

[OSEv3:vars]

openshift_metrics_install_metrics=true

The metrics public URL can be set during cluster installation using the openshift_metrics_hawkular_hostname Ansible variable, which defaults to:

https://hawkular-metrics.{{openshift_master_default_subdomain}}/hawkular/metrics

If you alter this variable, ensure the host name is accessible via your router.

openshift_metrics_hawkular_hostname=hawkular-metrics.{{openshift_master_default_subdomain}}

In accordance with upstream Kubernetes rules, metrics can be collected only on the default interface of eth0.

You must set an openshift_master_default_subdomain value to deploy metrics.

Configuring Metrics Storage

The openshift_metrics_cassandra_storage_type variable must be set in order to use persistent storage for metrics. If openshift_metrics_cassandra_storage_type is not set, then cluster metrics data is stored in an emptyDir volume, which will be deleted when the Cassandra pod terminates.

There are three options for enabling cluster metrics storage during cluster installation:

Option A: Dynamic

If your OpenShift Origin environment supports dynamic volume provisioning for your cloud provider, use the following variable:

[OSEv3:vars]

openshift_metrics_cassandra_storage_type=dynamic

If there are multiple default dynamically provisioned volume types, such as gluster-storage and glusterfs-storage-block, you can specify the provisioned volume type by variable. For example, openshift_logging_es_pvc_storage_class_name=glusterfs-storage-block openshift_metrics_cassandra_pvc_storage_class_name=glusterfs-storage-block.

Check Volume Configuration for more information on using DynamicProvisioningEnabled to enable or disable dynamic provisioning.

Option B: NFS Host Group

The use of NFS for metrics storage is not recommended in OpenShift Origin.

When the following variables are set, an NFS volume is created during cluster installation with path <nfs_directory>/<volume_name> on the host within the [nfs] host group. For example, the volume path using these options would be /exports/metrics:

[OSEv3:vars]

openshift_metrics_storage_kind=nfs
openshift_metrics_storage_access_modes=['ReadWriteOnce']
openshift_metrics_storage_nfs_directory=/exports
openshift_metrics_storage_nfs_options='*(rw,root_squash)'
openshift_metrics_storage_volume_name=metrics
openshift_metrics_storage_volume_size=10Gi

Option C: External NFS Host

The use of NFS for metrics storage is not recommended in OpenShift Origin.

To use an external NFS volume, one must already exist with a path of <nfs_directory>/<volume_name> on the storage host.

[OSEv3:vars]

openshift_metrics_storage_kind=nfs
openshift_metrics_storage_access_modes=['ReadWriteOnce']
openshift_metrics_storage_host=nfs.example.com
openshift_metrics_storage_nfs_directory=/exports
openshift_metrics_storage_volume_name=metrics
openshift_metrics_storage_volume_size=10Gi

The remote volume path using the following options would be nfs.example.com:/exports/metrics.

Upgrading or Installing OpenShift Origin with NFS

The use of NFS for the core OpenShift Origin components is not recommended, as NFS (and the NFS Protocol) does not provide the proper consistency needed for the applications that make up the OpenShift Origin infrastructure.

As a result, the installer and update playbooks require an option to enable the use of NFS with core infrastructure components.

# Enable unsupported configurations, things that will yield a partially
# functioning cluster but would not be supported for production use
#openshift_enable_unsupported_configurations=false

If you see the following messages when upgrading or installing your cluster, then an additional step is required.

TASK [Run variable sanity checks] **********************************************
fatal: [host.example.com]: FAILED! => {"failed": true, "msg": "last_checked_host: host.example.com, last_checked_var: openshift_hosted_registry_storage_kind;nfs is an unsupported type for openshift_hosted_registry_storage_kind. openshift_enable_unsupported_configurations=True mustbe specified to continue with this configuration."}

In your Ansible inventory file, specify the following parameter:

[OSEv3:vars]
openshift_enable_unsupported_configurations=True

Configuring Cluster Logging

Cluster logging is not set to automatically deploy by default. Set the following to enable cluster logging during cluster installation:

[OSEv3:vars]

openshift_logging_install_logging=true

Configuring Logging Storage

The openshift_logging_es_pvc_dynamic variable must be set in order to use persistent storage for logging. If openshift_logging_es_pvc_dynamic is not set, then cluster logging data is stored in an emptyDir volume, which will be deleted when the Elasticsearch pod terminates.

There are three options for enabling cluster logging storage during cluster installation:

Option A: Dynamic

If your OpenShift Origin environment supports dynamic volume provisioning for your cloud provider, use the following variable:

[OSEv3:vars]

openshift_logging_es_pvc_dynamic=true

If there are multiple default dynamically provisioned volume types, such as gluster-storage and glusterfs-storage-block, you can specify the provisioned volume type by variable. For example, openshift_logging_es_pvc_storage_class_name=glusterfs-storage-block openshift_metrics_cassandra_pvc_storage_class_name=glusterfs-storage-block.

Check Volume Configuration for more information on using DynamicProvisioningEnabled to enable or disable dynamic provisioning.

Option B: NFS Host Group

The use of NFS for logging storage is not recommended in OpenShift Origin.

When the following variables are set, an NFS volume is created during cluster installation with path <nfs_directory>/<volume_name> on the host within the [nfs] host group. For example, the volume path using these options would be /exports/logging:

[OSEv3:vars]

openshift_logging_storage_kind=nfs
openshift_logging_storage_access_modes=['ReadWriteOnce']
openshift_logging_storage_nfs_directory=/exports
openshift_logging_storage_nfs_options='*(rw,root_squash)'
openshift_logging_storage_volume_name=logging
openshift_logging_storage_volume_size=10Gi

Option C: External NFS Host

The use of NFS for logging storage is not recommended in OpenShift Origin.

To use an external NFS volume, one must already exist with a path of <nfs_directory>/<volume_name> on the storage host.

[OSEv3:vars]

openshift_logging_storage_kind=nfs
openshift_logging_storage_access_modes=['ReadWriteOnce']
openshift_logging_storage_host=nfs.example.com
openshift_logging_storage_nfs_directory=/exports
openshift_logging_storage_volume_name=logging
openshift_logging_storage_volume_size=10Gi

The remote volume path using the following options would be nfs.example.com:/exports/logging.

Upgrading or Installing OpenShift Origin with NFS

The use of NFS for the core OpenShift Origin components is not recommended, as NFS (and the NFS Protocol) does not provide the proper consistency needed for the applications that make up the OpenShift Origin infrastructure.

As a result, the installer and update playbooks require an option to enable the use of NFS with core infrastructure components.

# Enable unsupported configurations, things that will yield a partially
# functioning cluster but would not be supported for production use
#openshift_enable_unsupported_configurations=false

If you see the following messages when upgrading or installing your cluster, then an additional step is required.

TASK [Run variable sanity checks] **********************************************
fatal: [host.example.com]: FAILED! => {"failed": true, "msg": "last_checked_host: host.example.com, last_checked_var: openshift_hosted_registry_storage_kind;nfs is an unsupported type for openshift_hosted_registry_storage_kind. openshift_enable_unsupported_configurations=True mustbe specified to continue with this configuration."}

In your Ansible inventory file, specify the following parameter:

[OSEv3:vars]
openshift_enable_unsupported_configurations=True

Configuring the Service Catalog

The service catalog is enabled by default during installation. Enabling the service broker allows service brokers to be registered with the catalog.

To disable automatic deployment, set the following cluster variables in your inventory file:

openshift_enable_service_catalog=false

When the service catalog is enabled, the OpenShift Ansible broker and template service broker are both enabled as well; see Configuring the OpenShift Ansible Broker and Configuring the Template Service Broker for more information.

Configuring the OpenShift Ansible Broker

The OpenShift Ansible broker (OAB) is enabled by default during installation. However, further configuration may be required for use.

Table 10. Service broker customization variables
Variable Purpose

openshift_service_catalog_image_prefix

Specify the prefix for the service catalog component image.

Configuring Persistent Storage for the OpenShift Ansible Broker

The OAB deploys its own etcd instance separate from the etcd used by the rest of the OpenShift Origin cluster. The OAB’s etcd instance requires separate storage using persistent volumes (PVs) to function. If no PV is available, etcd will wait until the PV can be satisfied. The OAB application will enter a CrashLoop state until its etcd instance is available.

Some Ansible playbook bundles (APBs) also require a PV for their own usage in order to deploy. For example, each of the database APBs have two plans: the Development plan uses ephemeral storage and does not require a PV, while the Production plan is persisted and does require a PV.

APB PV Required?

postgresql-apb

Yes, but only for the Production plan

mysql-apb

Yes, but only for the Production plan

mariadb-apb

Yes, but only for the Production plan

mediawiki-apb

Yes

To configure persistent storage for the OAB:

The following example shows usage of an NFS host to provide the required PVs, but other persistent storage providers can be used instead.

  1. In your inventory file, add nfs to the [OSEv3:children] section to enable the [nfs] group:

    [OSEv3:children]
    masters
    nodes
    nfs
  2. Add a [nfs] group section and add the host name for the system that will be the NFS host:

    [nfs]
    master1.example.com
  3. Add the following in the [OSEv3:vars] section:

    openshift_hosted_etcd_storage_kind=nfs
    openshift_hosted_etcd_storage_nfs_options="*(rw,root_squash,sync,no_wdelay)"
    openshift_hosted_etcd_storage_nfs_directory=/opt/osev3-etcd (1)
    openshift_hosted_etcd_storage_volume_name=etcd-vol2 (1)
    openshift_hosted_etcd_storage_access_modes=["ReadWriteOnce"]
    openshift_hosted_etcd_storage_volume_size=1G
    openshift_hosted_etcd_storage_labels={'storage': 'etcd'}
    
    ansible_service_broker_registry_url=registry.access.redhat.com
    ansible_service_broker_registry_user=<user_name> (2)
    ansible_service_broker_registry_password=<password> (2)
    ansible_service_broker_registry_organization=<organization> (2)
    1 An NFS volume will be created with path <nfs_directory>/<volume_name> on the host within the [nfs] group. For example, the volume path using these options would be /opt/osev3-etcd/etcd-vol2.
    2 Only required if ansible_service_broker_registry_url is set to a registry that requires authentication for pulling APBs.

    These settings create a persistent volume that is attached to the OAB’s etcd instance during cluster installation.

Configuring the OpenShift Ansible Broker for Local APB Development

In order to do APB development with the OpenShift Container Registry in conjunction with the OAB, a whitelist of images the OAB can access must be defined. If a whitelist is not defined, the broker will ignore APBs and users will not see any APBs available.

By default, the whitelist is empty so that a user cannot add APB images to the broker without a cluster administrator configuring the broker. To whitelist all images that end in -apb:

  1. In your inventory file, add the following to the [OSEv3:vars] section:

    ansible_service_broker_local_registry_whitelist=['.*-apb$']

Configuring the Template Service Broker

The template service broker (TSB) is enabled by default during installation.

To configure the TSB, one or more projects must be defined as the broker’s source namespace(s) for loading templates and image streams into the service catalog. Set the desired projects by modifying the following in your inventory file’s [OSEv3:vars] section:

openshift_template_service_broker_namespaces=['openshift','myproject']
Table 11. Template service broker customization variables
Variable Purpose

template_service_broker_prefix

Specify the prefix for the template service broker component image.

ansible_service_broker_image_prefix

Specify the prefix for the ansible service broker component image.

Configuring Web Console Customization

The following Ansible variables set master configuration options for customizing the web console. See Customizing the Web Console for more details on these customization options.

Table 12. Web Console Customization Variables
Variable Purpose

openshift_web_console_install

Determines whether to install the web console. Can be set to true or false. Defaults to true.

openshift_web_console_prefix

Specify the prefix for the web console images.

openshift_master_logout_url

Sets clusterInfo.logoutPublicURL in the web console configuration. See Changing the Logout URL for details. Example value: https://example.com/logout

openshift_web_console_extension_script_urls

Sets extensions.scriptURLs in the web console configuration. See Loading Extension Scripts and Stylesheets for details. Example value: ['https://example.com/scripts/menu-customization.js','https://example.com/scripts/nav-customization.js']

openshift_web_console_extension_stylesheet_urls

Sets extensions.stylesheetURLs in the web console configuration. See Loading Extension Scripts and Stylesheets for details. Example value: ['https://example.com/styles/logo.css','https://example.com/styles/custom-styles.css']

openshift_master_oauth_template

Sets the OAuth template in the master configuration. See Customizing the Login Page for details. Example value: ['/path/to/login-template.html']

openshift_master_metrics_public_url

Sets metricsPublicURL in the master configuration. See Setting the Metrics Public URL for details. Example value: https://hawkular-metrics.example.com/hawkular/metrics

openshift_master_logging_public_url

Sets loggingPublicURL in the master configuration. See Kibana for details. Example value: https://kibana.example.com

openshift_web_console_inactivity_timeout_minutes

Configurate the web console to log the user out automatically after a period of inactivity. Must be a whole number greater than or equal to 5, or 0 to disable the feature. Defaults to 0 (disabled).

openshift_web_console_cluster_resource_overrides_enabled

Boolean value indicating if the cluster is configured for overcommit. When true, the web console will hide fields for CPU request, CPU limit, and memory request when editing resource limits since these values should be set by the cluster resource override configuration.