From cf6527239dbb2f606c1a935e93e055f8b859d3a1 Mon Sep 17 00:00:00 2001 From: Maysa Macedo Date: Wed, 21 Jun 2023 18:23:59 -0300 Subject: [PATCH] OpenStack: Add steps to enable dual-stack clusters The `MachinesSubnet` field has been reshaped as `controlPlanePort`, this commit updates the docs to ensure `controlPlanePort` is used. Also, this commit adds dual-stack documentation. --- docs/user/openstack/README.md | 2 +- docs/user/openstack/customization.md | 19 ++-- .../openstack/deploy_baremetal_workers.md | 14 ++- .../openstack/deploy_dual_stack_cluster.md | 94 +++++++++++++++++++ docs/user/openstack/external_load_balancer.md | 7 +- docs/user/openstack/provider_networks.md | 7 +- 6 files changed, 125 insertions(+), 18 deletions(-) create mode 100644 docs/user/openstack/deploy_dual_stack_cluster.md diff --git a/docs/user/openstack/README.md b/docs/user/openstack/README.md index c517189b52..3544cc8958 100644 --- a/docs/user/openstack/README.md +++ b/docs/user/openstack/README.md @@ -556,7 +556,7 @@ spec: #### Defining a MachineSet That Uses Multiple Networks -To define a MachineSet with multiple networks, the `primarySubnet` value in the `providerSpec` must be set to the OpenStack subnet that you want the Kubernetes endpoints of the nodes to be published on. For most use cases, this is the same subnet as the [machinesSubnet](./customization.md#cluster-scoped-properties) in the `install-config.yaml`. +To define a MachineSet with multiple networks, the `primarySubnet` value in the `providerSpec` must be set to the OpenStack subnet that you want the Kubernetes endpoints of the nodes to be published on. For most use cases, this is the same subnet(s) as the subnets listed in [controlPlanePort](./customization.md#cluster-scoped-properties) in the `install-config.yaml`. After you set the subnet, add all of the networks that you want to attach to your machines to the `Networks` list in `providerSpec`. You must also add the network that the primary subnet is part of to this list. diff --git a/docs/user/openstack/customization.md b/docs/user/openstack/customization.md index 4b92c4cb5b..b65ff44ea9 100644 --- a/docs/user/openstack/customization.md +++ b/docs/user/openstack/customization.md @@ -33,7 +33,7 @@ Beyond the [platform-agnostic `install-config.yaml` properties](../customization * `clusterOSImageProperties` (optional list of strings): a list of properties to be added to the installer-uploaded ClusterOSImage in Glance. The default is to not set any properties. `clusterOSImageProperties` is ignored when `clusterOSImage` points to an existing image in Glance. * `apiVIP` (optional string): An IP address on the machineNetwork that will be assigned to the API VIP. Be aware that the `10` and `11` of the machineNetwork will be taken by neutron dhcp by default, and wont be available. * `ingressVIP` (optional string): An IP address on the machineNetwork that will be assigned to the ingress VIP. Be aware that the `10` and `11` of the machineNetwork will be taken by neutron dhcp by default, and wont be available. -* `machinesSubnet` (optional string): the UUID of an OpenStack subnet to install the nodes of the cluster onto. For more information on how to install with a custom subnet, see the [custom subnets](#custom-subnets) section of the docs. +* `controlPlanePort` (optional object): the UUID and/or Name of an OpenStack Network and its Subnets where to install the nodes of the cluster onto. For more information on how to install with a custom subnet, see the [custom subnets](#custom-subnets) section of the docs. * `defaultMachinePlatform` (optional object): Default [OpenStack-specific machine pool properties](#machine-pools) which apply to [machine pools](../customization.md#machine-pools) that do not define their own OpenStack-specific properties. ## Machine pools @@ -161,18 +161,19 @@ platform: ## Custom Subnets -In the `install-config.yaml` file, the value of the `machinesSubnet` property is the subnet where the Kubernetes endpoints of the nodes in your cluster are published. The Ingress and API ports are created on this subnet, too. By default, the installer creates a network and subnet for these endpoints and ports. Alternatively, you can use a subnet of your own by setting the value of the `machinesSubnet` property to the UUID of an existing OpenStack subnet. To use this feature, you need to meet these requirements: +In the `install-config.yaml` file, the value of the `controlPlanePort` property contains the Name and/or UUID of the network and subnet(s) where the Kubernetes endpoints of the nodes in your cluster are published. The Ingress and API ports are created on the subnets, too. By default, the installer creates a network and subnet for these endpoints and ports. Alternatively, you can use a OpenStack network containing one subnet or two, in case of dual-stack, of your own by specifying their and/or in the `controlPlanePort` property. To use this feature, you need to meet these requirements: -* The subnet that is used by `machinesSubnet` has DHCP enabled. -* The CIDR of `machinesSubnet` matches the CIDR of `networks.machineNetwork`. -* The installer user must have permission to create ports on this network, including ports with fixed IP addresses. +* Any subnet used by `controlPlanePort` have DHCP enabled. +* The CIDR of any subnet listed in `controlPlanePort.fixedIPs` matches the CIDRs listed on `networks.machineNetwork`. +* When using dual-stack Network the api and ingress Ports needs to be pre-created by the user. Also, the installer user must have permission to add tags and security groups to those pre-created Ports. The value of the fixed IPs of the Ports needs to be specified at the `apiVIPs` and `ingressVIPs` options in the `install-config.yaml`. +* If not using dual-stack, the installer user must have permission to create ports on this network, including ports with fixed IP addresses. You should also be aware of the following limitations: -* If you plan to install a cluster that uses floating IPs, the `machinesSubnet` must be attached to a router that is connected to the `externalNetwork`. -* The installer will not create a private network or subnet for your OpenShift machines if the `machinesSubnet` is set in the `install-config.yaml`. -* By default, the API and Ingress VIPs use the .5 and .7 of your network CIDR. To prevent other services from taking the ports that are assigned to the API and Ingress VIPs, set the `apiVIP` and `ingressVIP` options in the `install-config.yaml` to addresses that are outside of the DHCP allocation pool. -* You cannot use the `externalDNS` property at the same time as a custom `machinesSubnet`. If you want to add a DNS to your cluster while using a custom subnet, [add it to the subnet in OpenStack](https://docs.openstack.org/neutron/rocky/admin/config-dns-res.html). +* If you plan to install a cluster that uses floating IPs, the `controlPlanePort` must be attached to a router that is connected to the `externalNetwork`. +* The installer will not create a private network or subnet for your OpenShift machines if the `controlPlanePort` is set in the `install-config.yaml`. +* By default when not using dual-stack, the API and Ingress VIPs use the .5 and .7 of your network CIDR. To prevent other services from taking the ports that are assigned to the API and Ingress VIPs, set the `apiVIP` and `ingressVIP` options in the `install-config.yaml` to addresses that are outside of the DHCP allocation pool. +* You cannot use the `externalDNS` property at the same time as a custom `controlPlanePort`. If you want to add a DNS to your cluster while using a custom subnet, [add it to the subnet in OpenStack](https://docs.openstack.org/neutron/rocky/admin/config-dns-res.html). ## Additional Networks diff --git a/docs/user/openstack/deploy_baremetal_workers.md b/docs/user/openstack/deploy_baremetal_workers.md index 72b492885a..afafe6bef9 100644 --- a/docs/user/openstack/deploy_baremetal_workers.md +++ b/docs/user/openstack/deploy_baremetal_workers.md @@ -94,7 +94,7 @@ a preexisting network. - Set `controlPlane.platform.openstack.type` to the VM flavor which will be used by the control plane nodes. - - Set `platform.openstack.machinesSubnet` to the UUID of the pre-provisioned subnet. + - Set `platform.openstack.controlPlanePort.fixedIPs.subnet.id` to the UUID of the pre-provisioned subnet and/or `platform.openstack.controlPlanePort.fixedIPs.subnet.name` to the name of pre-provisioned subnet. For example: @@ -118,7 +118,10 @@ a preexisting network. platform: openstack: - machinesSubnet: + controlPlanePort: + fixedIPs: + - subnet: + id: - Run the openshift installer: @@ -143,7 +146,7 @@ Plane and Compute nodes. The cluster is deployed to a preexisting network. - Set `controlPlane.platform.openstack.type` to a bare-metal server flavor. - - Set `platform.openstack.machinesSubnet` to the UUID of the pre-provisioned subnet. + - Set `platform.openstack.controlPlanePort.fixedIPs.subnet.id` to the UUID of the pre-provisioned subnet and/or `platform.openstack.controlPlanePort.fixedIPs.subnet.name` to the name of pre-provisioned subnet. For example: @@ -167,7 +170,10 @@ Plane and Compute nodes. The cluster is deployed to a preexisting network. platform: openstack: - machinesSubnet: + controlPlanePort: + fixedIPs: + - subnet: + id: - Run the openshift installer: diff --git a/docs/user/openstack/deploy_dual_stack_cluster.md b/docs/user/openstack/deploy_dual_stack_cluster.md new file mode 100644 index 0000000000..f152c4466c --- /dev/null +++ b/docs/user/openstack/deploy_dual_stack_cluster.md @@ -0,0 +1,94 @@ +# Creating a dual-stack cluster on OpenStack + +## Table of Contents + +- [Prerequisites](#prerequisites) +- [Creating DualStack Networks for the cluster](#creating-dualstack-networks-for-the-cluster) +- [Creating DualStack API and Ingress VIPs Ports for the cluster](#creating-dualstack-api-and-ingress-vips-for-the-cluster) +- [Deploy OpenShift](#deploy-openshift) + +## Prerequisites + +* Installation with dual-stack is only allowed when using one OpenStack network with one IPv4 and IPv6 subnet. +* API and Ingress VIPs ports needs to pre-created by the user and the addresses specified in the `install-config.yaml`. +* Add the IPv6 Subnet to a neutron router to provide router advertisements. +* The dualstack network MTU must accomodate the minimun MTU for IPv6, which is 1280, and OVN-Kubernetes encapsulation overhead, which is 100. + +Additional prerequisites are listed at the [OpenStack Platform Customization docs](./customization.md) + +**Note**: Converting a IPv4 single-stack cluster to a dual-network cluster network is not supported by OpenStack. + +## Creating Dual-Stack Networks for the cluster + +You must create one network and add one IPv4 subnet and another IPv6 subnet with either slaac, stateless or stateful modes. Also, +you must add the IPv6 subnet to a router. Here is an example: + +```sh +$ openstack network create --project --share --external --provider-physical-network --provider-network-type flat dualstack +$ openstack subnet create --project subnet-v4 --subnet-range 192.168.25.0/24 --dhcp --dns-nameserver --network dualstack +$ openstack subnet create --project subnet-v6 --subnet-range fd2e:6f44:5dd8:c956::/64 --dhcp --network dualstack --ip-version 6 --ipv6-ra-mode slaac --ipv6-address-mode slaac +$ openstack router add subnet subnet-v6 +``` + +Note the example above creates a provider network, but a creation of a tenant network is also supported, which must be connected to a router for external connectivity. + +## Creating Dual-Stack API and Ingress VIPs Ports for the cluster + +You must create the API and Ingress VIPs Ports with the following commands: + +```sh +$ openstack port create api --network dualstack +$ openstack port create ingress --network dualstack +``` + +## Deploy OpenShift + +Now that the Networking resources are pre-created you can deploy OpenShift. Here is an example of `install-config.yaml`: + +```yaml +apiVersion: v1 +baseDomain: mydomain.test +featureSet: TechPreviewNoUpgrade +compute: +- name: worker + platform: + openstack: + type: m1.xlarge + replicas: 3 +controlPlane: + name: master + platform: + openstack: + type: m1.xlarge + replicas: 3 +metadata: + name: mycluster +networking: + machineNetwork: + - cidr: "192.168.25.0/24" + - cidr: "fd2e:6f44:5dd8:c956::/64" + clusterNetwork: + - cidr: 10.128.0.0/14 + hostPrefix: 23 + - cidr: fd01::/48 + hostPrefix: 64 + serviceNetwork: + - 172.30.0.0/16 + - fd02::/112 +platform: + openstack: + ingressVIPs: ['192.168.25.79', 'fd2e:6f44:5dd8:c956:f816:3eff:fef1:1bad'] + apiVIPs: ['192.168.25.199', 'fd2e:6f44:5dd8:c956:f816:3eff:fe78:cf36'] + controlPlanePort: + fixedIPs: + - subnet: + name: subnet-v4 + - subnet: + name: subnet-v6 + network: + name: dualstack + +``` +There are important things to note: + +The subnets under `platform.openstack.controlPlanePort.fixedIPs` can contain both id or name. The same applies to the network `platform.openstack.controlPlanePort.network`. Dual-stack clusters are only supported with `featureSet: TechPreviewNoUpgrade` \ No newline at end of file diff --git a/docs/user/openstack/external_load_balancer.md b/docs/user/openstack/external_load_balancer.md index 0d595e5e3a..56f7a8117d 100644 --- a/docs/user/openstack/external_load_balancer.md +++ b/docs/user/openstack/external_load_balancer.md @@ -68,7 +68,10 @@ networking: platform: openstack: cloud: mycloud - machinesSubnet: 8586bf1a-cc3c-4d40-bdf6-c243decc603a + controlPlanePort: + fixedIPs: + - subnet: + id: 8586bf1a-cc3c-4d40-bdf6-c243decc603a apiVIPs: - 192.168.10.5 ingressVIPs: @@ -80,7 +83,7 @@ platform: There are some important things to note here: * `loadBalancer` is a new stanza created in OCP 4.13. The default type is `OpenShiftManagedDefault` (which will deploy HAproxy and Keepalived in OCP, known as the OpenShift managed load balancer). Setting it to `UserManaged` will allow a user managed load balancer to replace the OpenShift managed one. -* `machinesSubnet` is the subnet ID where both the OpenShift cluster and the user managed load balancer are deployed. +* `platform.openstack.controlPlanePort.fixedIPs.subnet.id` is the subnet ID where both the OpenShift cluster and the user managed load balancer are deployed. * In OCP 4.13 the feature had to be enabled as Technology Preview. This can be done by adding featureSet: `TechPreviewNoUpgrade` into the install-config.yaml. diff --git a/docs/user/openstack/provider_networks.md b/docs/user/openstack/provider_networks.md index 7fc2c4e5a6..53d4081cf1 100644 --- a/docs/user/openstack/provider_networks.md +++ b/docs/user/openstack/provider_networks.md @@ -78,7 +78,7 @@ on a tenant network: - Set `platform.openstack.apiVIP` to the IP address for the API VIP. - Set `platform.openstack.ingressVIP` to the IP address for the Ingress VIP. - - Set `platform.openstack.machinesSubnet` to the subnet ID of the provider network subnet. + - Set `platform.openstack.controlPlanePort.fixedIPs.subnet.id` to the subnet ID of the provider network subnet and/or `platform.openstack.controlPlanePort.fixedIPs.subnet.name` to the name of the provider network. - Set `networking.machineNetwork.cidr` to the CIDR of the provider network subnet. **Note:** @@ -93,7 +93,10 @@ address on the `networking.machineNetwork.cidr`. openstack: apiVIP: ingressVIP: - machinesSubnet: + controlPlanePort: + fixedIPs: + - subnet: + id: (...) networking: machineNetwork: