Set the default type EBS volume for machine pools:
- Controlplane, arbiter and worker pool default to gp3 volume.
- Edge pool default to gp2 volume.
The default decision is taken from existing code [0]. This commit just
makes the defaulting earlier.
This also loosens the validation to allow 0 for volume size and iops as
they are int type (i.e. not pointer).
Reference:
[0] fd5a518e49/pkg/asset/machines/worker.go (L102-L117)
GP3 volumes have the ability to configure throughput from 125 MiB/s to
2000 MiB/s. This allows the ability to set this at install time in the
install-config.
https://issues.redhat.com/browse/CORS-4212
GP3 volumes have the ability to configure throughput from 125 MiB/s to
2000 MiB/s. This allows the ability to set this at install time in the
install-config.
https://issues.redhat.com/browse/CORS-4212
Changes in the apimachinery and client-go packages broke our unit
tests after upgrade. client-go stopped serializing the empty
preferences: {} field for the kubeconfig. apimachinery, with
c58e197ee8
caused much more extensive breakage by changing the format with
which errors are returned, and our unit tests hard code the
expected error messages.
For the most part, I used claude to fix these issues, and here
is the summary it created:
Changes Made
Root Cause: The k8s apimachinery package (errors.go:93-96) now uses json.Marshal()
to format BadValue in error messages instead of Go's native format. Error Format Changes:
Arrays: []string{"a", "b"} → ["a","b"] (JSON format, no spaces)
Nil values: []string(nil) → null
Structs: aws.Subnet{ID:"x", Roles:...} → {"id":"x"} (JSON with lowercase keys)
Custom types: gcp.OSImage{Name:"x", Project:"y"} → {"name":"x","project":"y"}
pkg/types/aws/regions.go: Create a map of the region to the descriptive name
pkg/asset/installconfig/aws/platform.go: Add the description information to the survey.
pkg/types/aws:
** Remove the endpoints package from aws sdk v1.
** Remove the function to check for secret regions
pkg/asset/installconfig/aws:
** Add a function to check for secret regions.
** The function was moved here (and changed) because the types package
should not make api calls. As we migrated to aws sdk v2 the partitions are no
longer publicly available BUT we can access them through endpoints. To get the
partition we go through the ec2 api (make a proxy call) and get the endpoint
from that call then the endpoint will provide us with the partition.
This include static validations for the vpc.subnets field (no call to
AWS API is performed). The validation criteria can be found at jira
ticket [0] and enhancement proposal [1]. Summary of validation criteria:
- There are no duplicate subnet IDs.
- There can only be maximum 10 subnets assigned IngressControllerLB role.
- Either all subnets must have roles assigned or none at all.
- For a subnet that has defined roles
- Roles must be of supported types (i.e. from a set of defined roles)
- Roles must not be duplicate. This and * check naturally validates that a subnet can only have max 5 roles
- EdgeNode cannot be combined with any other roles
- ClusterNode, IngressControllerLB, ControlPlaneExternalLB (if cluster is external), and ControlPlaneInternalLB must be assigned to at least 1 subnet
- A subnet cannot have both role ControlPlaneExternalLB and ControlPlaneInternalLB *
- If the cluster is internal, ControlPlaneExternalLB must not be assigned to any subnets.
References:
[0] https://issues.redhat.com/browse/CORS-3869
[1] 30f44ee0cd/enhancements/installer/aws-lb-subnet-selection.md (installer-validation-rules)
The field platform.aws.subnets is deprecated but replaced by
platform.aws.vpc.subnets for more flexible configurations.
If the users use the old field (i.e. an existing install-config), the
install will convert that to the new one. However, only either one of
the field can be defined (i.e. exit error if both are specified).
Following proposal for selecting LB subnets, the field
platform.vpc.subnets will be introduced for more flexible
configurations. This enhancement proposal is available reference [0].
There are some adjustments to the API markers and descriptions in comparison to
the proposal.
- Organize field description for easier read.
- Correct kubebuilder:validation:MaxItems on array field
References:
[0] 2890cccf20/enhancements/installer/aws-lb-subnet-selection.md
Add the method to skip undesired zones when discoverying zones from
metadata.
This change targets explicity to skup the zone us-east-1e which provides
very limited offerings of EC2, with no EC2 supported by installer.
This change prevents creating infrastructure in such zone as it will
not be supported to run workloads (create instances) on it.
** AWS User tags support white space as valid characters.
** User tags should be validated every time that they are supplied not
only when propagateTags is True.
This field was introduced [1] before the Installer had support for
custom AMIs in machine pools [2]. Now that it does, the same
functionality is achieved via the defaultMachinePlatform field
`platform.aws.defaultMachinePlatform.amiID`
[1] fdf94e39ee
[2] bc47222576
The name doesn't really reflect the purpose of the field, but the
terraform implementation instead. Since the implementation has changed
in capi/capa, let's rename it so users don't expect the ignition object
to not be destroyed when there are enough permissions.
The old field is kept for the deprecation period but will be removed in
the future in favor of the new `BestEffortDeleteIgnition`.
** The go docs in the install-config's platform.aws.lbType is misleading as well as on the ingress object (oc explain ingresses.config.openshift.io.spec.loadBalancer.platform.aws.type).
Update the docs to further explain the ingress controller info.
AWS has a limit of 16 security groups. This limits the users to supply
10 security groups and reserves the additional 6 for OpenShift, which
leaves room for additional features such as using CAPI for installs.
AWS offers limited instance types in Wavelength Zones. It is required
to installer choose at least one supported (by RH, tested by QE) EC2 Type.
To do so, it is required to query AWS EC2 API to check offerings across
all Wavelength Zones in all regions.
The following data is insights about that:
2023-08-08 19:00:39.278646>> EC2 Offerings summary (total zones)
Total items: 4
t3.xlarge: 30
r5.2xlarge: 30
t3.medium: 30
g4dn.2xlarge: 30
2023-08-08 19:00:39.278646>> Zones by Family
g4dn.2x t3.medium t3.xlarge r5.2xlarge
zone
ap-northeast-1-wl1-kix-wlz-1 X X X X
ap-northeast-1-wl1-nrt-wlz-1 X X X X
ap-northeast-2-wl1-cjj-wlz-1 X X X X
ap-northeast-2-wl1-sel-wlz-1 X X X X
ca-central-1-wl1-yto-wlz-1 X X X X
eu-central-1-wl1-ber-wlz-1 X X X X
eu-central-1-wl1-dtm-wlz-1 X X X X
eu-central-1-wl1-muc-wlz-1 X X X X
eu-west-2-wl1-lon-wlz-1 X X X X
eu-west-2-wl1-man-wlz-1 X X X X
eu-west-2-wl2-man-wlz-1 X X X X
us-east-1-wl1-atl-wlz-1 X X X X
us-east-1-wl1-bna-wlz-1 X X X X
us-east-1-wl1-bos-wlz-1 X X X X
us-east-1-wl1-chi-wlz-1 X X X X
us-east-1-wl1-clt-wlz-1 X X X X
us-east-1-wl1-dfw-wlz-1 X X X X
us-east-1-wl1-dtw-wlz-1 X X X X
us-east-1-wl1-iah-wlz-1 X X X X
us-east-1-wl1-mia-wlz-1 X X X X
us-east-1-wl1-msp-wlz-1 X X X X
us-east-1-wl1-nyc-wlz-1 X X X X
us-east-1-wl1-tpa-wlz-1 X X X X
us-east-1-wl1-was-wlz-1 X X X X
us-west-2-wl1-den-wlz-1 X X X X
us-west-2-wl1-las-wlz-1 X X X X
us-west-2-wl1-lax-wlz-1 X X X X
us-west-2-wl1-phx-wlz-1 X X X X
us-west-2-wl1-sea-wlz-1 X X X X
us-west-2-wl1-sfo-wlz-1 X X X X
AWS Wavelength Zone is identified by wavelength-zone by API.
The same type is created internally to represent that zone
following the same convention of other zone types.
Making a copy of the AWS bootstrap S3 bucket in the cluster stage
and adding a field to the install config. If the user does not want
to delete the S3 bucket created for the bootstrap, they need to
set the new field to true.
This causes the S3 bucket to be created during the cluster stage
and not the boostrap stage which will then be ignored when the
bootstrap destroy code runs.
More details: https://issues.redhat.com/browse/CORS-2661
The AWS Zone provides a few attributes that is essential for the
edge compute pool when setting up AWS Local Zones.
The `GroupName` is used to create the labels that can be used
by user-workloads when deploying applications into specific locations.
The `Type` is used to aggregate edge and regular worker zones. Only
`local-zone` type is valid for edge compute pool. This field also
can be used on the user-workloads and it is exposed through Machine/node
label.
The `ParentZoneName` the zone in the region that the Local Zone is attached to.
It is used by installer to discover which VPC private route table the private
subnets on the Local Zones will be associated as t Local Zones does not
support Nat Gateways (and other network resources).
This change create the structure to store the zone attribute to installer
be used into different places (MachineSet creation, EC2 offerings, Gateway
map for tfvars, etc).
It also introduce functions used to discover and filter the zones in the
region.
According to
https://docs.aws.amazon.com/vpc/latest/userguide/amazon-vpc-limits.html,
the default Security groups number per network interface is 5 and could
be 16 at most, so we better have some pre-check on the number of
provided custom security groups.
When it's more than 15 (since the maximum is 16, but installer will also
create one (`${var.cluster_id}-master-sg/${var.cluster_id}-worker-sg`),
installer should quit and warn user about this.
