mirror of
https://github.com/openshift/openshift-docs.git
synced 2026-02-05 12:46:18 +01:00
430 lines
12 KiB
Plaintext
430 lines
12 KiB
Plaintext
:_mod-docs-content-type: ASSEMBLY
|
|
[id="cloud-experts-aws-load-balancer-operator"]
|
|
= Tutorial: AWS Load Balancer Operator on {product-title}
|
|
include::_attributes/attributes-openshift-dedicated.adoc[]
|
|
:context: cloud-experts-aws-load-balancer-operator
|
|
|
|
toc::[]
|
|
|
|
//Mobb content metadata
|
|
//Brought into ROSA product docs 2023-09-12
|
|
//---
|
|
//date: '2023-01-03T22:07:08.574151'
|
|
//title: AWS Load Balancer Operator On ROSA
|
|
//aliases: ['/docs/rosa/alb-sts']
|
|
//tags: ["AWS", "ROSA"]
|
|
//authors:
|
|
// - Shaozhen Ding
|
|
// - Paul Czarkowski
|
|
//---
|
|
|
|
include::snippets/mobb-support-statement.adoc[leveloffset=+1]
|
|
|
|
[TIP]
|
|
====
|
|
Load Balancers created by the AWS Load Balancer Operator cannot be used for xref:../networking/ingress_load_balancing/routes/nw-configuring-routes.adoc#route-configuration[OpenShift Routes], and should only be used for individual services or ingress resources that do not need the full layer 7 capabilities of an OpenShift Route.
|
|
====
|
|
|
|
The link:https://kubernetes-sigs.github.io/aws-load-balancer-controller/[AWS Load Balancer Controller] manages AWS Elastic Load Balancers for a {product-title} cluster. The controller provisions link:https://docs.aws.amazon.com/elasticloadbalancing/latest/application/introduction.html[AWS Application Load Balancers (ALB)] when you create Kubernetes Ingress resources and link:https://docs.aws.amazon.com/elasticloadbalancing/latest/network/introduction.html[AWS Network Load Balancers (NLB)] when implementing Kubernetes Service resources with a type of LoadBalancer.
|
|
|
|
Compared with the default AWS in-tree load balancer provider, this controller is developed with advanced annotations for both ALBs and NLBs. Some advanced use cases are:
|
|
|
|
* Using native Kubernetes Ingress objects with ALBs
|
|
* Integrate ALBs with the AWS Web Application Firewall (WAF) service
|
|
+
|
|
[NOTE]
|
|
====
|
|
WAFv1, WAF classic, is no longer supported. Use WAFv2.
|
|
====
|
|
+
|
|
* Specify custom NLB source IP ranges
|
|
* Specify custom NLB internal IP addresses
|
|
|
|
The link:https://github.com/openshift/aws-load-balancer-operator[AWS Load Balancer Operator] is used to used to install, manage and configure an instance of `aws-load-balancer-controller` in a {product-title} cluster.
|
|
|
|
[id="prerequisites_{context}"]
|
|
== Prerequisites
|
|
|
|
[NOTE]
|
|
====
|
|
AWS ALBs require a multi-AZ cluster, as well as three public subnets split across three AZs in the same VPC as the cluster. This makes ALBs unsuitable for many PrivateLink clusters. AWS NLBs do not have this restriction.
|
|
====
|
|
|
|
ifndef::openshift-rosa-hcp[]
|
|
* xref:../rosa_install_access_delete_clusters/rosa-sts-creating-a-cluster-quickly.adoc#rosa-sts-creating-a-cluster-quickly[A multi-AZ {product-title} cluster]
|
|
endif::openshift-rosa-hcp[]
|
|
ifdef::openshift-rosa-hcp[]
|
|
* xref:../rosa_hcp/rosa-hcp-sts-creating-a-cluster-quickly.adoc#rosa-hcp-sts-creating-a-cluster-quickly[A multi-AZ {product-title} cluster]
|
|
endif::openshift-rosa-hcp[]
|
|
* BYO VPC cluster
|
|
* AWS CLI
|
|
* OC CLI
|
|
|
|
[id="environment_{context}"]
|
|
=== Environment
|
|
|
|
* Prepare the environment variables:
|
|
+
|
|
[source,terminal]
|
|
----
|
|
$ export AWS_PAGER=""
|
|
$ export ROSA_CLUSTER_NAME=$(oc get infrastructure cluster -o=jsonpath="{.status.infrastructureName}" | sed 's/-[a-z0-9]\{5\}$//')
|
|
$ export REGION=$(oc get infrastructure cluster -o=jsonpath="{.status.platformStatus.aws.region}")
|
|
$ export OIDC_ENDPOINT=$(oc get authentication.config.openshift.io cluster -o jsonpath='{.spec.serviceAccountIssuer}' | sed 's|^https://||')
|
|
$ export AWS_ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)
|
|
$ export SCRATCH="/tmp/${ROSA_CLUSTER_NAME}/alb-operator"
|
|
$ mkdir -p ${SCRATCH}
|
|
$ echo "Cluster: ${ROSA_CLUSTER_NAME}, Region: ${REGION}, OIDC Endpoint: ${OIDC_ENDPOINT}, AWS Account ID: ${AWS_ACCOUNT_ID}"
|
|
----
|
|
|
|
[id="aws-vpc-subnets_{context}"]
|
|
=== AWS VPC and subnets
|
|
|
|
[NOTE]
|
|
====
|
|
This section only applies to clusters that were deployed into existing VPCs. If you did not deploy your cluster into an existing VPC, skip this section and proceed to the installation section below.
|
|
====
|
|
|
|
. Set the below variables to the proper values for your cluster deployment:
|
|
+
|
|
[source,terminal]
|
|
----
|
|
$ export VPC_ID=<vpc-id>
|
|
$ export PUBLIC_SUBNET_IDS=<public-subnets>
|
|
$ export PRIVATE_SUBNET_IDS=<private-subnets>
|
|
$ export CLUSTER_NAME=$(oc get infrastructure cluster -o=jsonpath="{.status.infrastructureName}")
|
|
----
|
|
+
|
|
. Add a tag to your cluster's VPC with the cluster name:
|
|
+
|
|
[source,terminal]
|
|
----
|
|
$ aws ec2 create-tags --resources ${VPC_ID} --tags Key=kubernetes.io/cluster/${CLUSTER_NAME},Value=owned --region ${REGION}
|
|
----
|
|
+
|
|
. Add a tag to your public subnets:
|
|
+
|
|
[source,terminal]
|
|
----
|
|
$ aws ec2 create-tags \
|
|
--resources ${PUBLIC_SUBNET_IDS} \
|
|
--tags Key=kubernetes.io/role/elb,Value='' \
|
|
--region ${REGION}
|
|
----
|
|
+
|
|
. Add a tag to your private subnets:
|
|
+
|
|
[source,terminal]
|
|
----
|
|
$ aws ec2 create-tags \
|
|
--resources "${PRIVATE_SUBNET_IDS}" \
|
|
--tags Key=kubernetes.io/role/internal-elb,Value='' \
|
|
--region ${REGION}
|
|
----
|
|
//subnets are tagged already after rosa create network
|
|
|
|
[id="installation_{context}"]
|
|
== Installation
|
|
|
|
. Create an AWS IAM policy for the AWS Load Balancer Controller:
|
|
+
|
|
[NOTE]
|
|
====
|
|
The policy is sourced from link:https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/v2.4.4/docs/install/iam_policy.json[the upstream AWS Load Balancer Controller policy] plus permission to create tags on subnets. This is required by the Operator to function.
|
|
====
|
|
+
|
|
[source,terminal]
|
|
----
|
|
$ oc new-project aws-load-balancer-operator
|
|
$ POLICY_ARN=$(aws iam list-policies --query \
|
|
"Policies[?PolicyName=='aws-load-balancer-operator-policy'].{ARN:Arn}" \
|
|
--output text)
|
|
$ if [[ -z "${POLICY_ARN}" ]]; then
|
|
wget -O "${SCRATCH}/load-balancer-operator-policy.json" \
|
|
https://raw.githubusercontent.com/rh-mobb/documentation/main/content/rosa/aws-load-balancer-operator/load-balancer-operator-policy.json
|
|
POLICY_ARN=$(aws --region "$REGION" --query Policy.Arn \
|
|
--output text iam create-policy \
|
|
--policy-name aws-load-balancer-operator-policy \
|
|
--policy-document "file://${SCRATCH}/load-balancer-operator-policy.json")
|
|
fi
|
|
$ echo $POLICY_ARN
|
|
----
|
|
+
|
|
. Create an AWS IAM trust policy for AWS Load Balancer Operator:
|
|
+
|
|
[source,terminal]
|
|
----
|
|
$ cat <<EOF > "${SCRATCH}/trust-policy.json"
|
|
{
|
|
"Version": "2012-10-17",
|
|
"Statement": [
|
|
{
|
|
"Effect": "Allow",
|
|
"Condition": {
|
|
"StringEquals" : {
|
|
"${OIDC_ENDPOINT}:sub": ["system:serviceaccount:aws-load-balancer-operator:aws-load-balancer-operator-controller-manager", "system:serviceaccount:aws-load-balancer-operator:aws-load-balancer-controller-cluster"]
|
|
}
|
|
},
|
|
"Principal": {
|
|
"Federated": "arn:aws:iam::$AWS_ACCOUNT_ID:oidc-provider/${OIDC_ENDPOINT}"
|
|
},
|
|
"Action": "sts:AssumeRoleWithWebIdentity"
|
|
}
|
|
]
|
|
}
|
|
EOF
|
|
----
|
|
+
|
|
. Create an AWS IAM role for the AWS Load Balancer Operator:
|
|
+
|
|
[source,terminal]
|
|
----
|
|
$ ROLE_ARN=$(aws iam create-role --role-name "${ROSA_CLUSTER_NAME}-alb-operator" \
|
|
--assume-role-policy-document "file://${SCRATCH}/trust-policy.json" \
|
|
--query Role.Arn --output text)
|
|
$ echo $ROLE_ARN
|
|
|
|
$ aws iam attach-role-policy --role-name "${ROSA_CLUSTER_NAME}-alb-operator" \
|
|
--policy-arn $POLICY_ARN
|
|
----
|
|
+
|
|
. Create a secret for the AWS Load Balancer Operator to assume our newly created AWS IAM role:
|
|
+
|
|
[source,terminal]
|
|
----
|
|
$ cat << EOF | oc apply -f -
|
|
apiVersion: v1
|
|
kind: Secret
|
|
metadata:
|
|
name: aws-load-balancer-operator
|
|
namespace: aws-load-balancer-operator
|
|
stringData:
|
|
credentials: |
|
|
[default]
|
|
role_arn = $ROLE_ARN
|
|
web_identity_token_file = /var/run/secrets/openshift/serviceaccount/token
|
|
EOF
|
|
----
|
|
+
|
|
. Install the AWS Load Balancer Operator:
|
|
+
|
|
[source,terminal]
|
|
----
|
|
$ cat << EOF | oc apply -f -
|
|
apiVersion: operators.coreos.com/v1
|
|
kind: OperatorGroup
|
|
metadata:
|
|
name: aws-load-balancer-operator
|
|
namespace: aws-load-balancer-operator
|
|
spec:
|
|
upgradeStrategy: Default
|
|
---
|
|
apiVersion: operators.coreos.com/v1alpha1
|
|
kind: Subscription
|
|
metadata:
|
|
name: aws-load-balancer-operator
|
|
namespace: aws-load-balancer-operator
|
|
spec:
|
|
channel: stable-v1.0
|
|
installPlanApproval: Automatic
|
|
name: aws-load-balancer-operator
|
|
source: redhat-operators
|
|
sourceNamespace: openshift-marketplace
|
|
startingCSV: aws-load-balancer-operator.v1.0.0
|
|
EOF
|
|
----
|
|
+
|
|
. Deploy an instance of the AWS Load Balancer Controller using the Operator:
|
|
+
|
|
[NOTE]
|
|
====
|
|
If you get an error here wait a minute and try again, it means the Operator has not completed installing yet.
|
|
====
|
|
+
|
|
[source,terminal]
|
|
----
|
|
$ cat << EOF | oc apply -f -
|
|
apiVersion: networking.olm.openshift.io/v1
|
|
kind: AWSLoadBalancerController
|
|
metadata:
|
|
name: cluster
|
|
spec:
|
|
credentials:
|
|
name: aws-load-balancer-operator
|
|
EOF
|
|
----
|
|
+
|
|
. Check the that the Operator and controller pods are both running:
|
|
+
|
|
[source,terminal]
|
|
----
|
|
$ oc -n aws-load-balancer-operator get pods
|
|
----
|
|
+
|
|
You should see the following, if not wait a moment and retry:
|
|
+
|
|
[source,terminal]
|
|
----
|
|
NAME READY STATUS RESTARTS AGE
|
|
aws-load-balancer-controller-cluster-6ddf658785-pdp5d 1/1 Running 0 99s
|
|
aws-load-balancer-operator-controller-manager-577d9ffcb9-w6zqn 2/2 Running 0 2m4s
|
|
----
|
|
|
|
[id="validating-the-deployment_{context}"]
|
|
== Validating the deployment
|
|
|
|
. Create a new project:
|
|
+
|
|
[source,terminal]
|
|
----
|
|
$ oc new-project hello-world
|
|
----
|
|
+
|
|
. Deploy a hello world application:
|
|
+
|
|
[source,terminal]
|
|
----
|
|
$ oc new-app -n hello-world --image=docker.io/openshift/hello-openshift
|
|
----
|
|
+
|
|
. Configure a NodePort service for the AWS ALB to connect to:
|
|
+
|
|
[source,terminal]
|
|
----
|
|
$ cat << EOF | oc apply -f -
|
|
apiVersion: v1
|
|
kind: Service
|
|
metadata:
|
|
name: hello-openshift-nodeport
|
|
namespace: hello-world
|
|
spec:
|
|
ports:
|
|
- port: 80
|
|
targetPort: 8080
|
|
protocol: TCP
|
|
type: NodePort
|
|
selector:
|
|
deployment: hello-openshift
|
|
EOF
|
|
----
|
|
+
|
|
. Deploy an AWS ALB using the AWS Load Balancer Operator:
|
|
+
|
|
[source,terminal]
|
|
----
|
|
$ cat << EOF | oc apply -f -
|
|
apiVersion: networking.k8s.io/v1
|
|
kind: Ingress
|
|
metadata:
|
|
name: hello-openshift-alb
|
|
namespace: hello-world
|
|
annotations:
|
|
alb.ingress.kubernetes.io/scheme: internet-facing
|
|
spec:
|
|
ingressClassName: alb
|
|
rules:
|
|
- http:
|
|
paths:
|
|
- path: /
|
|
pathType: Exact
|
|
backend:
|
|
service:
|
|
name: hello-openshift-nodeport
|
|
port:
|
|
number: 80
|
|
EOF
|
|
----
|
|
+
|
|
. Curl the AWS ALB Ingress endpoint to verify the hello world application is accessible:
|
|
+
|
|
[NOTE]
|
|
====
|
|
AWS ALB provisioning takes a few minutes. If you receive an error that says `curl: (6) Could not resolve host`, please wait and try again.
|
|
====
|
|
+
|
|
[source,termnial]
|
|
----
|
|
$ INGRESS=$(oc -n hello-world get ingress hello-openshift-alb \
|
|
-o jsonpath='{.status.loadBalancer.ingress[0].hostname}')
|
|
$ curl "http://${INGRESS}"
|
|
----
|
|
+
|
|
.Example output
|
|
[source,text]
|
|
----
|
|
Hello OpenShift!
|
|
----
|
|
|
|
. Deploy an AWS NLB for your hello world application:
|
|
+
|
|
[source,terminal]
|
|
----
|
|
$ cat << EOF | oc apply -f -
|
|
apiVersion: v1
|
|
kind: Service
|
|
metadata:
|
|
name: hello-openshift-nlb
|
|
namespace: hello-world
|
|
annotations:
|
|
service.beta.kubernetes.io/aws-load-balancer-type: external
|
|
service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: instance
|
|
service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
|
|
spec:
|
|
ports:
|
|
- port: 80
|
|
targetPort: 8080
|
|
protocol: TCP
|
|
type: LoadBalancer
|
|
selector:
|
|
deployment: hello-openshift
|
|
EOF
|
|
----
|
|
+
|
|
. Test the AWS NLB endpoint:
|
|
+
|
|
[NOTE]
|
|
====
|
|
NLB provisioning takes a few minutes. If you receive an error that says `curl: (6) Could not resolve host`, please wait and try again.
|
|
====
|
|
+
|
|
[source,terminal]
|
|
----
|
|
$ NLB=$(oc -n hello-world get service hello-openshift-nlb \
|
|
-o jsonpath='{.status.loadBalancer.ingress[0].hostname}')
|
|
$ curl "http://${NLB}"
|
|
----
|
|
+
|
|
.Example output
|
|
[source,text]
|
|
----
|
|
Hello OpenShift!
|
|
----
|
|
|
|
[id="cleaning-up_{context}"]
|
|
== Cleaning up
|
|
|
|
. Delete the hello world application namespace (and all the resources in the namespace):
|
|
+
|
|
[source,terminal]
|
|
----
|
|
$ oc delete project hello-world
|
|
----
|
|
+
|
|
. Delete the AWS Load Balancer Operator and the AWS IAM roles:
|
|
+
|
|
[source,terminal]
|
|
----
|
|
$ oc delete subscription aws-load-balancer-operator -n aws-load-balancer-operator
|
|
$ aws iam detach-role-policy \
|
|
--role-name "${ROSA_CLUSTER_NAME}-alb-operator" \
|
|
--policy-arn $POLICY_ARN
|
|
$ aws iam delete-role \
|
|
--role-name "${ROSA_CLUSTER_NAME}-alb-operator"
|
|
----
|
|
+
|
|
. Delete the AWS IAM policy:
|
|
+
|
|
[source,terminal]
|
|
----
|
|
$ aws iam delete-policy --policy-arn $POLICY_ARN
|
|
---- |