Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 35 additions & 4 deletions DEVELOPMENT.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,24 @@
# Setup Development Environment

**Tools used in this guide**
* [kubectl](https://docs.aws.amazon.com/eks/latest/userguide/install-kubectl.html)
* [aws](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html#getting-started-install-instructions) - version 2 is recommended
* [eksctl](https://docs.aws.amazon.com/eks/latest/userguide/eksctl.html)
* [jq](https://stedolan.github.io/jq/)
* [envsubst](https://www.gnu.org/software/gettext/manual/html_node/envsubst-Invocation.html)
* [wget](https://www.gnu.org/software/wget/)
* [Go](https://go.dev/dl/) - version 1.17+

## 1. Clone the repo

```sh
git clone --branch v2 https://github.com/aws/aws-node-termination-handler.git nthv2
cd nthv2
git clone --branch v2 https://github.com/aws/aws-node-termination-handler.git "${GOPATH}/src/nthv2"
cd "${GOPATH}/src/nthv2"

# Display all targets and the descriptions.
make help

make test
```

## 2. Specify an EKS Cluster
Expand All @@ -26,7 +40,13 @@ envsubst <resources/eks-cluster.yaml.tmpl | eksctl create cluster --kubeconfig "
export KUBECONFIG="$PWD/kubeconfig"
```

If you do not want to use `envsubst` you can copy the template file and substitute the referenced values.
As an alternative to using `envsubst` you can copy the template file and substitute the referenced values.

### 2.2. Create an IAM OIDC provider

*Note:* If you created your cluster using `resource/eks-cluster.yaml.tmpl` in the previous step then your cluster will already have an IAM OIDC provider.

This [guide](https://docs.aws.amazon.com/eks/latest/userguide/enable-iam-roles-for-service-accounts.html) will walk you through determining whether you cluster already has an IAM OIDC provider, and how to create one if it does not already exist.

## 3. Create Infrastructure

Expand Down Expand Up @@ -59,6 +79,10 @@ Resources created:
* `WebhookRespository` - ECR Repository for images of the Kubernetes admission webhook.

```sh
# Note: The queue-infrastructure.yaml template generates the names of EventBridge rules
# from the ClusterName and QueueName parameters. To avoid exceeding name length limits
# the combined length of ClusterName and QueueName parameters should not exceed 51
# characters.
export QUEUE_NAME=<name>
export QUEUE_STACK_NAME="${INFRASTRUCTURE_STACK_NAME}-queue-${QUEUE_NAME}"

Expand Down Expand Up @@ -124,6 +148,13 @@ The `apply` target sets some Helm chart values for you based on environment vari
make HELM_OPTS='--set logging.level=debug' apply
```

### 6.2. (Optional) List all deployed resources

```sh
kubectl api-resources --verbs=list --namespaced -o name | \
xargs -n 1 kubectl get --show-kind --ignore-not-found --namespace "${CLUSTER_NAMESPACE}"
```

## 7. Define and deploy a Terminator to EKS cluster

```sh
Expand All @@ -135,7 +166,7 @@ envsubst <resources/terminator.yaml.tmpl >terminator-${TERMINATOR_NAME}.yaml
kubectl apply -f terminator-${TERMINATOR_NAME}.yaml
```

If you do not want to use `envsubst` you can copy the template file and substitute the referenced values.
As an alternative to using `envsubst` you can copy the template file and substitute the referenced values.

## 8. Remove deployed controller from EKS cluster

Expand Down
8 changes: 5 additions & 3 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ $(GINKGO):
GOBIN="$(BIN_DIR)" go install github.com/onsi/ginkgo/v2/[email protected]

$(KO):
GOBIN="$(BIN_DIR)" go install github.com/google/[email protected]
@./scripts/download-ko.sh "$(BIN_DIR)"

$(SETUP_ENVTEST):
GOBIN="$(BIN_DIR)" go install sigs.k8s.io/controller-runtime/tools/[email protected]
Expand Down Expand Up @@ -105,7 +105,9 @@ $(KODATA):

.PHONY: apply
apply: $(KO) $(KODATA) ## Deploy the controller into the current kubernetes cluster.
helm upgrade --install dev charts/aws-node-termination-handler-2 --namespace nthv2 --create-namespace \
helm upgrade --install dev charts/aws-node-termination-handler-2 \
--namespace ${CLUSTER_NAMESPACE} \
--create-namespace \
$(HELM_BASE_OPTS) \
$(HELM_OPTS) \
--set controller.image=$(shell $(KO) publish -B github.com/aws/aws-node-termination-handler/cmd/controller) \
Expand All @@ -114,4 +116,4 @@ apply: $(KO) $(KODATA) ## Deploy the controller into the current kubernetes clus

.PHONY: delete
delete: ## Delete controller from current kubernetes cluster.
helm uninstall dev --namespace nthv2
helm uninstall dev --namespace ${CLUSTER_NAMESPACE}
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,8 @@ spec:
listKind: TerminatorList
plural: terminators
singular: terminator
categories:
- all
scope: Namespaced
versions:
- name: v1alpha1
Expand Down
2 changes: 1 addition & 1 deletion resources/eks-cluster.yaml.tmpl
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ kind: ClusterConfig
metadata:
name: ${CLUSTER_NAME}
region: ${AWS_REGION}
version: "1.21"
version: "1.22"
tags:
karpenter.sh/discovery: ${CLUSTER_NAME}
managedNodeGroups:
Expand Down
10 changes: 5 additions & 5 deletions resources/queue-infrastructure.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ Resources:
AutoScalingTerminateRule:
Type: AWS::Events::Rule
Properties:
Name: !Sub "nth_${ClusterName}_${QueueName}_auto-scaling-terminate"
Name: !Sub "nth_${ClusterName}_${QueueName}_asgtrm"
Description: !Sub "Route instance-terminate lifecycle actions from EC2 Auto Scaling to SQS Queue, ${QueueName}."
EventPattern:
source:
Expand All @@ -53,7 +53,7 @@ Resources:
RebalanceRecommendationRule:
Type: AWS::Events::Rule
Properties:
Name: !Sub "nth_${ClusterName}_${QueueName}_rebalance-recommendation"
Name: !Sub "nth_${ClusterName}_${QueueName}_rebrec"
Description: !Sub "Route rebalance recommendations from EC2 to SQS Queue, ${QueueName}."
EventPattern:
source:
Expand All @@ -70,7 +70,7 @@ Resources:
ScheduledChangeRule:
Type: AWS::Events::Rule
Properties:
Name: !Sub "nth_${ClusterName}_${QueueName}_scheduled-change"
Name: !Sub "nth_${ClusterName}_${QueueName}_schchg"
Description: !Sub "Route scheduled change health events from AWS Health to SQS Queue, ${QueueName}."
EventPattern:
source:
Expand All @@ -92,7 +92,7 @@ Resources:
SpotInterruptionRule:
Type: AWS::Events::Rule
Properties:
Name: !Sub "nth_${ClusterName}_${QueueName}_spot-interruption"
Name: !Sub "nth_${ClusterName}_${QueueName}_sptint"
Description: !Sub "Route spot interruption notices from EC2 to SQS Queue, ${QueueName}."
EventPattern:
source:
Expand All @@ -109,7 +109,7 @@ Resources:
StateChangeRule:
Type: AWS::Events::Rule
Properties:
Name: !Sub "nth_${ClusterName}_${QueueName}_state-change"
Name: !Sub "nth_${ClusterName}_${QueueName}_sttchg"
Description: !Sub "Route state change notifications from EC2 to SQS Queue, ${QueueName}."
EventPattern:
source:
Expand Down
1 change: 1 addition & 0 deletions resources/terminator.yaml.tmpl
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ apiVersion: node.k8s.aws/v1alpha1
kind: Terminator
metadata:
name: ${TERMINATOR_NAME}
namespace: ${CLUSTER_NAMESPACE}

spec:
sqs:
Expand Down
58 changes: 58 additions & 0 deletions scripts/download-ko.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
#!/usr/bin/env bash

set -euo pipefail

usage=$(cat << EOM
usage: $(basename "$0") -h | DIR_PATH

Download the ko binary to DIR_PATH.

Arguments:
-h Print usage message then exit.

EOM
)

while getopts "h" opt; do
case $opt in
h ) echo "${usage}"
exit 0
;;
\? ) echo "${usage}" 1>&2
exit 1
;;
esac
done

dir_path="$1"

if [[ -z "${dir_path}" ]]; then
echo "error: missing directory path" 1>&2
echo 1>&2
echo "${usage}" 1>&2
exit 1
fi

if ! which wget >/dev/null ; then
echo "error: wget not installed" 1>&2
exit 1
fi

version="0.11.2"
os="$(go env GOHOSTOS)"
arch="$(go env GOHOSTARCH)"

if [[ "${arch}" == "amd64" ]]; then
arch="x86_64"
elif [[ "${arch}" == "arm" ]]; then
arch="arm64"
elif [[ "${arch}" == "386" ]]; then
arch="i386"
fi

echo "Downloading github.com/google/ko@v${version} ..."

mkdir -p "${dir_path}"
cd "${dir_path}"
wget https://github.com/google/ko/releases/download/v${version}/ko_${version}_${os}_${arch}.tar.gz -O - | \
tar xzf - ko