The Theory

To make an Amazon Elastic Kubernetes Service (EKS) cluster private and allow nodes to join through a node group, you need to follow a few steps. By default, EKS creates a public cluster, but you can configure it to make it private for enhanced security. Here’s an overview of the process:

  1. Create a VPC: Start by creating a Virtual Private Cloud (VPC) in your AWS account if you haven’t already. This VPC will be used to host your private EKS cluster.
  2. Create private subnets: Within the VPC, create one or more private subnets. These subnets will provide the network isolation required for a private cluster. Make sure the subnets have no direct internet access and that their route tables do not have an internet gateway attached.
  3. Create security groups: Create security groups to define the inbound and outbound traffic rules for your EKS cluster and nodes. These security groups should allow communication between the control plane and the worker nodes, as well as any other necessary network traffic.
  4. Create a NAT gateway: Since the private subnets don’t have direct internet access, you need to set up a Network Address Translation (NAT) gateway in a public subnet to enable outbound internet connectivity for resources in the private subnets.
  5. Configure VPC endpoints: Create VPC endpoints for EKS and EC2 to allow private communication between your EKS cluster control plane and the worker nodes. These endpoints will ensure that the control plane and nodes can communicate without requiring access to the public internet.
  6. Create a private EKS cluster: Now, create a private EKS cluster using the AWS Management Console, AWS CLI, or AWS SDKs. During the cluster creation, specify the private subnets, security groups, and VPC endpoints you created earlier. This will ensure that the cluster is deployed within the private subnets and can communicate with the nodes via the VPC endpoints.
  7. Create a node group: Once the cluster is created, you can proceed to create a node group. When creating the node group, specify the private subnets and security groups that you set up earlier. The node group will be deployed in the private subnets and join the private EKS cluster.

Following these steps will result in a private EKS cluster where the control plane and worker nodes communicate privately through the VPC endpoints. The private nature of the cluster enhances security by reducing exposure to the public internet.

Note that these steps provide a high-level overview of the process, and there may be additional considerations or customizations based on your specific requirements. For detailed instructions and the most up-to-date information, it’s recommended to refer to the official AWS EKS documentation.

How to do this in Terraform

To create a private Amazon EKS cluster and allow nodes to join through a node group using Terraform, you can follow the steps outlined below:

  1. Set up the necessary Terraform files: Create a new directory for your Terraform configuration and create the main.tf file inside it.
  2. Configure the AWS provider: In the main.tf file, configure the AWS provider to define your AWS access credentials and the desired region:
1
2
3

provider "aws" {
  region = "your_region"
}
  1. Create a VPC: Define a VPC resource to create the Virtual Private Cloud:
1
2
3

resource "aws_vpc" "my_vpc" {
  cidr_block = "10.0.0.0/16"
}
  1. Create private subnets: Define private subnets within the VPC to host your EKS cluster:
1
2
3
4
5

resource "aws_subnet" "private_subnet" {
  count = 2
  vpc_id     = aws_vpc.my_vpc.id
  cidr_block = "10.0.${count.index}.0/24"
}
  1. Create security groups: Define security groups to allow inbound and outbound traffic for the EKS cluster:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

resource "aws_security_group" "eks_cluster_sg" {
  vpc_id = aws_vpc.my_vpc.id

  # Define inbound and outbound rules as per your requirements
  # Example:
  ingress {
    from_port   = 22
    to_port     = 22
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
}
  1. Create a NAT gateway: Configure a NAT gateway to provide outbound internet access to resources in the private subnets:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

resource "aws_eip" "nat_eip" {
  vpc      = true
}

resource "aws_nat_gateway" "nat_gateway" {
  allocation_id = aws_eip.nat_eip.id
  subnet_id     = aws_subnet.private_subnet[0].id
}

# Create a route table entry for the NAT gateway
resource "aws_route" "private_subnet_nat_route" {
  route_table_id         = aws_subnet.private_subnet[0].route_table_id
  destination_cidr_block = "0.0.0.0/0"
  nat_gateway_id         = aws_nat_gateway.nat_gateway.id
}
  1. Configure VPC endpoints: Create VPC endpoints for EKS and EC2 to enable private communication:
1
2
3
4
5
6
7
8
9

resource "aws_vpc_endpoint" "eks_endpoint" {
  vpc_id       = aws_vpc.my_vpc.id
  service_name = "com.amazonaws.${var.region}.eks"
}

resource "aws_vpc_endpoint" "ec2_endpoint" {
  vpc_id       = aws_vpc.my_vpc.id
  service_name = "com.amazonaws.${var.region}.ec2"
}
  1. Create a private EKS cluster: Define the EKS cluster resource with the appropriate settings:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

resource "aws_eks_cluster" "my_eks_cluster" {
  name     = "my-cluster"
  role_arn = aws_iam_role.my_eks_role.arn

  vpc_config {
    subnet_ids          = aws_subnet.private_subnet[*].id
    security_group_ids  = [aws_security_group.eks_cluster_sg.id]
    endpoint_private_access = true
    endpoint_public_access  = false
  }

  depends_on = [
    aws_vpc_endpoint.eks_endpoint,
    aws_vpc_endpoint.ec2_endpoint
  ]
}
  1. Create a node group: Define the EKS node group resource to join the private EKS cluster:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

resource "aws_eks_node_group" "my_eks_nodegroup" {
  cluster_name    = aws_eks_cluster.my_eks_cluster.name
  node_group_name = "my-nodegroup"

  node_group_config {
    instance_type = "your_instance_type"
    desired_size  = 1
    min_size      = 1
    max_size      = 1
    subnet_ids    = aws_subnet.private_subnet[*].id
    ami_type      = "AL2_x86_64"
  }
}
  1. Apply the Terraform configuration: Initialize the Terraform working directory and apply the configuration:
1
2

terraform init
terraform apply

This configuration will create a private VPC, subnets, security groups, NAT gateway, VPC endpoints, EKS cluster, and a node group that joins the private cluster.

Make sure to customize the configuration according to your specific requirements, such as VPC CIDR blocks, security group rules, EKS cluster name, node group instance type, etc.

Note: This is a simplified example, and there may be additional resources or configuration options you need to consider based on your specific needs. It’s recommended to refer to the Terraform AWS provider documentation for detailed information on each resource and its attributes.

How to do this in CloudFormation

To create a private Amazon EKS cluster and allow nodes to join through a node group using AWS CloudFormation, you can use AWS CloudFormation templates to define the infrastructure as code. Here’s an outline of the steps to accomplish this:

  1. Create an AWS CloudFormation template: Create a new CloudFormation template in YAML or JSON format. This template will define the resources required for your private EKS cluster.
  2. Define the VPC and subnets: Specify the VPC and private subnets where your EKS cluster will reside:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

Resources:
  MyVPC:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock: 10.0.0.0/16

  PrivateSubnet1:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref MyVPC
      CidrBlock: 10.0.0.0/24

  PrivateSubnet2:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref MyVPC
      CidrBlock: 10.0.1.0/24
  1. Create security groups: Define the security groups to control inbound and outbound traffic for your EKS cluster:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

Resources:
  EKSSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupDescription: EKS security group
      VpcId: !Ref MyVPC
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: 22
          ToPort: 22
          CidrIp: 0.0.0.0/0
      SecurityGroupEgress:
        - IpProtocol: "-1"
          FromPort: 0
          ToPort: 0
          CidrIp: 0.0.0.0/0
  1. Create a NAT gateway: Set up a NAT gateway to enable outbound internet access for the private subnets:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

Resources:
  MyEIP:
    Type: AWS::EC2::EIP
    Properties:
      Domain: vpc

  MyNATGateway:
    Type: AWS::EC2::NatGateway
    Properties:
      AllocationId: !GetAtt MyEIP.AllocationId
      SubnetId: !Ref PrivateSubnet1

  PrivateSubnet1RouteTable:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref MyVPC

  PrivateSubnet1Route:
    Type: AWS::EC2::Route
    DependsOn: MyNATGateway
    Properties:
      RouteTableId: !Ref PrivateSubnet1RouteTable
      DestinationCidrBlock: 0.0.0.0/0
      NatGatewayId: !Ref MyNATGateway

  PrivateSubnet1Association:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      SubnetId: !Ref PrivateSubnet1
      RouteTableId: !Ref PrivateSubnet1RouteTable
  1. Configure VPC endpoints: Create VPC endpoints for EKS and EC2 to enable private communication:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

Resources:
  EKSEndpoint:
    Type: AWS::EC2::VPCEndpoint
    Properties:
      VpcId: !Ref MyVPC
      ServiceName: com.amazonaws.<region>.eks

  EC2Endpoint:
    Type: AWS::EC2::VPCEndpoint
    Properties:
      VpcId: !Ref MyVPC
      ServiceName: com.amazonaws.<region>.ec2
  1. Create a private EKS cluster: Define the EKS cluster resource with the appropriate settings:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

Resources:
  MyEKSCluster:
    Type: AWS::EKS::Cluster
    Properties:
      Name: my-cluster
      ResourcesVpcConfig:
        SecurityGroupIds:
          - !Ref EKSSecurityGroup
        SubnetIds:
          - !Ref PrivateSubnet1
          - !Ref PrivateSubnet2
      Version: "1.21"
      RoleArn: arn:aws:iam::123456789012:role/MyEKSClusterRole
      KubernetesNetworkConfig:
        ServiceIpv4Cidr: "10.100.0.0/16"
  1. Create a node group: Define the EKS node group resource to join the private EKS cluster:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

Resources:
  MyEKSNodeGroup:
    Type: AWS::EKS::Nodegroup
    Properties:
      ClusterName: !Ref MyEKSCluster
      NodegroupName: my-nodegroup
      ScalingConfig:
        DesiredSize: 1
        MinSize: 1
        MaxSize: 3
      Subnets:
        - !Ref PrivateSubnet1
        - !Ref PrivateSubnet2
      InstanceTypes:
        - t3.medium
      RemoteAccess:
        Ec2SshKey: my-key-pair
  1. Deploy the CloudFormation stack: Use the AWS Management Console, AWS CLI, or AWS SDKs to deploy the CloudFormation stack with your template.

Ensure that you customize the configuration based on your specific requirements, such as VPC CIDR blocks, security group rules, EKS cluster name, node group instance type, etc.

Please note that this is a simplified example, and additional considerations and customization may be required based on your specific needs. For more detailed information on each resource and its properties, consult the AWS CloudFormation documentation.