Handling Custom VPCs on AWS

As you aware, when you are dealing with the AWS cloud, we always tend to work with “default” Virtual Private Clouds (VPCs). This is fine for initial testings but for production setups. If you are planning to move to real production setups, you are required to build your VPCs in a more secyre manner. As AWS recommends, having a custom VPCs in a more controlled environment is essential.

However, creating a custom VPC is not straight forward and it needs some knowledge on networking and AWS. Though there are easy ways such as using the VPC Wizard, it is always better to learn it in hard way. That will give you a lot of confidence in all aspects of AWS VPC networking.

As a start, this blog (Part 01) will discuss the initial setup of creating a Custom VPC including the steps of creating two Subnets (public and private) and two EC2 instances within them (See the following diagram).

The initial Custom VPC Setup

Step 1: Create a Custom VPC

Move to the VPC dashboard → Your VPCs → Click “Create VPC”

[Use 10.0.0.0/16 as the CIDR block, Tenancy = Default, IPV6 = No IPV6 CIDR block]

In step 1, you will see the following being created along it.

1. Route Table (Main)
2. NACL
3. Security Group

That means we need to manually create the following for the newly created custom VPC.

1. Internet Gateway (IGW)
2. Subnets

Step 2: Create Subnets

Here we create two subnets 1) Public Subnet 2) Private Subnet

Creating the Public Subnet

Creating the Public Subnet

Fill the relevant fields as follows:

1. Name Tag: public-subnet
2. VPC: Select the custom VPC you created
3. Availability Zone: It is your preference. Here we use ap-southeast-1a (A Singapore AZ)
4. IPv4 CIDR block: 10.0.1.0/24
5. Make the public subnet public: select the create public subnet and Actions → Modify auto-assigned IP settings → tick the check box “Auto-assign IPv4”

Creating the Private Subnet

Creating the Private Subnet

Fill the relevant fields as follows:

1. Name Tag: private-subnet
2. VPC: Select the custom VPC you created
3. Availability Zone: It is your preference. Here we use ap-southeast-1b (A Singapore AZ)
4. IPv4 CIDR block: 10.0.2.0/24

Step 3: Create an Internet Gateway

Since the Step 1 does not create an Internet Gateway for you, it is required to create one now.

Select Internet Gateways → Create Internet Gateway → Give a name tag for the IGW. Lets call it custom-IGW.

Step 4: Attach the Custom VPC to Internet Gateway

Select the Internet Gateway (custom-IGW) → Actions → Attach a VPC.

Step 5: Create a Route Table for the public subnet

The main route table, which was created during Step 1, is the main route table for the custom VPC (custom-vpc). If we add the custom VPC Internet Gateway (custom-IGW) as a route out entry in the main route table, that means all the subnets which are associated to the main route table will be exposed as public subnets. Since that approach is not secure, it is essential to create a secondary route table to expose only the public subnets to the Internet. This step will do that task.

Click “Create Route Table” → Specify a Name Tag and select the custom VPC → Hit Create.

Step 6: Connect the Internet Gateway via Route Table (Secondary)

Editing the Routing Table (Secondary) — For public Subnets

Then select the created route table and select the “Routes” tab and add the following entry as a route.

Destination : 0.0.0.0/0

Target: Select Internet Gateway → Then select the IGW created for the custom VPC (custom-IGW)

Step 7: Associate the public subnet to route table (secondary)

Then you are required to associate the public subnet to the secondary route table.

Associating the Public Subnet to the Routing Table (Secondary)

Step 8: Create two EC2 instances on each subnet (public and private)

You may create two EC2 instances in both subnets.

Make sure to select the correct VPC and the subnet while creating each instance. For the public subnet EC2, make sure to have HTTP 80 port open at the Security Group level (create a new SG with HTTP 80). The private subnet does not require to open port 80 and can proceed with the default Security Group for the moment (We will change this later at certain times, but this is fine for the moment).

SSH the public instance

After creating both EC2 instances, you may try out the public instance by doing a SSH to it. If all ok, then do a sudo yum update to see it can download and update the repositories from the Internet. If it can then you are successful so far in the implementation.

SSH the private instance

Now, you may try to access the private instance from the public instance. Since private instance does not have a public instance, you cannot ssh from outside of the VPC. To do this, you can use the .pem file that you used to SSH to the public instance to SSH to the private instance. Though it is not recommended way, before we start learning about NAT Gateways this would be a good start to understand it in the hard way. For that you need to copy the .pem file to the public instance and try and SSH to the private instance.

However, since private instance security group still does not have the SSH port opened as an inbound port, you may have to create a new security group and have the following ports open.

1. SSH -> Port 22 -> 10.0.1.0/24
2. HTTP ->Port 80 ->10.0.1.0/24

That will allow you to SSH into the private instance. If you are successful doing a SSH to the private instance, you should try doing a sudo yum update within the instance. Now you would see that you are unable to do this task. That is understandable since the private subnet does not have a route table entry to the Internet Gateway. So how can we achieve this? Next step will explain it.

Step 9: Create a NAT instance

In order to get the Internet access to the private instance / subnet, you are required to create a NAT instance (an EC2 instance) within the public subnet. (See the image below)

The Custom VPC with the NAT instance

While creating the EC2 instance, make sure to select a NAT instance, which is selected from the community AMI(s) by searching “nat”.

Make sure you adhere to the following while doing the NAT instance creation.

1. Select the Custom VPC (custom-vpc)

2. Select the public subnet.

3. Select the same Security Group, that you created

Disable the Source Destination Check

This should be done after creating the instance. [Actions → Networking → Change Source Destination Check — Disable]

Source Destination Check: Each EC2 instance performs the Source Destination Check by default. This means that the instance must be the source or destination of any traffic it sends or receives. However, a NAT instance must be able to send and receive traffic when the source or destination is not itself. Therefore you must disable this check on the NAT instance.

Add an Route entry to the Custom Main Route Table

By adding a route entry (See image below) to the Main Route table (where private subnet is associated), it gives a way to connect to public subnet by targeting the NAT instance (Use 0.0.0.0/0 as the Destination and use the NAT Instance as the Target).

Adding the NAT Instance Routing Entry to the Main Route Table

That means, now this creates an indirect connection from private subnet instances to public subnet instances. Having connection to the public subnet means that you have Internet connectivity by default. So, NAT instance basically working as a dummy instance, which bridges the public and private instances in order to get connectivity to the Internet. However, the private subnet is not anyway exposed to the public by not connecting to the secondary Route Table.

Now if you SSH to the private instance and do a sudo yum update, you can see it working!.

Step 10: Create a NAT Gateway

Custom VPC with NAT Gateway

Though we created a NAT instance on Step 09, it is kind of an old approach, due to many reasons.

1. It is a bit of a long process
2. It is not scalable
3. There is no fail-over

Creating a NAT Gateway

To overcome this, AWS came up with the NAT Gateway solution. You just only have to set the Main Routing Table pointing to the NAT Gateway, just the way you followed on Step 09.

While creating the NAT Gateway, make sure you to do the following

1. Select the public subnet
2. An Elastic IP (If you do not have one, create one)
3. Change the Main Route Table target to NAT Gateway, that you created. (The destination is same as 0.0.0.0/0)

Editing the Main Route Table

If all fine, you should be able to access Internet from the private instance again.

Step 11: Bastion Hosts

Bastion hosts are instances that sit within your public subnet and are typically accessed using SSH (for Linux) or RDP (for Windows).

This basically solves the problems of private key handling, which we faced in the previous steps while creating the custom VPCs to access private subnets from the public subnets.

Please do refer my blog post for more on this subject: https://medium.com/@crishantha/handing-bastion-hosts-on-aws-via-ssh-agent-forwarding-f1d2d4e8622a