All about AWS Direct Connect (DX)

AWS Advanced Networking Series — Part 03

The Background

In my last two blog articles on AWS advanced networking concepts, I tried to explain two main aspects in AWS hybrid connectivity.

Article 1: AWS Site-to-Site VPN — https://crishantha.medium.com/aws-site-to-site-vpn-c4baf45703fd

Article 2: Transitive Peering with Transitive Gateway — https://crishantha.medium.com/transitive-peering-between-networks-using-aws-transit-gateway-4d3550bea2dc

As the third step of the series, I will be focusing on the AWS Direct Connect (DX) connections.

So. Lets dive in!

AWS Direct Connect (DX) Connection

As we all know, AWS DX connection is a dedicated network connection between your client data centers and your AWS region.

It provides a consistent performance and reduced bandwidth costs compared to the well known Site-to-Site VPN connectivity which is wired through the public Internet.

In order to create a DX connection, we need to first setup the physical connectivity.

AWS Direct Connect — Physical Connection

You will find three basic physical components when you create an AWS DX connection (See Figure 01).

1. AWS Region / Network that you want to connect your data center
2. The customer data center (on-premise network)
3. Direct Connect location (closer to your customer data center)

Figure 01 — Direct Connect (DX) Connection Physical Components

Once these physical components are setup, you are required to follow two basic steps in order to fulfill a single physical connection.

1. Creating a “port” at the Direct Connect (DX) location
2. Creating a “Cross Connect” — Connecting DX location AWS Router and the DX location Customer Gateway.

In theory, Direct Connect (DX) connection is just a “port” operating at a certain speed, which belongs to a certain AWS account. By default, the requested “port”consists of a 1 Gbps (1000-Base-LX) or a 10 Gbps (10GBASE-LR) speed connection.

These connections are single mode fiber optic cables (Cross Connect Cables) allocated to you from the DX location port, which will be further extended to your customer data centers for a complete connection.

Getting this single fiber optic cable can be done in two ways.

1. Using your own customer router via a main telco — More suitable for a larger implementation.
2. Using a partner router via a partner — More suited for a small or medium implementations. These partners are basically partnered with main telcos.

Cross Connect Cable — The cable goes from DX / AWS Router to your Customer Router / Partner Router is called the Cross Connect Cable. These are fiber optic cables which are not Highly Available (HA) and not Encrypted.

See figure 02 for more expanded DX connection physical connectivity options.

Figure 02 — Direct Connect Physical Connectivity (Expanded)

Creating a Direct Connect (DX) Connection

The DX connection can be created in two options (connection ordering types) — See Figure 03.

1. Classic — Can create connections one at a time
2. Connection Wizard / Resiliency Toolkit — Can create connections using resiliency recommendations. (Recommended)

Out of these two, it is recommended to follow the connection wizard / resiliency toolkit, since it gives you three options for resiliency.

Figure 03 — Connection ordering types

Once the connection ordering type is selected, you can proceed with the DX connection creation.

The Classic Approach

Lets assume we selected the “classic” approach here (See Figure 04).

Figure 04 — Creating a connection

While creating the connection, you are required to select a “closer” location as the Direct Connect Location. In the above figure, I have selected “STT, GDC INDIA PVT, LTD in Chennai. You may adjust your location depending on your customer data center location.

Once the “location” is selected, you will be able to connect to a port within 72 hours. If you have the connectivity for the port defined, you can extend that connection to your customer data center without any issue. If the selected location is not able to find a connection to your customer data center, AWS partners can help you on this process.

Selecting this partners can be done while creating the connection itself (See Figure 04). You can see a check box checked by default. You may select the “Service Provider” drop down to choose your preferred partner. In this example, I have chosen “Tata Communication” as the partner . It is your decision to select a partner of your choice.

The Resiliency Toolkit Approach

Launched in 2019. With this approach, you can have multiple resiliency approaches. Depending on your resiliency requirement, you may chose the best available option (See Figure 05).

1. Development and Test
2. High Resiliency
3. Maximum Resiliency

Figure 05 — Resiliency Levels

Development and Test Resiliency

This is used for non-critical workloads or development workloads. Provides resiliency by having separate devices in a single location (See Figure 06).

This provides resiliency against device failures but does not provide resiliency against location failures.

Figure 06 — Device level resiliency

High Resiliency

This is used for critical workloads. Provides resiliency by having two separate connections in multiple locations (See Figure 07).

Therefore, it provides resiliency at the device and the location levels.

Figure 07 — Device and Location level resiliency

Maximum Resiliency

This is used for critical workloads. Provides resiliency by having two separate connections in multiple locations, where each location have multiple devices providing maximum level of resiliency (See Figure 08).

Figure 08 — Device level and location level maximum resiliency

AWS Direct Connect (DX) — Logical Connection

Once the physical connectivity is established, we are now ready to implement the logical connectivity.

The logical connectivity primarily implemented through Virtual Interfaces (VIFs).

Virtual Interfaces (VIFs)

Figure 09 — DX Virtual Interfaces (VIFs)

There are two types of VIFs (See Figure 09).

1. Public VIF — This lets you connect to AWS services, which are in the public space such as S3, DynamoDB, etc
2. Private VIF — This lets you connect to AWS VPCs. Each private VIF connects to a separate VPC in AWS. For example, if there are two VPCs, there will be two private VIFs.

There needs to be some routing configurations should happen between AWS Router and the Customer Gateway within the DX location for every VIF that you are planning to configure. Primarily it will be a VLAN connection with BGP routing.

AWS Direct Connect vs Site-to-Site VPN

1. Direct Connect (DX) connection takes more time to provision compared to the VPN
2. Much better bandwidth in DX compared to VPN. DX can provide you up to 40Gbps (4*10Gbps ports), where VPN is limited to 1.25Gbps.
3. DX uses the private fiber physical connection where VPN uses the Internet. Hence, DX has low latency and a dedicated bandwidth unlike VPN.
4. DX connections are not “natively” encrypted, unlike VPN connections can be encrypted. However, you are able to encrypt the DX channel by encrypting a public VIF channel which runs on top of DX connection.

References

1. AWS Re:Invent 2019: Connectivity to AWS and Hybrid AWS Network Architectures — https://youtu.be/eqW6CPb58gs
2. Testing AWS Direct Connect Resiliency with Resiliency Toolkit — https://aws.amazon.com/blogs/networking-and-content-delivery/testing-aws-direct-connect-resiliency-with-resiliency-toolkit-failover-testing/