Creating EC2 Custom Metrics via CloudWatch Monitoring Scripts

EC2 Monitoring with CloudWatch

We all know EC2 Monitoring can be done in two levels in AWS.

1. Using AWS Provided Metrics (Basic and Detailed Monitoring)
2. Using Custom Metrics (CloudWatch Agents and CloudWatch Monitoring Scripts)

Out of these two,

AWS provided metrics will give us information related to CPU Utilization, CPU Credit Balance, Network IN/ OUT and Status Checks (Instance Status and System Status)

Custom Metrics primarily there to extract RAM details, Instance Swap details or any other custom metric.

EC2 Monitoring via Custom Metrics

As mentioned, there are two types of Custom Metrics in AWS

1. Custom Metrics via CloudWatch Agents (Recommended way)
2. Custom Metrics via CloudWatch Monitoring Scripts

EC2 Monitoring via CloudWatch Monitoring Scripts

In this article, we explain only the second option, which is Custom Metrics via CloudWatch Monitoring Scripts for EC2 instances.

Steps

Step 1: Create an Amazon Linux EC2 instance that you need to monitor.

Step 2: SSH into the EC2 instance and set the environment in order to execute EC2 CloudWatch Custom Scripts.

$ sudo yum update

$ sudo yum install -y perl-Switch perl-DateTime perl-Sys-Syslog perl-LWP-Protocol-https perl-Digest-SHA.x86_64

Step 3: Now download, install and configure monitoring scripts to a preferred folder in the EC2 instance.

$ curl https://aws-cloudwatch.s3.amazonaws.com/downloads/CloudWatchMonitoringScripts-1.2.2.zip -O

Step 4: Unzip the downloaded Zip file and move to the aws-scripts-mon folder

$ unzip CloudWatchMonitoringScripts-1.2.2.zip

$ cd aws-scripts-mon

You should see the following scripts in the aws-scripts-mon folder now (See Figure 01).

Figure 01

Step 5: Create an IAM Role to push CloudWatch metrics from EC2 (You need to execute this step in AWS IAM).

You may use CloudWatchFullAccess AWS policy to create this role. After creating the IAM Role, you may attach the role to the EC2 instance.

Step 6: Again go back to the EC2 instance, where you intend to execute the Custom scripts and execute the following script (mon-put-instance-data.pl) to push them to CloudWatch (See Figure 2).

Figure 02

Step 7: If you wish, you can even schedule these CloudWatch metrics pushing job in every 5 min via crontab.

$ crontab -e

*/5 * * * * ~/aws-scripts-mon/mon-put-instance-data.pl — mem-used-incl-cache-buff — mem-util — disk-space-util — disk-path=/ — from-cron

Step 8: You may now go to CloudWatch in the AWS Console to see the populated Memory related metrics populated by CloudWatch Custom Metrics Scripts.

Go to CloudWatch → Metrics → Select “All Metrics” → Select “Custom Namespaces” → Select “System/ Linux” → Select “Instance Id”

Figure 03

Figure 04

You will see three EC2 instance metrics shown on it.

Figure 05

Finally, you can create CloudWatch Alarms out of these metrics now. Well Done!

References:

1. Monitoring Memory and Disk Metrics for Amazon EC2 Linux Instance: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/mon-scripts.html