CloudWatch Agent Quickstart Guide

CloudWatch Agent Quickstart Guide

The purpose of this quick-start guide is to help system administrators, and developers get the AWS CloudWatch Agent installed, configured, and running on an active EC2 instance as quickly as possible. This guide is not a comprehensive guide on CloudWatch metrics. The AWS CloudWatch Agent supports on-premise servers, Linux, and Windows operating systems. However, this guide only covers Linux on EC2 instances. These instructions are a starting point for host-based metrics on EC2 and not intended to be a guide on how to monitor production systems. For more information on topics not covered here, please refer to the AWS Documentation.

Why install the Cloud Watch Agent?

By default, AWS automatically provides 17 CloudWatch Metrics: StatusCheckFailed_System, StatusCheckFailed_Instance, StatusCheckFailed, NetworkPacketsIn, NetworkPacketsOut, CPUUtilization, MetadataNoToken, NetworkIn, NetworkOut, DiskReadBytes, DiskWriteBytes, DiskReadOps, DiskWriteOps, CPUCreditUsage, CPUCreditBalance, CPUSurplusCreditBalance, CPUSurplusCreditsCharged

The AWS CLI can be used to list the names of these metrics:

1
2
3

aws cloudwatch list-metrics --namespace AWS/EC2 \
  --dimensions Name=InstanceId,Value=instance-id \
  --query 'Metrics[*].[MetricName]' --output text

Replace “instance-id” with the instance id of one of your EC2 instances.

Refer to the AWS documentation for a full description of the metrics.

Operating system based metrics like memory utilization, disk space usage, and system load are not available by default. Monitoring these and other metrics is crucial to accessing the health of a server. EC2 instances are virtual machines, and the hypervisor of the host machine running the EC2 instances does not have access to the operating system running on the virtual machine, nor does it have access to host-level metrics provided by the operating system running within the virtual machine. For CloudWatch to have access to the host level metrics within an EC2 instance, a script or application running on the EC2 instance must send the metrics to CloudWatch. AWS recommends using the Cloud Watch agent to collect metrics and logs instead of using the monitoring scripts.

The CloudWatch agent enables you to do the following:

• Collect more system-level metrics from Amazon EC2 instances across operating systems. The metrics can include in-guest metrics, in addition to the metrics for EC2 instances.
• Collect system-level metrics from on-premises servers. These can include servers in a hybrid environment as well as servers not managed by AWS.
• Retrieve custom metrics from your applications or services using the StatsD and collectd protocols. StatsD is supported on both Linux servers and servers running Windows Server. collectd is supported only on Linux servers.
• Collect logs from Amazon EC2 instances and on-premises servers, running either Linux or Windows Server.

1. Configuring the Permissions

As with any AWS service, the first consideration is authentication and authorization. For the CloudWatch agent to send metrics to the CloudWatch service, the agent must be authenticated and have permission to write metrics to CloudWatch. Creating an IAM role with a trust policy for the EC2 service is the best way to enable access for the CloudWatch agent. CloudWatchAgentServerPolicy and CloudWatchAgentAdminPolicy are AWS managed policies created by Amazon for this purpose. After creating the role, attach the role to an instance profile. Then associate the instance profile to the EC2 instance that will be running the CloudWatch agent. Creating the role and instance profile can be accomplished via the AWS console or the CLI. This guide will only show how to accomplish this task via the CLI. If the EC2 instance you want to monitor already has a role assigned to it with the required permissions, skip this step.

Create a text file named ec2-cw-agent-server-role-trust-policy.json with the text of the trust policy:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

cat <<EOP >> ec2-cw-agent-server-role-trust-policy.json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "ec2.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}
EOP

The credentials used by the CLI must have permissions to create an IAM role.

The CLI command for creating the role:

1
2
3

aws iam create-role --role-name CloudWatchAgentServerRole \
  --assume-role-policy-document \
  file://ec2-cw-agent-server-role-trust-policy.json

Attached the managed policies:

1
2
3
4
5
6

aws iam attach-role-policy \
  --policy-arn arn:aws:iam::aws:policy/CloudWatchAgentServerPolicy \
  --role-name CloudWatchAgentServerRole
aws iam attach-role-policy \
  --policy-arn arn:aws:iam::aws:policy/CloudWatchAgentAdminPolicy \
  --role-name CloudWatchAgentServerRole

The CloudWatchAgentAdminPolicy will allow the EC2 instance to store the configuration for the CloudWatch agent in SSM instead of locally on the instance. Remove the CloudWatchAgentAdminPolicy managed policy from the role after saving the configuration to SSM. The CloudWatchAgentServerPolicy managed policy gives access to reading the configuration from SSM.

Create the instance profile and attach it to the instance:

1
2
3
4
5
6
7
8
9

aws iam create-instance-profile --instance-profile-name \
  CloudWatchAgentServerRole
aws iam wait instance-profile-exists --instance-profile-name \
  CloudWatchAgentServerRole
aws iam add-role-to-instance-profile --role-name \
  CloudWatchAgentServerRole --instance-profile-name \
  CloudWatchAgentServerRole
aws ec2 associate-iam-instance-profile --iam-instance-profile \
  Name=CloudWatchAgentServerRole --instance-id instance-id

Replace “instance-id” with the id of the instance.

2. Installing and Configuring the Cloud Watch Agent

The next step is to install and configure the CloudWatch agent on the EC2 instance. First, SSH into the EC2 instance. Install the CloudWatch agent using the CLI command:

1
2

sudo rpm -Uvh \
  https://s3.amazonaws.com/amazoncloudwatch-agent/amazon_linux/amd64/latest/amazon-cloudwatch-agent.rpm

Before running the CloudWatch agent on any servers, create a CloudWatch agent configuration file. Use the wizard to create the configuration file and then modify it manually if needed.

1

sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-config-wizard

Entering y for Statsd will enable the statsd protocol in the CloudWatch agent and allow additional host and application-level metrics to be sent to CloudWatch.

Finally start the CloudWatch Agent:

1
2

sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl \
-a fetch-config -m ec2 -c ssm:AmazonCloudWatch-linux -s

What custom metrics are available from the CloudWatch Agent? The full list of metrics is available at: Metrics Collected by the CloudWatch Agent on Linux Instances

With statsd enabled, custom metrics not provided by CloudWatch agent can be sent to the CloudWatch agent via the statsd protocol. Here is an example script that is sending the CPU load to the CloudWatch Agent via the statsd protocol:

1
2
3
4
5
6
7
8

#!/bin/bash
set -e
LOADAVGS=( $(cat /proc/loadavg) )
echo "load_avg_1:${LOADAVGS[0]}|g" | nc -w1 -Cu 127.0.0.1 8125 &
echo "load_avg_5:${LOADAVGS[1]}|g" | nc -w1 -Cu 127.0.0.1 8125 &
echo "load_avg_10:${LOADAVGS[2]}|g" | nc -w1 -Cu 127.0.0.1 8125 &
wait
echo "done"

Here is an example crontab entry to send the system lod stats every 5 minutes:

1

*/5 * * * * /home/ec2-user/send-load-metrics.sh &> /dev/null

Conclusion

This guide is just an introduction to the capabilities of the CloudWatch agent. Read the following resources for more details.

Suggested Resources

• Download and Configure the CloudWatch Agent Using the Command Line
• Create IAM Roles and Users for Use with the CloudWatch Agent
• Collecting Metrics and Logs from Amazon EC2 Instances and On-Premises Servers with the CloudWatch Agent
• Create the CloudWatch Agent Configuration File
• Amazon CloudWatch pricing