Getting Started

Step by step setup of the Rusty AWS CloudWatch Exporter
Tags:
Categories:

Prerequisites

  • Have a valid license in a file. The default name is license.json but a different name can be used.

AWS Policy

The Rusty AWS CloudWatch Exporter leverages the AWS API to obtain resource information that is then used for labeling the metrics. In order to use them, the exporter requires authenticating against the API and the profile used needs to have the proper permissions that enables it to use the APIs.

The following is the minimum set of permissions needed in the profile:

  • ec2:DescribeInstances
  • dynamodb:DescribeTable
  • dynamodb:DescribeGlobalTable
  • dynamodb:ListTagsOfResource
  • elasticloadbalancing:DescribeLoadBalancers
  • elasticloadbalancing:DescribeTags
  • elasticloadbalancing:DescribeTargetGroups
  • lambda:GetFunction
  • sqs:GetQueueUrl
  • sqs:ListQueueTags

We recommend creating a policy with them and then attaching said policy to the role used to authenticate or run the service with.

Rusty AWS CloudWatch Exporter Setup

Configuration File

Create a configuration file - usually named config.yaml from the following template:

host_url: 'localhost'
host_port: 3000
ingestion_token: 'some random string'
aws_account_id: '1234567890'
aws_region: 'us-west-2'
aws_profile: 'default'

The meaning of each field is:

  • host_url and host_port: The IP and port, respectively, where the service will bind for listening to incoming requests. (Required)
  • ingestion_token: String to use for incoming metric ingestion request validation. This will be needed when setting up the AWS Firehose. Metric ingestion requests that do not contain the expected ingestion token in the X-Amz-Firehose-Access-Key header will be rejected by the exporter. (Required)
  • aws_account_id: The AWS account the resources being monitored belong to. It is used for the AWS API. (Required)
  • aws_region: The default region to use for the AWS API client. (Required)
  • aws_profile: Profile name to use to acquire the authentication for the AWS API Client. (Required)

Service Setup

The Rusty AWS CloudWatch Exporter is distributed via a Docker image through the following repository.

The simplest command to get the container up and running is by executing the following command:

$ docker run \
    --name rusty-aws-cloudwatch-exporter
    -v /path/to/config.yaml:/root/config.yaml
    -v /path/to/license.json:/root/license.json
    -p 127.0.0.1:3000:3000
    public.ecr.aws/z0k1a5l6/rusty-aws-cloudwatch-exporter:v0.2.0

If the location for the config file or license file are different the following command line arguments can be passed in to change where the service looks for them:

  • --config-file <config-file-path>: Configures the location to search for the configuration file. (Optional)
  • --license-file <license-file-path>: Configures the location to look for the license file. (Optional)

We do not make any assumptions on the way this is going to be deployed as it is too specific to the user’s environment and setup. This can be done in multiple ways such as executing it on a specific host, deploy it as part of k8s pod, etc.

AWS API Credentials

The service relies on the AWS API to retrieve resource information. In order to authenticate against it, it uses the AWS SDK Authentication mechanisms.

Even though you can pick up the best option that suits you, we recommend running the application where it directly inherits a role that has the appropriate permissions/policy already assigned. For example, running the service within an EC2 instance.

Reachable Endpoint

The HTTP endpoint opened by the exporter service needs to be reachable from the Firehose and it needs to be an HTTPS endpoint.

The service itself DOES NOT support TLS on the endpoint natively. The provisioning of that is left to the user of the service to setup.

Depending on the way the service is deployed, the options that will be available to achieve this.

As an example, if the exporter is setup to run as a docker container within a specific EC2 instance, then this can be achieved by using a reverse proxy such as nginx with a proper certificate.

Metric Streaming Setup

This assumes that the Rusty AWS CloudWatch exporter is already setup as above and up and running.

What remains to be done is make AWS send metrics to it. For this we need to setup a pipeline that consists of a CloudWatch Stream and a Kinesis Firehose Delivery Stream.

AWS Kinesis Firehose Delivery Stream

This is the first component that needs to be created as the CloudWatch Stream needs to reference it in its creation.

This AWS Blogpost is a good guide on how to create it.

The important details are that the Source is set to be Direct PUT and the destination is an HTTP Endpoint.

For the HTTP Endpoint URL configuration option make sure that you use https://your-reachable-exporter-url/ingestion/firehose.

In the Access Key use the same string as what was set for the ingestion_token configuration in the exporter service.

The rest of the settings are optional and discretionary based on the environment and setup. A few notable mentions are:

  • Setting the encoding to be gzipped is not necessary but recommended.
  • The Buffer Hints can be tuned in order to minimize latency or improve request efficiency.
  • The selected Service Access role needs to be capable of reaching the endpoint and making the request.

AWS CloudWatch Metric Stream

Once the AWS Kinesis Delivery Stream has been setup, the metric faucet needs to be created.

Reading the AWS Documentation on metric streams is recommended.

The type of metric stream that needs to be created is the Custom setup with firehose using the previously created AWS Firehose as the Kinesis Data Firehose stream to use.

The Output Format must be OpenTelemetry 1.0.

The rest of the configuration options are used to control the set of metrics to be sent by the Metric Stream and are completely discretionary to the end user based on their needs.