Kong Mesh on Amazon ECS

This page describes running Kong Mesh on ECS and offers guidelines for integrating Kong Mesh into your deployment process.

For a demo of Kong Mesh on ECS, see the example repository for Cloudformation. This demo covers bootstrapping an ECS cluster from scratch, deploying Kong Mesh, and deploying some services into the mesh.

Overview

On ECS, Kong Mesh runs in Universal mode. Every ECS task runs with an Envoy sidecar. Kong Mesh supports tasks on the following launch types:

Fargate
EC2

The control plane itself also runs as an ECS service in the cluster.

Data plane authentication

As part of joining and synchronizing with the mesh, every sidecar needs to authenticate with the control plane.

With Kong Mesh, this is typically accomplished by using a data plane token. In Universal mode, creating and managing data plane tokens is a manual step for the mesh operator.

With Kong Mesh 2.0.0, you can instead configure the sidecar to authenticate using the identity of the ECS task it’s running as.

Mesh communication

With Kong Mesh on ECS, each service enumerates other mesh services it contacts in the Dataplane specification.

Deployment

This section covers ECS-specific parts of running Kong Mesh, using the example Cloudformation as a guide.

Control plane

Kong Mesh runs in Universal mode on ECS. The example setup repository uses an AWS RDS database as a PostgreSQL backend. It also uses ECS service discovery to enable ECS tasks to communicate with the Kong Mesh control plane.

The example Cloudformation includes two Cloudformation stacks for creating a cluster and deploying Kong Mesh

Workload identity

The data plane proxy attempts to authenticate using the IAM role of the ECS task it’s running under. The control plane assumes that if this role has been tagged with certain kuma.io/ tags, it can be authorized to run as the corresponding Kuma resource identity.

In particular, every role must be tagged at a minimum with kuma.io/type set to either dataplane, ingress, or egress. For dataplane, i.e. a normal data plane proxy, the kuma.io/mesh tag is also required to be set.

This means that the setting of these two tags on IAM roles must be restricted accordingly for your AWS account (which must be explicitly given to the CP, see below).

The control plane must have the following options enabled. The example Cloudformation sets them via environment variables:

- Name: KUMA_DP_SERVER_AUTHN_DP_PROXY_TYPE
  Value: aws-iam
- Name: KUMA_DP_SERVER_AUTHN_ZONE_PROXY_TYPE
  Value: aws-iam
- Name: KUMA_DP_SERVER_AUTHN_ENABLE_RELOADABLE_TOKENS
  Value: "true"
- Name: KMESH_AWSIAM_AUTHORIZEDACCOUNTIDS
  Value: !Ref AWS::AccountId # this tells the CP which accounts can be used by DPs to authenticate

Every sidecar must have the --auth-type=aws flag set as well.

Services

When deploying an ECS task to be included in the mesh, the following must be considered.

Outbounds

Services are bootstrapped with a Dataplane specification.

Transparent proxy is not supported on ECS, so the Dataplane resource for a service must enumerate all other mesh services this service contacts and include them in the Dataplane specification as outbounds.

See the example repository to learn how to handle the Dataplane template with Cloudformation.

IAM role

The ECS task IAM role must also have some tags set in order to authenticate. It must always have the kuma.io/type tag set to either "dataplane", "ingress", or "egress".

If it’s a "dataplane" type, then it must also have the kuma.io/mesh tag set. Additionally, you can set the kuma.io/service tag to further restrict its identity.

Sidecar

The sidecar must run as a container in the ECS task.

See the example repository for an example container definition.