このページは、まだ日本語ではご利用いただけません。翻訳中です。
Traffic Log
New to Kuma? Don’t use this, check the
MeshAccessLog
policy instead.
With the Traffic Log policy you can easily set up access logs on every data plane in a mesh.
This policy only records outbound traffic. It doesn’t record inbound traffic.
To configure access logs in Kong Mesh you need to:
In the rest of this page we assume you have already configured your observability tools to work with Kong Mesh. If you haven’t already read the observability docs.
Add a logging backend
A logging backend is essentially a sink for access logs.
Currently, it can be either a file
or a TCP log collector
, such as Logstash, Splunk or other.
apiVersion: kuma.io/v1alpha1
kind: Mesh
metadata:
name: default
spec:
logging:
# TrafficLog policies may leave the `backend` field undefined.
# In that case the logs will be forwarded into the `defaultBackend` of that Mesh.
defaultBackend: file
# List of logging backends that can be referred to by name
# from TrafficLog policies of that Mesh.
backends:
- name: logstash
# Use `format` field to adjust the access log format to your use case.
format: '{"start_time": "%START_TIME%", "source": "%KUMA_SOURCE_SERVICE%", "destination": "%KUMA_DESTINATION_SERVICE%", "source_address": "%KUMA_SOURCE_ADDRESS_WITHOUT_PORT%", "destination_address": "%UPSTREAM_HOST%", "duration_millis": "%DURATION%", "bytes_received": "%BYTES_RECEIVED%", "bytes_sent": "%BYTES_SENT%"}'
type: tcp
# Use `config` field to co configure a TCP logging backend.
conf:
# Address of a log collector (like logstash, splunk or other).
address: 127.0.0.1:5000
- name: file
type: file
# Use `file` field to configure a file-based logging backend.
conf:
path: /tmp/access.log
# When `format` field is omitted, the default access log format will be used.
Add a TrafficLog resource
You need to create a TrafficLog
policy to select a subset of traffic and write its access logs into one of the backends configured for that mesh.
apiVersion: kuma.io/v1alpha1
kind: TrafficLog
metadata:
name: all-traffic
mesh: default
spec:
# This TrafficLog policy applies all traffic in that Mesh.
sources:
- match:
kuma.io/service: "*"
destinations:
- match:
kuma.io/service: "*"
apiVersion: kuma.io/v1alpha1
kind: TrafficLog
metadata:
name: backend-to-database-traffic
mesh: default
spec:
# This TrafficLog policy applies only to traffic from service `backend` to service `database`.
sources:
- match:
kuma.io/service: backend_kuma-example_svc_8080
destinations:
- match:
kuma.io/service: database_kuma-example_svc_5432
conf:
# Forward the logs into the logging backend named `logstash`.
backend: logstash
When
backend
field is omitted, the logs will be forwarded into thedefaultBackend
of thatMesh
.
Matching
TrafficLog
is an Outbound Connection Policy.
For this reason the only supported value for destinations.match
is kuma.io/service
.
Logging external services
When running Kong Mesh on Kubernetes you can also log the traffic to external services. To do it, the matched destination section has to have wildcard *
value.
In such case %KUMA_DESTINATION_SERVICE%
will have value external
and %UPSTREAM_HOST%
will have an IP of the service.
Builtin Gateway support
Traffic Log is a Kong Mesh outbound connection policy, so Kong Mesh chooses a Traffic Log policy by matching the service tag of the data plane’s outbounds.
Since a builtin gateway data plane does not have outbounds, Kong Mesh always uses the builtin service name pass_through
to match the Traffic Log policy for Gateways.
Access Log Format
Kong Mesh gives you full control over the format of the access logs.
The shape of a single log record is defined by a template string that uses command operators to extract and format data about a TCP
connection or an HTTP
request.
E.g.,
%START_TIME% %KUMA_SOURCE_SERVICE% => %KUMA_DESTINATION_SERVICE% %DURATION%
where %START_TIME%
and %KUMA_SOURCE_SERVICE%
are examples of available command operators.
All command operators defined by Envoy are supported, along with additional command operators defined by Kong Mesh:
Command Operator | Description |
---|---|
%KUMA_MESH% |
name of the mesh in which traffic is flowing |
%KUMA_SOURCE_SERVICE% |
name of a service that is the source of traffic |
%KUMA_DESTINATION_SERVICE% |
name of a service that is the destination of traffic |
%KUMA_SOURCE_ADDRESS_WITHOUT_PORT% |
address of a Dataplane that is the source of traffic |
%KUMA_TRAFFIC_DIRECTION% |
direction of the traffic, INBOUND , OUTBOUND or UNSPECIFIED
|
Access Logs for TCP and HTTP traffic
All access log command operators are valid to use with both TCP
and HTTP
traffic.
If a command operator is specific to HTTP
traffic, such as %REQ(X?Y):Z%
or %RESP(X?Y):Z%
, it will be replaced by a symbol “-
” in case of TCP
traffic.
Internally, Kong Mesh determines traffic protocol based on the value of kuma.io/protocol
tag on the inbound
interface of a destination
Dataplane
.
The default format string for TCP
traffic is:
[%START_TIME%] %RESPONSE_FLAGS% %KUMA_MESH% %KUMA_SOURCE_ADDRESS_WITHOUT_PORT%(%KUMA_SOURCE_SERVICE%)->%UPSTREAM_HOST%(%KUMA_DESTINATION_SERVICE%) took %DURATION%ms, sent %BYTES_SENT% bytes, received: %BYTES_RECEIVED% bytes
The default format string for HTTP
traffic is:
[%START_TIME%] %KUMA_MESH% "%REQ(:METHOD)% %REQ(X-ENVOY-ORIGINAL-PATH?:PATH)% %PROTOCOL%" %RESPONSE_CODE% %RESPONSE_FLAGS% %BYTES_RECEIVED% %BYTES_SENT% %DURATION% %RESP(X-ENVOY-UPSTREAM-SERVICE-TIME)% "%REQ(X-FORWARDED-FOR)%" "%REQ(USER-AGENT)%" "%REQ(X-REQUEST-ID)%" "%REQ(:AUTHORITY)%" "%KUMA_SOURCE_SERVICE%" "%KUMA_DESTINATION_SERVICE%" "%KUMA_SOURCE_ADDRESS_WITHOUT_PORT%" "%UPSTREAM_HOST%"
To provide different format for TCP and HTTP logging you can define two separate logging backends with the same address and different format. Then define two TrafficLog entity, one for TCP and one for HTTP with matching
kuma.io/protocol
selector.
JSON format
If you need an access log with entries in JSON
format, you have to provide a template string that is a valid JSON
object, e.g.
{
"start_time": "%START_TIME%",
"source": "%KUMA_SOURCE_SERVICE%",
"destination": "%KUMA_DESTINATION_SERVICE%",
"source_address": "%KUMA_SOURCE_ADDRESS_WITHOUT_PORT%",
"destination_address": "%UPSTREAM_HOST%",
"duration_millis": "%DURATION%",
"bytes_received": "%BYTES_RECEIVED%",
"bytes_sent": "%BYTES_SENT%"
}
All policy options
$schema: http://json-schema.org/draft-04/schema#
$ref: #/definitions/TrafficLog
definitions
TrafficLog
- ## Traffic Log
- TrafficLog defines log for traffic between dataplanes.
- Type:
object
- This schema accepts additional properties.
- Properties
- sources
- List of selectors to match dataplanes that are sources of traffic.
- Type:
array
- Items
- $ref: #/definitions/kuma.mesh.v1alpha1.Selector
- destinations
- List of selectors to match services that are destinations of traffic.
- Type:
array
- Items
- $ref: #/definitions/kuma.mesh.v1alpha1.Selector
- conf
- Configuration of the logging.
- $ref: #/definitions/kuma.mesh.v1alpha1.TrafficLog.Conf
- sources
kuma.mesh.v1alpha1.Selector
- ## Selector
- Selector defines structure for selecting tags for given dataplane
- Type:
object
- This schema accepts additional properties.
- Properties
- match
- Tags to match, can be used for both source and destinations
- Type:
object
- This schema accepts additional properties.
- Properties
- match
kuma.mesh.v1alpha1.TrafficLog.Conf
- ## Conf
- Configuration defines settings of the logging.
- Type:
object
- This schema accepts additional properties.
- Properties
- backend
- Backend defined in the Mesh entity.
- Type:
string
- backend
Generated with json-schema-md-doc