AI Rate Limiting Advanced Configuration

このページは、まだ日本語ではご利用いただけません。翻訳中です。

構成

このプラグインはDB-lessモードと部分的に互換性があります。

The cluster strategy is not supported in DB-less and hybrid modes. For Kong Gateway in DB-less or hybrid mode, use the redis strategy.

互換性のあるプロトコル

AI Rate Limiting Advancedプラグインは以下のプロトコルに対応しています：

grpc, grpcs, http, https

パラメータ

このプラグインの設定で使用できるすべてのパラメータのリストは次のとおりです。

• name or plugin

  string required

  プラグイン名。この場合は ai-rate-limiting-advanced 。

  • Kong Admin API、Kong Konnect API、宣言型構成、または decK ファイルを使用する場合、フィールドはnameです。
  • Kubernetes で KongPlugin オブジェクトを使用する場合、フィールドはpluginです。

• instance_name

  string

  プラグインのインスタンスを識別するための任意のカスタム名 (例: ai-rate-limiting-advanced_my-service 。

  インスタンス名はKong ManagerとKonnectに表示されるので、 例えば複数のサービスで同じプラグインを複数のコンテキストで実行する場合に便利です。また、Kong Admin API経由で特定のプラグインインスタンスに アクセスするためにも使用できます。

  インスタンス名は、次のコンテキスト内で一意である必要があります。

  • Kong Gateway Enterpriseのワークスペース内
  • Konnectのコントロールプレーン（CP）またはコントロールプレーン（CP）グループ内
  • Kong Gateway (OSS)の全世界

• service.name or service.id

  string

  プラグインが対象とするサービス名または ID。最上位の /plugins エンドポイント. からプラグインをサービスに追加する場合は、これらのパラメータのいずれかを設定してください /services/{serviceName|Id}/plugins を使用する場合は必要ありません。

• route.name or route.id

  string

  プラグインがターゲットとするルート名または ID。最上位の /plugins エンドポイント. を通るルートにプラグインを追加する場合は、これらのパラメータのいずれかを設定してください /routes/{routeName|Id}/plugins を使用する場合は必要ありません。

• consumer.name or consumer.id

  string

  プラグインがターゲットとするコンシューマーの名前または ID。 最上位の /plugins エンドポイント. からコンシューマーにプラグインを追加する場合は、これらのパラメーターのいずれかを設定してください /consumers/{consumerName|Id}/pluginsを使用する場合は必要ありません。

• consumer_group.name or consumer_group.id

  string

  プラグインが対象とするコンシューマグループの名前または ID。 設定されている場合、プラグインは指定されたグループが認証されているリクエストに対してのみアクティブになります/plugins エンドポイント. /consumer_groups/{consumerGroupName|Id}/pluginsを使用する場合は必要ありません。

• enabled

  boolean default: true

  このプラグインが適用されるかどうか。

• config

  record required
  • identifier

    string required default: consumer Must be one of: ip, credential, consumer, service, header, path, consumer-group

    The type of identifier used to generate the rate limit key. Defines the scope used to increment the rate limiting counters. Can be ip, credential, consumer, service, header, path or consumer-group. Note if identifier is consumer-group, the plugin must be applied on a consumer group entity. Because a consumer may belong to multiple consumer groups, the plugin needs to know explicitly which consumer group to limit the rate.

  • window_type

    string default: sliding Must be one of: fixed, sliding

    Sets the time window type to either sliding (default) or fixed. Sliding windows apply the rate limiting logic while taking into account previous hit rates (from the window that immediately precedes the current) using a dynamic weight. Fixed windows consist of buckets that are statically assigned to a definitive time range, each request is mapped to only one fixed window based on its timestamp and will affect only that window’s counters.

  • sync_rate

    number

    How often to sync counter data to the central data store. A value of 0 results in synchronous behavior; a value of -1 ignores sync behavior entirely and only stores counters in node memory. A value greater than 0 will sync the counters in the specified number of seconds. The minimum allowed interval is 0.02 seconds (20ms).

  • llm_providers

    array of type record required

    The provider config. Takes an array of name, limit and window size values.

    • window_size

      array of type number required

      One or more window sizes to apply a limit to (defined in seconds). There must be a matching number of window limits and sizes specified.

    • name

      string required Must be one of: openai, azure, anthropic, cohere, mistral, llama2, bedrock, gemini, huggingface, requestPrompt

      The LLM provider to which the rate limit applies.

    • limit

      array of type number required

      One or more requests-per-window limits to apply. There must be a matching number of window limits and sizes specified.

  • strategy

    string required default: local Must be one of: cluster, redis, local

    The rate-limiting strategy to use for retrieving and incrementing the limits. Available values are: local and cluster.

  • dictionary_name

    string required default: kong_rate_limiting_counters

    The shared dictionary where counters are stored. When the plugin is configured to synchronize counter data externally (that is config.strategy is cluster or redis and config.sync_rate isn’t -1), this dictionary serves as a buffer to populate counters in the data store on each synchronization cycle.

  • hide_client_headers

    boolean default: false

    Optionally hide informative response headers that would otherwise provide information about the current status of limits and counters.

  • retry_after_jitter_max

    number default: 0

    The upper bound of a jitter (random delay) in seconds to be added to the Retry-After header of denied requests (status = 429) in order to prevent all the clients from coming back at the same time. The lower bound of the jitter is 0; in this case, the Retry-After header is equal to the RateLimit-Reset header.

  • header_name

    string

    A string representing an HTTP header name.

  • path

    string starts_with: /

    A string representing a URL path, such as /path/to/resource. Must start with a forward slash (/) and must not contain empty segments (i.e., two consecutive forward slashes).

  • redis

    record required
    • host

      string default: 127.0.0.1

      A string representing a host name, such as example.com.

    • port

      integer default: 6379 between: 0 65535

      An integer representing a port number between 0 and 65535, inclusive.

    • connect_timeout

      integer default: 2000 between: 0 2147483646

      An integer representing a timeout in milliseconds. Must be between 0 and 2^31-2.

    • send_timeout

      integer default: 2000 between: 0 2147483646

      An integer representing a timeout in milliseconds. Must be between 0 and 2^31-2.

    • read_timeout

      integer default: 2000 between: 0 2147483646

      An integer representing a timeout in milliseconds. Must be between 0 and 2^31-2.

    • username

      string referenceable

      Username to use for Redis connections. If undefined, ACL authentication won’t be performed. This requires Redis v6.0.0+. To be compatible with Redis v5.x.y, you can set it to default.

    • password

      string referenceable encrypted

      Password to use for Redis connections. If undefined, no AUTH commands are sent to Redis.

    • sentinel_username

      string referenceable

      Sentinel username to authenticate with a Redis Sentinel instance. If undefined, ACL authentication won’t be performed. This requires Redis v6.2.0+.

    • sentinel_password

      string referenceable encrypted

      Sentinel password to authenticate with a Redis Sentinel instance. If undefined, no AUTH commands are sent to Redis Sentinels.

    • database

      integer default: 0

      Database to use for the Redis connection when using the redis strategy

    • keepalive_pool_size

      integer default: 256 between: 1 2147483646

      The size limit for every cosocket connection pool associated with every remote server, per worker process. If neither keepalive_pool_size nor keepalive_backlog is specified, no pool is created. If keepalive_pool_size isn’t specified but keepalive_backlog is specified, then the pool uses the default value. Try to increase (e.g. 512) this value if latency is high or throughput is low.

    • keepalive_backlog

      integer between: 0 2147483646

      Limits the total number of opened connections for a pool. If the connection pool is full, connection queues above the limit go into the backlog queue. If the backlog queue is full, subsequent connect operations fail and return nil. Queued operations (subject to set timeouts) resume once the number of connections in the pool is less than keepalive_pool_size. If latency is high or throughput is low, try increasing this value. Empirically, this value is larger than keepalive_pool_size.

    • sentinel_master

      string

      Sentinel master to use for Redis connections. Defining this value implies using Redis Sentinel.

    • sentinel_role

      string Must be one of: master, slave, any

      Sentinel role to use for Redis connections when the redis strategy is defined. Defining this value implies using Redis Sentinel.

    • sentinel_nodes

      array of type record len_min: 1

      Sentinel node addresses to use for Redis connections when the redis strategy is defined. Defining this field implies using a Redis Sentinel. The minimum length of the array is 1 element.

      • host

        string required default: 127.0.0.1

        A string representing a host name, such as example.com.

      • port

        integer default: 6379 between: 0 65535

        An integer representing a port number between 0 and 65535, inclusive.

    • cluster_nodes

      array of type record len_min: 1

      Cluster addresses to use for Redis connections when the redis strategy is defined. Defining this field implies using a Redis Cluster. The minimum length of the array is 1 element.

      • ip

        string required default: 127.0.0.1

        A string representing a host name, such as example.com.

      • port

        integer default: 6379 between: 0 65535

        An integer representing a port number between 0 and 65535, inclusive.

    • ssl

      boolean default: false

      If set to true, uses SSL to connect to Redis.

    • ssl_verify

      boolean default: false

      If set to true, verifies the validity of the server SSL certificate. If setting this parameter, also configure lua_ssl_trusted_certificate in kong.conf to specify the CA (or server) certificate used by your Redis server. You may also need to configure lua_ssl_verify_depth accordingly.

    • server_name

      string

      A string representing an SNI (server name indication) value for TLS.

    • cluster_max_redirections

      integer default: 5

      Maximum retry attempts for redirection.

    • connection_is_proxied

      boolean default: false

      If the connection to Redis is proxied (e.g. Envoy), set it true. Set the host and port to point to the proxy address.

  • disable_penalty

    boolean default: false

    If set to true, this doesn’t count denied requests (status = 429). If set to false, all requests, including denied ones, are counted. This parameter only affects the sliding window_type and the request prompt provider.

  • request_prompt_count_function

    string

    If defined, it use custom function to count requests for the request prompt provider

  • error_code

    number default: 429

    Set a custom error code to return when the rate limit is exceeded.

  • error_message

    string default: AI token rate limit exceeded for provider(s):

    Set a custom error message to return when the rate limit is exceeded.

  • error_hide_providers

    boolean default: false

    Optionally hide informative response that would otherwise provide information about the provider in the error message.

  • tokens_count_strategy

    string required default: total_tokens Must be one of: total_tokens, prompt_tokens, completion_tokens, cost

    What tokens to use for cost calculation. Available values are: total_tokens prompt_tokens, completion_tokens or cost.

  • llm_format

    string default: openai Must be one of: openai, bedrock, gemini

    LLM input and output format and schema to use