AI Rate Limiting Advanced設定 - Plugin - v3.8.x

古いプラグインバージョンのドキュメントを閲覧しています。

構成

このプラグインはDB-lessモードと部分的に互換性があります。

The cluster strategy is not supported in DB-less and hybrid modes. For Kong Gateway in DB-less or hybrid mode, use the redis strategy.

互換性のあるプロトコル

AI Rate Limiting Advancedプラグインは以下のプロトコルに対応しています：

grpc, grpcs, http, https

パラメータ

このプラグインの設定で使用できるすべてのパラメータのリストは次のとおりです。

name or plugin

string required
プラグイン名。この場合はai-rate-limiting-advanced 。
- Kong Admin API、Kong Konnect API、宣言型構成、または decK ファイルを使用する場合、フィールドはnameです。
- Kubernetes で KongPlugin オブジェクトを使用する場合、フィールドはpluginです。
instance_name

string
プラグインのインスタンスを識別するための任意のカスタム名 (例: ai-rate-limiting-advanced_my-service 。

インスタンス名はKong ManagerとKonnectに表示されるので、例えば複数のサービスで同じプラグインを複数のコンテキストで実行する場合に便利です。また、Kong Admin API経由で特定のプラグインインスタンスにアクセスするためにも使用できます。

インスタンス名は、次のコンテキスト内で一意である必要があります。
- Kong Gateway Enterpriseのワークスペース内
- Konnectのコントロールプレーン（CP）またはコントロールプレーン（CP）グループ内
- Kong Gateway (OSS)の全世界
service.name or service.id

string

プラグインが対象とするサービス名または ID。最上位の /plugins エンドポイント. からプラグインをサービスに追加する場合は、これらのパラメータのいずれかを設定してください /services/{serviceName|Id}/plugins を使用する場合は必要ありません。
route.name or route.id

string

プラグインがターゲットとするルート名または ID。最上位の /plugins エンドポイント. を通るルートにプラグインを追加する場合は、これらのパラメータのいずれかを設定してください /routes/{routeName|Id}/plugins を使用する場合は必要ありません。
consumer.name or consumer.id

string

プラグインがターゲットとするコンシューマーの名前または ID。最上位の /plugins エンドポイント. からコンシューマーにプラグインを追加する場合は、これらのパラメーターのいずれかを設定してください /consumers/{consumerName|Id}/pluginsを使用する場合は必要ありません。
consumer_group.name or consumer_group.id

string

プラグインが対象とするコンシューマグループの名前または ID。設定されている場合、プラグインは指定されたグループが認証されているリクエストに対してのみアクティブになります/plugins エンドポイント. /consumer_groups/{consumerGroupName|Id}/pluginsを使用する場合は必要ありません。
enabled

boolean default: true

このプラグインが適用されるかどうか。
config

record required
- identifier
  
  string required default: consumer Must be one of: ip, credential, consumer, service, header, path, consumer-group
  
  The type of identifier used to generate the rate limit key. Defines the scope used to increment the rate limiting counters. Can be ip, credential, consumer, service, header, path or consumer-group.
- window_type
  
  string default: sliding Must be one of: fixed, sliding
  
  Sets the time window type to either sliding (default) or fixed. Sliding windows apply the rate limiting logic while taking into account previous hit rates (from the window that immediately precedes the current) using a dynamic weight. Fixed windows consist of buckets that are statically assigned to a definitive time range, each request is mapped to only one fixed window based on its timestamp and will affect only that window’s counters.
- sync_rate
  
  number
  
  How often to sync counter data to the central data store. A value of 0 results in synchronous behavior; a value of -1 ignores sync behavior entirely and only stores counters in node memory. A value greater than 0 will sync the counters in the specified number of seconds. The minimum allowed interval is 0.02 seconds (20ms).
- llm_providers
  
  array of type record required
  The provider config. Takes an array of name, limit and window size values.
  
  window_size
  
  number required
  
  The window size to apply a limit (defined in seconds).
  
  name
  
  string required Must be one of: openai, azure, anthropic, cohere, mistral, llama2, bedrock, gemini, requestPrompt
  
  The LLM provider to which the rate limit applies.
  
  limit
  
  number required
  
  The limit applies to the LLM provider within the defined window size. It used the query cost from the tokens to increment the counter.
- strategy
  
  string required default: local Must be one of: cluster, redis, local
  
  The rate-limiting strategy to use for retrieving and incrementing the limits. Available values are: local and cluster.
- dictionary_name
  
  string required default: kong_rate_limiting_counters
  
  The shared dictionary where counters are stored. When the plugin is configured to synchronize counter data externally (that is config.strategy is cluster or redis and config.sync_rate isn’t -1), this dictionary serves as a buffer to populate counters in the data store on each synchronization cycle.
- hide_client_headers
  
  boolean default: false
  
  Optionally hide informative response headers that would otherwise provide information about the current status of limits and counters.
- retry_after_jitter_max
  
  number default: 0
  
  The upper bound of a jitter (random delay) in seconds to be added to the Retry-After header of denied requests (status = 429) in order to prevent all the clients from coming back at the same time. The lower bound of the jitter is 0; in this case, the Retry-After header is equal to the RateLimit-Reset header.
- header_name
  
  string
  
  A string representing an HTTP header name.
- path
  
  string starts_with: /
  
  A string representing a URL path, such as /path/to/resource. Must start with a forward slash (/) and must not contain empty segments (i.e., two consecutive forward slashes).
- redis
  
  record required
  host
  
  string
  
  A string representing a host name, such as example.com.
  
  port
  
  integer between: 0 65535
  
  An integer representing a port number between 0 and 65535, inclusive.
  
  connect_timeout
  
  integer default: 2000 between: 0 2147483646
  
  An integer representing a timeout in milliseconds. Must be between 0 and 2^31-2.
  
  send_timeout
  
  integer default: 2000 between: 0 2147483646
  
  An integer representing a timeout in milliseconds. Must be between 0 and 2^31-2.
  
  read_timeout
  
  integer default: 2000 between: 0 2147483646
  
  An integer representing a timeout in milliseconds. Must be between 0 and 2^31-2.
  
  username
  
  string referenceable
  
  Username to use for Redis connections. If undefined, ACL authentication won’t be performed. This requires Redis v6.0.0+. To be compatible with Redis v5.x.y, you can set it to default.
  
  password
  
  string referenceable encrypted
  
  Password to use for Redis connections. If undefined, no AUTH commands are sent to Redis.
  
  sentinel_username
  
  string referenceable
  
  Sentinel username to authenticate with a Redis Sentinel instance. If undefined, ACL authentication won’t be performed. This requires Redis v6.2.0+.
  
  sentinel_password
  
  string referenceable encrypted
  
  Sentinel password to authenticate with a Redis Sentinel instance. If undefined, no AUTH commands are sent to Redis Sentinels.
  
  database
  
  integer default: 0
  
  Database to use for the Redis connection when using the redis strategy
  
  keepalive_pool_size
  
  integer default: 256 between: 1 2147483646
  
  The size limit for every cosocket connection pool associated with every remote server, per worker process. If neither keepalive_pool_size nor keepalive_backlog is specified, no pool is created. If keepalive_pool_size isn’t specified but keepalive_backlog is specified, then the pool uses the default value. Try to increase (e.g. 512) this value if latency is high or throughput is low.
  
  keepalive_backlog
  
  integer between: 0 2147483646
  
  Limits the total number of opened connections for a pool. If the connection pool is full, connection queues above the limit go into the backlog queue. If the backlog queue is full, subsequent connect operations fail and return nil. Queued operations (subject to set timeouts) resume once the number of connections in the pool is less than keepalive_pool_size. If latency is high or throughput is low, try increasing this value. Empirically, this value is larger than keepalive_pool_size.
  
  sentinel_master
  
  string
  
  Sentinel master to use for Redis connections. Defining this value implies using Redis Sentinel.
  
  sentinel_role
  
  string Must be one of: master, slave, any
  
  Sentinel role to use for Redis connections when the redis strategy is defined. Defining this value implies using Redis Sentinel.
  
  sentinel_nodes
  
  array of type record len_min: 1
  
  Sentinel node addresses to use for Redis connections when the redis strategy is defined. Defining this field implies using a Redis Sentinel. The minimum length of the array is 1 element.
  
  host
  
  string required default: 127.0.0.1
  
  A string representing a host name, such as example.com.
  
  port
  
  integer default: 6379 between: 0 65535
  
  An integer representing a port number between 0 and 65535, inclusive.
  
  cluster_nodes
  
  array of type record len_min: 1
  
  Cluster addresses to use for Redis connections when the redis strategy is defined. Defining this field implies using a Redis Cluster. The minimum length of the array is 1 element.
  
  ip
  
  string required default: 127.0.0.1
  
  A string representing a host name, such as example.com.
  
  port
  
  integer default: 6379 between: 0 65535
  
  An integer representing a port number between 0 and 65535, inclusive.
  
  ssl
  
  boolean default: false
  
  If set to true, uses SSL to connect to Redis.
  
  ssl_verify
  
  boolean default: false
  
  If set to true, verifies the validity of the server SSL certificate. If setting this parameter, also configure lua_ssl_trusted_certificate in kong.conf to specify the CA (or server) certificate used by your Redis server. You may also need to configure lua_ssl_verify_depth accordingly.
  
  server_name
  
  string
  
  A string representing an SNI (server name indication) value for TLS.
  
  cluster_max_redirections
  
  integer default: 5
  
  Maximum retry attempts for redirection.
  
  connection_is_proxied
  
  boolean default: false
  
  If the connection to Redis is proxied (e.g. Envoy), set it true. Set the host and port to point to the proxy address.
- disable_penalty
  
  boolean default: false
  
  If set to true, this doesn’t count denied requests (status = 429). If set to false, all requests, including denied ones, are counted. This parameter only affects the sliding window_type and the request prompt provider.
- request_prompt_count_function
  
  string
  
  If defined, it use custom function to count requests for the request prompt provider
- error_code
  
  number default: 429
  
  Set a custom error code to return when the rate limit is exceeded.
- error_message
  
  string default: API rate limit exceeded for provider(s):
  
  Set a custom error message to return when the rate limit is exceeded.
- error_hide_providers
  
  boolean default: false
  
  Optionally hide informative response that would otherwise provide information about the provider in the error message.
- tokens_count_strategy
  
  string required default: total_tokens Must be one of: total_tokens, prompt_tokens, completion_tokens, cost
  
  What tokens to use for cost calculation. Available values are: total_tokens prompt_tokens, completion_tokens or cost.

前へ AI Rate Limiting Advanced

次へ AI Rate Limiting Advancedの基本構成例

構成

互換性のあるプロトコル

パラメータ

name or plugin

instance_name

service.name or service.id

route.name or route.id

consumer.name or consumer.id

consumer_group.name or consumer_group.id

enabled

config

identifier

window_type

sync_rate

llm_providers

window_size

name

limit

strategy

dictionary_name

hide_client_headers

retry_after_jitter_max

header_name

path

redis

host

port

connect_timeout

send_timeout

read_timeout

username

password

sentinel_username

sentinel_password

database

keepalive_pool_size

keepalive_backlog

sentinel_master

sentinel_role

sentinel_nodes

host

port

cluster_nodes

ip

port

ssl

ssl_verify

server_name

cluster_max_redirections

connection_is_proxied

disable_penalty

request_prompt_count_function

error_code

error_message

error_hide_providers

tokens_count_strategy