Collapse navigation tree

Collapse

Manage Throughput

Redpanda throttles throughput on ingress and egress independently, and you can configure limits at the broker and client levels. This prevents clients from causing unbounded network and disk usage on brokers.

You can configure limits at two levels:

  • Broker limits: These apply to all clients connected to the broker and restrict total traffic on the broker. See Broker-wide throughput limits.

  • Client limits: These apply to authenticated users or clients defined by their client ID. You can manage client quotas with rpk cluster quotas, with the Redpanda Cloud UI, with the Redpanda Cloud Data Plane API, or with the Kafka API. When no quotas apply, the client has unlimited throughput.

Throughput throttling is supported for BYOC and Dedicated clusters only.

After reading this page, you will be able to:

  • Set user-based throughput quotas

  • Set client ID-based quotas

  • Monitor quota usage and throttling behavior

View connected client details

Before configuring throughput quotas, check the current produce and consume throughput of a client.

Use the rpk cluster connections list command or the GET /v1/monitoring/kafka/connections Data Plane API endpoint to view detailed information about active Kafka client connections.

For example, to view a cluster’s connected clients in order of highest current produce throughput, run:

  • rpk

  • Data Plane API

rpk cluster connections list --order-by="recent_request_statistics.produce_bytes desc"
UID                                   STATE  USER             CLIENT-ID             IP:PORT           NODE  SHARD  OPEN-TIME  IDLE           PROD-TPUT/SEC  FETCH-TPUT/SEC  REQS/MIN
b20601a3-624c-4a8c-ab88-717643f01d56  OPEN   UNAUTHENTICATED  perf-producer-client  127.0.0.1:55012   0     0      9s         0s             78.9MB         0B              292
36338ca5-86b7-4478-ad23-32d49cfaef61  OPEN   UNAUTHENTICATED  rpk                   127.0.0.1:49722   0     0      13s        13.694243104s  0B             0B              1
7e277ef6-0176-4007-b100-6581bfde570f  OPEN   UNAUTHENTICATED  rpk                   127.0.0.1:49736   0     0      13s        10.093957335s  0B             0B              2
567d9918-d3dc-4c74-ab5d-85f70cd3ee35  OPEN   UNAUTHENTICATED  rpk                   127.0.0.1:49748   0     0      13s        0.591413542s   0B             0B              5
08616f21-08f9-46e7-8f06-964bd8240d9b  OPEN   UNAUTHENTICATED  rpk                   127.0.0.1:49764   0     0      13s        10.094602845s  0B             0B              2
e4d5b57e-5c76-4975-ada8-17a88d68a62d  OPEN   UNAUTHENTICATED  rpk                   127.0.0.1:54992   0     0      10s        0.302090085s   0B             14.5MB          27
b41584f3-2662-4185-a4b8-0d8510f5c780  OPEN   UNAUTHENTICATED  perf-producer-client  127.0.0.1:55002   0     0      8s         7.743592270s   0B             0B              1
62fde947-411d-4ea8-9461-3becc2631b46  CLOSED UNAUTHENTICATED  rpk                   127.0.0.1:48578   0     0      26s        0.000737836s   0B             0B              1
95387e2e-2ec4-4040-aa5e-4257a3efa1a2  CLOSED UNAUTHENTICATED  rpk                   127.0.0.1:48564   0     0      26s        0.208180826s   0B             0B              1
curl \
  --request GET 'https://<dataplane-api-url>/v1/monitoring/kafka/connections' \
  --header "Authorization: Bearer $ACCESS_TOKEN" \
  --data '{
    "filter": "",
    "order_by": "recent_request_statistics.produce_bytes desc"
  }'
Show example API response 
{
  "connections": [
    {
      "node_id": 0,
      "shard_id": 0,
      "uid": "b20601a3-624c-4a8c-ab88-717643f01d56",
      "state": "KAFKA_CONNECTION_STATE_OPEN",
      "open_time": "2025-10-15T14:15:15.755065000Z",
      "close_time": "1970-01-01T00:00:00.000000000Z",
      "authentication_info": {
        "state": "AUTHENTICATION_STATE_UNAUTHENTICATED",
        "mechanism": "AUTHENTICATION_MECHANISM_UNSPECIFIED",
        "user_principal": ""
      },
      "listener_name": "",
      "tls_info": {
        "enabled": false
      },
      "source": {
        "ip_address": "127.0.0.1",
        "port": 55012
      },
      "client_id": "perf-producer-client",
      "client_software_name": "apache-kafka-java",
      "client_software_version": "3.9.0",
      "transactional_id": "my-tx-id",
      "group_id": "",
      "group_instance_id": "",
      "group_member_id": "",
      "api_versions": {
        "18": 4,
        "22": 3,
        "3": 12,
        "24": 3,
        "0": 7
      },
      "idle_duration": "0s",
      "in_flight_requests": {
        "sampled_in_flight_requests": [
          {
            "api_key": 0,
            "in_flight_duration": "0.000406892s"
          }
        ],
        "has_more_requests": false
      },
      "total_request_statistics": {
        "produce_bytes": "78927173",
        "fetch_bytes": "0",
        "request_count": "4853",
        "produce_batch_count": "4849"
      },
      "recent_request_statistics": {
        "produce_bytes": "78927173",
        "fetch_bytes": "0",
        "request_count": "4853",
        "produce_batch_count": "4849"
      }
    },
    ...
  ],
  "total_size": "9"
}

To view connections for a specific client, you can use a filter expression:

  • rpk

  • Data Plane API

rpk cluster connections list --client-id="perf-producer-client"
UID                                   STATE  USER             CLIENT-ID             IP:PORT           NODE  SHARD  OPEN-TIME  IDLE          PROD-TPUT/SEC  FETCH-TPUT/SEC  REQS/MIN
b41584f3-2662-4185-a4b8-0d8510f5c780  OPEN   UNAUTHENTICATED  perf-producer-client  127.0.0.1:55002   0     0      8s         7.743592270s  0B             0B              1
b20601a3-624c-4a8c-ab88-717643f01d56  OPEN   UNAUTHENTICATED  perf-producer-client  127.0.0.1:55012   0     0      9s         0s            78.9MB         0B              292

The USER field in the connection list shows the authenticated principal. Unauthenticated connections show UNAUTHENTICATED, which corresponds to an empty user principal (user="") in quota configurations, not user=<default>.

curl \
  --request GET 'https://<dataplane-api-url>/v1/monitoring/kafka/connections' \
  --header "Authorization: Bearer $ACCESS_TOKEN" \
  --data '{
    "filter": "client_id = \"perf-producer-client\""
  }'
Show example API response 
{
  "connections": [
    {
      "node_id": 0,
      "shard_id": 0,
      "uid": "b41584f3-2662-4185-a4b8-0d8510f5c780",
      "state": "KAFKA_CONNECTION_STATE_OPEN",
      "open_time": "2025-10-15T14:15:15.219538000Z",
      "close_time": "1970-01-01T00:00:00.000000000Z",
      "authentication_info": {
        "state": "AUTHENTICATION_STATE_UNAUTHENTICATED",
        "mechanism": "AUTHENTICATION_MECHANISM_UNSPECIFIED",
        "user_principal": ""
      },
      "listener_name": "",
      "tls_info": {
        "enabled": false
      },
      "source": {
        "ip_address": "127.0.0.1",
        "port": 55002
      },
      "client_id": "perf-producer-client",
      "client_software_name": "apache-kafka-java",
      "client_software_version": "3.9.0",
      "transactional_id": "",
      "group_id": "",
      "group_instance_id": "",
      "group_member_id": "",
      "api_versions": {
        "18": 4,
        "3": 12,
        "10": 4
      },
      "idle_duration": "7.743592270s",
      "in_flight_requests": {
        "sampled_in_flight_requests": [],
        "has_more_requests": false
      },
      "total_request_statistics": {
        "produce_bytes": "0",
        "fetch_bytes": "0",
        "request_count": "3",
        "produce_batch_count": "0"
      },
      "recent_request_statistics": {
        "produce_bytes": "0",
        "fetch_bytes": "0",
        "request_count": "3",
        "produce_batch_count": "0"
      }
    },
    ...
  ],
  "total_size": "2"
}

The user principal field in the connection list shows the authenticated principal. Unauthenticated connections show AUTHENTICATION_STATE_UNAUTHENTICATED, which corresponds to an empty user principal (user="") in quota configurations, not user=<default>.

To view connections for a specific authenticated user:

rpk cluster connections list --user alice

This shows all connections from user alice, useful for monitoring clients that are subject to user-based quotas.

Broker-wide throughput limits

Broker-wide throughput limits account for all Kafka API traffic going into or out of the broker, as data is produced to or consumed from a topic. The limit values represent the allowed rate of data in bytes per second passing through in each direction. Redpanda also provides administrators the ability to exclude clients from throughput throttling and to fine-tune which Kafka request types are subject to throttling limits.

Client throughput limits

Redpanda provides configurable throughput quotas for individual clients or authenticated users. Quotas are managed through the Kafka-compatible AlterClientQuotas and DescribeClientQuotas APIs, accessible with rpk, Redpanda Console, or Kafka client libraries.

Redpanda supports two types of client throughput quotas:

  • Client ID-based quotas: Limit throughput based on the self-declared client-id field.

  • User-based quotas: Limit throughput based on authenticated user principal. Requires authentication.

You can also combine both types for fine-grained control (for example, limiting a specific user when using a specific client application).

For conceptual information about quota types, entity hierarchy, precedence rules, and how Redpanda tracks and enforces quotas through throttling, see About Client Throughput Quotas.

Set user-based quotas

User-based quotas require authentication to be enabled. To set up authentication, see Authentication.

Quota for a specific user

To limit throughput for a specific authenticated user across all clients:

rpk cluster quotas alter --add producer_byte_rate=2000000 --name user=alice

This limits user alice to 2 MB/s for produce requests regardless of the client ID used.

To view quotas for a user:

rpk cluster quotas describe --name user=alice

Expected output:

user=alice
	producer_byte_rate=2000000

Default quota for all users

To set a fallback quota for any user without a more specific quota:

rpk cluster quotas alter --add consumer_byte_rate=5000000 --default user

This applies a 5 MB/s fetch quota to all authenticated users who don’t have a more specific quota configured.

Remove a user quota

To remove a quota for a specific user:

rpk cluster quotas alter --delete consumer_byte_rate --name user=alice

To remove all quotas for a user:

rpk cluster quotas delete --name user=alice

Set client ID-based quotas

Client ID-based quotas apply to all users using a specific client ID. These quotas do not require authentication. Because the client ID is self-declared, client ID-based quotas are not suitable for guaranteeing isolation between tenants.

For multi-tenant environments, Redpanda recommends user-based quotas for per-tenant isolation.

Individual client ID throughput limit

The following sections show how to manage throughput with rpk. You can also manage throughput with the Redpanda Cloud Data Plane API.

To view current throughput quotas set through the Kafka API, run rpk cluster quotas describe.

For example, to see the quotas for client ID consumer-1:

rpk cluster quotas describe --name client-id=consumer-1
client-id=consumer-1
	producer_byte_rate=140000

To set a throughput quota for a single client, use the rpk cluster quotas alter command.

rpk cluster quotas alter --add consumer_byte_rate=200000 --name client-id=consumer-1
ENTITY                STATUS
client-id=consumer-1  OK

Group of clients throughput limit

Alternatively, you can view or configure throughput quotas for a group of clients based on a match on client ID prefix. The following example sets the consumer_byte_rate quota to client IDs prefixed with consumer-:

rpk cluster quotas alter --add consumer_byte_rate=200000 --name client-id-prefix=consumer-
A client-id-prefix quota group is not related to Kafka consumer groups. The client ID is an application-defined identifier sent with every request. Client libraries typically default to their own name (such as kgo, rdkafka, sarama, or perf-producer-client), but applications can set it using the client.id configuration property. This makes prefix-based quotas useful for grouping related applications (for example, inventory-service- to match inventory-service-1, inventory-service-2, etc.).

Default client throughput limit

You can apply default throughput limits to clients. Redpanda applies the default limits if no quotas are configured for a specific client ID or prefix.

To specify a produce quota of 1 GB/s through the Kafka API (applies across all produce requests to a single broker), run:

rpk cluster quotas alter --default client-id --add producer_byte_rate=1000000000

Set combined user and client quotas

You can set quotas for specific (user, client ID) combinations for fine-grained control.

User with specific client

To limit a specific user when using a specific client:

rpk cluster quotas alter --add consumer_byte_rate=1000000 --name user=alice --name client-id=consumer-1

User alice using client-id=consumer-1 is limited to a 1 MB/s fetch rate. The same user with a different client ID would use a different quota (or fall back to less specific matches).

To view combined quotas:

rpk cluster quotas describe --name user=alice --name client-id=consumer-1

User with client prefix

To set a shared quota for a user across multiple clients matching a prefix:

rpk cluster quotas alter --add producer_byte_rate=3000000 --name user=bob --name client-id-prefix=app-

All clients used by user bob with a client ID starting with app- share a combined 3 MB/s produce quota.

Default user with specific client

To set a quota for a specific client across all users:

rpk cluster quotas alter --add producer_byte_rate=500000 --default user --name client-id=payment-processor

Any user using client-id=payment-processor is limited to a 500 KB/s produce rate, unless they have a more specific quota configured.

Bulk manage client throughput limits

To more easily manage multiple quotas, you can use the cluster quotas describe and cluster quotas import commands to do a bulk export and update.

For example, to export all client quotas in JSON format:

rpk cluster quotas describe --format json

rpk cluster quotas import accepts the output string from rpk cluster quotas describe --format <json/yaml>:

rpk cluster quotas import --from '{"quotas":[{"entity":[{"name":"analytics-consumer","type":"client-id"}],"values":[{"key":"consumer_byte_rate","values":"10000000"}]},{"entity":[{"name":"analytics-","type":"client-id-prefix"}],"values":[{"key":"producer_byte_rate","values":"10000000"},{"key":"consumer_byte_rate","values":"5000000"}]}]}'

You can also save the JSON or YAML output to a file and pass the file path in the --from flag.

View throughput limits in Redpanda Cloud

You can also use Redpanda Cloud to view enforced limits. In the side menu, go to Quotas.

Monitor client throughput

The following metrics provide insights into client throughput quota usage:

To identify which clients are actively connected and generating traffic, see View connected client details.

Quota metrics use the redpanda_quota_rule label to identify which quota was applied to a request. The label distinguishes between different entity types (user, client, or combinations). See the label values in redpanda_kafka_quotas_client_quota_throughput.

See also