4.6 Rate-limiting

Rate-limiting is a technique that controls the number of requests processed by a service in a second. Pound contains mechanism to implement rate-limiting using token bucket filter algorithm.

The token bucket filter decides whether a request can be accepted based on the presence of abstract entities called tokens in a container called bucket. Each token represents an ability to process one incoming request. The algorithm works as follows:

• Tokens are added to the bucket at a constant rate of r tokens per second.
• A bucket can host at most M tokens. Any tokens arriving when the bucket is full are discarded.
• If a request arrives when the bucket holds some tokens, one token is removed and the request is accepted.
• If the bucket is empty, the request is not accepted.

This algorithm makes sure requests are accepted at a constant rate or r requests per seconds with bursts of up to M requests. Such bursts occur when no requests arrive for M/r or more seconds.

The special conditional statement TBF provides an interface to this algorithm. The statement takes three arguments: key, a string identifying the bucket, and the values of r and M. It evaluates to true if the algorithm accepts the request and to false otherwise. Placing that conditional in a Service section ensures that requests will be processed at the configured rate. For each such service section, there is normally another one to be used for requests that arrive faster than allowed. The selection criteria for the two sections differ only by the absence of TBF conditional in the second section. Consider the following example:

Service "git"
   Host "example.org"
   Path -re "^/git/([^/]+).*"
   TBF "$1" 5 10
   Backend
       Address 192.0.2.10
       Port 80
   End
End
Service "block"
   Host "example.org"
   Path -re "^/git/([^/]+).*"
   Error 429
End

This configuration limits request rate to any path beginning with /git/ (presumably, a git repository) to 5 requests per second with bursts of up to 10 requests. The bucket to use is identified by the first directory component after the /git/ prefix, so that rates are computed for each repository individually. Requests that conform to that policy are routed to the backend at ‘192.0.2.10:80’. This is configured by the service section ‘git’.

The requests that arrive quicker than permitted by that service will be served by service ‘block’, which will respond to them with HTTP status 429 (‘Too Many Requests’). See Error, for a discussion of Error backend.

To compute rates for each connection individually, use the remoteip accessor (see Request Accessor Interpretation). In that case the corresponding fragment from the example above becomes:

   Path -re "^/git/([^/]+).*"
   TBF "%[remoteip 0]-$1" 5 10

If the rate limit should apply to several services, the recommended course of action is to declare such error service first, using a common subset of conditions for all services it is intended to cover and a negation of TBF. E.g.:

Service "rate-limit"
   Host "example.org"
   Path -re "^/git/([^/]+).*"
   Not TBF "$1" 5 10
   Error 429
End

This service section shall be followed by the services it protects.