Class RateLimiter
- Direct Known Subclasses:
SmoothRateLimiter
acquire() blocks if necessary until a permit is available, and then takes it. Once
acquired, permits need not be released.
RateLimiter is safe for concurrent use: It will restrict the total rate of calls from
all threads. Note, however, that it does not guarantee fairness.
Rate limiters are often used to restrict the rate at which some physical or logical resource
is accessed. This is in contrast to Semaphore which restricts the
number of concurrent accesses instead of the rate (note though that concurrency and rate are
closely related, e.g. see Little's
Law).
A RateLimiter is defined primarily by the rate at which permits are issued. Absent
additional configuration, permits will be distributed at a fixed rate, defined in terms of
permits per second. Permits will be distributed smoothly, with the delay between individual
permits being adjusted to ensure that the configured rate is maintained.
It is possible to configure a RateLimiter to have a warmup period during which time
the permits issued each second steadily increases until it hits the stable rate.
As an example, imagine that we have a list of tasks to execute, but we don't want to submit more than 2 per second:
final RateLimiter rateLimiter = RateLimiter.create(2.0); // rate is "2 permits per second"
void submitTasks(List<Runnable> tasks, Executor executor) {
for (Runnable task : tasks) {
rateLimiter.acquire(); // may wait
executor.execute(task);
}
}
As another example, imagine that we produce a stream of data, and we want to cap it at 5kb per second. This could be accomplished by requiring a permit per byte, and specifying a rate of 5000 permits per second:
final RateLimiter rateLimiter = RateLimiter.create(5000.0); // rate = 5000 permits per second
void submitPacket(byte[] packet) {
rateLimiter.acquire(packet.length);
networkService.send(packet);
}
It is important to note that the number of permits requested never affects the
throttling of the request itself (an invocation to acquire(1) and an invocation to
acquire(1000) will result in exactly the same throttling, if any), but it affects the throttling
of the next request. I.e., if an expensive task arrives at an idle RateLimiter, it will be
granted immediately, but it is the next request that will experience extra throttling,
thus paying for the cost of the expensive task.
- Since:
- 13.0
-
Nested Class Summary
Nested Classes -
Field Summary
FieldsModifier and TypeFieldDescriptionprivate Objectprivate final RateLimiter.SleepingStopwatchThe underlying timer; used both to measure elapsed time and sleep as necessary. -
Constructor Summary
Constructors -
Method Summary
Modifier and TypeMethodDescriptiondoubleacquire()Acquires a single permit from thisRateLimiter, blocking until the request can be granted.doubleacquire(int permits) Acquires the given number of permits from thisRateLimiter, blocking until the request can be granted.private booleancanAcquire(long nowMicros, long timeoutMicros) private static voidcheckPermits(int permits) static RateLimitercreate(double permitsPerSecond) Creates aRateLimiterwith the specified stable throughput, given as "permits per second" (commonly referred to as QPS, queries per second).static RateLimiterCreates aRateLimiterwith the specified stable throughput, given as "permits per second" (commonly referred to as QPS, queries per second), and a warmup period, during which theRateLimitersmoothly ramps up its rate, until it reaches its maximum rate at the end of the period (as long as there are enough requests to saturate it).(package private) static RateLimitercreate(double permitsPerSecond, long warmupPeriod, TimeUnit unit, double coldFactor, RateLimiter.SleepingStopwatch stopwatch) (package private) static RateLimitercreate(double permitsPerSecond, RateLimiter.SleepingStopwatch stopwatch) static RateLimiterCreates aRateLimiterwith the specified stable throughput, given as "permits per second" (commonly referred to as QPS, queries per second), and a warmup period, during which theRateLimitersmoothly ramps up its rate, until it reaches its maximum rate at the end of the period (as long as there are enough requests to saturate it).(package private) abstract double(package private) abstract voiddoSetRate(double permitsPerSecond, long nowMicros) final doublegetRate()Returns the stable rate (aspermits per seconds) with which thisRateLimiteris configured with.private Objectmutex()(package private) abstract longqueryEarliestAvailable(long nowMicros) Returns the earliest time that permits are available (with one caveat).(package private) final longreserve(int permits) Reserves the given number of permits from thisRateLimiterfor future use, returning the number of microseconds until the reservation can be consumed.(package private) final longreserveAndGetWaitLength(int permits, long nowMicros) Reserves next ticket and returns the wait time that the caller must wait for.(package private) abstract longreserveEarliestAvailable(int permits, long nowMicros) Reserves the requested number of permits and returns the time that those permits can be used (with one caveat).final voidsetRate(double permitsPerSecond) Updates the stable rate of thisRateLimiter, that is, thepermitsPerSecondargument provided in the factory method that constructed theRateLimiter.toString()booleanAcquires a permit from thisRateLimiterif it can be acquired immediately without delay.booleantryAcquire(int permits) Acquires permits from thisRateLimiterif it can be acquired immediately without delay.booleantryAcquire(int permits, long timeout, TimeUnit unit) Acquires the given number of permits from thisRateLimiterif it can be obtained without exceeding the specifiedtimeout, or returnsfalseimmediately (without waiting) if the permits would not have been granted before the timeout expired.booleantryAcquire(int permits, Duration timeout) Acquires the given number of permits from thisRateLimiterif it can be obtained without exceeding the specifiedtimeout, or returnsfalseimmediately (without waiting) if the permits would not have been granted before the timeout expired.booleantryAcquire(long timeout, TimeUnit unit) Acquires a permit from thisRateLimiterif it can be obtained without exceeding the specifiedtimeout, or returnsfalseimmediately (without waiting) if the permit would not have been granted before the timeout expired.booleantryAcquire(Duration timeout) Acquires a permit from thisRateLimiterif it can be obtained without exceeding the specifiedtimeout, or returnsfalseimmediately (without waiting) if the permit would not have been granted before the timeout expired.
-
Field Details
-
stopwatch
The underlying timer; used both to measure elapsed time and sleep as necessary. A separate object to facilitate testing. -
mutexDoNotUseDirectly
-
-
Constructor Details
-
RateLimiter
RateLimiter(RateLimiter.SleepingStopwatch stopwatch)
-
-
Method Details
-
create
Creates aRateLimiterwith the specified stable throughput, given as "permits per second" (commonly referred to as QPS, queries per second).The returned
RateLimiterensures that on average no more thanpermitsPerSecondare issued during any given second, with sustained requests being smoothly spread over each second. When the incoming request rate exceedspermitsPerSecondthe rate limiter will release one permit every(1.0 / permitsPerSecond)seconds. When the rate limiter is unused, bursts of up topermitsPerSecondpermits will be allowed, with subsequent requests being smoothly limited at the stable rate ofpermitsPerSecond.- Parameters:
permitsPerSecond- the rate of the returnedRateLimiter, measured in how many permits become available per second- Throws:
IllegalArgumentException- ifpermitsPerSecondis negative or zero
-
create
-
create
Creates aRateLimiterwith the specified stable throughput, given as "permits per second" (commonly referred to as QPS, queries per second), and a warmup period, during which theRateLimitersmoothly ramps up its rate, until it reaches its maximum rate at the end of the period (as long as there are enough requests to saturate it). Similarly, if theRateLimiteris left unused for a duration ofwarmupPeriod, it will gradually return to its "cold" state, i.e. it will go through the same warming up process as when it was first created.The returned
RateLimiteris intended for cases where the resource that actually fulfills the requests (e.g., a remote server) needs "warmup" time, rather than being immediately accessed at the stable (maximum) rate.The returned
RateLimiterstarts in a "cold" state (i.e. the warmup period will follow), and if it is left unused for long enough, it will return to that state.- Parameters:
permitsPerSecond- the rate of the returnedRateLimiter, measured in how many permits become available per secondwarmupPeriod- the duration of the period where theRateLimiterramps up its rate, before reaching its stable (maximum) rate- Throws:
IllegalArgumentException- ifpermitsPerSecondis negative or zero orwarmupPeriodis negative- Since:
- 28.0
-
create
Creates aRateLimiterwith the specified stable throughput, given as "permits per second" (commonly referred to as QPS, queries per second), and a warmup period, during which theRateLimitersmoothly ramps up its rate, until it reaches its maximum rate at the end of the period (as long as there are enough requests to saturate it). Similarly, if theRateLimiteris left unused for a duration ofwarmupPeriod, it will gradually return to its "cold" state, i.e. it will go through the same warming up process as when it was first created.The returned
RateLimiteris intended for cases where the resource that actually fulfills the requests (e.g., a remote server) needs "warmup" time, rather than being immediately accessed at the stable (maximum) rate.The returned
RateLimiterstarts in a "cold" state (i.e. the warmup period will follow), and if it is left unused for long enough, it will return to that state.- Parameters:
permitsPerSecond- the rate of the returnedRateLimiter, measured in how many permits become available per secondwarmupPeriod- the duration of the period where theRateLimiterramps up its rate, before reaching its stable (maximum) rateunit- the time unit of the warmupPeriod argument- Throws:
IllegalArgumentException- ifpermitsPerSecondis negative or zero orwarmupPeriodis negative
-
create
static RateLimiter create(double permitsPerSecond, long warmupPeriod, TimeUnit unit, double coldFactor, RateLimiter.SleepingStopwatch stopwatch) -
mutex
-
setRate
public final void setRate(double permitsPerSecond) Updates the stable rate of thisRateLimiter, that is, thepermitsPerSecondargument provided in the factory method that constructed theRateLimiter. Currently throttled threads will not be awakened as a result of this invocation, thus they do not observe the new rate; only subsequent requests will.Note though that, since each request repays (by waiting, if necessary) the cost of the previous request, this means that the very next request after an invocation to
setRatewill not be affected by the new rate; it will pay the cost of the previous request, which is in terms of the previous rate.The behavior of the
RateLimiteris not modified in any other way, e.g. if theRateLimiterwas configured with a warmup period of 20 seconds, it still has a warmup period of 20 seconds after this method invocation.- Parameters:
permitsPerSecond- the new stable rate of thisRateLimiter- Throws:
IllegalArgumentException- ifpermitsPerSecondis negative or zero
-
doSetRate
abstract void doSetRate(double permitsPerSecond, long nowMicros) -
getRate
public final double getRate()Returns the stable rate (aspermits per seconds) with which thisRateLimiteris configured with. The initial value of this is the same as thepermitsPerSecondargument passed in the factory method that produced thisRateLimiter, and it is only updated after invocations to setRate(double). -
doGetRate
abstract double doGetRate() -
acquire
public double acquire()Acquires a single permit from thisRateLimiter, blocking until the request can be granted. Tells the amount of time slept, if any.This method is equivalent to
acquire(1).- Returns:
- time spent sleeping to enforce rate, in seconds; 0.0 if not rate-limited
- Since:
- 16.0 (present in 13.0 with
voidreturn type})
-
acquire
public double acquire(int permits) Acquires the given number of permits from thisRateLimiter, blocking until the request can be granted. Tells the amount of time slept, if any.- Parameters:
permits- the number of permits to acquire- Returns:
- time spent sleeping to enforce rate, in seconds; 0.0 if not rate-limited
- Throws:
IllegalArgumentException- if the requested number of permits is negative or zero- Since:
- 16.0 (present in 13.0 with
voidreturn type})
-
reserve
final long reserve(int permits) Reserves the given number of permits from thisRateLimiterfor future use, returning the number of microseconds until the reservation can be consumed.- Returns:
- time in microseconds to wait until the resource can be acquired, never negative
-
tryAcquire
Acquires a permit from thisRateLimiterif it can be obtained without exceeding the specifiedtimeout, or returnsfalseimmediately (without waiting) if the permit would not have been granted before the timeout expired.This method is equivalent to
tryAcquire(1, timeout).- Parameters:
timeout- the maximum time to wait for the permit. Negative values are treated as zero.- Returns:
trueif the permit was acquired,falseotherwise- Throws:
IllegalArgumentException- if the requested number of permits is negative or zero- Since:
- 28.0
-
tryAcquire
Acquires a permit from thisRateLimiterif it can be obtained without exceeding the specifiedtimeout, or returnsfalseimmediately (without waiting) if the permit would not have been granted before the timeout expired.This method is equivalent to
tryAcquire(1, timeout, unit).- Parameters:
timeout- the maximum time to wait for the permit. Negative values are treated as zero.unit- the time unit of the timeout argument- Returns:
trueif the permit was acquired,falseotherwise- Throws:
IllegalArgumentException- if the requested number of permits is negative or zero
-
tryAcquire
public boolean tryAcquire(int permits) Acquires permits from thisRateLimiterif it can be acquired immediately without delay.This method is equivalent to
tryAcquire(permits, 0, anyUnit).- Parameters:
permits- the number of permits to acquire- Returns:
trueif the permits were acquired,falseotherwise- Throws:
IllegalArgumentException- if the requested number of permits is negative or zero- Since:
- 14.0
-
tryAcquire
public boolean tryAcquire()Acquires a permit from thisRateLimiterif it can be acquired immediately without delay.This method is equivalent to
tryAcquire(1).- Returns:
trueif the permit was acquired,falseotherwise- Since:
- 14.0
-
tryAcquire
Acquires the given number of permits from thisRateLimiterif it can be obtained without exceeding the specifiedtimeout, or returnsfalseimmediately (without waiting) if the permits would not have been granted before the timeout expired.- Parameters:
permits- the number of permits to acquiretimeout- the maximum time to wait for the permits. Negative values are treated as zero.- Returns:
trueif the permits were acquired,falseotherwise- Throws:
IllegalArgumentException- if the requested number of permits is negative or zero- Since:
- 28.0
-
tryAcquire
Acquires the given number of permits from thisRateLimiterif it can be obtained without exceeding the specifiedtimeout, or returnsfalseimmediately (without waiting) if the permits would not have been granted before the timeout expired.- Parameters:
permits- the number of permits to acquiretimeout- the maximum time to wait for the permits. Negative values are treated as zero.unit- the time unit of the timeout argument- Returns:
trueif the permits were acquired,falseotherwise- Throws:
IllegalArgumentException- if the requested number of permits is negative or zero
-
canAcquire
private boolean canAcquire(long nowMicros, long timeoutMicros) -
reserveAndGetWaitLength
final long reserveAndGetWaitLength(int permits, long nowMicros) Reserves next ticket and returns the wait time that the caller must wait for.- Returns:
- the required wait time, never negative
-
queryEarliestAvailable
abstract long queryEarliestAvailable(long nowMicros) Returns the earliest time that permits are available (with one caveat).- Returns:
- the time that permits are available, or, if permits are available immediately, an arbitrary past or present time
-
reserveEarliestAvailable
abstract long reserveEarliestAvailable(int permits, long nowMicros) Reserves the requested number of permits and returns the time that those permits can be used (with one caveat).- Returns:
- the time that the permits may be used, or, if the permits may be used immediately, an arbitrary past or present time
-
toString
-
checkPermits
private static void checkPermits(int permits)
-