1.7 KiB
1.7 KiB
Rate Limiting
Control replay speed to avoid overwhelming downstream systems.
Configuration
await foreach (var @event in replayService.ReplayFromOffsetAsync(
"orders",
startOffset: 0,
options: new ReplayOptions
{
MaxEventsPerSecond = 1000 // Limit to 1000 events/sec
}))
{
await ProcessEventAsync(@event);
}
Token Bucket Algorithm
The framework uses a token bucket algorithm for smooth rate limiting:
Bucket capacity: MaxEventsPerSecond
Refill rate: MaxEventsPerSecond tokens/second
For each event:
1. Wait for available token
2. Consume token
3. Process event
Use Cases
Gradual Replay
// Slow replay to avoid spike in database load
options.MaxEventsPerSecond = 100; // 100 events/sec
Fast Replay (Off-Peak)
// Fast replay during off-peak hours
if (DateTime.UtcNow.Hour >= 2 && DateTime.UtcNow.Hour <= 6)
{
options.MaxEventsPerSecond = 10000; // 10k events/sec
}
else
{
options.MaxEventsPerSecond = 1000; // 1k events/sec during peak
}
Unlimited (Maximum Speed)
// No rate limiting - replay as fast as possible
options.MaxEventsPerSecond = null;
Monitoring
long eventsProcessed = 0;
var startTime = DateTime.UtcNow;
await foreach (var @event in replayService.ReplayFromOffsetAsync(...))
{
await ProcessEventAsync(@event);
eventsProcessed++;
if (eventsProcessed % 1000 == 0)
{
var elapsed = DateTime.UtcNow - startTime;
var rate = eventsProcessed / elapsed.TotalSeconds;
Console.WriteLine($"{rate:F0} events/sec");
}
}