155 lines
3.6 KiB
Markdown
155 lines
3.6 KiB
Markdown
# Retention Policies
|
|
|
|
Automatic event cleanup based on age, size, or count limits.
|
|
|
|
## Overview
|
|
|
|
Retention policies automatically delete old events from streams to manage storage costs and compliance requirements. Policies can be based on event age, stream size, or event count.
|
|
|
|
**Key Features:**
|
|
|
|
- ✅ **Time-Based** - Delete events older than N days
|
|
- ✅ **Size-Based** - Keep only last N events
|
|
- ✅ **Wildcard Policies** - Default policy for all streams
|
|
- ✅ **Cleanup Windows** - Run during specific hours
|
|
- ✅ **Per-Stream Config** - Override defaults per stream
|
|
|
|
## Quick Start
|
|
|
|
```csharp
|
|
using Svrnty.CQRS.Events.PostgreSQL;
|
|
|
|
var builder = WebApplication.CreateBuilder(args);
|
|
|
|
// Enable retention policies
|
|
builder.Services.AddPostgresRetentionPolicies(options =>
|
|
{
|
|
options.Enabled = true;
|
|
options.CleanupInterval = TimeSpan.FromHours(1);
|
|
options.UseCleanupWindow = true;
|
|
options.CleanupWindowStart = TimeSpan.FromHours(2); // 2 AM UTC
|
|
options.CleanupWindowEnd = TimeSpan.FromHours(6); // 6 AM UTC
|
|
});
|
|
|
|
var app = builder.Build();
|
|
app.Run();
|
|
```
|
|
|
|
## Policy Types
|
|
|
|
### Time-Based Retention
|
|
|
|
Delete events older than specified age:
|
|
|
|
```csharp
|
|
var policy = new RetentionPolicyConfig
|
|
{
|
|
StreamName = "orders",
|
|
MaxAge = TimeSpan.FromDays(90), // Keep 90 days
|
|
Enabled = true
|
|
};
|
|
|
|
await _policyStore.SetPolicyAsync(policy);
|
|
```
|
|
|
|
### Size-Based Retention
|
|
|
|
Keep only the last N events:
|
|
|
|
```csharp
|
|
var policy = new RetentionPolicyConfig
|
|
{
|
|
StreamName = "analytics",
|
|
MaxEventCount = 1_000_000, // Keep last 1 million events
|
|
Enabled = true
|
|
};
|
|
|
|
await _policyStore.SetPolicyAsync(policy);
|
|
```
|
|
|
|
### Combined Retention
|
|
|
|
Apply both time and size limits:
|
|
|
|
```csharp
|
|
var policy = new RetentionPolicyConfig
|
|
{
|
|
StreamName = "logs",
|
|
MaxAge = TimeSpan.FromDays(7),
|
|
MaxEventCount = 500_000,
|
|
Enabled = true
|
|
};
|
|
|
|
await _policyStore.SetPolicyAsync(policy);
|
|
```
|
|
|
|
**Events deleted if EITHER condition met.**
|
|
|
|
### Wildcard Policy
|
|
|
|
Default policy for all streams:
|
|
|
|
```csharp
|
|
var defaultPolicy = new RetentionPolicyConfig
|
|
{
|
|
StreamName = "*", // Applies to all streams
|
|
MaxAge = TimeSpan.FromDays(365),
|
|
Enabled = true
|
|
};
|
|
|
|
await _policyStore.SetPolicyAsync(defaultPolicy);
|
|
```
|
|
|
|
Stream-specific policies override wildcard policy.
|
|
|
|
## Features
|
|
|
|
### [Time-Based Retention](time-based-retention.md)
|
|
Delete events older than specified age with MaxAge configuration.
|
|
|
|
### [Size-Based Retention](size-based-retention.md)
|
|
Limit stream size with MaxEventCount configuration.
|
|
|
|
### [Cleanup Windows](cleanup-windows.md)
|
|
Schedule retention cleanup during specific time windows.
|
|
|
|
### [Wildcard Policies](wildcard-policies.md)
|
|
Set default retention policies for all streams.
|
|
|
|
## Configuration
|
|
|
|
```csharp
|
|
public class RetentionServiceOptions
|
|
{
|
|
public bool Enabled { get; set; } = true;
|
|
public TimeSpan CleanupInterval { get; set; } = TimeSpan.FromHours(1);
|
|
public bool UseCleanupWindow { get; set; } = false;
|
|
public TimeSpan CleanupWindowStart { get; set; } = TimeSpan.FromHours(2);
|
|
public TimeSpan CleanupWindowEnd { get; set; } = TimeSpan.FromHours(6);
|
|
}
|
|
```
|
|
|
|
## Best Practices
|
|
|
|
### ✅ DO
|
|
|
|
- Set appropriate retention periods
|
|
- Use cleanup windows for off-peak hours
|
|
- Monitor cleanup statistics
|
|
- Test policies on non-production first
|
|
- Document compliance requirements
|
|
|
|
### ❌ DON'T
|
|
|
|
- Don't delete critical audit logs prematurely
|
|
- Don't run cleanup during peak hours
|
|
- Don't set very short retention (< 7 days)
|
|
- Don't forget backup strategy
|
|
- Don't ignore cleanup errors
|
|
|
|
## See Also
|
|
|
|
- [Event Streaming Overview](../README.md)
|
|
- [PostgreSQL Storage](../storage/postgresql-storage.md)
|
|
- [Stream Configuration](../stream-configuration/retention-config.md)
|