dotnet-cqrs/docs/event-streaming/retention-policies/README.md

Retention Policies

Automatic event cleanup based on age, size, or count limits.

Overview

Retention policies automatically delete old events from streams to manage storage costs and compliance requirements. Policies can be based on event age, stream size, or event count.

Key Features:

• ✅ Time-Based - Delete events older than N days
• ✅ Size-Based - Keep only last N events
• ✅ Wildcard Policies - Default policy for all streams
• ✅ Cleanup Windows - Run during specific hours
• ✅ Per-Stream Config - Override defaults per stream

Quick Start

using Svrnty.CQRS.Events.PostgreSQL;

var builder = WebApplication.CreateBuilder(args);

// Enable retention policies
builder.Services.AddPostgresRetentionPolicies(options =>
{
    options.Enabled = true;
    options.CleanupInterval = TimeSpan.FromHours(1);
    options.UseCleanupWindow = true;
    options.CleanupWindowStart = TimeSpan.FromHours(2);   // 2 AM UTC
    options.CleanupWindowEnd = TimeSpan.FromHours(6);     // 6 AM UTC
});

var app = builder.Build();
app.Run();

Policy Types

Time-Based Retention

Delete events older than specified age:

var policy = new RetentionPolicyConfig
{
    StreamName = "orders",
    MaxAge = TimeSpan.FromDays(90),  // Keep 90 days
    Enabled = true
};

await _policyStore.SetPolicyAsync(policy);

Size-Based Retention

Keep only the last N events:

var policy = new RetentionPolicyConfig
{
    StreamName = "analytics",
    MaxEventCount = 1_000_000,  // Keep last 1 million events
    Enabled = true
};

await _policyStore.SetPolicyAsync(policy);

Combined Retention

Apply both time and size limits:

var policy = new RetentionPolicyConfig
{
    StreamName = "logs",
    MaxAge = TimeSpan.FromDays(7),
    MaxEventCount = 500_000,
    Enabled = true
};

await _policyStore.SetPolicyAsync(policy);

Events deleted if EITHER condition met.

Wildcard Policy

Default policy for all streams:

var defaultPolicy = new RetentionPolicyConfig
{
    StreamName = "*",  // Applies to all streams
    MaxAge = TimeSpan.FromDays(365),
    Enabled = true
};

await _policyStore.SetPolicyAsync(defaultPolicy);

Stream-specific policies override wildcard policy.

Features

Time-Based Retention

Delete events older than specified age with MaxAge configuration.

Size-Based Retention

Limit stream size with MaxEventCount configuration.

Cleanup Windows

Schedule retention cleanup during specific time windows.

Wildcard Policies

Set default retention policies for all streams.

Configuration

public class RetentionServiceOptions
{
    public bool Enabled { get; set; } = true;
    public TimeSpan CleanupInterval { get; set; } = TimeSpan.FromHours(1);
    public bool UseCleanupWindow { get; set; } = false;
    public TimeSpan CleanupWindowStart { get; set; } = TimeSpan.FromHours(2);
    public TimeSpan CleanupWindowEnd { get; set; } = TimeSpan.FromHours(6);
}

Best Practices

✅ DO

• Set appropriate retention periods
• Use cleanup windows for off-peak hours
• Monitor cleanup statistics
• Test policies on non-production first
• Document compliance requirements

❌ DON'T

• Don't delete critical audit logs prematurely
• Don't run cleanup during peak hours
• Don't set very short retention (< 7 days)
• Don't forget backup strategy
• Don't ignore cleanup errors

See Also

• Event Streaming Overview
• PostgreSQL Storage
• Stream Configuration