dotnet-cqrs/roadmap-2026/compression.md

gRPC Compression Support - Implementation Plan

Status: Planned Target: Q1 2026 Priority: High Complexity: Medium

Overview

Implement intelligent message compression for gRPC services with automatic threshold detection, per-handler control via attributes, and configurable compression levels. This will reduce bandwidth costs, improve performance over slow networks, and provide fine-grained control for developers.

Goals

1. Automatic Compression: Compress messages larger than a configurable threshold (default: 1KB)
2. Attribute-Based Control: Allow developers to opt-out ([NoCompression]) or opt-in ([CompressAlways]) per handler
3. Configurable Levels: Support Fastest, Optimal, and SmallestSize compression levels
4. Smart Defaults: Don't compress small messages or already-compressed data
5. Backward Compatible: No breaking changes to existing APIs
6. Zero Configuration: Works out-of-the-box with sensible defaults

Technical Design

Architecture Components

┌─────────────────────────────────────────────────────────┐
│                   gRPC Service Layer                     │
│  (Generated CommandServiceImpl/QueryServiceImpl)        │
└─────────────────┬───────────────────────────────────────┘
                  │
                  ▼
┌─────────────────────────────────────────────────────────┐
│            Compression Interceptor                       │
│  - Automatically registered via UseCompression()        │
│  - Checks handler metadata for compression attributes   │
│  - Measures message size                                │
│  - Applies compression based on policy                  │
└─────────────────┬───────────────────────────────────────┘
                  │
                  ▼
┌─────────────────────────────────────────────────────────┐
│                Handler Execution                         │
│  ICommandHandler / IQueryHandler                        │
└─────────────────────────────────────────────────────────┘

New Abstractions

1. Compression Attributes

File: Svrnty.CQRS.Grpc.Abstractions/Attributes/CompressionAttribute.cs

namespace Svrnty.CQRS.Grpc.Abstractions;

[AttributeUsage(AttributeTargets.Class, AllowMultiple = false, Inherited = true)]
public sealed class NoCompressionAttribute : Attribute
{
}

[AttributeUsage(AttributeTargets.Class, AllowMultiple = false, Inherited = true)]
public sealed class CompressAlwaysAttribute : Attribute
{
}

[AttributeUsage(AttributeTargets.Class, AllowMultiple = false, Inherited = true)]
public sealed class CompressionLevelAttribute : Attribute
{
    public CompressionLevel Level { get; }

    public CompressionLevelAttribute(CompressionLevel level)
    {
        Level = level;
    }
}

public enum CompressionLevel
{
    Fastest = 0,
    Optimal = 1,
    SmallestSize = 2
}

2. Compression Configuration

File: Svrnty.CQRS.Grpc/Configuration/GrpcCompressionOptions.cs

namespace Svrnty.CQRS.Grpc;

public class GrpcCompressionOptions
{
    public bool EnableAutomaticCompression { get; set; } = true;

    public int CompressionThresholdBytes { get; set; } = 1024;

    public CompressionLevel DefaultCompressionLevel { get; set; } = CompressionLevel.Optimal;

    public string CompressionAlgorithm { get; set; } = "gzip";

    public bool EnableCompressionMetrics { get; set; } = false;
}

3. Compression Metadata

File: Svrnty.CQRS.Grpc/Metadata/ICompressionMetadata.cs

namespace Svrnty.CQRS.Grpc;

public interface ICompressionMetadata
{
    Type HandlerType { get; }
    CompressionPolicy Policy { get; }
    CompressionLevel? CustomLevel { get; }
}

public enum CompressionPolicy
{
    Automatic,
    Never,
    Always
}

internal class CompressionMetadata : ICompressionMetadata
{
    public Type HandlerType { get; init; }
    public CompressionPolicy Policy { get; init; }
    public CompressionLevel? CustomLevel { get; init; }
}

Implementation Details

1. Metadata Discovery

File: Svrnty.CQRS.Grpc/Discovery/CompressionMetadataProvider.cs

namespace Svrnty.CQRS.Grpc;

internal class CompressionMetadataProvider
{
    private readonly Dictionary<Type, ICompressionMetadata> _cache = new();
    private readonly object _lock = new();

    public ICompressionMetadata GetMetadata(Type handlerType)
    {
        if (_cache.TryGetValue(handlerType, out var metadata))
            return metadata;

        lock (_lock)
        {
            if (_cache.TryGetValue(handlerType, out metadata))
                return metadata;

            metadata = BuildMetadata(handlerType);
            _cache[handlerType] = metadata;
            return metadata;
        }
    }

    private ICompressionMetadata BuildMetadata(Type handlerType)
    {
        var noCompression = handlerType.GetCustomAttribute<NoCompressionAttribute>();
        var compressAlways = handlerType.GetCustomAttribute<CompressAlwaysAttribute>();
        var customLevel = handlerType.GetCustomAttribute<CompressionLevelAttribute>();

        if (noCompression != null && compressAlways != null)
        {
            throw new InvalidOperationException(
                $"Handler {handlerType.Name} cannot have both [NoCompression] and [CompressAlways] attributes.");
        }

        var policy = CompressionPolicy.Automatic;
        if (noCompression != null)
            policy = CompressionPolicy.Never;
        else if (compressAlways != null)
            policy = CompressionPolicy.Always;

        return new CompressionMetadata
        {
            HandlerType = handlerType,
            Policy = policy,
            CustomLevel = customLevel?.Level
        };
    }
}

2. Compression Interceptor

File: Svrnty.CQRS.Grpc/Interceptors/CompressionInterceptor.cs

namespace Svrnty.CQRS.Grpc;

internal class CompressionInterceptor : Interceptor
{
    private readonly GrpcCompressionOptions _options;
    private readonly CompressionMetadataProvider _metadataProvider;
    private readonly ILogger<CompressionInterceptor> _logger;

    public CompressionInterceptor(
        IOptions<GrpcCompressionOptions> options,
        CompressionMetadataProvider metadataProvider,
        ILogger<CompressionInterceptor> logger)
    {
        _options = options.Value;
        _metadataProvider = metadataProvider;
        _logger = logger;
    }

    public override async Task<TResponse> UnaryServerHandler<TRequest, TResponse>(
        TRequest request,
        ServerCallContext context,
        UnaryServerMethod<TRequest, TResponse> continuation)
    {
        var response = await continuation(request, context);

        if (_options.EnableAutomaticCompression)
        {
            ApplyCompressionPolicy(context, response);
        }

        return response;
    }

    private void ApplyCompressionPolicy<TResponse>(ServerCallContext context, TResponse response)
    {
        if (!context.UserState.TryGetValue("HandlerType", out var handlerTypeObj)
            || handlerTypeObj is not Type handlerType)
        {
            ApplyAutomaticCompression(context, response);
            return;
        }

        var metadata = _metadataProvider.GetMetadata(handlerType);

        switch (metadata.Policy)
        {
            case CompressionPolicy.Never:
                break;

            case CompressionPolicy.Always:
                SetCompression(context, metadata.CustomLevel ?? _options.DefaultCompressionLevel);
                break;

            case CompressionPolicy.Automatic:
                ApplyAutomaticCompression(context, response, metadata.CustomLevel);
                break;
        }
    }

    private void ApplyAutomaticCompression<TResponse>(
        ServerCallContext context,
        TResponse response,
        CompressionLevel? customLevel = null)
    {
        var messageSize = EstimateMessageSize(response);

        if (messageSize >= _options.CompressionThresholdBytes)
        {
            SetCompression(context, customLevel ?? _options.DefaultCompressionLevel);

            if (_options.EnableCompressionMetrics)
            {
                _logger.LogDebug(
                    "Compressing response of {Size} bytes (threshold: {Threshold})",
                    messageSize, _options.CompressionThresholdBytes);
            }
        }
    }

    private void SetCompression(ServerCallContext context, CompressionLevel level)
    {
        var grpcLevel = level switch
        {
            CompressionLevel.Fastest => System.IO.Compression.CompressionLevel.Fastest,
            CompressionLevel.Optimal => System.IO.Compression.CompressionLevel.Optimal,
            CompressionLevel.SmallestSize => System.IO.Compression.CompressionLevel.SmallestSize,
            _ => System.IO.Compression.CompressionLevel.Optimal
        };

        context.WriteOptions = new WriteOptions(flags: WriteFlags.NoCompress);
        context.ResponseTrailers.Add("grpc-encoding", _options.CompressionAlgorithm);
    }

    private int EstimateMessageSize<TResponse>(TResponse response)
    {
        try
        {
            var json = System.Text.Json.JsonSerializer.Serialize(response);
            return System.Text.Encoding.UTF8.GetByteCount(json);
        }
        catch
        {
            return 0;
        }
    }
}

3. gRPC Configuration Builder

File: Svrnty.CQRS.Grpc/Configuration/GrpcOptionsBuilder.cs

namespace Svrnty.CQRS.Grpc;

public class GrpcOptionsBuilder
{
    private readonly IServiceCollection _services;
    internal bool CompressionEnabled { get; private set; }

    internal GrpcOptionsBuilder(IServiceCollection services)
    {
        _services = services;
    }

    public GrpcOptionsBuilder UseCompression(Action<GrpcCompressionOptions>? configure = null)
    {
        CompressionEnabled = true;

        _services.AddSingleton<CompressionMetadataProvider>();

        if (configure != null)
        {
            _services.Configure(configure);
        }

        _services.AddGrpc(options =>
        {
            options.Interceptors.Add<CompressionInterceptor>();
        });

        return this;
    }

    public GrpcOptionsBuilder EnableReflection()
    {
        _services.AddGrpcReflection();
        return this;
    }
}

4. CQRS Configuration Builder

File: Svrnty.CQRS/Configuration/CqrsOptionsBuilder.cs

namespace Svrnty.CQRS;

public class CqrsOptionsBuilder
{
    private readonly IServiceCollection _services;

    internal CqrsOptionsBuilder(IServiceCollection services)
    {
        _services = services;
    }

    public CqrsOptionsBuilder AddGrpc(Action<GrpcOptionsBuilder> configure)
    {
        var grpcBuilder = new GrpcOptionsBuilder(_services);
        configure(grpcBuilder);
        return this;
    }

    public CqrsOptionsBuilder AddMinimalApi()
    {
        return this;
    }
}

5. Service Registration

File: Svrnty.CQRS/ServiceCollectionExtensions.cs

namespace Svrnty.CQRS;

public static class ServiceCollectionExtensions
{
    public static IServiceCollection AddSvrntyCqrs(
        this IServiceCollection services,
        Action<CqrsOptionsBuilder>? configure = null)
    {
        services.AddDefaultCommandDiscovery();
        services.AddDefaultQueryDiscovery();

        if (configure != null)
        {
            var builder = new CqrsOptionsBuilder(services);
            configure(builder);
        }

        return services;
    }
}

6. Source Generator Updates

The source generator needs to be updated to pass handler type information to the service implementation.

File: Svrnty.CQRS.Grpc.Generators/ServiceImplementationGenerator.cs

// In the generated service implementation, add handler type to context:

public override async Task<ExecuteCommandResponse> ExecuteCommand(
    ExecuteCommandRequest request,
    ServerCallContext context)
{
    context.UserState["HandlerType"] = typeof(THandler);

    // ... rest of implementation
}

Usage Examples

Example 1: Basic Setup with Default Compression

using Svrnty.CQRS;
using Svrnty.CQRS.Grpc;

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddCommand<AddUserCommand, int, AddUserCommandHandler>();
builder.Services.AddQuery<FetchUserQuery, User, FetchUserQueryHandler>();

builder.Services.AddSvrntyCqrs(cqrs =>
{
    cqrs.AddGrpc(grpc =>
    {
        grpc.UseCompression();
        grpc.EnableReflection();
    });
});

var app = builder.Build();

app.UseSvrntyCqrs();

app.Run();

Example 2: Custom Compression Configuration

builder.Services.AddSvrntyCqrs(cqrs =>
{
    cqrs.AddGrpc(grpc =>
    {
        grpc.UseCompression(compression =>
        {
            compression.CompressionThresholdBytes = 5 * 1024;
            compression.DefaultCompressionLevel = CompressionLevel.Fastest;
            compression.EnableCompressionMetrics = true;
        });

        grpc.EnableReflection();
    });
});

Example 3: Handler with No Compression (Binary Data)

using Svrnty.CQRS.Grpc.Abstractions;

[NoCompression]
public class GetProfileImageQueryHandler : IQueryHandler<GetProfileImageQuery, byte[]>
{
    public async Task<byte[]> HandleAsync(GetProfileImageQuery query, CancellationToken ct)
    {
        return await LoadImage(query.UserId);
    }
}

Example 4: Handler with Always Compress (Large Text)

[CompressAlways]
[CompressionLevel(CompressionLevel.SmallestSize)]
public class ExportDataQueryHandler : IQueryHandler<ExportDataQuery, string>
{
    public async Task<string> HandleAsync(ExportDataQuery query, CancellationToken ct)
    {
        return await GenerateCsvExport();
    }
}

Example 5: Default Automatic Compression

public class GetUsersQueryHandler : IQueryHandler<GetUsersQuery, List<User>>
{
    public async Task<List<User>> HandleAsync(GetUsersQuery query, CancellationToken ct)
    {
        return await GetUsers();
    }
}

Example 6: Both gRPC and HTTP (Compression Only for gRPC)

builder.Services.AddSvrntyCqrs(cqrs =>
{
    cqrs.AddGrpc(grpc =>
    {
        grpc.UseCompression(compression =>
        {
            compression.CompressionThresholdBytes = 2 * 1024;
        });
        grpc.EnableReflection();
    });

    cqrs.AddMinimalApi();
});

Migration & Backward Compatibility

Breaking Changes

None. This feature is fully backward compatible.

Default Behavior

• Compression is NOT enabled by default
• Must call .UseCompression() to enable
• Once enabled, uses automatic compression with 1KB threshold
• Existing handlers work without modification

Migration Path

1. Update to new version
2. Add .UseCompression() to gRPC configuration
3. Optionally add [NoCompression] to handlers with binary data
4. Optionally add [CompressAlways] to handlers with large text
5. Optionally configure custom threshold/levels

Success Criteria

Functional Requirements

• ✅ Automatic compression for messages > 1KB (configurable)
• ✅ [NoCompression] attribute prevents compression
• ✅ [CompressAlways] attribute forces compression
• ✅ [CompressionLevel] attribute sets custom level
• ✅ Configuration via fluent API: AddSvrntyCqrs().AddGrpc().UseCompression()
• ✅ No breaking changes to existing APIs
• ✅ Interceptor automatically registered when calling UseCompression()

Performance Requirements

• ✅ Compression overhead < 5ms for 100KB messages
• ✅ 50-80% size reduction for text-heavy payloads
• ✅ No measurable impact for small messages (< 1KB)

Documentation Requirements

User Documentation

• Update README.md with compression feature in 2026 roadmap
• Update CLAUDE.md with compression configuration details
• Add compression examples to sample project

Implementation Checklist

Phase 1: Core Implementation

• Create compression attributes in Svrnty.CQRS.Grpc.Abstractions
• Create GrpcCompressionOptions configuration class
• Create GrpcOptionsBuilder with UseCompression() method
• Update CqrsOptionsBuilder to support AddGrpc()
• Implement CompressionMetadataProvider for discovering attributes
• Implement CompressionInterceptor for applying compression
• Ensure interceptor is automatically registered via UseCompression()

Phase 2: Source Generator

• Update source generator to pass handler type to context
• Set HandlerType in ServerCallContext.UserState

Phase 3: Documentation

• Update README.md with 2026 roadmap item
• Update CLAUDE.md with compression details
• Update sample project with compression examples
• Create migration guide

Phase 4: Release

• Code review and feedback
• Merge to main branch
• Release as part of next minor version
• Publish NuGet packages

Dependencies

NuGet Packages

• Grpc.AspNetCore >= 2.68.0 (already referenced)
• System.IO.Compression (part of .NET runtime)

Internal Dependencies

• Svrnty.CQRS (for configuration builders)
• Svrnty.CQRS.Grpc.Abstractions (for attributes)
• Svrnty.CQRS.Grpc (for implementation)
• Svrnty.CQRS.Grpc.Generators (source generator updates)

Open Questions

1. Message Size Estimation: Should we use protobuf serialization size or JSON estimation?

   • Proposed: Use protobuf for accuracy, fallback to JSON estimate if not available

2. HTTP Compression: Should we also support compression for Minimal API endpoints?

   • Proposed: Phase 2 - HTTP already has built-in compression via ASP.NET Core

3. Compression Metrics: What metrics should we expose?

   • Proposed: Compressed vs uncompressed size, compression ratio, compression time

4. Per-Message Override: Should clients be able to request no compression via metadata?

   • Proposed: Phase 2 - respect grpc-accept-encoding header

Future Enhancements (Beyond 2026)

1. Multiple Compression Algorithms: Support Brotli, LZ4, Snappy
2. Adaptive Compression: Machine learning to predict best compression strategy
3. Client Hints: Honor client compression preferences
4. Compression Metrics Dashboard: Visual metrics for compression effectiveness
5. HTTP Compression: Extend to Minimal API endpoints with same attribute model
6. Streaming Compression: Apply compression to streaming RPCs

---

Document Version: 1.0 Last Updated: 2025-01-08 Author: Svrnty Team