dotnet-cqrs/Svrnty.Sample/README-RABBITMQ.md

RabbitMQ Integration Example

This sample project demonstrates Phase 4: Cross-Service Communication using RabbitMQ for event streaming between microservices.

Overview

The sample shows how events emitted by command handlers are automatically published to RabbitMQ, enabling cross-service communication without any RabbitMQ-specific code in your handlers.

Architecture

┌─────────────────────┐
│  AddUserCommand     │
│     Handler         │
└──────────┬──────────┘
           │
           │ workflow.Emit(UserAddedEvent)
           │
           ▼
┌─────────────────────┐
│  UserWorkflow       │
│  (External Scope)   │
└──────────┬──────────┘
           │
           │ Auto-publish to RabbitMQ
           │
           ▼
┌─────────────────────┐      ┌─────────────────────┐
│ RabbitMQ Exchange   │      │ Internal Event      │
│ svrnty-sample.      │      │ Store (PostgreSQL)  │
│ user-events         │      │                     │
└──────────┬──────────┘      └─────────────────────┘
           │                           │
           │                           │
    ┌──────┴──────┐           ┌────────┴────────┐
    │             │           │                 │
    ▼             ▼           ▼                 ▼
┌─────────┐ ┌──────────┐ ┌──────────┐ ┌──────────────┐
│RabbitMQ │ │ Email    │ │Internal  │ │ Analytics    │
│Consumer │ │ Service  │ │Consumer  │ │ Service      │
│(Sample) │ │(External)│ │(Sample)  │ │ (External)   │
└─────────┘ └──────────┘ └──────────┘ └──────────────┘

Key Features Demonstrated

1. Zero RabbitMQ Code in Handlers: Command handlers emit events via workflows without any knowledge of RabbitMQ
2. Automatic Topology Management: Framework creates exchanges, queues, and bindings automatically
3. Dual Delivery: Events are published to both internal store and RabbitMQ (External scope)
4. Consumer Groups: Multiple consumers can load-balance message processing
5. Type-Safe Event Handling: Events are deserialized with full type information

Configuration

Enable/Disable RabbitMQ

Edit appsettings.json:

{
  "EventStreaming": {
    "RabbitMQ": {
      "Enabled": true,  // Set to false to disable RabbitMQ
      "ConnectionString": "amqp://guest:guest@localhost:5672/"
    }
  }
}

Stream Configuration

In Program.cs, streams are configured with StreamScope.External to publish to RabbitMQ:

streaming.AddStream<UserWorkflow>(stream =>
{
    stream.Type = StreamType.Ephemeral;
    stream.DeliverySemantics = DeliverySemantics.AtLeastOnce;
    stream.Scope = StreamScope.External;  // Publish to RabbitMQ
    stream.ExternalStreamName = "user-events";  // RabbitMQ stream name
});

Running the Sample

Prerequisites

1. Docker and Docker Compose installed
2. .NET 10 SDK installed

Steps

1. Start Infrastructure

# From repository root
docker-compose up -d

# Verify services are running
docker ps

Expected services:

• PostgreSQL (port 5432)
• RabbitMQ (ports 5672, 15672)
• pgAdmin (port 5050) - optional

2. Build and Run Sample

cd Svrnty.Sample
dotnet build
dotnet run

3. Run Automated Test

cd Svrnty.Sample
chmod +x test-rabbitmq-integration.sh
./test-rabbitmq-integration.sh

This script will:

• Start the application
• Execute commands that emit events
• Verify events are published to RabbitMQ
• Show consumer logs

Manual Testing

1. Execute Command via HTTP API

curl -X POST http://localhost:6001/api/command/addUser \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Alice Johnson",
    "email": "alice@example.com",
    "age": 30
  }'

2. Verify in RabbitMQ Management UI

Open http://localhost:15672 (guest/guest)

Check Exchanges:

• Name: svrnty-sample.user-events
• Type: topic
• Durable: yes

Check Queues:

• Name: svrnty-sample.email-service
• Consumers: 1
• Messages: Should show activity

Check Bindings:

• Exchange: svrnty-sample.user-events
• Queue: svrnty-sample.email-service
• Routing key: # (wildcard)

3. View Application Logs

Watch for these log entries:

[Information] Subscribing to 'user-events' stream from RabbitMQ...
[Information] [RABBITMQ] Received external event: UserAddedEvent (EventId: xxx, CorrelationId: xxx)
[Information] [RABBITMQ] Sending welcome email to alice@example.com (UserId: 123)

Event Flow

When you execute AddUserCommand:

1. Handler Emits Event

   workflow.EmitAdded(new UserAddedEvent
   {
       UserId = userId,
       Name = command.Name,
       Email = command.Email
   });
   

2. Framework Publishes to RabbitMQ

   • Serializes event to JSON
   • Adds metadata headers (event-type, event-id, correlation-id, timestamp)
   • Publishes to exchange: svrnty-sample.user-events
   • Uses routing key based on event type

3. RabbitMQ Routes Message

   • Exchange routes message to bound queues
   • Queue: svrnty-sample.email-service receives message

4. Consumer Receives Event

   [Information] [RABBITMQ] Received external event: UserAddedEvent
   [Information] [RABBITMQ] Sending welcome email to alice@example.com
   

RabbitMQ Topology

The framework automatically creates:

Exchanges

• Name: {ExchangePrefix}.{StreamName}
  • Example: svrnty-sample.user-events
• Type: topic (configurable)
• Durable: true
• Auto-delete: false

Queues

• Name: {ExchangePrefix}.{SubscriptionId}
  • Example: svrnty-sample.email-service
• Durable: true
• Prefetch: 10 (configurable)
• Mode: Consumer Group (multiple consumers share queue)

Bindings

• Exchange: svrnty-sample.user-events
• Queue: svrnty-sample.email-service
• Routing Key: # (receives all events)

Consumer Implementation

See RabbitMQEventConsumerBackgroundService.cs:

protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
    await _rabbitMq.SubscribeExternalAsync(
        streamName: "user-events",
        subscriptionId: "email-service",
        consumerId: $"rabbitmq-consumer-{Guid.NewGuid():N}",
        eventHandler: ProcessEventAsync,
        cancellationToken: stoppingToken);
}

private Task ProcessEventAsync(
    ICorrelatedEvent @event,
    IDictionary<string, string> metadata,
    CancellationToken cancellationToken)
{
    switch (@event)
    {
        case UserAddedEvent userAdded:
            _logger.LogInformation(
                "[RABBITMQ] Sending welcome email to {Email}",
                userAdded.Email);
            // Send email logic here
            break;
    }
    return Task.CompletedTask;
}

Comparison: Internal vs External Event Consumption

The sample demonstrates two consumption patterns:

1. Internal Event Consumption (EventConsumerBackgroundService)

• Consumes from internal PostgreSQL event store
• Same process/service
• Uses IEventSubscriptionClient
• Good for: Same-service event handlers, sagas, process managers

2. External Event Consumption (RabbitMQEventConsumerBackgroundService)

• Consumes from RabbitMQ
• Cross-service communication
• Uses IExternalEventDeliveryProvider
• Good for: Microservices, distributed systems, event-driven architecture

Performance Considerations

Publisher

• Throughput: ~10,000 events/second (local RabbitMQ)
• Latency: ~5-10ms per publish
• Reliability: Publisher confirms disabled by default (can be enabled for critical events)

Consumer

• Throughput: Limited by prefetch (default: 10) and handler processing time
• Prefetch 10: ~1,000 events/second (lightweight handlers)
• Prefetch 100: ~10,000 events/second (lightweight handlers)

Configuration for High Throughput

{
  "EventStreaming": {
    "RabbitMQ": {
      "PrefetchCount": 100,
      "EnablePublisherConfirms": false,
      "PersistentMessages": false
    }
  }
}

Configuration for Reliability

{
  "EventStreaming": {
    "RabbitMQ": {
      "PrefetchCount": 10,
      "EnablePublisherConfirms": true,
      "PersistentMessages": true,
      "DurableQueues": true,
      "DurableExchanges": true
    }
  }
}

Troubleshooting

Events Not Appearing in RabbitMQ

1. Check RabbitMQ is running: docker ps | grep rabbitmq
2. Check RabbitMQ logs: docker logs svrnty-rabbitmq
3. Verify Enabled: true in appsettings.json
4. Check stream scope is External in Program.cs
5. Look for errors in application logs

Consumer Not Receiving Events

1. Check consumer is subscribed: Look for "Subscribing to 'user-events' stream" in logs
2. Check queue has consumers: RabbitMQ Management UI → Queues
3. Verify routing keys match: Exchange bindings should show #
4. Check for exceptions in consumer logs

Connection Failures

1. Verify connection string: amqp://guest:guest@localhost:5672/
2. Check RabbitMQ is accessible: telnet localhost 5672
3. Review RabbitMQ credentials (default: guest/guest)
4. Check firewall rules

Next Steps

• Add more event types and consumers
• Implement cross-service workflows (Service A → Service B → Service C)
• Add integration tests with TestContainers
• Explore consumer group scaling (multiple instances)
• Implement dead letter queue handling
• Add monitoring and observability

Related Documentation

• RABBITMQ-GUIDE.md - Comprehensive RabbitMQ integration guide
• PHASE4-COMPLETE.md - Phase 4 completion summary
• docker-compose.yml - Infrastructure setup

Support

For questions or issues, see: https://git.openharbor.io/svrnty/dotnet-cqrs