dotnet-cqrs/Svrnty.Sample/README-RABBITMQ.md

# RabbitMQ Integration Example

This sample project demonstrates **Phase 4: Cross-Service Communication** using RabbitMQ for event streaming between microservices.

## Overview

The sample shows how events emitted by command handlers are automatically published to RabbitMQ, enabling cross-service communication without any RabbitMQ-specific code in your handlers.

## Architecture

```
┌─────────────────────┐
│  AddUserCommand     │
│     Handler         │
└──────────┬──────────┘
           │
           │ workflow.Emit(UserAddedEvent)
           │
           ▼
┌─────────────────────┐
│  UserWorkflow       │
│  (External Scope)   │
└──────────┬──────────┘
           │
           │ Auto-publish to RabbitMQ
           │
           ▼
┌─────────────────────┐      ┌─────────────────────┐
│ RabbitMQ Exchange   │      │ Internal Event      │
│ svrnty-sample.      │      │ Store (PostgreSQL)  │
│ user-events         │      │                     │
└──────────┬──────────┘      └─────────────────────┘
           │                           │
           │                           │
    ┌──────┴──────┐           ┌────────┴────────┐
    │             │           │                 │
    ▼             ▼           ▼                 ▼
┌─────────┐ ┌──────────┐ ┌──────────┐ ┌──────────────┐
│RabbitMQ │ │ Email    │ │Internal  │ │ Analytics    │
│Consumer │ │ Service  │ │Consumer  │ │ Service      │
│(Sample) │ │(External)│ │(Sample)  │ │ (External)   │
└─────────┘ └──────────┘ └──────────┘ └──────────────┘
```

## Key Features Demonstrated

1. **Zero RabbitMQ Code in Handlers**: Command handlers emit events via workflows without any knowledge of RabbitMQ
2. **Automatic Topology Management**: Framework creates exchanges, queues, and bindings automatically
3. **Dual Delivery**: Events are published to both internal store and RabbitMQ (External scope)
4. **Consumer Groups**: Multiple consumers can load-balance message processing
5. **Type-Safe Event Handling**: Events are deserialized with full type information

## Configuration

### Enable/Disable RabbitMQ

Edit `appsettings.json`:

```json
{
  "EventStreaming": {
    "RabbitMQ": {
      "Enabled": true,  // Set to false to disable RabbitMQ
      "ConnectionString": "amqp://guest:guest@localhost:5672/"
    }
  }
}
```

### Stream Configuration

In `Program.cs`, streams are configured with `StreamScope.External` to publish to RabbitMQ:

```csharp
streaming.AddStream<UserWorkflow>(stream =>
{
    stream.Type = StreamType.Ephemeral;
    stream.DeliverySemantics = DeliverySemantics.AtLeastOnce;
    stream.Scope = StreamScope.External;  // Publish to RabbitMQ
    stream.ExternalStreamName = "user-events";  // RabbitMQ stream name
});
```

## Running the Sample

### Prerequisites

1. Docker and Docker Compose installed
2. .NET 10 SDK installed

### Steps

1. **Start Infrastructure**

```bash
# From repository root
docker-compose up -d

# Verify services are running
docker ps
```

Expected services:
- PostgreSQL (port 5432)
- RabbitMQ (ports 5672, 15672)
- pgAdmin (port 5050) - optional

2. **Build and Run Sample**

```bash
cd Svrnty.Sample
dotnet build
dotnet run
```

3. **Run Automated Test**

```bash
cd Svrnty.Sample
chmod +x test-rabbitmq-integration.sh
./test-rabbitmq-integration.sh
```

This script will:
- Start the application
- Execute commands that emit events
- Verify events are published to RabbitMQ
- Show consumer logs

## Manual Testing

### 1. Execute Command via HTTP API

```bash
curl -X POST http://localhost:6001/api/command/addUser \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Alice Johnson",
    "email": "alice@example.com",
    "age": 30
  }'
```

### 2. Verify in RabbitMQ Management UI

Open http://localhost:15672 (guest/guest)

**Check Exchanges:**
- Name: `svrnty-sample.user-events`
- Type: `topic`
- Durable: `yes`

**Check Queues:**
- Name: `svrnty-sample.email-service`
- Consumers: `1`
- Messages: Should show activity

**Check Bindings:**
- Exchange: `svrnty-sample.user-events`
- Queue: `svrnty-sample.email-service`
- Routing key: `#` (wildcard)

### 3. View Application Logs

Watch for these log entries:

```
[Information] Subscribing to 'user-events' stream from RabbitMQ...
[Information] [RABBITMQ] Received external event: UserAddedEvent (EventId: xxx, CorrelationId: xxx)
[Information] [RABBITMQ] Sending welcome email to alice@example.com (UserId: 123)
```

## Event Flow

When you execute `AddUserCommand`:

1. **Handler Emits Event**
   ```csharp
   workflow.EmitAdded(new UserAddedEvent
   {
       UserId = userId,
       Name = command.Name,
       Email = command.Email
   });
   ```

2. **Framework Publishes to RabbitMQ**
   - Serializes event to JSON
   - Adds metadata headers (event-type, event-id, correlation-id, timestamp)
   - Publishes to exchange: `svrnty-sample.user-events`
   - Uses routing key based on event type

3. **RabbitMQ Routes Message**
   - Exchange routes message to bound queues
   - Queue: `svrnty-sample.email-service` receives message

4. **Consumer Receives Event**
   ```csharp
   [Information] [RABBITMQ] Received external event: UserAddedEvent
   [Information] [RABBITMQ] Sending welcome email to alice@example.com
   ```

## RabbitMQ Topology

The framework automatically creates:

### Exchanges

- **Name**: `{ExchangePrefix}.{StreamName}`
  - Example: `svrnty-sample.user-events`
- **Type**: `topic` (configurable)
- **Durable**: `true`
- **Auto-delete**: `false`

### Queues

- **Name**: `{ExchangePrefix}.{SubscriptionId}`
  - Example: `svrnty-sample.email-service`
- **Durable**: `true`
- **Prefetch**: `10` (configurable)
- **Mode**: Consumer Group (multiple consumers share queue)

### Bindings

- **Exchange**: `svrnty-sample.user-events`
- **Queue**: `svrnty-sample.email-service`
- **Routing Key**: `#` (receives all events)

## Consumer Implementation

See `RabbitMQEventConsumerBackgroundService.cs`:

```csharp
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
    await _rabbitMq.SubscribeExternalAsync(
        streamName: "user-events",
        subscriptionId: "email-service",
        consumerId: $"rabbitmq-consumer-{Guid.NewGuid():N}",
        eventHandler: ProcessEventAsync,
        cancellationToken: stoppingToken);
}

private Task ProcessEventAsync(
    ICorrelatedEvent @event,
    IDictionary<string, string> metadata,
    CancellationToken cancellationToken)
{
    switch (@event)
    {
        case UserAddedEvent userAdded:
            _logger.LogInformation(
                "[RABBITMQ] Sending welcome email to {Email}",
                userAdded.Email);
            // Send email logic here
            break;
    }
    return Task.CompletedTask;
}
```

## Comparison: Internal vs External Event Consumption

The sample demonstrates **two consumption patterns**:

### 1. Internal Event Consumption (EventConsumerBackgroundService)

- Consumes from internal PostgreSQL event store
- Same process/service
- Uses `IEventSubscriptionClient`
- Good for: Same-service event handlers, sagas, process managers

### 2. External Event Consumption (RabbitMQEventConsumerBackgroundService)

- Consumes from RabbitMQ
- Cross-service communication
- Uses `IExternalEventDeliveryProvider`
- Good for: Microservices, distributed systems, event-driven architecture

## Performance Considerations

### Publisher

- **Throughput**: ~10,000 events/second (local RabbitMQ)
- **Latency**: ~5-10ms per publish
- **Reliability**: Publisher confirms disabled by default (can be enabled for critical events)

### Consumer

- **Throughput**: Limited by prefetch (default: 10) and handler processing time
- **Prefetch 10**: ~1,000 events/second (lightweight handlers)
- **Prefetch 100**: ~10,000 events/second (lightweight handlers)

### Configuration for High Throughput

```json
{
  "EventStreaming": {
    "RabbitMQ": {
      "PrefetchCount": 100,
      "EnablePublisherConfirms": false,
      "PersistentMessages": false
    }
  }
}
```

### Configuration for Reliability

```json
{
  "EventStreaming": {
    "RabbitMQ": {
      "PrefetchCount": 10,
      "EnablePublisherConfirms": true,
      "PersistentMessages": true,
      "DurableQueues": true,
      "DurableExchanges": true
    }
  }
}
```

## Troubleshooting

### Events Not Appearing in RabbitMQ

1. Check RabbitMQ is running: `docker ps | grep rabbitmq`
2. Check RabbitMQ logs: `docker logs svrnty-rabbitmq`
3. Verify `Enabled: true` in appsettings.json
4. Check stream scope is `External` in Program.cs
5. Look for errors in application logs

### Consumer Not Receiving Events

1. Check consumer is subscribed: Look for "Subscribing to 'user-events' stream" in logs
2. Check queue has consumers: RabbitMQ Management UI → Queues
3. Verify routing keys match: Exchange bindings should show `#`
4. Check for exceptions in consumer logs

### Connection Failures

1. Verify connection string: `amqp://guest:guest@localhost:5672/`
2. Check RabbitMQ is accessible: `telnet localhost 5672`
3. Review RabbitMQ credentials (default: guest/guest)
4. Check firewall rules

## Next Steps

- Add more event types and consumers
- Implement cross-service workflows (Service A → Service B → Service C)
- Add integration tests with TestContainers
- Explore consumer group scaling (multiple instances)
- Implement dead letter queue handling
- Add monitoring and observability

## Related Documentation

- [RABBITMQ-GUIDE.md](../RABBITMQ-GUIDE.md) - Comprehensive RabbitMQ integration guide
- [PHASE4-COMPLETE.md](../PHASE4-COMPLETE.md) - Phase 4 completion summary
- [docker-compose.yml](../docker-compose.yml) - Infrastructure setup

## Support

For questions or issues, see: https://git.openharbor.io/svrnty/dotnet-cqrs