svrnty-mcp-gateway/docs/deployment/https-setup.md

# HTTPS/TLS Setup Guide - OpenHarbor.MCP.Gateway

**Purpose**: Production-grade HTTPS/TLS configuration for MCP gateway/proxy deployment
**Audience**: DevOps engineers, system administrators
**Last Updated**: 2025-10-19
**Version**: 1.0.0

---

## Table of Contents

1. [Overview](#overview)
2. [Prerequisites](#prerequisites)
3. [Gateway HTTPS Configuration](#gateway-https-configuration)
4. [Backend Server Connections](#backend-server-connections)
5. [Load Balancer Integration](#load-balancer-integration)
6. [TLS Termination Strategies](#tls-termination-strategies)
7. [Security Headers](#security-headers)
8. [Testing](#testing)
9. [Troubleshooting](#troubleshooting)

---

## Overview

OpenHarbor.MCP.Gateway acts as a reverse proxy/load balancer between MCP clients and servers. It supports multiple TLS termination strategies and secure backend connections.

**Architecture Options:**
1. **TLS Termination at Gateway** (recommended)
   - Client → HTTPS → Gateway → HTTP → Backend Servers
2. **TLS Passthrough**
   - Client → HTTPS → Gateway (proxy) → HTTPS → Backend Servers
3. **End-to-End TLS**
   - Client → HTTPS → Gateway (HTTPS) → HTTPS → Backend Servers

**Security Features:**
- Centralized certificate management
- Multiple backend server support with connection pooling
- Circuit breakers and health checks
- Rate limiting and authentication
- Security headers (HSTS, CSP, etc.)

---

## Prerequisites

### Development
- .NET 8.0 SDK
- Docker + Docker Compose (optional)
- TLS certificates (dev or production)

### Production
- Valid TLS certificate (from CA or Let's Encrypt)
- Load balancer (optional): Nginx, HAProxy, AWS ALB
- Backend MCP servers configured
- Monitoring/logging infrastructure

---

## Gateway HTTPS Configuration

### Configuration via appsettings.json

**appsettings.Production.json:**

```json
{
  "Kestrel": {
    "Endpoints": {
      "Http": {
        "Url": "http://*:8080"
      },
      "Https": {
        "Url": "https://*:8443",
        "Certificate": {
          "Path": "/app/certs/gateway.pfx",
          "Password": ""  // From environment variable
        },
        "Protocols": "Http1AndHttp2"
      }
    }
  },

  "Gateway": {
    "BackendServers": [
      {
        "Name": "mcp-server-1",
        "Url": "http://mcp-server-1:5050",  // HTTP backend (TLS terminated at gateway)
        "HealthCheckPath": "/health",
        "MaxConnections": 10
      },
      {
        "Name": "mcp-server-2",
        "Url": "http://mcp-server-2:5050",
        "HealthCheckPath": "/health",
        "MaxConnections": 10
      }
    ],

    "LoadBalancing": {
      "Strategy": "RoundRobin",  // RoundRobin, LeastConnections, or ToolBased
      "HealthCheckInterval": "00:00:30",
      "CircuitBreaker": {
        "FailureThreshold": 5,
        "SuccessThreshold": 2,
        "Timeout": "00:01:00"
      }
    },

    "Security": {
      "RequireHttps": true,
      "AllowedOrigins": ["https://app.example.com"],
      "RateLimiting": {
        "RequestsPerMinute": 100,
        "BurstSize": 20
      }
    }
  }
}
```

### Environment Variables

```bash
# Certificate password
export KESTREL__CERTIFICATES__DEFAULT__PASSWORD="ProductionSecurePassword"

# Backend server URLs (override config)
export GATEWAY__BACKENDSERVERS__0__URL="https://mcp-server-1.internal:5051"
export GATEWAY__BACKENDSERVERS__1__URL="https://mcp-server-2.internal:5051"

# API keys
export GATEWAY__SECURITY__APIKEY="gateway-api-key"
```

---

## Backend Server Connections

### Option 1: HTTP Backends (TLS Termination at Gateway)

**Recommended for**: Internal networks, simplified certificate management

```json
{
  "Gateway": {
    "BackendServers": [
      {
        "Name": "server-1",
        "Url": "http://10.0.1.10:5050",  // HTTP only
        "HealthCheckPath": "/health"
      }
    ]
  }
}
```

**Architecture:**
```
Internet → HTTPS (443) → Gateway (TLS termination) → HTTP → Backend Servers
```

**Pros:**
- Single certificate to manage
- Lower backend server CPU usage
- Simpler debugging (plain HTTP)

**Cons:**
- Backend traffic unencrypted (use private network)

### Option 2: HTTPS Backends (End-to-End Encryption)

**Recommended for**: Security-sensitive deployments, zero-trust networks

```json
{
  "Gateway": {
    "BackendServers": [
      {
        "Name": "server-1",
        "Url": "https://mcp-server-1.internal:5051",
        "HealthCheckPath": "/health",
        "TlsOptions": {
          "ValidateCertificate": true,
          "AllowedCertificateThumbprints": [
            "A1B2C3D4E5F6..."  // Pin backend certificates
          ]
        }
      }
    ]
  }
}
```

**Architecture:**
```
Internet → HTTPS (443) → Gateway (TLS re-encryption) → HTTPS → Backend Servers
```

**Pros:**
- End-to-end encryption
- Backend servers independently secured
- Compliance with zero-trust architecture

**Cons:**
- More certificates to manage
- Higher CPU usage (double TLS)
- More complex troubleshooting

### Custom Backend Certificate Validation

**Program.cs Configuration:**

```csharp
services.AddHttpClient("BackendClient", client =>
{
    client.Timeout = TimeSpan.FromSeconds(30);
})
.ConfigurePrimaryHttpMessageHandler(() =>
{
    var handler = new HttpClientHandler();

    // Custom certificate validation for backends
    handler.ServerCertificateCustomValidationCallback = (request, cert, chain, errors) =>
    {
        if (errors == SslPolicyErrors.None)
            return true;

        // Accept specific backend certificates (certificate pinning)
        var allowedThumbprints = new[]
        {
            "A1B2C3D4E5F6...",  // Backend server 1
            "B2C3D4E5F6A1..."   // Backend server 2
        };

        return cert != null && allowedThumbprints.Contains(cert.Thumbprint);
    };

    return handler;
});
```

---

## Load Balancer Integration

### Option 1: Gateway Behind Load Balancer (TLS at LB)

**Architecture:**
```
Internet → ALB/NLB (TLS) → Gateway (HTTP) → Backends (HTTP/HTTPS)
```

**AWS Application Load Balancer Example:**

**Target Group:**
```json
{
  "TargetType": "ip",
  "Protocol": "HTTP",
  "Port": 8080,
  "HealthCheck": {
    "Protocol": "HTTP",
    "Path": "/health",
    "Interval": 30,
    "Timeout": 5
  }
}
```

**Listener:**
```json
{
  "Protocol": "HTTPS",
  "Port": 443,
  "Certificates": [
    {
      "CertificateArn": "arn:aws:acm:us-east-1:123456789012:certificate/abc123..."
    }
  ],
  "DefaultActions": [
    {
      "Type": "forward",
      "TargetGroupArn": "arn:aws:elasticloadbalancing:..."
    }
  ]
}
```

**Gateway Configuration (HTTP only, LB handles TLS):**

```json
{
  "Kestrel": {
    "Endpoints": {
      "Http": {
        "Url": "http://*:8080"
      }
    }
  },

  "Gateway": {
    "Security": {
      "RequireHttps": false,  // LB enforces HTTPS
      "TrustForwardedHeaders": true  // Trust X-Forwarded-Proto from ALB
    }
  }
}
```

**Program.cs - Forward Headers:**

```csharp
app.UseForwardedHeaders(new ForwardedHeadersOptions
{
    ForwardedHeaders = ForwardedHeaders.XForwardedFor | ForwardedHeaders.XForwardedProto
});
```

### Option 2: Gateway as Primary TLS Termination

**Architecture:**
```
Internet → Gateway (TLS) → Backends (HTTP/HTTPS)
```

**Use Cases:**
- On-premises deployment
- Direct internet exposure
- Custom routing logic

**Configuration:**

```json
{
  "Kestrel": {
    "Endpoints": {
      "Https": {
        "Url": "https://*:443",
        "Certificate": {
          "Path": "/app/certs/gateway.pfx",
          "Password": ""
        }
      }
    }
  }
}
```

---

## TLS Termination Strategies

### Strategy 1: TLS Termination (Recommended for Most Cases)

**Diagram:**
```
Client --HTTPS--> Gateway --HTTP--> Backend Servers
                  [TLS
                 Termination]
```

**Benefits:**
- Single certificate management point
- Lower backend CPU usage
- Easy to inspect/log traffic
- Simpler troubleshooting

**Configuration:**
```json
{
  "Kestrel": {
    "Endpoints": {
      "Https": { "Url": "https://*:8443" }
    }
  },
  "Gateway": {
    "BackendServers": [
      { "Url": "http://backend:5050" }  // HTTP to backends
    ]
  }
}
```

### Strategy 2: TLS Passthrough (Transparent Proxy)

**Diagram:**
```
Client --HTTPS--> Gateway --HTTPS--> Backend Servers
                 [Proxy Only,
                  No TLS
                 Termination]
```

**Benefits:**
- End-to-end encryption
- Gateway never sees decrypted traffic
- Backend servers control TLS config

**Limitations:**
- Cannot inspect traffic
- Cannot do intelligent routing based on content
- More complex certificate management

**Not Recommended** for MCP Gateway (requires layer 7 routing)

### Strategy 3: TLS Re-encryption (Maximum Security)

**Diagram:**
```
Client --HTTPS--> Gateway --HTTPS--> Backend Servers
                 [TLS Termination
                  + Re-encryption]
```

**Benefits:**
- Can inspect/route traffic
- End-to-end encryption
- Zero-trust architecture

**Configuration:**
```json
{
  "Kestrel": {
    "Endpoints": {
      "Https": { "Url": "https://*:8443" }
    }
  },
  "Gateway": {
    "BackendServers": [
      { "Url": "https://backend:5051" }  // HTTPS to backends
    ]
  }
}
```

---

## Security Headers

**Add to Program.cs:**

```csharp
app.Use(async (context, next) =>
{
    // HSTS (HTTP Strict Transport Security)
    context.Response.Headers.Add("Strict-Transport-Security", "max-age=31536000; includeSubDomains; preload");

    // Content Security Policy
    context.Response.Headers.Add("Content-Security-Policy", "default-src 'self'; script-src 'self'; style-src 'self'");

    // X-Frame-Options (prevent clickjacking)
    context.Response.Headers.Add("X-Frame-Options", "DENY");

    // X-Content-Type-Options (prevent MIME sniffing)
    context.Response.Headers.Add("X-Content-Type-Options", "nosniff");

    // Referrer Policy
    context.Response.Headers.Add("Referrer-Policy", "strict-origin-when-cross-origin");

    // Permissions Policy
    context.Response.Headers.Add("Permissions-Policy", "geolocation=(), microphone=(), camera=()");

    await next();
});

// Redirect HTTP to HTTPS
if (!app.Environment.IsDevelopment())
{
    app.UseHttpsRedirection();
    app.UseHsts();
}
```

---

## Testing

### 1. Test Gateway HTTPS Endpoint

```bash
# Health check
curl https://gateway.example.com/health

# Expected output
{
  "status": "Healthy",
  "service": "MCP Gateway",
  "backends": [
    {
      "name": "mcp-server-1",
      "healthy": true,
      "lastCheck": "2025-10-19T12:00:00Z"
    }
  ]
}
```

### 2. Test MCP Tool Invocation Through Gateway

```bash
curl -X POST https://gateway.example.com/mcp/invoke \
  -H "Content-Type: application/json" \
  -H "X-API-Key: gateway-api-key" \
  -d '{
    "jsonrpc": "2.0",
    "method": "tools/call",
    "params": {
      "name": "search_documents",
      "arguments": {
        "query": "test"
      }
    },
    "id": "1"
  }'
```

### 3. Verify TLS Configuration

```bash
# Check TLS version and cipher suites
nmap --script ssl-enum-ciphers -p 8443 gateway.example.com

# Check certificate
echo | openssl s_client -connect gateway.example.com:8443 -servername gateway.example.com 2>/dev/null | openssl x509 -noout -text
```

### 4. Test Load Balancing

```bash
# Send multiple requests, verify distribution
for i in {1..10}; do
  curl -s https://gateway.example.com/health | jq -r '.backend'
done

# Should see round-robin distribution across backends
```

### 5. Test Circuit Breaker

```bash
# Stop one backend server
docker stop mcp-server-1

# Verify gateway routes to healthy server
curl https://gateway.example.com/health

# Should show mcp-server-2 only
```

---

## Troubleshooting

### Issue: "Unable to configure HTTPS endpoint"

**Symptoms:**
```
System.InvalidOperationException: Unable to configure HTTPS endpoint
```

**Solution:**
1. Verify certificate file exists
2. Check password is correct
3. Ensure certificate is PFX format

```bash
# Verify PFX
openssl pkcs12 -info -in gateway.pfx -noout
```

### Issue: Gateway Cannot Connect to Backend Servers

**Symptoms:**
```
All backends unavailable
Circuit breaker: OPEN
```

**Solution:**
1. Check backend server URLs
2. Verify network connectivity
3. Check firewall rules

```bash
# Test backend connectivity from gateway container
docker exec gateway-container curl http://mcp-server-1:5050/health
```

### Issue: Clients Receive "Certificate Invalid"

**Symptoms:**
Clients reject gateway certificate.

**Solution:**
1. Ensure certificate CN/SAN matches gateway domain
2. Verify certificate is from trusted CA
3. Check certificate not expired

```bash
# Check certificate details
openssl x509 -in gateway.crt -noout -text | grep -A1 "Subject Alternative Name"
```

### Issue: Slow Response Times

**Symptoms:**
High latency through gateway.

**Solution:**
1. Check backend server performance
2. Increase connection pool size
3. Enable HTTP/2

```json
{
  "Gateway": {
    "BackendServers": [
      {
        "MaxConnections": 50  // Increase from 10
      }
    ]
  },
  "Kestrel": {
    "Endpoints": {
      "Https": {
        "Protocols": "Http1AndHttp2"  // Enable HTTP/2
      }
    }
  }
}
```

---

## Production Deployment Checklist

- [ ] Valid TLS certificate from trusted CA
- [ ] Certificate includes full chain
- [ ] Private key secured (Kubernetes secret, vault, etc.)
- [ ] HSTS header enabled
- [ ] HTTP → HTTPS redirect configured
- [ ] Backend server health checks working
- [ ] Circuit breakers configured
- [ ] Connection pooling optimized
- [ ] Security headers configured
- [ ] Rate limiting enabled
- [ ] Monitoring/alerting set up
- [ ] Certificate expiry monitoring configured
- [ ] Load testing completed
- [ ] Firewall rules configured
- [ ] TLS 1.2+ enforced

---

## References

**OpenHarbor.MCP Documentation:**
- [Gateway README](../../README.md)
- [Architecture](../../docs/architecture.md)
- [Load Balancing Strategies](../../docs/load-balancing.md)

**ASP.NET Core Documentation:**
- [Kestrel HTTPS Configuration](https://learn.microsoft.com/en-us/aspnet/core/fundamentals/servers/kestrel/endpoints)
- [Reverse Proxy](https://learn.microsoft.com/en-us/aspnet/core/host-and-deploy/proxy-load-balancer)
- [Security Headers](https://learn.microsoft.com/en-us/aspnet/core/security/enforcing-ssl)

**Load Balancers:**
- [Nginx Reverse Proxy](https://docs.nginx.com/nginx/admin-guide/web-server/reverse-proxy/)
- [AWS Application Load Balancer](https://docs.aws.amazon.com/elasticloadbalancing/latest/application/)
- [Traefik](https://doc.traefik.io/traefik/)

---

**Document Version**: 1.0.0
**Last Updated**: 2025-10-19
**Maintained By**: Svrnty Development Team
**Related**: [Server HTTPS Setup](../../OpenHarbor.MCP.Server/docs/deployment/https-setup.md), [Client HTTPS Setup](../../OpenHarbor.MCP.Client/docs/deployment/https-setup.md)