Scaling WebSockets on AWS: Why We Abandoned API Gateway for ECS and NGINX

When building a real-time collaborative application on AWS, the "serverless" path of least resistance is usually Amazon API Gateway's WebSocket API. It integrates cleanly with Lambda and requires zero infrastructure management. But as our client's active concurrent users scaled past 100,000, API Gateway transformed from a convenience into an existential financial threat.

API Gateway WebSockets charge $1.00 per million messages and $0.25 per million connection minutes. For a highly interactive chat application sending frequent typing indicators and presence updates, the monthly bill quickly skyrocketed past $15,000—just for the gateway layer.

The Breaking Point

Beyond the exorbitant costs, we were running into hard physical limitations of the managed service:

The 2-Hour Disconnect: API Gateway enforces a hard 2-hour maximum connection duration. At exactly 120 minutes, connections are violently dropped, forcing clients to reconnect and lose state.
Throttling: We consistently hit the hard limit of 10,000 connections per second per region.
Latency: The architecture forced every single tiny WebSocket message to invoke an AWS Lambda function, introducing unnecessary cold starts and cold-path latency.

The Migration: ECS, NGINX, and Redis Pub/Sub

We designed a radically simpler, fundamentally cheaper architecture using Amazon ECS (Fargate) and a custom NGINX reverse proxy fleet backing Node.js Socket.io servers.

# NGINX Configuration for WebSocket proxying
map $http_upgrade $connection_upgrade {
    default upgrade;
    ''      close;
}

upstream websocket_backend {
    server 127.0.0.1:3000;
    keepalive 64;
}

server {
    listen 80;
    
    location /socket.io/ {
        proxy_pass http://websocket_backend;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection $connection_upgrade;
        
        # Crucial for long-lived connections
        proxy_read_timeout 3600s;
        proxy_send_timeout 3600s;
    }
}

Engineering Reality

"Moving off a fully managed service means you take ownership of the scaling mechanics. We had to implement aggressive ECS Target Tracking scaling policies based on memory utilization and active TCP connections to ensure the Fargate tasks could gracefully handle the thundering herd during a massive reconnect event."

State Management at the Edge

To replace API Gateway's built-in connection management, we deployed a highly available Amazon ElastiCache (Redis) cluster. We used Redis Pub/Sub to broadcast messages across the horizontally scaled ECS Fargate fleet. When a user connected to Fargate Task A but needed to message a user on Fargate Task B, Redis seamlessly bridged the gap in sub-millisecond time.

The Outcomes

By trading managed convenience for architectural control, we dramatically altered the unit economics of the application.

85%

Cost Reduction

The monthly WebSocket infrastructure bill dropped from $15,000 to just under $2,200.

Forced Disconnects

Connections now remain open for days at a time, limited only by the client's network stability.

Verdict

Serverless offerings like API Gateway WebSockets are phenomenal for MVPs, low-traffic workloads, or bursty intermittent payloads. However, for continuous, high-volume real-time data streams, the premium you pay for AWS to manage your idle TCP connections is rarely justifiable at scale.