<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"><channel><title><![CDATA[Architecture of a Self-Hosted Transactional Email Gateway: Decoupled Workers, Argon2id, and Key Rotation]]></title><description><![CDATA[Architecture of a Self-Hosted Transactional Email Gateway: Decoupled Workers, Argon2id, and Key Rotation]]></description><link>https://hermesruanlopes.hashnode.dev</link><image><url>https://cdn.hashnode.com/uploads/logos/6aad8e4c1fa3162e6cd84369/0fe078e5-77df-4427-94c6-9a254d691cc5.png</url><title>Architecture of a Self-Hosted Transactional Email Gateway: Decoupled Workers, Argon2id, and Key Rotation</title><link>https://hermesruanlopes.hashnode.dev</link></image><generator>RSS for Node</generator><lastBuildDate>Thu, 24 Sep 2026 15:32:09 GMT</lastBuildDate><atom:link href="https://hermesruanlopes.hashnode.dev/rss.xml" rel="self" type="application/rss+xml"/><language><![CDATA[en]]></language><ttl>60</ttl><item><title><![CDATA[Architecture of a Self-Hosted Transactional Email Gateway: Decoupled Workers, Argon2id, and Key Rotation]]></title><description><![CDATA[When multiple internal applications need to send transactional emails, embedding SMTP transport logic and template rendering directly into each service introduces tight coupling, unpredictable request]]></description><link>https://hermesruanlopes.hashnode.dev/architecture-of-a-self-hosted-transactional-email-gateway-decoupled-workers-argon2id-and-key-rotation</link><guid isPermaLink="true">https://hermesruanlopes.hashnode.dev/architecture-of-a-self-hosted-transactional-email-gateway-decoupled-workers-argon2id-and-key-rotation</guid><category><![CDATA[Node.js]]></category><category><![CDATA[TypeScript]]></category><category><![CDATA[MJML]]></category><category><![CDATA[smtp]]></category><category><![CDATA[handlebars]]></category><category><![CDATA[Docker]]></category><category><![CDATA[Mail]]></category><category><![CDATA[nodemailer]]></category><category><![CDATA[Microservices]]></category><category><![CDATA[npm]]></category><dc:creator><![CDATA[Ruan Lopes]]></dc:creator><pubDate>Fri, 18 Sep 2026 19:34:21 GMT</pubDate><content:encoded><![CDATA[<p>When multiple internal applications need to send transactional emails, embedding SMTP transport logic and template rendering directly into each service introduces tight coupling, unpredictable request latency, and scattered credentials.</p>
<p><strong>Hermes</strong> is an open-source, multi-tenant transactional email gateway designed to centralize delivery, template management, and access control. To prevent SMTP network latency (often 500ms to 2s per handshake) and third-party throttling from blocking upstream callers, the system strictly separates HTTP request intake from email execution using an asynchronous BullMQ/Redis queue. Sensitive data is protected using two distinct cryptographic models: stored SMTP credentials and OAuth tokens are encrypted at rest using <strong>AES-256-GCM</strong>, while programmatic API keys are validated using <strong>Argon2id</strong> with indexed prefix lookups.</p>
<p>The project is structured as an ecosystem consisting of a REST API gateway, background workers, an administrative dashboard, and a client SDK:</p>
<ul>
<li><p><strong>API Gateway &amp; Workers:</strong> <a href="https://github.com/RuanLopes1350/hermes-api">github.com/RuanLopes1350/hermes-api</a></p>
</li>
<li><p><strong>Admin Dashboard:</strong> <a href="https://github.com/RuanLopes1350/hermes-front">github.com/RuanLopes1350/hermes-front</a></p>
</li>
<li><p><strong>TypeScript Client SDK:</strong> <a href="https://github.com/RuanLopes1350/hermes-client">github.com/RuanLopes1350/hermes-client</a></p>
</li>
<li><p><strong>NPM Package:</strong> <a href="https://www.npmjs.com/package/@ruanlopes1350/hermes-client"><code>@ruanlopes1350/hermes-client</code></a></p>
</li>
</ul>
<h2><strong>1. Problem Statement and Architectural Motivation</strong></h2>
<p>In monolithic architectures or microservice networks without a dedicated gateway, handling transactional emails presents three recurring technical issues:</p>
<ol>
<li><p><strong>HTTP Blocking &amp; Latency Spikes:</strong> Connecting directly to an SMTP server (e.g., via Nodemailer in Express or NestJS) during a user sign-up or password reset forces the HTTP thread to wait for DNS resolution, TLS handshakes, and SMTP <code>RCPT TO</code>/<code>DATA</code> acknowledgments before responding to the user.</p>
</li>
<li><p><strong>Failure Handling and Retries:</strong> If an SMTP server experiences transient network timeouts, services without persistent queues either drop the email or require complex local retry logic that can trigger duplicate dispatches or database locking.</p>
</li>
<li><p><strong>Key and Credential Management:</strong> Rotating SMTP credentials or granting sending permissions across multiple distinct services creates configuration drift. If an integration API key leaks, revoking it manually across multiple deployment environments causes unexpected downtime.</p>
</li>
</ol>
<p>Hermes addresses these issues by acting as an intermediary service with logical tenant isolation, asynchronous worker execution, and automated cryptographic key rotation.</p>
<pre><code class="language-plaintext">┌─────────────────────────────────────────────────────────────┐
│           CLIENT SERVICES (@ruanlopes1350/hermes-client)    │
└──────────────────────────────┬──────────────────────────────┘
   (POST /api/emails + X-API-Key)│       ▲ (HMAC Signed Webhook
                                 │       │  for Key Rotation)
                                 ▼       │
┌────────────────────────────────────────┴────────────────────┐
│                    HERMES REST API                          │
│        (Express + Better Auth + Drizzle ORM)                │
└──────────────┬──────────────────────────────┬───────────────┘
               │                              │
(Insert Email record as 'pending')            │ (Enqueue BullMQ Job)
               ▼                              ▼
┌──────────────────────────────┐ ┌────────────────────────────┐
│         POSTGRESQL           │ │        REDIS STORE         │
│  (Tenancy, Keys, Templates)  │ │ (Queues, PubSub, SSE Rate) │
└──────────────────────────────┘ └─────────────┬──────────────┘
                                               │
                                               ▼
                                 ┌────────────────────────────┐
                                 │       HERMES WORKER        │
                                 │   (Nodemailer / OAuth2)    │
                                 └─────────────┬──────────────┘
                                               │
                                               ▼ (TLS / SMTP)
                                 ┌────────────────────────────┐
                                 │        SMTP SERVERS        │
                                 │    (Gmail, Custom SMTP)    │
                                 └────────────────────────────┘
</code></pre>
<h2><strong>2. Decoupling HTTP Requests from Email Delivery</strong></h2>
<p>The system runs as separate Node.js processes:</p>
<h3><strong>The API Gateway (</strong><code>server.ts</code><strong>)</strong></h3>
<p>The API accepts incoming requests, performs input validation, authenticates the client via API Key or session cookie, records the message in PostgreSQL with a <code>pending</code> status, and pushes the job payload onto a BullMQ Redis queue. Once the job is pushed, the API immediately responds with a <code>201 Created</code> status code and the unique email record identifier.</p>
<p>Average API latency remains under 25ms, independent of whether the underlying SMTP server takes 200ms or 4000ms to respond.</p>
<h3><strong>The Background Worker (</strong><code>worker.ts</code><strong>)</strong></h3>
<p>The worker process listens to the queue, fetches the email payload, and resolves the required credential. It handles:</p>
<ul>
<li><p>Decrypting the SMTP password or generating fresh access tokens via Google OAuth2 (Gmail API).</p>
</li>
<li><p>Fetching and compiling the MJML template with dynamic Handlebars variables.</p>
</li>
<li><p>Dispatching the email via Nodemailer.</p>
</li>
<li><p>Updating the status in PostgreSQL (<code>sent</code> or <code>failed</code>) with execution metadata, logs, and error stacks.</p>
</li>
<li><p>Emitting Redis Pub/Sub events that feed Server-Sent Events (SSE) on the frontend for live monitoring.</p>
</li>
</ul>
<p>If the delivery fails due to network instability, BullMQ manages exponential backoff retries without blocking new incoming requests.</p>
<h2><strong>3. Cryptographic Implementation</strong></h2>
<p>Security requirements for a shared gateway require safeguarding credentials at rest and validating high-frequency API calls securely.</p>
<h3><strong>3.1 API Keys: Prefix Indexing + Argon2id</strong></h3>
<p>Traditional API key storage often uses SHA-256 (vulnerable to fast offline dictionary attacks if leaked) or standard bcrypt (which is compute-heavy when verifying every single HTTP request).</p>
<p>Hermes uses a structured key format:</p>
<pre><code class="language-plaintext">hm_b5c92a10.e4d3c2b1a0f9e8d7c6b5a4938271605f4581902...
└───┬──────┘ └───────────────────┬───────────────────┘
    │                            └─ 64-char Hex Secret (Argon2id hashed)
    └─ 8-char Hex Prefix (Indexed plain-text)
</code></pre>
<p><strong>Verification Flow:</strong></p>
<ol>
<li><p>The request provides the header <code>X-API-Key: hm_&lt;prefix&gt;.&lt;secret&gt;</code>.</p>
</li>
<li><p>The middleware parses the prefix (<code>hm_b5c92a10</code>) and queries the database:</p>
<pre><code class="language-sql">SELECT * FROM credential WHERE prefix = 'hm_b5c92a10' AND is_active = true;
</code></pre>
<p>This keeps database lookup at O(1)<em>O</em>(1) complexity via an indexed column, avoiding full-table scans.</p>
</li>
<li><p>Once the single matching record is retrieved, the secret is checked using <code>argon2.verify(key_hash, secret)</code>.</p>
</li>
<li><p>Argon2id provides memory-hard resistance against GPU-accelerated brute force without incurring the penalty of hashing against every record in the table.</p>
</li>
</ol>
<h3><strong>3.2 Sensitive Credentials at Rest: AES-256-GCM</strong></h3>
<p>SMTP passwords and OAuth2 refresh tokens are stored in the database as encrypted strings formatted as:</p>
<pre><code class="language-plaintext">&lt;iv_hex&gt;:&lt;auth_tag_hex&gt;:&lt;ciphertext_hex&gt;
</code></pre>
<ul>
<li><p><strong>Algorithm:</strong> <code>aes-256-gcm</code> (Galois/Counter Mode).</p>
</li>
<li><p><strong>IV:</strong> 16-byte initialization vector generated randomly for each encryption operation.</p>
</li>
<li><p><strong>Authentication Tag:</strong> 16-byte tag used to detect ciphertext tampering or corruption prior to decryption.</p>
</li>
<li><p><strong>Master Key (</strong><code>MASTER_KEY</code><strong>):</strong> Provided via environment variables, never committed or exposed over the API.</p>
</li>
</ul>
<hr />
<h2><strong>4. Zero-Downtime Automated Key Rotation</strong></h2>
<p>API keys should rotate periodically, but updating keys in production often leads to service interruptions if the client application is not updated in sync.</p>
<p>Hermes handles rotation via an asynchronous scheduled process:</p>
<ol>
<li><p><strong>Daily Cron Job (</strong><code>system.ts</code><strong>):</strong> BullMQ runs a daily check (<code>0 0 * * *</code>) searching for active credentials approaching their expiration date (<code>rotate_threshold_days</code>).</p>
</li>
<li><p><strong>Webhook-First Strategy:</strong> When a key is marked for rotation, the system generates the new key and immediately sends an HTTP POST request to the service's configured webhook URL.</p>
</li>
<li><p><strong>HMAC SHA-256 Signature:</strong> To prevent webhook spoofing, payloads are signed with an <code>X-Hermes-Signature</code> header calculated using the service's private webhook secret:</p>
<pre><code class="language-typescript">const signature = crypto
  .createHmac('sha256', webhookSecret)
  .update(JSON.stringify(payload))
  .digest('hex');
</code></pre>
</li>
<li><p><strong>Guaranteed Delivery:</strong> If the client application's webhook returns an error (or times out), the key rotation is aborted, the database is <strong>not</strong> updated, and BullMQ schedules a retry. The active key continues to work.</p>
</li>
<li><p><strong>Database Commit:</strong> Only when the client's webhook responds with <code>200 OK</code> does Hermes update the hash and expiration date in the database.</p>
</li>
</ol>
<h2><strong>5. Ecosystem Components</strong></h2>
<h3><strong>5.1 The Admin Dashboard (</strong><code>hermes-front</code><strong>)</strong></h3>
<p>Built with Next.js (App Router), Tailwind CSS, and Shadcn UI:</p>
<ul>
<li><p><strong>Tenancy Management:</strong> Create isolated services, add team members with role-based permissions (<code>admin</code>, <code>member</code>), and manage sending quotas.</p>
</li>
<li><p><strong>Credential Storage:</strong> Configure standard SMTP servers or authorize Gmail via Google OAuth2 with automatic token refresh.</p>
</li>
<li><p><strong>MJML Editor:</strong> Integrated Monaco Editor with real-time compilation preview and Handlebars syntax highlighting.</p>
</li>
<li><p><strong>Live Monitoring:</strong> Real-time throughput metrics, queue status, and email delivery states driven by Server-Sent Events (SSE).</p>
</li>
</ul>
<h3><strong>5.2 The Client SDK (</strong><code>@ruanlopes1350/hermes-client</code><strong>)</strong></h3>
<p>Published to NPM as a lightweight TypeScript client designed for integration:</p>
<ul>
<li><p><strong>Fluent Builder:</strong> Construct dispatches with chaining (<code>.to()</code>, <code>.subject()</code>, <code>.useTemplate()</code>, <code>.send()</code>).</p>
</li>
<li><p><strong>Framework Middleware:</strong> Built-in webhook handlers for Express, Fastify, and Next.js that verify signatures and apply new keys in memory or <code>.env</code> automatically.</p>
</li>
<li><p><strong>Storage Adapters:</strong> Decoupled storage interfaces (<code>MemoryAdapter</code>, <code>EnvAdapter</code>, or custom implementations for Redis/DynamoDB).</p>
</li>
<li><p><strong>Resilience:</strong> Automatic retries with jitter and exponential backoff for transient HTTP errors (408, 429, 5xx).</p>
</li>
</ul>
<pre><code class="language-typescript">import { HermesClient, MemoryAdapter } from '@ruanlopes1350/hermes-client';

const hermes = new HermesClient({
  baseUrl: 'https://hermes.internal.domain',
  storageAdapter: new MemoryAdapter(process.env.HERMES_API_KEY!),
});

await hermes.email()
  .to('engineer@company.com')
  .subject('Weekly Digest')
  .useTemplate('cltmpl_engineering_digest', { name: 'Alex' })
  .send();
</code></pre>
<h2><strong>6. Technical Tradeoffs and Limitations</strong></h2>
<p>No architecture is without tradeoffs. Building Hermes revealed several constraints that should be evaluated before deploying:</p>
<ol>
<li><p><strong>Operational Footprint vs. SaaS:</strong> Running Hermes requires managing a Node.js runtime, PostgreSQL, Redis, and a background worker container. For small projects that send fewer than 50 emails a day, using a managed API (such as Resend or Postmark) has lower maintenance overhead. Hermes is suited for organizations needing internal data residency, multi-service governance, and private SMTP infrastructure.</p>
</li>
<li><p><strong>Argon2id CPU Overhead:</strong> While prefix indexing limits lookups to O(1)<em>O</em>(1), Argon2id verification is deliberately CPU-intensive to resist brute-forcing. Under heavy traffic spikes (&gt;500 requests/sec directly on the API), CPU utilization on the API container rises significantly. For extreme throughput, request-rate caching or offloading verification to an edge reverse-proxy may be necessary.</p>
</li>
<li><p><strong>Docker Compose Auto-Scaler Constraints:</strong> The included worker scaling script (<code>scaler.ts</code>) evaluates Redis queue depth and adjusts Worker replica counts by communicating with the local Docker daemon socket. While sufficient for single-host VPS deployments, this model does not natively orchestrate across distributed clusters (e.g., Kubernetes HPA or Nomad).</p>
</li>
<li><p><strong>Deliverability Responsibility:</strong> Hermes handles routing, queuing, formatting, and retries. However, it does not replace email deliverability fundamentals. SPF, DKIM, DMARC records, and IP reputation must still be configured on the destination SMTP server or upstream relay provider.</p>
</li>
</ol>
<h2><strong>7. Conclusion</strong></h2>
<p>By separating API ingestion from transport execution and applying modern cryptography to key management, Hermes provides a structured pattern for transactional email routing in self-hosted environments.</p>
<p>All components of the project are open-source:</p>
<ul>
<li><p>Review the source code and documentation: <a href="https://github.com/RuanLopes1350/hermes-api">hermes-api</a></p>
</li>
<li><p>Explore the frontend: <a href="https://github.com/RuanLopes1350/hermes-front">hermes-front</a></p>
</li>
<li><p>Inspect the client library: <a href="https://github.com/RuanLopes1350/hermes-client">hermes-client</a> on GitHub and <a href="https://www.npmjs.com/package/@ruanlopes1350/hermes-client">NPM</a>.</p>
</li>
</ul>
]]></content:encoded></item></channel></rss>