intertwingly

It’s just data

Unified Logging for Multi-Region Rails with Navigator, Vector, and NATS

The showcase application runs across three environments: 8 Fly.io machines globally, a Hetzner bare metal server in Helsinki, and a Mac Mini at my home in Raleigh. Each environment has different constraints, but all need the same thing: a unified way to view logs.

Fly.io uses Vector internally and provides NATS access as a built-in feature. All logs from Fly.io machines automatically flow to NATS subjects that you can subscribe to. No additional configuration needed—it just works via fly logs and NATS proxy.

Hetzner setup is already documented in Monitoring with Vector and NATS. Vector collects Docker logs and forwards them to Fly.io's NATS instance, unifying logs across both clouds.

This post focuses on completing the picture by adding logging to rubymini, bringing the home development/admin server into the same unified logging system.

Overall Design

1. NATS as Universal Broker

NATS is a lightweight, cloud-native messaging system that's perfect for log distribution:

For Fly.io machines, logs flow directly to Fly's NATS instance. For Hetzner and rubymini, Vector forwards logs to NATS.

2. Vector for Log Collection

Vector is a high-performance observability pipeline that collects, transforms, and routes logs:

For Fly.io, Vector is already running—you just consume from NATS. For rubymini, Navigator's built-in Vector integration means zero external configuration—Navigator starts and manages the Vector process.

3. Bun Logger App for Viewing

A simple web application provides the viewing interface:

The logger app runs as a separate Fly.io application for global access, plus locally on rubymini for development/debugging.

Implementation: Mac Mini (rubymini)

The Mac Mini runs Navigator via launchd, with logs forwarded to a local NATS instance for development.

NATS Installation

Install NATS server via Homebrew:

brew install nats-server

Create launchd plist for NATS:

<!-- ~/Library/LaunchAgents/showcase-nats.plist -->
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"
  "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
  <key>Label</key>
  <string>net.intertwingly.showcase-nats</string>

  <key>ProgramArguments</key>
  <array>
    <string>/opt/homebrew/bin/nats-server</string>
    <string>--config</string>
    <string>/Users/rubys/showcase-nats.conf</string>
  </array>

  <key>RunAtLoad</key>
  <true/>

  <key>KeepAlive</key>
  <true/>

  <key>StandardOutPath</key>
  <string>/Users/rubys/Library/Logs/showcase-nats.log</string>

  <key>StandardErrorPath</key>
  <string>/Users/rubys/Library/Logs/showcase-nats.log</string>
</dict>
</plist>

Load the service:

launchctl load ~/Library/LaunchAgents/showcase-nats.plist

Navigator automatically starts and manages Vector when enabled in the configuration. The application generates config/navigator.yml dynamically with:

logging:
  format: json
  vector:
    enabled: true
    socket: /tmp/navigator-vector.sock
    config: /Users/rubys/git/showcase/config/vector.toml

When vector.enabled is true, Navigator automatically:

Create config/vector.toml to configure Vector's log forwarding:

# Global configuration
data_dir = "/Users/rubys/git/showcase/log/vector"

# Receive logs from Navigator via Unix socket
[sources.navigator]
type = "socket"
mode = "unix"
path = "/tmp/navigator-vector.sock"

# Parse and wrap JSON logs from Navigator
[transforms.parse_json]
type = "remap"
inputs = ["navigator"]
source = '''
  # Parse the JSON message from Navigator
  original = parse_json!(string!(.message))

  # Add source field if missing (for HTTP access logs)
  if !exists(original.source) {
    if exists(original.client_ip) {
      original.source = "access"
    } else {
      original.source = "unknown"
    }
  }

  # Wrap in logfiler-expected format
  .timestamp = original."@timestamp"
  .message = encode_json(original)
  .log = {"level": "info"}
  .fly = {
    "app": {
      "name": "showcase",
      "instance": "rubymini"
    },
    "region": "rdu"
  }
  .source = original.source
'''

# Forward to NATS
[sinks.nats]
type = "nats"
inputs = ["parse_json"]
url = "nats://localhost:4222"
subject = "logs.showcase.rubymini."
encoding.codec = "json"

The transform does two things:

  1. Adds source identification - Navigator access logs don't have a source field, so Vector adds source="access" for HTTP logs
  2. Wraps in logfiler format - The logger app expects logs in a specific structure with timestamp, message, fly.region, and log.level fields. Navigator's pure JSON logs need to be wrapped in this format.

Navigator now forwards logs to the local NATS instance, making them available to the logger app.

Logger App Deployment

Deploy the Bun logger app as a launchd service:

<!-- ~/Library/LaunchAgents/showcase-logger.plist -->
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"
  "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
  <key>Label</key>
  <string>net.intertwingly.showcase-logger</string>

  <key>ProgramArguments</key>
  <array>
    <string>/opt/homebrew/bin/bun</string>
    <string>run</string>
    <string>/Users/rubys/showcase-logger/index.js</string>
  </array>

  <key>WorkingDirectory</key>
  <string>/Users/rubys/showcase-logger</string>

  <key>EnvironmentVariables</key>
  <dict>
    <key>PORT</key>
    <string>9001</string>
    <key>NATS_URL</key>
    <string>nats://localhost:4222</string>
  </dict>

  <key>RunAtLoad</key>
  <true/>

  <key>KeepAlive</key>
  <true/>

  <key>StandardOutPath</key>
  <string>/Users/rubys/Library/Logs/showcase-logger.log</string>

  <key>StandardErrorPath</key>
  <string>/Users/rubys/Library/Logs/showcase-logger.log</string>
</dict>
</plist>

Load the service:

launchctl load ~/Library/LaunchAgents/showcase-logger.plist

The logger can be accessed two ways:

Logger Application

The logger app is a TypeScript/Bun application that subscribes to NATS and streams logs to a web interface. The actual implementation is more sophisticated than shown here, with separate modules for log filtering, formatting, WebSocket broadcasting, and Sentry integration.

Below is a simplified conceptual example showing the core pattern. For the complete implementation, see the showcase repository.

Backend (Simplified Example)

import { connect } from 'nats';
import { serve } from 'bun';

const nats = await connect({
  servers: process.env.NATS_URL || 'nats://localhost:4222'
});

// WebSocket clients
const clients = new Set();

// Subscribe to all showcase logs
const sub = nats.subscribe('logs.showcase.*.*');
(async () => {
  for await (const msg of sub) {
    const log = JSON.parse(msg.data);
    // Broadcast to all connected WebSocket clients
    for (const client of clients) {
      client.send(JSON.stringify(log));
    }
  }
})();

// Web server
serve({
  port: process.env.PORT || 9001,
  fetch(req, server) {
    const url = new URL(req.url);

    if (url.pathname === '/logs') {
      // Serve HTML viewer
      return new Response(HTML_VIEWER, {
        headers: { 'Content-Type': 'text/html' }
      });
    }

    if (url.pathname === '/ws') {
      // Upgrade to WebSocket
      if (server.upgrade(req)) {
        return; // WebSocket upgraded
      }
      return new Response('WebSocket upgrade failed', { status: 400 });
    }

    return new Response('Not found', { status: 404 });
  },

  websocket: {
    open(ws) {
      clients.add(ws);
    },
    close(ws) {
      clients.delete(ws);
    }
  }
});

Frontend (HTML Viewer)

<!DOCTYPE html>
<html>
<head>
  <title>Showcase Logs</title>
  <style>
    body { font-family: monospace; padding: 20px; background: #1e1e1e; color: #d4d4d4; }
    #logs { white-space: pre-wrap; }
    .error { color: #f44747; }
    .warn { color: #ff8c00; }
    .info { color: #4ec9b0; }
    .debug { color: #858585; }
    input { width: 400px; padding: 5px; margin: 10px 0; }
  </style>
</head>
<body>
  <h1>Showcase Logs</h1>
  <input type="text" id="filter" placeholder="Filter logs (regex)..." />
  <button onclick="clearLogs()">Clear</button>
  <div id="logs"></div>

  <script>
    const ws = new WebSocket('ws://' + location.host + '/ws');
    const logsDiv = document.getElementById('logs');
    const filterInput = document.getElementById('filter');

    ws.onmessage = (event) => {
      const log = JSON.parse(event.data);

      // Apply filter
      const filter = filterInput.value;
      if (filter && !new RegExp(filter).test(JSON.stringify(log))) {
        return;
      }

      // Format log entry
      const time = new Date(log.timestamp).toLocaleTimeString();
      const level = log.level || 'info';
      const message = log.message || JSON.stringify(log);

      const line = document.createElement('div');
      line.className = level;
      line.textContent = `[${time}] [${log.app || 'unknown'}] ${message}`;

      logsDiv.appendChild(line);

      // Auto-scroll to bottom
      logsDiv.scrollTop = logsDiv.scrollHeight;

      // Limit to 1000 lines
      while (logsDiv.children.length > 1000) {
        logsDiv.removeChild(logsDiv.firstChild);
      }
    };

    function clearLogs() {
      logsDiv.innerHTML = '';
    }
  </script>
</body>
</html>

Features

Try It Yourself

All components are open source:

Conclusion

Distributed applications need centralized logs. When your Rails app runs across Fly.io, Hetzner, and a Mac Mini at home, unified logging eliminates context-switching and speeds up debugging.

The pattern is straightforward:

  1. NATS as universal broker - Publish logs from all sources
  2. Vector for collection - Navigator integration makes it zero-config
  3. Simple web viewer - Real-time streaming to browser

The result is unified logging without operational complexity: no vendor lock-in, no complex configuration, no logging fees.

Start simple. Add Vector to Navigator. Deploy NATS and logger. View logs from all environments in one interface.

Debugging multi-region issues shouldn't require juggling three log systems—it should just require opening a browser.

For the broader architectural context of showcase's multi-region deployment, see Shared-Nothing Multi-Tenancy with SQLite, TurboCable, and Navigator.

For complete implementation details including launchd plists, Vector configuration templates, and logger app source code, see the showcase repository.