Opsgenie shuts down April 2027. Faultline replaces paging and adds the detection layer Opsgenie never had. Migration guide →
Skip to content
Docs

CLI and heartbeats

Some failures are invisible from outside: a cron job that stopped running, a worker stuck on an empty queue. Heartbeats catch them by expecting a ping and alerting when it stops.

By · Published · Updated

How do heartbeats work?

  • You create a heartbeat check with an interval (how often the job should ping) and a grace period (extra slack for slow runs).
  • Your job pings a unique URL each time it succeeds.
  • If no ping arrives within interval plus grace, the check fails and goes through the same thresholds, incidents, escalation, and runbooks as any other monitor. The next ping recovers it.
  • The token in the URL is the only credential, so the job itself needs no API key. Treat the URL as a secret.
Timeline diagram: a job pings, then pings again one interval later. The next expected ping is missing, and once the grace period after it also passes with no ping, the check fails and goes through thresholds, incident, escalation, and runbooks. Example: a nightly backup with interval 86400 and grace 1800 fails 24.5 hours after its last ping.
A missed ping only counts once interval plus grace has passed.

How do I install the CLI?

With npm, on Node.js 18 or later. Then save an API key once, or set it as an environment variable.

npm install -g faultline-cli     # Node.js 18 or later

# One-time setup: saves your API key to ~/.faultline/config.json
faultline configure

# Or, on servers and in CI, use environment variables instead
export FAULTLINE_API_KEY=flt_your_key_here
export FAULTLINE_API_URL=https://api.fltln.io

Creating checks needs an API key (Settings, API Keys, on Pro and above). Pinging does not.

How do I create a heartbeat check?

Run faultline check create with an interval and a grace period, in seconds. It prints the ping URL.

faultline check create nightly-backup --type heartbeat \
  --interval 86400 --grace 1800

# Created heartbeat check "nightly-backup" (service clx8m2k3p0000abc123def456).
#
# Ping URL, call this at the end of your cron or worker run:
#   curl -fsS https://api.fltln.io/hb/hb_AbC123...
FlagDefaultMeaning
-t, --typeheartbeatheartbeat, or http to create a URL monitor (with --url)
-i, --interval60Seconds between expected pings
-g, --grace60Extra seconds before a missed ping counts as a failure
-d, --descriptionnoneOptional description

Push syntax

faultline push hb_AbC123...                          # bare token
faultline push https://api.fltln.io/hb/hb_AbC123...  # or the full ping URL
  • No API key needed. For a bare token it uses FAULTLINE_API_URL, or https://api.fltln.io if unset.
  • Gives up after 10 seconds, so a network problem never hangs your job.
  • Exits 1 on an unknown token, any HTTP error, or a network failure, so && chains and healthchecks see it. The token is masked in output.
  • The ping URL also accepts a plain GET or POST from curl, wget, or any HTTP client.

How do I monitor a cron job?

Chain the ping after the job with &&, so it fires only when the job succeeds. The schedule syntax is standard crontab(5).

# crontab -e
# Ping only if the backup succeeded. Cron has a minimal PATH, so use the
# full path to the CLI (find it with: command -v faultline).
0 3 * * * /usr/local/bin/backup.sh && /usr/local/bin/faultline push hb_AbC123...

# No Node.js on the box? curl does the same job:
0 3 * * * /usr/local/bin/backup.sh && curl -fsS -m 10 -X POST https://api.fltln.io/hb/hb_AbC123...

For a daily job, set --interval 86400 and a grace that covers your longest normal run, for example --grace 1800.

How do I monitor a queue worker?

A long-running worker never exits, so ping from inside the loop on a timer. This catches a crashed process and also a loop that is stuck, which a process monitor would miss.

# Python queue worker: prove the consume loop is alive, even when the queue
# is empty. Create the check with --interval 60 --grace 60.
import os, time, urllib.request

PING_URL = os.environ["FAULTLINE_HEARTBEAT_URL"]
PING_EVERY = 60
last_ping = 0.0

while True:
    if time.monotonic() - last_ping >= PING_EVERY:
        try:
            urllib.request.urlopen(urllib.request.Request(PING_URL, method="POST"), timeout=10)
        except Exception as e:
            print(f"heartbeat failed: {e}")   # never let a ping break the worker
        last_ping = time.monotonic()

    job = queue.receive(wait_seconds=20)      # your queue client here
    if job:
        handle(job)
// Node.js queue worker: same idea, throttled to one ping a minute.
const PING_URL = process.env.FAULTLINE_HEARTBEAT_URL;
let lastPing = 0;

for (;;) {
  if (Date.now() - lastPing >= 60_000) {
    fetch(PING_URL, { method: "POST", signal: AbortSignal.timeout(10_000) })
      .catch((e) => console.error("heartbeat failed:", e.message));
    lastPing = Date.now();
  }
  const job = await queue.receive({ waitSeconds: 20 }); // your queue client here
  if (job) await handle(job);
}

How do I monitor a Docker container?

Let Docker's own healthcheck send the ping, using the Dockerfile HEALTHCHECK instruction or the Compose healthcheck attribute. It runs only while the container is running, so a crashed or stopped container stops the pings too.

# Node-based image: install the CLI and let Docker's healthcheck push the
# heartbeat. If the app's own health endpoint fails, or the container stops,
# pings stop and Faultline opens an incident.
FROM node:22-slim
RUN apt-get update && apt-get install -y --no-install-recommends curl  && rm -rf /var/lib/apt/lists/*  && npm install -g faultline-cli
# ... your app ...
HEALTHCHECK --interval=60s --timeout=15s \
  CMD curl -fsS http://localhost:8080/health && faultline push "$FAULTLINE_HEARTBEAT_TOKEN" || exit 1
# docker-compose.yml, for an image without Node.js (uses curl instead of the CLI)
services:
  worker:
    image: my-worker:latest
    environment:
      FAULTLINE_HEARTBEAT_URL: https://api.fltln.io/hb/hb_AbC123...
    healthcheck:
      test: ["CMD-SHELL", "curl -fsS http://localhost:8080/health && curl -fsS -m 10 -X POST \"$FAULTLINE_HEARTBEAT_URL\" || exit 1"]
      interval: 60s
      timeout: 15s

To watch a container from outside instead (no changes to the image), add a Docker monitor in the dashboard, which reads container state through the Docker remote API.

Other CLI commands

CommandWhat it does
faultline ingest "msg" --severity MAJORReport an internal failure; opens or updates an incident
faultline maintenance start|end <serviceId>Pause and resume checks around a deploy
faultline ask "why is checkout-api degraded?"Ask the AI assistant, using your own Anthropic API key

More docs

Try it on a real service.

5 monitors free, no card required.