Skip to main content
Version: 0.16

Alerts

Alerts notify administrators when a server metric, or a combination of metrics, crosses a defined threshold. The condition is evaluated continuously against the live metrics stream, and when it holds, a notification is emitted. Notifications can be delivered as an in-server event (which a webhook can forward downstream) or as an email message.

The trigger is an expression that can reference one or more metrics. Expressions support logical operators, so conditions such as store_foundationdb_error > 100 || store_s3_error > 100 can be built up directly. Metric identifiers in the expression use underscores in place of dots and hyphens (security_brute_force_ban rather than security.brute-force-ban) because the expression language restricts variable names to alphanumeric characters and underscores.

Enterprise feature

This feature is available exclusively in the Enterprise Edition of Stalwart and is not included in the Community Edition.

Configuration​

Each alert is represented by an Alert object (found in the WebUI under Settings › Telemetry › Alerts). The relevant fields are:

  • enable: whether the alert is active. Default true.
  • condition: the expression evaluated against incoming metrics. The alert fires when the expression evaluates to true.
  • eventAlert: event notification settings. A nested type with variants Disabled and Enabled.
  • emailAlert: email notification settings. A nested type with variants Disabled and Enabled.

Event notification​

When the Enabled variant of eventAlert is selected, the alert emits a telemetry.alert event whenever condition holds. The event can be captured by a WebHook so that downstream systems receive the notification.

The fields on the Enabled variant are:

  • eventMessage: message carried by the emitted event. May reference metric values using placeholders.

Email notification​

When the Enabled variant of emailAlert is selected, the server sends an email each time the condition becomes true. The fields on the Enabled variant are:

  • fromName: optional display name of the sender.
  • fromAddress: sender email address.
  • to: list of recipient addresses.
  • subject: subject line. May reference metric values using placeholders.
  • body: message body. May reference metric values using placeholders.

Example​

The following Alert fires when the count of FoundationDB or S3 errors exceeds one hundred. It raises an event notification carrying a message with the current counts and also sends an email with the same information:

{
  "enable": true,
  "condition": {
    "else": "store_foundationdb_error > 100 || store_s3_error > 100"
  },
  "eventAlert": {
    "@type": "Enabled",
    "eventMessage": "Database errors: FDB %{store.foundationdb-error}%, S3 %{store.s3-error}%"
  },
  "emailAlert": {
    "@type": "Enabled",
    "fromName": "Alert Subsystem",
    "fromAddress": "[email protected]",
    "to": ["[email protected]"],
    "subject": "Found %{store.foundationdb-error}% FDB and %{store.s3-error}% S3 errors",
    "body": "We found %{store.foundationdb-error}% FDB and %{store.s3-error}% S3 errors."
  }
}