Go to file
Fabian Reinartz cc5aab6cff provider/boltmem: add alert subscription.
This commit implements the Subscribe() and GetPending()
methods for the bolt based alert provider.
2016-05-03 12:46:34 +02:00
config Move SMTP auth to the config file 2016-04-16 16:41:55 -04:00
doc Fix a typo in simple.yml 2016-04-20 22:10:28 -04:00
notify Add resolved alerts to pushover template 2016-04-27 10:55:06 +02:00
provider provider/boltmem: add alert subscription. 2016-05-03 12:46:34 +02:00
template provider/boltmem: add alert subscription. 2016-05-03 12:46:34 +02:00
test Deal with changed webhook format in tests 2016-02-12 11:00:51 +01:00
types provider/boltmem: Add Put/Get for BoltDB provider 2016-05-03 12:46:34 +02:00
ui provider/boltmem: add alert subscription. 2016-05-03 12:46:34 +02:00
vendor vendor: add godebug/pretty 2016-05-03 12:39:22 +02:00
version Merge pull request #297 from prometheus/goversion-buildinfo 2016-04-23 15:18:17 +02:00
.dockerignore New release process using docker, circleci and a centralized 2016-04-22 21:46:53 +02:00
.gitignore New release process using docker, circleci and a centralized 2016-04-22 21:46:53 +02:00
.promu.yml *: bump default Go version to 1.6.2 2016-05-03 12:39:21 +02:00
.travis.yml *: bump default Go version to 1.6.2 2016-05-03 12:39:21 +02:00
api.go Merge pull request #251 from prometheus/typo 2016-02-23 14:03:19 -05:00
AUTHORS.md Update Fabian's email address 2016-03-24 17:02:42 +01:00
CHANGELOG.md Merge remote-tracking branch 'origin/release-0.1' 2016-04-08 16:31:01 -04:00
circle.yml ci: fix inconsistent indentation 2016-05-02 14:51:26 +02:00
CONTRIBUTING.md License cleanup. 2015-01-22 15:45:23 +01:00
dispatch_test.go Move send_resolved parameter to notifier configuration 2015-12-17 13:43:56 +01:00
dispatch.go Move send_resolved parameter to notifier configuration 2015-12-17 13:43:56 +01:00
Dockerfile New release process using docker, circleci and a centralized 2016-04-22 21:46:53 +02:00
inhibit.go PR with changes after code review 2015-11-23 18:24:57 +01:00
LICENSE License cleanup. 2015-01-22 15:45:23 +01:00
main.go Merge pull request #301 from mpchadwick/listening-port 2016-04-12 02:13:23 -04:00
Makefile New release process using docker, circleci and a centralized 2016-04-22 21:46:53 +02:00
NOTICE License cleanup. 2015-01-22 15:45:23 +01:00
README.md New release process using docker, circleci and a centralized 2016-04-22 21:46:53 +02:00
route_test.go Move send_resolved parameter to notifier configuration 2015-12-17 13:43:56 +01:00
route.go Move send_resolved parameter to notifier configuration 2015-12-17 13:43:56 +01:00
web.go Enable debug endpoint 2016-03-15 20:07:58 +01:00

Alertmanager Build Status

CircleCI Docker Stars Docker Pulls Image Size Image Layers

The Alertmanager handles alerts sent by client applications such as the Prometheus server. It takes care of deduplicating, grouping, and routing them to the correct receiver integration such as email, PagerDuty, or OpsGenie. It also takes care of silencing and inhibition of alerts.

Installation

Build dependencies

These dependencies are necessary for building Alertmanager. There are no runtime dependencies, as the resulting binary is statically linked.

Debian family:

sudo apt-get install build-essential libc6-dev

Red Hat family:

sudo yum install glibc-static

Compiling the binary

The current version has to be run from the repository folder as UI assets and notification templates are not yet statically compiled into the binary.

You can either go get it:

$ GO15VENDOREXPERIMENT=1 go get github.com/prometheus/alertmanager
# cd $GOPATH/src/github.com/prometheus/alertmanager
$ alertmanager -config.file=<your_file>

Or checkout the source code and build manually:

$ mkdir -p $GOPATH/src/github.com/prometheus
$ cd $GOPATH/src/github.com/prometheus
$ git clone https://github.com/prometheus/alertmanager.git
$ cd alertmanager
$ make build
$ ./alertmanager -config.file=<your_file>

Example

This is an example configuration that should cover most relevant aspects of the new YAML configuration format. Authoritative source for now is the code.

global:
  # The smarthost and SMTP sender used for mail notifications.
  smtp_smarthost: 'localhost:25'
  smtp_from: 'alertmanager@example.org'

# The root route on which each incoming alert enters.
route:
  # The root route must not have any matchers as it is the entry point for
  # all alerts. It needs to have a receiver configured so alerts that do not
  # match any of the sub-routes are sent to someone.
  receiver: 'team-X-mails'
  
  # The labels by which incoming alerts are grouped together. For example,
  # multiple alerts coming in for cluster=A and alertname=LatencyHigh would
  # be batched into a single group.
  group_by: ['alertname', 'cluster']

  # When a new group of alerts is created by an incoming alert, wait at
  # least 'group_wait' to send the initial notification.
  # This way ensures that you get multiple alerts for the same group that start
  # firing shortly after another are batched together on the first
  # notification.
  group_wait: 30s

  # When the first notification was sent, wait 'group_interval' to send a batch
  # of new alerts that started firing for that group.
  group_interval: 5m

  # If an alert has successfully been sent, wait 'repeat_interval' to
  # resend them.
  repeat_interval: 3h

  # All the above attributes are inherited by all child routes and can 
  # overwritten on each.

  # The child route trees.
  routes:
  # This routes performs a regular expression match on alert labels to
  # catch alerts that are related to a list of services.
  - match_re:
      service: ^(foo1|foo2|baz)$
    receiver: team-X-mails

    # The service has a sub-route for critical alerts, any alerts
    # that do not match, i.e. severity != critical, fall-back to the
    # parent node and are sent to 'team-X-mails'
    routes:
    - match:
        severity: critical
      receiver: team-X-pager

  - match:
      service: files
    receiver: team-Y-mails

    routes:
    - match:
        severity: critical
      receiver: team-Y-pager

  # This route handles all alerts coming from a database service. If there's
  # no team to handle it, it defaults to the DB team.
  - match:
      service: database

    receiver: team-DB-pager
    # Also group alerts by affected database.
    group_by: [alertname, cluster, database]

    routes:
    - match:
        owner: team-X
      receiver: team-X-pager

    - match:
        owner: team-Y
      receiver: team-Y-pager


# Inhibition rules allow to mute a set of alerts given that another alert is
# firing.
# We use this to mute any warning-level notifications if the same alert is
# already critical.
inhibit_rules:
- source_match:
    severity: 'critical'
  target_match:
    severity: 'warning'
  # Apply inhibition if the alertname is the same.
  equal: ['alertname']


receivers:
- name: 'team-X-mails'
  email_configs:
  - to: 'team-X+alerts@example.org'

- name: 'team-X-pager'
  email_configs:
  - to: 'team-X+alerts-critical@example.org'
  pagerduty_configs:
  - service_key: <team-X-key>

- name: 'team-Y-mails'
  email_configs:
  - to: 'team-Y+alerts@example.org'

- name: 'team-Y-pager'
  pagerduty_configs:
  - service_key: <team-Y-key>

- name: 'team-DB-pager'
  pagerduty_configs:
  - service_key: <team-DB-key>

Testing

If you want to test the new Alertmanager while running the current version, you can mirror traffic to the new one with a simple nginx configuration similar to this:

server {
  server_name <your_current_alertmanager>;
  location / {
    proxy_pass  http://localhost:9093;
    post_action @forward;
  }
  location @forward {
    proxy_pass http://<your_new_alertmanager>:9093;
  }
}

Architecture