安装Prometheus以及其相关组件

Prometheus的核心组件

Prometheus Server，主要用于抓取数据和存储时序数据，另外还提供查询和 Alert Rule 配置管理。
client libraries，用于对接 Prometheus Server, 可以查询和上报数据。
push gateway ，用于批量，短期的监控数据的汇总节点，主要用于业务数据汇报等。
各种汇报数据的 exporters ，例如汇报机器数据的 node_exporter, 汇报 MongoDB 信息的 MongoDB exporter 等等。
用于告警通知管理的 alertmanager 。

开始安装

安装Prometheus

安装的方式很多，例如docker和编译。这里是下载官网编译好的程序。

  $ cd /usr/local/src
  $ wget https://github.com/prometheus/prometheus/releases/download/v2.3.2/prometheus-2.3.2.linux-amd64.tar.gz

  $ tar xf  prometheus-2.3.2.linux-amd64.tar.gz -C /usr/local
  $ ln -s /usr/local/prometheus-2.3.2.linux-amd64 /usr/local/prometheus

  $ cd /usr/local/prometheus
  $ ./prometheus --version

  $ groupadd prometheus
  $ useradd -g prometheus -m -d /var/lib/prometheus -s /sbin/nologin prometheus

添加启动脚本prometheus.service

  $ vim /etc/systemd/system/prometheus.service

  [Unit]
  Description=prometheus.service
  After=network.target
  [Service]
  Type=simple
  User=prometheus
  ExecStart=/usr/local/prometheus/prometheus --config.file=/usr/local/prometheus/prometheus.yml --storage.tsdb.path=/var/lib/prometheus
  Restart=on-failure
  [Install]
  WantedBy=multi-user.target

修改prometheus.yml

  $ cp -rvp prometheus.yml{,.old}
  $ vim prometheus.yml
  scrape_configs:
      - job_name: 'prometheus'
          static_configs:
          - targets: ['192.168.99.90:9090']

启动prometheus

  $ systemctl daemon-reload
  $ systemctl enable prometheus.service
  $ systemctl start prometheus.service

Prometheus Server 默认端口是 9090，当 Prometheus 启动后，你可以通过浏览器来访问 http://IP:9090。

安装node_exporter

开始安装服务(有两种方式)

方法一：

  $ wget https://github.com/prometheus/node_exporter/releases/download/v0.16.0/node_exporter-0.16.0.linux-amd64.tar.gz
  $ tar -zxvf node_exporter-0.16.0.linux-amd64.tar.gz
  $ mv node_exporter-0.16.0.linux-amd64 /usr/local/prometheus/node_exporter-0.16.0.linux-amd64
  $ ln -s /usr/local/prometheus/node_exporter-0.16.0.linux-amd64 /usr/local/prometheus/node_exporter

添加启动脚本node_exporter.service

  $ vim /etc/systemd/system/node_exporter.service

  [Unit]
  Description=node_exporter
  After=network.target
  [Service]
  Type=simple
  User=prometheus
  Group=prometheus
  ExecStart=/usr/local/prometheus/node_exporter/node_exporter
  Restart=on-failure
  [Install]
  WantedBy=multi-user.target

  $ systemctl daemon-reload
  $ systemctl enable node_exporter.service
  $ systemctl start node_exporter.service

方法二：

  $ docker pull prom/node-exporter:v0.16.0
  $ docker run --name node_exporter -d -p 9100:9100 prom/node-exporter:v0.16.0

修改prometheus.yml

安装完之后，打开prometheus.yml 文件, 在 scrape_configs 中添加如下配置：

  scrape_configs:
  - job_name: 'linux-game-XX.XX.XX.XX'
      static_configs:
      - targets: ["XXX.XXX.XXX.XXX:9100"]
          labels:
          instance: node-game-01

安装alertmanager

  $ groupadd alertmanager
  $ useradd -g alertmanager -s /sbin/nologin alertmanager

  $ wget https://github.com/prometheus/alertmanager/releases/download/v0.15.2/alertmanager-0.15.2.linux-amd64.tar.gz
  $ tar xf alertmanager-0.15.2.linux-amd64.tar.gz

  $ mv alertmanager-0.15.2.linux-amd64 /usr/local/prometheus/alertmanager-0.15.2.linux-amd64
  $ ln -s /usr/local/prometheus/alertmanager-0.15.2.linux-amd64 /usr/local/prometheus/alertmanager

  $ vim /etc/systemd/system/alertmanager.service

  [Unit]
  Description=alertmanager.service
  After=network.target
  [Service]
  Type=simple
  User=alertmanager
  Group=alertmanager
  ExecStart=/usr/local/prometheus/alertmanager/alertmanager --config.file="/usr/local/prometheus/alertmanager/alertmanager.yml"
  Restart=on-failure
  [Install]
  WantedBy=multi-user.target

  $ systemctl daemon-reload
  $ systemctl enable alertmanager.service
  $ systemctl start alertmanager.service

数据可视化(可选)

Prometheus自带Web UI，如果想更加直观的给用户展示数据，那么可以安装Grafana，即使 GPE (Grafana+Prometheus+Exporter）

具体的架构如下图：

安装Grafana

$ groupadd grafana
$ useradd -g grafana -m -s /sbin/nologin grafana

$ wget https://s3-us-west-2.amazonaws.com/grafana-releases/release/grafana-5.2.3.linux-amd64.tar.gz
$ tar xf grafana-5.2.3.linux-amd64.tar.gz -C /usr/local
$ ln -s /usr/local/grafana-5.2.3 /usr/local/grafana

$ chown grafana:grafana -R /usr/local/grafana

启动grafana

$ vim /etc/systemd/system/grafana-server.service

[Unit]
Description=grafana-server.service
After=network.target
[Service]
Type=notify
User=grafana
Group=grafana
ExecStart=/usr/local/grafana/bin/grafana-server --homepath /usr/local/grafana --config=/usr/local/grafana/conf/defaults.ini --pidfile= cfg:default.paths.logs=/usr/local/grafana/data/log cfg:default.paths.data=/usr/local/grafana/data cfg:default.paths.plugins=/usr/local/grafana/data/plugins
Restart=on-failure
[Install]
WantedBy=multi-user.target

$ systemctl daemon-reload
$ systemctl enable grafana-server.service
$ systemctl start grafana-server.service