试试Beats系列

在2015年的最后一次Elastic Search学习会上，我了解了Beats。
由于对于这次也抱有对elastic能够表现不错的期望，我决定亲自动手试试。

基本上可以按照官方安装指南中的步骤进行构建。
https://www.elastic.co/guide/en/beats/libbeat/current/index.html

【环境】
服务器（Elasticsearch、Kibana、Logstash/ CentOS7）
客户端（Packetbeat、Filebeat、Topbeat/ CentOS7，在运行Chef的服务器上）

【建立并注意到的要点】
・弹性搜索（Elasticsearch）的位置似乎从/opt移至/etc
・日志管道（Logstash）将位于/opt目录下创建
・封包跟踪器（Packetbeat）、性能分析器（Topbeat）、文件跟踪器（Filebeat）将位于/etc目录下创建

安装和配置ElasticSearch

#yum install java-1.7.0-openjdk
#curl -L -O #https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-2.1.1.rpm
#rpm -i elasticsearch-2.1.1.rpm
#systemctl daemon-reload
#systemctl enable elasticsearch.service
#systemctl start elasticsearch.service
#systemctl status elasticsearch.service
jsonで値が取れるか確認
#curl http://127.0.0.1:9200

network.host: 192.168.22.70　コメントアウト
http.port: 9200　コメントアウト

#systemctl stop elasticsearch
#systemctl status elasticsearch
#systemctl start elasticsearch
#systemctl status elasticsearch


#curl http://192.168.22.70:9200
 {
   "name" : "Malekith the Accursed",
   "cluster_name" : "elasticsearch",
   "version" : {
     "number" : "2.1.1",
     "build_hash" : "40e2c53a6b6c2972b3d13846e450e66f4375bd71",
     "build_timestamp" : "2015-12-15T13:05:55Z",
     "build_snapshot" : false,
     "lucene_version" : "5.3.1"
   },
   "tagline" : "You Know, for Search"
 }

安装和配置Logstash。

#curl -L -O #https://download.elastic.co/logstash/logstash/packages/centos/logstash-2.1.1-1.noarch.rpm
#rpm -i logstash-2.1.1-1.noarch.rpm
#rpm -qa | grep log
#cd /opt/logstash/
#ls
#./bin/plugin install logstash-input-beats

input {
  beats {
    port => 5044
  }
}

output {
  elasticsearch {
    hosts => "192.168.22.70:9200"
    sniffing => true
    manage_template => false
    index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
    document_type => "%{[@metadata][type]}"
  }
}

#systemctl enable logstash
#systemctl start logstash
#systemctl status logstash


#systemctl start logstash.service
#systemctl status logstash.service

Kibana的安装和配置

#curl -L -O https://download.elastic.co/kibana/kibana/kibana-4.3.1-linux-x64.tar.gz
#tar xzvf kibana-4.3.1-linux-x64.tar.gz
#cd kibana-4.3.1-linux-x64/
#./bin/kibana &

http://127.0.0.1:5601確認

beats用ダッシュボードを適用
#curl -L -O http://download.elastic.co/beats/dashboards/beats-dashboards-1.0.1.tar.gz
#tar xzvf beats-dashboards-1.0.1.tar.gz
#cd beats-dashboards-1.0.1/
#ls
#./load.sh
#cd kibana-4.3.1-linux-x64
#cp kibana.yml kibana.yml.org

server.port: 5601 コメントを取る
server.host: "192.168.22.70"　コメント取る
elasticsearch.url: "http://192.168.22.70:9200"
pid.file: /var/run/kibana.pid　コメントとる
logging.verbose: true 一度確認でコメントを外して後でfalseにする

客户端（packet beat，file beat，top beat）分别在yml文件中指定了使用Elasticsearch（以注释形式）。
■安装packet beat

#yum install libpcap
#curl -L -O https://download.elastic.co/beats/packetbeat/packetbeat-1.0.1-x86_64.rpm
#rpm -vi packetbeat-1.0.1-x86_64.rpm
#rpm -qa | grep packet
#cp packetbeat.yml packetbeat.yml.org

以下の該当箇所をコメントアウト、記載変更
ports: [80, 8080, 8081, 5000, 8002]
 hosts: ["192.168.22.70:9200"]

#curl -XPUT 'http://192.168.22.70:9200/_template/packetbeat' -d@/etc/packetbeat/packetbeat.template.json

#systemctl start packetbeat
#systemctl status packetbeat

#curl -XGET 'http://192.168.22.70:9200/packetbeat-*/_search?pretty'
 {
   "took" : 31,
   "timed_out" : false,
   "_shards" : {
     "total" : 15,
     "successful" : 15,
     "failed" : 0
   },
   "hits" : {
     "total" : 285387,
     "max_score" : 1.0,
     "hits" : [ {
       "_index" : "packetbeat-2016.01.04",
       "_type" : "pgsql",
       "_id" : "AVILHywD_QULozThr4Lw",
       "_score" : 1.0,
       "_source":{"@timestamp":"2016-01-04T05:32:18.239Z","beat":{"hostname":"chef01","name":"chef01"},"bytes_in":35,"bytes_out":59,"client_ip":"127.0.0.1","client_port":32847,"client_proc":"","client_server":"chef01","count":1,"direction":"out","ip":"127.0.0.1","method":"SELECT","pgsql":{"error_code":"","error_message":"","error_severity":"","iserror":false,"num_fields":1,"num_rows":1},"port":5432,"proc":"","query":"SELECT 'pong' as ping LIMIT 1","responsetime":2,"server":"chef01","status":"OK","type":"pgsql"}
     }, 以下略
}

#systemctl start topbeat
#systemctl status topbeat

■ 安装Filebeat

#curl -L -O https://download.elastic.co/beats/filebeat/filebeat-1.0.1-x86_64.rpm
#rpm -vi filebeat-1.0.1-x86_64.rpm
#cp filebeat.yml filebeat.yml.org

 - /var/log/messages　指定
hosts: ["192.168.22.70:9200"]　指定

#curl -XPUT 'http://192.168.22.70:9200/_template/filebeat?pretty' -d@/etc/filebeat/filebeat.template.json

#systemctl start filebeat
#systemctl status filebeat

■ 今后
暂时停止。虽然适用产品数量较少，但有主要的度量指标，因此还需再试试看。