ELK 通过 logstash 收集单个/多个日志文件

一、收集单个日志文件

注意事项：

1. logstah 服务默认启动用户和组是 logstash
2. 被收集的日志文件有读的权限并对写入的文件有写权限
3. 而 logstash 是普通用户

1.1 编辑 logstash 配置文件

vim /etc/logstash/conf.d/test.conf
input {
  file {
    path => "/var/log/syslog"
    type => "systemlog"
  }
}

output {
  elasticsearch {
    hosts => ["10.0.0.31:9200"] 
    index => "logstash-lck-testindex"
  }
}

1.2 检测配置文件语法是否正确和启动

#检测配置文件语法是否正确
root@ubuntu1804:~# /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/test.conf -t
#启动
root@ubuntu1804:~# /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/test.conf

1.3 生成数据并验证

root@logstash1:~# echo "test" >> /var/log/syslog

二、收集多个日志文件

2.1 编辑 logstash 配置文件

vim /etc/logstash/conf.d/test.conf
input {
  file {
    path => "/var/log/syslog"
    type => "systemlog"
    start_position => "beginning"
    stat_interval => "3 second"
  } 
  file {
    path => "/var/log/vmware*.log"
    type => "vmwarelog"
    start_position => "beginning"
    stat_interval => "3 second"
  }
}

output {
  if [type] == "systemlog" {
    elasticsearch {
      hosts => ["10.0.0.31:9200"] 
      index => "logstash-lck-testindex"
    }
  }
  if [type] == "vmwarelog" {
    elasticsearch {
      hosts => ["10.0.0.31:9200"] 
      index => "logstash-lck-vmwarelog-%{+YYYY.MM.dd}"
    }
  }
}

2.2 检测配置文件语法是否正确和启动

#检测配置文件语法是否正确
root@ubuntu1804:~# /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/test.conf -t
#启动
root@ubuntu1804:~# /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/test.conf

2.3 启动服务，并验证

#注册成系统服务的启动
systemctl restart logstash.service
#压缩包方式的启动
root@ubuntu1804:~# /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/test.conf

2.4 创建索引方便查询日志