003 SpringCloud整合-LogStash安装及ELK日志收集

SpringCloud整合-LogStash安装及ELK日志收集

1.安装ElasticSearch和kibana

ElasticSearch7.x 、IK分词器、Kibana 环境搭建、安装

2.Docker安装logstash

1.拉取docker镜像

docker pull logstash:6.6.0

2.创建外部挂载目录

mkdir  /dockerData/logstash/config

3.拷贝容器内部文件到宿主机

也可直接创建第四步文件

#拷贝容器内部文件到宿主机
docker container cp logstash:/usr/share/logstash/pipeline/logstash.conf  /dockerData/logstash/config
docker container cp logstash:/usr/share/logstash/config/logstash.yml   /dockerData/logstash/config

4.修改外部挂载文件

logstash.yml

http.host: "0.0.0.0"
xpack.monitoring.elasticsearch.url: http://127.0.0.1:9200
#es认证配置，没有就不用配
xpack.monitoring.elasticsearch.username: logstash_system
xpack.monitoring.elasticsearch.password: logstash_system

logstash.conf

input {
  tcp {
    mode => "server"
    host => "0.0.0.0"
    port => 4560
    #日志文件json输出
    codec => json_lines
 }
}
output {
  elasticsearch {
	action => "index"
    # ES地址
    hosts  => ["127.1.0.1:9200"]
    #es认证配置，没有就不用配
    user => "logstash_system"
    password => "logstash_system"
    # 输出到的es索引
    index  => "index-logstash"
  }	
  stdout {
    codec => rubydebug
  }
}

5.运行docker容器

docker run  -p 9600:9600 -p 4560:4560 \
--name logstash \
-v /dockerData/logstash/config/logstash.conf:/usr/share/logstash/pipeline/logstash.conf \
-v /dockerData/logstash/config/logstash.yml:/usr/share/logstash/config/logstash.yml \
-d --restart=always logstash:6.6.0 

3.整合kibana

1.进入kibana

2.点击Create index pattern创建es索引的匹配模式

3.进入discover查看创建的Index pattern

4.SpringCloud整合ELK日志采集

1.引入logstash依赖

        <!--集成logstash-->
        <dependency>
            <groupId>net.logstash.logback</groupId>
            <artifactId>logstash-logback-encoder</artifactId>
            <version>6.6</version>
        </dependency>

2.配置logback日志文件

logback-spring.xml

<?xml version="1.0" encoding="UTF-8"?>
<configuration debug="false" scan="true" scanPeriod="1 seconds">
    <include resource="org/springframework/boot/logging/logback/base.xml" />
    <contextName>logback</contextName>

    <appender name="stash" class="net.logstash.logback.appender.LogstashTcpSocketAppender">
        <destination>127.0.0.1:4560</destination>
        <!-- encoder必须配置,有多种可选 -->
        <encoder charset="UTF-8" class="net.logstash.logback.encoder.LogstashEncoder" />
        <!-- 日志输出编码 -->
        <!--<encoder charset="UTF-8" class="net.logstash.logback.encoder.LoggingEventCompositeJsonEncoder">
            <providers>
                <timestamp>
                    <timeZone>UTC</timeZone>
                </timestamp>
                <pattern>
                    <pattern>
                        {
                        "logLevel": "%level",
                        "serviceName": "${springAppName:-}",
                        "pid": "${PID:-}",
                        "thread": "%thread",
                        "class": "%logger{40}",
                        "rest": "%message"
                        }
                    </pattern>
                </pattern>
            </providers>
        </encoder>-->
    </appender>

    <root level="info">
        <appender-ref ref="stash" />
    </root>
</configuration>

yml引用

#日志收集
logging:
  config: classpath:logback-spring.xml

3.启动测试，查看kibana