ELK filebeat 설치(docker)

도커버전

# docker -v
Docker version 1.13.1, build 0be3e21/1.13.1

elasticsearch image 다운

# docker pull docker.elastic.co/elasticsearch/elasticsearch:7.6.2
Trying to pull repository docker.elastic.co/elasticsearch/elasticsearch ... 
7.6.2: Pulling from docker.elastic.co/elasticsearch/elasticsearch
c808caf183b6: Pull complete 
d6caf8e15a64: Pull complete 
b0ba5f324e82: Pull complete 
d7e8c1e99b9a: Pull complete 
85c4d6c81438: Pull complete 
3119218fac98: Pull complete 
914accf214bb: Pull complete 
Digest: sha256:59342c577e2b7082b819654d119f42514ddf47f0699c8b54dc1f0150250ce7aa
Status: Downloaded newer image for docker.elastic.co/elasticsearch/elasticsearch:7.6.2

다운된 이미지 확인

# docker images
REPOSITORY                                      TAG                 IMAGE ID            CREATED             SIZE
docker.elastic.co/elasticsearch/elasticsearch   7.6.2               f29a1ee41030        12 months ago       791 MB

elasticsearch container구동

docker run -d --name elasticsearch -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" docker.elastic.co/elasticsearch/elasticsearch:7.6.2

프로세스 확인

# docker ps -a
CONTAINER ID        IMAGE                                                 COMMAND                  CREATED             STATUS                      PORTS                                            NAMES
288020a92902        docker.elastic.co/elasticsearch/elasticsearch:7.6.2   "/usr/local/bin/do..."   18 seconds ago      Up 17 seconds               0.0.0.0:9200->9200/tcp, 0.0.0.0:9300->9300/tcp   elasticsearch

포트확인

# netstat -anp | grep LIST | grep docker
tcp6       0      0 :::9200                 :::*                    LISTEN      47354/docker-proxy- 
tcp6       0      0 :::9300                 :::*                    LISTEN      47341/docker-proxy- 
tcp6       0      0 :::9000                 :::*                    LISTEN      9740/docker-proxy-c 

설치테스트 확인

# curl -X GET "http://127.0.0.1:9200/"
{
  "name" : "288020a92902",
  "cluster_name" : "docker-cluster",
  "cluster_uuid" : "cEPRQY2ZQEmln7gmqkspKA",
  "version" : {
    "number" : "7.6.2",
    "build_flavor" : "default",
    "build_type" : "docker",
    "build_hash" : "ef48eb35cf30adf4db14086e8aabd07ef6fb113f",
    "build_date" : "2020-03-26T06:34:37.794943Z",
    "build_snapshot" : false,
    "lucene_version" : "8.4.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

kibana이미지 다운로드

docker pull docker.elastic.co/kibana/kibana:7.6.2

이미지 확인

# docker images
REPOSITORY                                      TAG                 IMAGE ID            CREATED             SIZE
docker.elastic.co/kibana/kibana                 7.6.2               f70986bc5191        12 months ago       1.01 GB
docker.elastic.co/elasticsearch/elasticsearch   7.6.2               f29a1ee41030        12 months ago       791 MB

기존 elasticsearch 컨테이너 중지 및 삭제

# docker stop elasticsearch
elasticsearch

# docker rm elasticsearch
elasticsearch

docker 네트워크 추가 및 확인

# docker network create elastic
4d701d5f8a1cfa5532e890cbf78eb34a045072ce0af777b3e6e107e15236176b

# docker network ls
NETWORK ID          NAME                DRIVER              SCOPE
9b2a88c85b9c        bridge              bridge              local
4d701d5f8a1c        elastic             bridge              local
be9d55b57ff5        host                host                local
141df44101a1        none                null                local

elasticsearch 컨테이너 추가 및 구동

# docker run -d --network=elastic --name elasticsearch -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" docker.elastic.co/elasticsearch/elasticsearch:7.6.2
3987f3d4aebf4f2df07636d50fcb637d30e76fc77cf4f5404b212e4bf4388946

# docker ps
CONTAINER ID        IMAGE                                                 COMMAND                  CREATED             STATUS              PORTS                                            NAMES
3987f3d4aebf        docker.elastic.co/elasticsearch/elasticsearch:7.6.2   "/usr/local/bin/do..."   51 seconds ago      Up 50 seconds       0.0.0.0:9200->9200/tcp, 0.0.0.0:9300->9300/tcp   elasticsearch

logstash 이미지 다운

# docker pull docker.elastic.co/logstash/logstash:7.6.2

logstash config설정

# mkdir -p /root/apps/logstash

# vi /root/apps/logstash/logstash.conf
input {

  beats {

    # 지정된 포트로 filebeat의 데이터를 전송받음

    port => 5044

  }

}

 

output {

  elasticsearch {

    hosts => ["elasticsearch:9200"]

    user => elastic

    password =>changeme

    manage_template => false

    index => "access-log" #DB의 데이터베이스 명

    document_type => "log" #DB의 테이블 명

  }
}

logstash 컨테이너 구동

# docker run -d --name logstash --network elastic -p 5044:5044 -v /apps/logstash/logstash.conf:/usr/share/logstash/pipeline/logstash.conf docker.elastic.co/logstash/logstash:7.6.2

kibana 컨테이너 추가 및 구동

# docker run -d --network=elastic --name kibana -p 5601:5601 docker.elastic.co/kibana/kibana:7.6.2
e2cf8ba36d8296216bf9a47a82e9957bbbcdc64ff0fa8d5ad6709d3c0278a357

# docker ps
CONTAINER ID        IMAGE                                                 COMMAND                  CREATED             STATUS              PORTS                                            NAMES
e2cf8ba36d82        docker.elastic.co/kibana/kibana:7.6.2                 "/usr/local/bin/du..."   5 seconds ago       Up 5 seconds        0.0.0.0:5601->5601/tcp                           kibana

filebeat이미지다운 및 filebeat config설정

# docker pull docker.elastic.co/beats/filebeat:7.6.2
Trying to pull repository docker.elastic.co/beats/filebeat ... 
7.6.2: Pulling from docker.elastic.co/beats/filebeat
c808caf183b6: Already exists 
c5d26ebed11d: Pull complete 
2b0a2e030582: Pull complete 
ffde1e12d35a: Pull complete 
4503b4f6065d: Pull complete 
1e67ef889e4e: Pull complete 
d845627f6df4: Pull complete 
Digest: sha256:24211654fbe1ce3866583d7ae385feffbfaa77d4598d189fdec46111133811a9
Status: Downloaded newer image for docker.elastic.co/beats/filebeat:7.6.2

# mkdir -p /root/apps/filebeat

# vi /root/apps/filebeat/filebeat.yml
filebeat.prospectors:

- input_type: log

  paths:

    - /usr/share/filebeat/logs/access_log

output:

  logstash:

    hosts: ["logstash:5044"]

아파치 로그 링크

# ln -s /apps/apache/logs/access_log /apps/filebeat/test.log

filebeat 컨테이너 실행

# docker run -d --name filebeat --network elastic -v /apps/filebeat/test.log:/usr/share/filebeat/logs/access_log docker.elastic.co/beats/filebeat:7.6.2

웹접속
웹브라우저에서 http://설치ip:5601[kibana포트]

설치완료!