apache error log logstash grok pattern
# DataBase/Elasticsearch2021. 4. 14. 15:24apache error log logstash grok pattern

아파치 에러로그 [Sun Jan 31 16:07:40.266665 2021] [access_compat:error] [pid 14356] [client 18.183.46.74:50067] AH01797: client denied by server configuration: /DATA/darksharavim/html/xmlrpc.php grok filter 설정값 \[(?%{DAY:day} %{MONTH:month} %{MONTHDAY} %{TIME} %{YEAR})\] \[.*:%{LOGLEVEL:loglevel}\] \[pid %{NUMBER:pid}\] \[client %{IP:clientip}:.*\] %{GREEDYDATA:errormsg} 결과값 { "timestamp": [ "Sun Jan 3..

[trouble shooting]master_not_discovered_exception
# DataBase/Elasticsearch2021. 4. 7. 13:28[trouble shooting]master_not_discovered_exception

엘라스틱 health 확인시 아래와 같이 상태값이 503 # curl -XGET "localhost:9200/_cluster/health?pretty" { "error" : { "root_cause" : [ { "type" : "master_not_discovered_exception", "reason" : null } ], "type" : "master_not_discovered_exception", "reason" : null }, "status" : 503 } 추가확인시 클러스터 uuid값이 나오지 않았음 # curl http://localhost:9200/ { "name" : "node-1", "cluster_name" : "elasticsearch", "cluster_uuid" : "_na_",..

[trouble shooting]memory locking requested for elasticsearch process but memory is not locked
# DataBase/Elasticsearch2021. 4. 7. 10:44[trouble shooting]memory locking requested for elasticsearch process but memory is not locked

엘라스틱서치 실행시 아래와 같이 오류발생확인 [2021-04-07T09:46:19,467][ERROR][o.e.b.Bootstrap] [darksharavim.com] node validation exception [1] bootstrap checks failed. You must address the points described in the following [1] lines before starting Elasticsearch. bootstrap check failure [1] of [1]: memory locking requested for elasticsearch process but memory is not locked ulimit상태값 # ulimit -aS core file size (bl..

logstash debug test
# DataBase/Elasticsearch2021. 4. 6. 10:29logstash debug test

JSON데이터 준비 # vi cat SendMessage.json {"Hello": "World"} logstash debug test 파일생성 # vi /etc/logstash/conf.d/test.conf input { tcp { port => 5050 codec => json } } output { file { path => "/var/log/logstash/logstash.log" codec => rubydebug } } 5050포트로 json데이터를 받으면 output경로에 로그가 쌓이게끔 설정 실행 및 포트 확인 # /usr/share/logstash/bin/logstash -f test.conf Using JAVA_HOME defined java: /apps/java WARNING, usin..

logstash - grok debugger를 이용해서 apache log 파싱
# DataBase/Elasticsearch2021. 4. 3. 23:02logstash - grok debugger를 이용해서 apache log 파싱

어떤 패턴을 사용해야할지 모를때 확인가능한 사이트 grokdebug.herokuapp.com/discover?# Grok Debugger grokdebug.herokuapp.com grokconstructor.appspot.com/do/match?example=2 Test grok patterns Test grok patterns 10.121.123.104 - - [01/Nov/2012:21:01:04 +0100] "GET /cluster HTTP/1.1" 200 1272 MATCHED httpversion 1.1 request /cluster timestamp 01/Nov/2012:21:01:04·+0100 auth clientip 10.121.123.104 bytes 1272 response 200 ..

image