From 240a357dc88ef675bb7cf1e8b06eca7c1089ca0f Mon Sep 17 00:00:00 2001 From: Germain Masse Date: Sun, 22 Mar 2020 15:46:36 +0100 Subject: [PATCH] Filebeat container (v7.0.0) --- docs/content/configure-elk.md | 68 ++++++++++++++++++++++++++++++++--- 1 file changed, 64 insertions(+), 4 deletions(-) diff --git a/docs/content/configure-elk.md b/docs/content/configure-elk.md index b5ca504b..335ff029 100644 --- a/docs/content/configure-elk.md +++ b/docs/content/configure-elk.md @@ -1,7 +1,9 @@ From [Wikipedia](https://en.wikipedia.org/wiki/Elasticsearch): >Elasticsearch can be used to search all kinds of documents. It provides scalable search, has near real-time search, and supports multitenancy. "Elasticsearch is distributed, which means that indices can be divided into shards and each shard can have zero or more replicas. Each node hosts one or more shards, and acts as a coordinator to delegate operations to the correct shard(s). Rebalancing and routing are done automatically [...]" -This implements sends mail logs to a ELK stack via filebeat client. +This implements sending mail logs to a ELK stack via filebeat client. + +:construction: In the next release (v7.0.0), Filebeat client will not be included inside mailserver container anymore. The recommended practice is to run Filebeat in its own container (documented [below](#filebeat-container)). # Environment variables: **ENABLE_ELK_FORWARDER** @@ -56,8 +58,6 @@ input { } ``` - - # Create Index on Kibana Go http://localhost:5601. The first time needs create default index. Steps: @@ -68,4 +68,64 @@ Steps: - Select **Time-field name** (refresh fields): @timestamp (appears until process some log) - Create -1. Go to Discover and filter by fields. +2. Go to Discover and filter by fields. + +--- +--- +:warning: The following documentation only apply for the next release (v7.0.0). + +# Filebeat container +Configuration for each container is mainly done through _Hints based autodiscover_ (following [Elastic](https://www.elastic.co/guide/en/beats/filebeat/current/configuration-autodiscover-hints.html) practice). +By default, filebeat will **not** retrieve logs from any containers, you must enable logging using Docker labels as documented below. + +## Configuration +### (Method 1) Using existing ELK + +Update `config/filebeat.docker.yml` with your existing logstash endpoint: +``` +output.logstash: + hosts: ["elk_host_or_ip:5044"] +``` + +Adapt your Docker Compose file or use the one provided: +``` +cp docker-compose.filebeat.yml.dist docker-compose.yml +docker-compose up -d +``` + +### (Method 2) Run ELK embedded on the same host + +Filebeat will use the configuration file `config/filebeat.docker.yml`. For basic needs, you don't need to update the file (logstash endpoint is `127.0.0.1:5044'`). + +#### Maxmind GeoIP license + +As of Dec 30 2019, Maxming GeoIP database is no more publicly available. You must first [sign-up](https://www.maxmind.com/en/geolite2/signup) (it's free) and request for a GeoLite2-City license. +Then update the ELK build environment variables: +``` +cp elk/.env.dist elk/.env +``` +With your license number: +``` +MAXMIND_LICENSE=your_license_number +``` + +#### Run containers +Adapt your Docker Compose file or use the one provided: +``` +cp docker-compose.elk.yml.dist docker-compose.yml +docker-compose up -d +``` + +#### Create Index on Kibana +Go http://localhost:5601. The first time needs create default index. +Steps: + +1. Create Index pattern + + - **Index name or pattern** * + - Select **Time-field name** (refresh fields): @timestamp (appears until process some log) + - Create + +2. Go to Discover and filter by fields. + +:bangbang: This ELK image is provided for testing purpose without any security measure. Please follow these [hardening procedures](https://elk-docker.readthedocs.io/#security-considerations). \ No newline at end of file