Kibana

Elasticsearch and Kibana are used to index logs to make them available for querying, visualisation etc. The central log aggregation facility (on rs236235) uses the ​Elasticsearch output plugin to feed into the facilities of Elasticsearch to index (logstash) logs.

Access via ​logs.clarin.eu. Account request via sysops or Twan.

State: under evaluation

Setup

See the ​compose_fluent_kibana compose project.

Deployment

Currently running on rs236235 on basis of the compose project in /home/twagoo/fluent-kibana. It using the proxy provided by the compose project that also adds security through basic authenticaiton. The service can be accessed through the central proxy via ​https://logs.clarin.eu.

Maintenance

The ​docker-elastic-curator image is used to close old indices. A configuration is prepared in the ​curator.yml configuration of compose_fluent_kibana. This cleanup can be triggered by calling the project's control script with the curate command:

./control.sh curate