Kibana

Elasticsearch and Kibana are used to index logs to make them available for querying, visualisation etc. The central log aggregation facility (on eosc-cesnet-vps1) uses the ​Elasticsearch output plugin to feed into the facilities of Elasticsearch to index (logstash) logs.

Access via ​logs.clarin.eu. Account request via sysops.

Setup

See the ​compose_fluent_kibana compose project.

Deployment

Currently running on eosc-cesnet-vps1?. It uses a proxy (provided by the compose project) that adds security through basic authentication. The service can be accessed through the central proxy via ​https://logs.clarin.eu.

Access

Basic authentication with locally stored user/password pairs has been configured. To request access, send the output of htpasswd -n -B username_of_choise to the system administrators to request access to this service.

Instructions for admin

Add the new password hashline to the file compose_fluent_kibana/clarin/nginx/htpasswd (a symlink to compose_fluent_kibana/../htpasswd). Note that the service may have to be restarted for the newly added credentials to work. The restart process can take a while, this is normal.

Maintenance

The ​docker-elastic-curator image is used to close old indices. A configuration is prepared in the ​curator.yml configuration of compose_fluent_kibana. This cleanup can be triggered by calling the project's control script with the curate command:

./control.sh curate