RedHat OpenShift Logging (EFK) Stack as Logging Solution

Problem

We need to store user logs and allow users a way to visualize them.

Proposals

  1. Built-in RedHat OpenShift Logging

    Works out-of-the-box with multitenancy. Performant and has good visualization (Kibana).

    Optimized for short-term storage. It is not possible to query usage data, such as log storage sizes, by user, namespaces, or organizations. It is not possible to implement retention policies by storage size but only by time. Resource hungry and thus expensive to run.

  2. Custom Solution using Loki

    There were some prototypes using Loki for cluster logging synsights-logging (internal) Loki allows us querying storage used by tenant. This way we can implement retention policies by storage size and not only by time. Cheaper to run than ES. Performant and has good visualization (Grafana).

    Integration of Loki into the ElasticSearch console and into concepts like OpenShift Projects would require quite a bit of engineering.

    There is a proposal with OpenShift to use Loki as the log store.

Decision

Built-in RedHat OpenShift Logging with short log retention.

Rationale

RedHat OpenShift Logging is easy to install and supports multitenancy by default. We have experience with it on OpenShift clusters.

A short retention period is chosen to ensure good performance and ease of operation and maintenance. If more retention is needed log forwarding to a custom system can be configured.