Log Shipping in Kubernetes with EFK stack

I first worked on log shipping with ELK stack three years ago. In the context of Kubernetes cluster, log shipping has similar challenges. In this post I will discuss the set up of log shipping with Kubernetes cluster. Logging Architecture As discussed, if the Kubernetes cluster has a runtime in compliant with CRI (e.g. containerd), … Read moreLog Shipping in Kubernetes with EFK stack

Log shipping through ELK

A common devops task is build logging pipeline with ELK stack (Elasticsearch, Logstash, Kibana). Suppose the application is written in Java and currently use log4j’s RollingFileAppender to generate log files locally. We can use log4j’s socket appender to write to Logstash, which further pushes the log stream to Elasticsearch cluster. In this model, failure to … Read moreLog shipping through ELK