Log Shipping in Kubernetes with EFK stack

I first worked on log shipping with ELK stack three years ago. In the context of Kubernetes cluster, log shipping has similar challenges. In this post I will discuss the set up of log shipping with Kubernetes cluster. Logging Architecture As discussed, if the Kubernetes cluster has a runtime in compliant with CRI (e.g. containerd), … Read moreLog Shipping in Kubernetes with EFK stack

AWS Certified DevOps Engineer Exam Tips

The last 30 days have been exhausting for me. I studied hard on the new AWS Certified DevOps Engineer exam and thank goodness I passed (750 out of 1000 is required). This was the hardest professional certification I ever worked on. The exam was re-launched recently in March 2019 so there is still a shortage … Read moreAWS Certified DevOps Engineer Exam Tips

Log shipping through ELK

A common devops task is build logging pipeline with ELK stack (Elasticsearch, Logstash, Kibana). Suppose the application is written in Java and currently use log4j’s RollingFileAppender to generate log files locally. We can use log4j’s socket appender to write to Logstash, which further pushes the log stream to Elasticsearch cluster. In this model, failure to … Read moreLog shipping through ELK