In the 60s, automobiles manufactured in Japan consistently beats their competitors in American market. Many refers to the lean manufacturing methodology in the automation as the secret sauce. The software industries borrowed a lot of similar methodologies from TPS (Toyota Production System) into software development industry, which brought about agile software development.
For software to deliver value, it is not just about developing software in agile methodologies. A full SDLC (software development life cycle) includes build, release and upgrades too, some of which are managed in a different department in the organization. DevOps extends agile methodology across departments. DevOps has now become a buzzword. Some even refers to it as a culture but none of these are possible without automation.
Automation Pipelines
The power horse of the DevOps tooling is automation pipeline (e.g. Jenkins, Azure DevOps, GitHub). These pipelines expedites iterations with frequent feedback about software quality, whether it is common conventional SDLC workflow or more recent infrastructure as code worklfow. For SDLC, the goal is to establish continuous integration (CI) and ultimately continuous deployment (CD).
Serverless Deployment
With serverless deployment, the operation of managing computing resources is abstracted away. Serverless deployment models further simplifies SDLCs and are ideal for some common use cases such as API services, IoT, scheduled and event-driven tasks.
DataOps
Another creative use of automation pipelines is the data pipelines. Data engineering tasks includes ingestion, ETL, integration, and storage and automation pipelines are ideal automation tools for these tasks.
Observability
Observability setup enables instant feedback, an important construct of DevOps. An observability stack consists of metrics collection, log shipping, performance monitoring, request tracing and visualization etc.
More on automation
- IAM Roles for any workload - Background A few month back a client of mine wanted to use GitLab pipeline to deploy infrastructure on AWS with Terraform. The key question is how to authenticate the Terraform process running in the pipeline to AWS with temporary credential. Having worked it out on GitHub, my proposal at time…
- Managing EC2 instances across AWS accounts - I regard AWS Systems Manager highly as omnipotent. Nonetheless, there are a few reasons that makes Ansible still a prevalent VM (EC2) management tool over Systems Manager (SSM). First, organizations already vested in their custom Ansible roles and playbooks want to reuse, and expand their assets in Ansible. The benefit…
- WordPress Security Basics - Background In 2019, I moved this site to WordPress hosted on an Amazon Lightsail instance. There were few visits at that time so I lived with the single-server architecture. The website traffic has since been in steady growth but I have been too busy to catch up with the WordPress…
- AWS Systems Manager is an Omnipotent Hodgepodge - Introduction to Systems Manager AWS Systems Manager addresses a lot of SysOps requirements for configuration management, including server automation. In this domain, there is another AWS service called OpsWorks. However, with OpsWorks Stack, OpsWorks Chef and OpsWorks Puppet all coming EOL in 2024, the entire OpsWorks service is mostly deprecated.…
- Authentication to kube-apiserver via OIDC - Background There are many benefits of using OIDC to authenticate to kube-api server, especially with multiple clusters that need consistent log-in experience. With the last post on how OIDC Authorization Code Flow works, now I will discuss options with authentication for kubectl to connect to kube API server. To start,…
Contact Digi Hunch for Professional Services.