Post

DevOps - Kubernetes: Orchestrating Containerized Applications at Scale

Posted
By harryvasanth
4 min read
DevOps - Kubernetes: Orchestrating Containerized Applications at Scale

Intro

Kubernetes (K8s) has emerged as the de facto standard for container orchestration, enabling organizations to deploy, scale, and manage containerized applications efficiently. This guide explores advanced concepts in Kubernetes, including custom resource definitions, operators, advanced scheduling, and security best practices to help you leverage the full power of Kubernetes in your DevOps practices.

Step 1: Setting Up a Kubernetes Cluster

Before diving into advanced topics, ensure you have a Kubernetes cluster ready.

1.1 Install Minikube for Local Development

1
2

curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64
sudo install minikube-linux-amd64 /usr/local/bin/minikube

Start Minikube:

1

minikube start --driver=docker

1.2 Install kubectl

1
2

curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl

Verify the installation:

1

kubectl version --client

Step 2: Deploying Applications with Advanced Configurations

2.1 Create a Deployment with Resource Limits

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
        - name: nginx
          image: nginx:1.14.2
          resources:
            limits:
              cpu: "500m"
              memory: "128Mi"
            requests:
              cpu: "250m"
              memory: "64Mi"

Apply the deployment:

1

kubectl apply -f nginx-deployment.yaml

2.2 Implement Horizontal Pod Autoscaling

Create an HPA for the nginx deployment:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

apiVersion: autoscaling/v2beta1
kind: HorizontalPodAutoscaler
metadata:
  name: nginx-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: nginx-deployment
  minReplicas: 1
  maxReplicas: 10
  metrics:
    - type: Resource
      resource:
        name: cpu
        targetAverageUtilization: 50

Apply the HPA:

1

kubectl apply -f nginx-hpa.yaml

Step 3: Advanced Scheduling Techniques

3.1 Node Affinity

Schedule pods on specific nodes based on labels:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

apiVersion: v1
kind: Pod
metadata:
  name: nginx-pod
spec:
  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
          - matchExpressions:
              - key: disktype
                operator: In
                values:
                  - ssd
  containers:
    - name: nginx
      image: nginx

3.2 Pod Topology Spread Constraints

Distribute pods across nodes or zones:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

apiVersion: v1
kind: Pod
metadata:
  name: nginx-spread
spec:
  topologySpreadConstraints:
    - maxSkew: 1
      topologyKey: topology.kubernetes.io/zone
      whenUnsatisfiable: DoNotSchedule
      labelSelector:
        matchLabels:
          app: nginx
  containers:
    - name: nginx
      image: nginx

Step 4: Custom Resource Definitions (CRDs) and Operators

4.1 Create a Custom Resource Definition

Define a custom resource for managing databases:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  name: databases.example.com
spec:
  group: example.com
  versions:
    - name: v1
      served: true
      storage: true
      schema:
        openAPIV3Schema:
          type: object
          properties:
            spec:
              type: object
              properties:
                engine:
                  type: string
                version:
                  type: string
  scope: Namespaced
  names:
    plural: databases
    singular: database
    kind: Database
    shortNames:
      - db

Apply the CRD:

1

kubectl apply -f database-crd.yaml

4.2 Implement a Kubernetes Operator

Create a simple operator using the Operator SDK:

1
2

operator-sdk init --domain example.com --repo github.com/example/database-operator
operator-sdk create api --group example --version v1 --kind Database --resource --controller

Implement the controller logic in controllers/database_controller.go.

Step 5: Advanced Networking

5.1 Network Policies

Implement network segmentation:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-frontend-to-backend
spec:
  podSelector:
    matchLabels:
      app: backend
  ingress:
    - from:
        - podSelector:
            matchLabels:
              app: frontend
      ports:
        - protocol: TCP
          port: 8080

5.2 Service Mesh with Istio

Install Istio:

1

istioctl install --set profile=demo -y

Enable Istio injection for a namespace:

1

kubectl label namespace default istio-injection=enabled

Step 6: Security Best Practices

6.1 Pod Security Policies

Enforce security standards for pods:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  name: restricted
spec:
  privileged: false
  seLinux:
    rule: RunAsAny
  runAsUser:
    rule: MustRunAsNonRoot
  fsGroup:
    rule: RunAsAny
  volumes:
    - "configMap"
    - "emptyDir"
    - "projected"
    - "secret"
    - "downwardAPI"

6.2 RBAC (Role-Based Access Control)

Create a role and role binding:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: default
  name: pod-reader
rules:
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["get", "watch", "list"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: read-pods
  namespace: default
subjects:
  - kind: User
    name: jane
    apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: pod-reader
  apiGroup: rbac.authorization.k8s.io

Step 7: Monitoring and Logging

7.1 Prometheus for Monitoring

Deploy Prometheus using Helm:

1
2

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm install prometheus prometheus-community/prometheus

7.2 EFK Stack for Logging

Deploy the Elasticsearch, Fluentd, and Kibana (EFK) stack:

1
2
3
4
5

kubectl create namespace logging
helm repo add elastic https://helm.elastic.co
helm install elasticsearch elastic/elasticsearch -n logging
helm install kibana elastic/kibana -n logging
kubectl apply -f https://raw.githubusercontent.com/fluent/fluentd-kubernetes-daemonset/master/fluentd-daemonset-elasticsearch.yaml

Conclusion

Kubernetes provides a powerful platform for orchestrating containerized applications at scale. By leveraging advanced features like custom resources, operators, advanced scheduling, and implementing security best practices, you can build robust, scalable, and secure applications.

devops, kubernetes
kubernetes orchestration containerization
This post is licensed under CC BY 4.0 by the author.