Kubernetes High-Level Hardening Guide – DZone Security

This is an article from DZone’s 2021 Kubernetes and Enterprise Trend Report.

For more:

Read the report

As more organizations begin to adopt cloud-native technologies, adoption of Kubernetes has become the industry standard for orchestration of containers. This shift toward Kubernetes has automated and simplified the deployment, scaling, and management of containerized applications, providing numerous advantages over legacy management protocols for traditional monolithic systems. However, managing Kubernetes securely at scale comes with a unique set of challenges, including cluster strengthening, supply chain security, and runtime threat detection.

This introduction to Kubernetes security combines best practices from the Cloud Native Computing Foundation (CNCF), National Security Agency (NSA), and Cybersecurity and Infrastructure Security Agency (CISA) to help organizations mitigate risks and adopt a multi-layered approach to security.

Mass preparation and solidification

Securing a Kubernetes environment begins with hardening the cluster. For users of the managed Kubernetes service (for example, GKE, EKS, AKS), the respective cloud provider manages the security of the master node and implements many default security settings for the group. GKE Autopilot takes additional measures, implementing GKE Enhancement Guidelines and GCP Security Best Practices. But even for users of the GKE or EKS/AKS standard, there are guidelines that cloud providers maintain to secure access to the Kubernetes API server, container access to cloud resources, and Kubernetes upgrades:

For self-managed Kubernetes clusters (eg, kube-admAnd kops), kube-bench can be used to test whether a block meets the security guidelines set forth in the CIS Kubernetes Benchmark. The main recommendations include encrypting secrets stored in files etcd At rest, protect control level connections with TLS certificates and turn on audit logging.

Network policies and resources

By default, Kubernetes allows connection from an iPod to another pod within the same group. While this is ideal for service discovery, it provides zero separation from the network, allowing malicious actors or compromised systems to have unlimited access to all resources. This becomes very difficult for teams that use namespaces as the primary means of multitenancy within Kubernetes.

To control the flow of traffic between pods, namespaces, and external endpoints, use a CNI plug-in that supports the NetworkPolicy API (eg Calico, Flannel, or the cloud’s CNI) to isolate the network. Following the zero-trust model, the best practice is to implement a default Deny All policy to block all in and out traffic unless another policy specifically allows it.

In addition to network policies, Kubernetes provides two resource-level policies: LimitRange And ResourceQuotas. LimitRanges They can be used to limit the use of individual resources (for example, a maximum of two CPUs per pod), while ResourceQuota It controls the overall usage of resources (for example, a total of 20 CPUs in the development namespace).

RBAC and Service Accounts

With strong network and resource policies in place, the next step is to enforce the RBAC’s mandate to restrict access. Kubernetes administrators can force RBAC for users and groups to access the group, as well as to restrict services from accessing resources inside and outside the group (for example, databases hosted on the cloud).

Be careful using the default service account that is installed on each case when creating. Depending on the permissions granted to the default service account, the pod may be granted more permissions than required. If the service does not require any specific connection to the Kubernetes service, set automountServiceAccountToken set to false to prevent compositing.

system hardening

Now that the block is secure, the next step is to reduce the attack surface on the systems. This applies to the operating system running on nodes as well as to the kernel on containers. Instead of general purpose Linux nodes, choose a specialized operating system optimized to run containers, such as AWS Bottlerocket or GKE COS.

Next, take advantage of the Linux kernel’s security features, such as SELinux, AppArmor (beta since 1.4), and/or seccomp (stable since 1.19). AppArmor defines permissions for a Linux user or group to limit programs to a limited set of resources. Once you select an AppArmor profile, pods containing AppArmor comments will enforce these rules.

apiVersion: v1
kind: Pod
metadata:
	name: apparmor
	annotations:
		container.apparmor.security.beta.kubernetes.io/hello: localhost/k8s-apparmor-example-deny-write
spec:
	containers:
	- name: hello
	  image: busybox
	  command: [ "sh", "-c", "echo 'Hello AppArmor!' && sleep 1h" ]

Seccomp, on the other hand, restricts container syscalls. As long as seccomp profiles are available on the core Kubernetes node, seccomp profiles can be specified under securityContext Section:

apiVersion: v1
kind: Pod
metadata:
  name: audit-pod
  labels:
	app: audit-pod
spec:
  securityContext:
	seccompProfile:
		type: Localhost
		localhostProfile: profiles/audit.json
  containers:
  - name: test-container
 	image: hashicorp/http-echo:0.2.3
 	args:
	- "-text=just made some syscalls!"

Even when no seccomp profiles are available, users can still restrict the container from various privilege escalation attacks. Within security contexts, Kubernetes allows you to configure whether a container can be run as privileged, root, or escalated to root. Users can also restrict hostPIDAnd hostIPCAnd hostNetworkAnd hostPaths. All of these settings can be enforced via the Pod Security Policy (deprecated in version 1.21) or by using other open source tools, such as K-Rail, Kyverno, and OPA/Gatekeeper.

Finally, if additional security assurance is required, custom RuntimeClass It can be configured to take advantage of virtual machines (eg gVisor, Kata). Determine RuntimeClass At the node level and specifying it within the pod definition:

apiVersion: node.k8s.io/v1 # RuntimeClass is defined in the node.k8s.io API group
kind: RuntimeClass
metadata:
  name: myclass # The name the RuntimeClass will be referenced by
  # RuntimeClass is a non-namespaced resource
handler: myconfiguration # The name of the corresponding CRI configuration
---
apiVersion: v1
kind: Pod
metadata:
  name: mypod
spec:
  runtimeClassName: myclass

supply chain security

Even if the assembly and system are safe, to ensure the overall security of the entire application, the supply chain must also be considered. For in-house developed applications, follow best practices for creating containers. Specifically, use a minimal base image to reduce the attack surface, pin bundle versions, and use multi-stage builds to create small images. Also, specify a non-root user with which the container should run, or create non-rooted containers with podman To restrict root access.

Next, scan all images for vulnerabilities using open source tools (for example, Trivy, Clair, Anchore) or commercial tools (for example, Xray from Artifactory or scanning containers in the process of building a cloud provider). Some tools also allow image signature and signature verification to ensure that containers are not tampered with during the build and load process. Finally, select the whitelisted records that Kubernetes can pull images from. ImagePolicyWebhook or any policy enforcement tool listed above.

Monitoring, recording and runtime security

At this point we have a secure suite with a locked supply chain that produces clean, verified images with limited permissions. However, environments are dynamic, and security teams must be able to respond to incidents in operating environments. First of all, make sure the containers are stable at runtime by setting readOnlyRootFilesystem to true and store the tmp log files in a file emptyDir.

On top of typical application monitoring (eg Prometheus / Grafana) or logging (eg EFK), analyze syscall processes and Kubernetes API logs using Falco or Sysdig. Both tools can analyze Linux kernel calls at runtime and trigger alerts when the rule is violated. Examples of rules include alerting when privilege escalation occurs, when read/write events are detected in known directories, or when the shell is called. Finally, combine the Kubernetes API audit logs with existing log collectors and alert tools to monitor all activities in the cluster. This includes API request history, performance metrics, deployments, resource consumption, operating system calls, and network traffic.

conclusion

Due to the complex nature of cloud-native systems, a multi-layered approach is required to secure a Kubernetes environment. Kubernetes recommends 4Cs for cloud-native security: cloud, cluster, container, and token. Start by strengthening the cluster and following cloud security best practices. Next, seal the container, reduce the attack surface, limit access, and ensure runtime stability. Next, secure the supply chain and analyze the code and container for vulnerabilities. Finally, monitor all activities at runtime to build defense into each layer of your programs running within Kubernetes.

This is an article from DZone’s 2021 Kubernetes and Enterprise Trend Report.

For more:

Read the report

.

Leave a Comment