Main notes taken from the book The Kubernetes Book
Examples of higher-level controllers include Deployment
s, DaemonSet
s, and StatefulSet
s.
Every node runs a service called the kubelet that registers it with the cluster and communicates with the API server.
Pods are the atomic unit of scheduling in Kubernetes.
Single-container Pods are the simplest. However, multi-container Pods are ideal for co-locating tightly coupled workloads and are fundamental to service meshes.
Pods get scheduled on nodes (host physical servers, VMs, cloud instances) -- a single Pod can't be scheduled to span multiple nodes.
Pods are defined declaratively in manifest files which are posted to the API server.
Pods are almost always deployed via higher-level controllers.
Pods augment containers:
- Labels and annotations
- Restart policies
- Probes (startup probes, readiness probes, liveness probes, and potentially more)
- Affinity and anti-affinity rules
- Termination control
- Security policies
- Resource requests and limits
Some Pod properties:
- Labels: group Pods and associate them with other objects.
- Annotations: allows adding experimental features and integrations with 3rd-party tools and services.
- Probes: allows testing the health and status of Pods, enabling advanced scheduling, updates, and more.
- Affinity and anti-affinity rules: give control over where Pods run.
- Termination control: allows to gracefully terminate Pods and the applications they run.
- Security policies: allows to enforce security features.
- Resource requests and limits: allows to specify minimum and maximum values for things like CPU, memory and disk IO.
Introspect pod:
kubectl get pods <pod> -o yaml
kubectl describe pods <pod>
kubectl logs <pod>
(example:kubectl logs multipod --container syncer
)
kubectl get namespaces
: list namespaceskubectl describe ns <namespace>
: inspect a namespacekubectl get svc --namespace <namespace>
: list services by filtering by a specific namespacekubectl get svc --all-namespaces
: list services from all namespaceskubectl create ns <namespace>
: create a namespacekubectl delete ns <namespace>
: delete a namespacekubectl config set-context --current --namespace <namespace>
: configurekubectl
to run all future commands against a specific namespace
The Deployment controller is specifically designed for stateless apps.
kubectl get deploy <deployment>
andkubectl describe deploy <deployment>
: see details of a deployment and the ReplicaSet
s.kubectl get rs
andkubectl get rs <replicaset>
: getReplicaSet
s infokubectl rollout pause deploy <deployment>
: pause a rolloutkubectl rollout resume deploy <deployment>
: resume a rolloutkubectl rollout undo deployment <deployment> --to-revision=<revision-number>
: rollback a deployment
kubectl describe svc <service-name>
: describe a service
where
Selector
is the list of labels the Service looks for when building its list of Pods to send traffic toIP
is the permanent internal ClusterIP (VIP) of the ServicePort
is the port the Service listens on inside the clusterTargetPort
is the port the application is listening onNodePort
is the cluster-wide port that can be used for external accessEndpoints
is the dynamic list of healthy Pod IPs that match the selector
kubectl get endpointslices
: get the service endpoint slices
Ingress is a way to expose multiple applications and Kubernetes Services via a single cloud load-balancer.
When running a service mesh an Ingress may not be needed.
Most Kubernetes clusters require us to install an Ingress controller.
An Ingress controller governs how incoming traffic is routed to applications and Services. It supports host-based and path-based HTTP routing.
Two major components to service discovery:
- Registration: process of an application posting its connection details to a service registry so other apps can find it and consume it
- Discovery
Kubernetes uses its internal DNS as a service registry.
All Kubernetes Services automatically register their details with DNS, and all containers are pre-configured to use the cluster DNS for service discovery.
The cluster DNS resolves Service names to ClusterIPs. These are stable virtual IPs on a special network called the
service network. Although there are no routes to this network, the kube-proxy
configures all cluster nodes to
redirect traffic destined for the service network to Pod IPs on the Pod network.
Persistent volume subsystem: Kubernetes storage subsystem.
Container Storage Interface (CSI) is an open standard aimed at providing a clean storage interface for container orchestrators such as Kubernetes.
- Persistent Volumes (PV): mapped to external storage assets
- Persistent Volume Claims (PVC): authorizes applications (Pods) to use PVs
- Storage Classes (SC): dynamically creates physical back-end storage resources that get automatically mapped to Persistent Volumes (PV)
Each provider (a.k.a. provisioner) needs a CSI plugin to expose their storage assets to Kubernetes. The plugin
usually runs as a set of Pods in the kube-system
Namespace.
Access modes for volumes:
- ReadWriteOnce (RWO): the PV can only be bound as R/W by a single PVC
- ReadWriteMany (RWM): the PV can be bound as R/W by multiple PVCs
- ReadOnlyMany (ROM): the PV can be bound as R/O by multiple PVCs
ConfigMaps and Secrets are the Kubernetes native way of decoupling applications and config data.
They’re both first-class object in the Kubernetes API and can be created and manipulated with kubectl apply
,
kubectl get
, and kubectl describe
commands.
ConfigMaps are designed for application configuration parameters and even entire configuration files, whereas Secrets are designed for sensitive configuration data.
Both can be injected into containers at run-time via various constructs, with volumes being the preferred method, as they allow updates to eventually be reflected in running containers.
Secrets are not encrypted in the cluster store, not encrypted in-flight on the network, and not encrypted when surfaced in a container.
StatefulSet
s are for applications that need Pods to be predictable and long-lived.
StatefulSet
s guarantee:
- Predictable and persistent Pod names
- Predictable and persistent DNS hostnames
- Predictable and persistent volume bindings
Example: failed Pods managed by a StatefulSet will be replaced by new Pods with the exact same Pod name, the exact same DNS hostname, and the exact same volumes. This is true even if the replacement Pod is started on a different cluster node. The same is not true of Pods managed by a Deployment.
Usually, rollbacks in replicas managed by StatefulSet
s require manual attention.
Each Pod replica spawned by a StatefulSet
gets a predictable and persistent name, DNS hostname, and unique set of
volumes. These stay with the Pod for its entire lifecycle, including failures, restarts, scaling, and other scheduling
operations. In fact, StatefulSet Pod names are integral to scaling operations and connecting to storage volumes.
All requests to the API server include credentials and pass through authentication, authorization, and admission control. The connection between the client and the API server is also secured with TLS.
The authentication layer is responsible for validating the identity of requests. Client certificates are commonly used, and integration with AD and other IAM services is recommended for production clusters. Kubernetes does not have its own identity database, meaning it doesn't store or manage user accounts.
The authorization layer checks whether the authenticated user is authorized to carry out the action in the request. This layer is also pluggable and the most common module is RBAC, which is enabled on most Kubernetes clusters and has been stable/GA since Kubernetes 1.8.
It's a least-privilege deny-by-default system. Meaning, all actions are denied by default, and specific actions are enabled by creating allow rules.
Kubernetes doesn't support deny rules, it only supports allow rules.
Admission control kicks in after authorization and is responsible for enforcing policies.
Types of admission controllers:
- Mutating: check for compliance and can modify requests
- Validating: check for policy compliance but cannot modify requests
Kubernetes is API centric, and the API is exposed internally and externally via the API server.
The API server runs as a control plane service, and all internal and external clients interact with each other and the API, via the API server.
The API itself is a modern resource-based RESTful API that accepts CRUD-style operations via uniform HTTP methods such
as POST
, GET
, PUT
, PATCH
, and DELETE
.