Running Stateful Apps in Kubernetes

With Kubernetes you will eventually, have the need to run stateful applications in Kubernetes. This is more common than you think. If you have never run stateful apps on Kubernetes before this can be a scary thing adding more moving parts to a Kubernetes cluster, deploying the app, as well as managing your stateful application/s on Kubernetes when it requires state.

In this blog post I am going to take you on a short journey to gain an understanding of Stateless vs Stateful applications, how storage works in Kubernetes touching on volumes, storage classes, persistent volumes (PC), and persistent volume claims (PVC), what Stateful Sets are, about Persistent state with pods, and good practices for running Stateful Apps on Kubernetes.

Stateless

A stateless app is an application program that does not save client data generated in one session for use in the next session with that client.

Stateful

A stateful app is a program that saves client data from the activities of one session for use in the next session.

The data that is saved is called the application’s state. Here is a visual covering the differences between Stateless and Stateful applications:

Volumes

Here is a breakdown of what volumes are:

  • A volume is a directory, typically with data in it, that is accessible to the containers in a pod.
    • A volume represents a way to store, retrieve, and persist data across pods through an applications lifecycle.
    • Volume modes in Kubernetes supports are Filesystem or Block.
    • Volumes are backed by different types of storage such as NFS, iSCSI, or other cloud storage (i.e. awsElasticBlockStore, azureDisk, gcePersistentDisk etc..).
    • When pods ceases to exist, Kubernetes destroys ephemeral volumes; however, Kubernetes does not destroy persistent volumes.

StorageClasses

Here is a breakdown of what volumes are:

  • Define types of storage tiers like Premium and Standard through Storage Classes in Kubernetes.
    • Give K8s admins a way to describe the “classes” of storage they offer.
    • StorageClasses define the provisioner, parameters, and reclaimPolicy used when a PersistentVolume is provisioned.
    • When a pod is deleted the underlying storage resource can either be deleted or kept for use with a future pod.
    • A reclaim Policy controls the behavior of the underlying storage resource when pod & the its persistent volume are no longer required.

Example of a configuration file for a StorageClass:

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: managed-premium-retain
provisioner: kubernetes.io/azure-disk
reclaimPolicy: Retain
parameters:
  storageaccounttype: Premium_LRS
  kind: Managed

Reclaim Policy

Here is a breakdown of what Reclaim Policies:

  • Retain –
    • Allows for manual reclamation of the resource. The PV is not available for another claim due to previous claimant’s data remaining on the volume. A K8s admin must manually reclaim the volume.
    • Delete –
      • The delete reclaim policy removes the PV resource from the K8s cluster, & the associated storage asset such as cloud storage, NFS etc…
    • Recycle –
      • Performs a basic scrub on the volume & makes it available again for a new PVC.

Persistent Volumes (PVs)

Here is a breakdown of what Persistent Volumes are:

  • A persistent volume (PV) is a storage resource created and managed by the Kubernetes API that can exist beyond the lifetime of an individual pod.
    • A Persistent Volume can be manually provisioned by an Kubernetes admin or dynamically provisioned using Storage Classes by the Kubernetes API server.
    • Dynamic provisioning uses a StorageClass to identify what type of storage (NFS, iSCSI, or cloud-based) needs to be created.

Example of a configuration file for the PersistentVolume:

apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv0010
spec:
  capacity:
   storage: 40Gi
  volumeMode: Filesystem
  accessModes:
   - ReadWriteOnce
  persistentVolumeReclaimPolicy: Recycle
  storageClassName: slow
  mountOptions:
   - hard
   - nfsvers=4.1
  nfs:
   path: /tmp
   server: 172.19.0.22

Persistent Volume Claims (PVCs)

Here is a breakdown of what Persistent Volumes Claims are:

  • A PersistentVolumeClaim (PVC) is a request for storage by a user.
    • A PersistentVolumeClaim specifies the volume mode of either Block or File storage from a StorageClass, the access mode, and the capacity needed.
    • PVC Access Modes Are:
      • ReadOnlyMany (ROX) allows being mounted by multiple nodes in read-only mode.
      • ReadWriteOnce (RWO) allows being mounted by a single node in read-write mode.
      • ReadWriteMany (RWX) allows multiple nodes to be mounted in read-write mode.

Example of a configuration file for the PersistentVolumeClaim:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: pvc0002
spec:
  storageClassName: manual
  accessModes:
   - ReadWriteOnce
  resources:
   requests:
    storage: 10Gi

Lifecycle of a Volume & Claim

Let’s take a look at how the lifecycle of volumes and claims flow:

StatefulSets

Here is a breakdown of what Stateful Sets are:

  • StaefulSets are Kubernetes objects that are used when we need each pod to have its own independent state & use its own individual volume.
    • With StatefulSets each pod is assigned a unique name & the unique name stays with it even if the pod is deleted & recreated.
    • Headless services are primarily used when we deploy statefulset applications. Headless services don’t operate like load balancers. Headless services are not assigned IPs like a regular service is.

StatefulSets are typically used when the following is needed:

  • unique network identifiers for pods
    • persistent storage for retaining data
    • Ordered, graceful deployment, & scaling of pods
    • Ordered, & automated rolling updates of the app

Some Good Practices When Running Stateful Apps on Kubernetes

That wraps up this blog post! Thanks for reading and stay tuned to my blog for more content on Kubernetes soon.

Read more

GitOps Fundamentals Certification

Recently Codefresh launched the 1st certification in its GitOps certification path. This one is called “GitOps Fundamentals“. You can find it here: https://codefresh.learnworlds.com .

It takes you through the basics of GitOps to gain theoretical knowledge, and how to utilize Argo CD as the GitOps operator to gain hands-on knowledge. You will learn about both and will have questions on both in the quizzes and final exam.

They also touch on Argo Rollouts to go over Progressive Delivery with topics such as blue/green deployments and canary deployments. This is the 1st ever GitOps certification and it’s free! They do have plans for GitOps at Edge and GitOps at Scale certifications.

You can find more information about the GitOps certification and Codefresh’s future plans for it on this blog by Hannah Seligson (one of the authors of the course and exam) here: https://codefresh.io/blog/get-gitops-certified-argo.

I jumped all over this opportunity to get certified on GitOps, by signing up for the course, taking the training, and the exam! I passed and now I am GitOps certified.

Here is the certification:

GitOps is gaining adoption more and more every day in the Kubernetes space. Also, Argo CD is growing extremely fast as one of the top if not the top GitOps operator. I recommend you check this Codefresh GitOps certification out and get GitOps certified as this pattern and the technology behind it are growing at a super fast rate.

Also note, it looks like Weaveworks is planning to launch a “Certified GitOps Practitioner (CGP)” certification soon. I would guess the Weaveworks GitOps certification will contain content on Flux another GitOps operator. You can learn more about their coming GitOps certification here: https://www.weave.works/certified-gitops-practitioner

Also for more training on GitOps and Argo CD be sure to check out my GitOps and Argo CD courses on Pluralsight here:

GitOps: The Big Picture“:

https://app.pluralsight.com/library/courses/gitops-the-big-picture

Getting Started with Argo CD“:

https://app.pluralsight.com/library/courses/argo-cd-getting-started

And here is the link to my Pluralsight profile to follow me for future GitOps, Kubernetes, Cloud, and DevOps content: https://app.pluralsight.com/profile/author/steve-buchanan

Read more

Cloudbusting Podcast Guest EP130 – Diving into Jucy Lucy, & Root Canal’s with a little Kubernetes, Azure Arc, & GitOps on the side

This was a fun podcast with Jez Ward, and Dave Chapman of Cloudreach. They run a podcast called Cloudbusting. On the podcast they focus on transformation, leadership, ways of working and emerging technology they explore the significant impact that cloud is having on people and businesses.

On this podcast episode, we set out to answer very important questions such as what are Jucy Lucy’s?, what are root Canals like today? oh, and yeah we also spend some time talking about what Containers, Kubernetes, & GitOps are and how they fit in the cloud.

Check out the podcast episode here:

https://www.cloudreach.com/en/podcasts/diving-into-kubernetes-azure-arc-gitops-episode-130

Read more

New Rancher Course Published for Multiple Kubernetes Management

My 14th course has been published on Pluralsight! This course is titled “Getting Started with Rancher” and can be found here: https://app.pluralsight.com/library/courses/rancher-getting-started

With the growth of Kubernetes, the complexity & needs have also grown. IT Professionals need help with the operational & security challenges of managing Kubernetes clusters across multiple clouds, on-premises, & the edge.

My new course will teach you how to use Rancher for multi-Kubernetes cluster management, streamlining Kubernetes cluster deployments, & unified multi-Kubernetes cluster app management. When you’re finished with this course, you’ll have the skills and knowledge of Rancher needed for multi-K8s cluster management.

There is a learning path on Pluralsight focused on Kubernetes management. This is my 3rd course in the Kubernetes Management path titled “Kubernetes Tooling and Techniques” on Pluralsight. My other courses in the path are: “GitOps: The Big Picture” and “Getting Started with Argo CD“. You can get to the path using this link: https://app.pluralsight.com/paths/skills/kubernetes-tooling-and-techniques

Again you can check out my new Rancher course here: https://app.pluralsight.com/library/courses/mulesoft4-big-picture

I hope you find value in this new Getting Started with Rancher course. Be sure to follow my profile on Pluralsight so you will be notified as I release new courses related to Kubernetes and other topics! 

Here is the link to my Pluralsight profile to follow mehttps://app.pluralsight.com/profile/author/steve-buchanan

Read more

Get started with Argo CD & Azure Kubernetes Service

As Kubernetes adoption continues to grow so does GitOps. GitOps has been increasing in adoption and popularity among enterprises at a fast rate as well. Here is what GitOps is: “GitOps is an operating model pattern for cloud-native applications & Kubernetes storing application & declarative infrastructure code in Git as the source of truth used for automated continuous delivery.” GitOps puts Git at the center of continuous delivery making git the Source of Truth describing the desired state of your entire system. For a deeper dive into GitOps check out my GitOps course on Pluralsight here: https://app.pluralsight.com/library/courses/gitops-the-big-picture

In the GitOps model, you need GitOps operators. GitOps Operators are software agents that continuously monitor your apps running on your Kubernetes clusters comparing the live state of your app against the desired state you have defined in your Git repository. These GitOps Operators ensure the desired state is in place on your Kubernetes clusters performing create, update, delete activities on your Kubernetes clusters as needed.

This is where Argo CD comes into the picture. Argo CD is one of the top GitOps Operators. It provides declarative, continuous delivery to your Kubernetes clusters. It was created by a team at Intuit in 2018 and later open-sourced. I am going to write a few blogs exploring the use of Argo CD with AKS. This will be the first of the series walking through the deployment of Argo CD to AKS. In the next post, we will deploy an app to Kubernetes using Argo CD and see where the topic goes from there. Now let’s dive into deploying Argo CD to AKS. Here are the steps:

-DEPLOYING ARGO CD ON AKS-

Log onto the Azure portal (https://portal.azure.com)

Launch Azure Cloud Shell

Connect to your AKS cluster from the Azure Cloud Shell by running the following:

# Set your subscription

az account set –subscription YOURSUBSCRIPTIONIDHERE

# Connect to your KS cluster

az aks get-credentials –resource-group YOURRESOURCEGROUPNAME –name CLUSTER NAME

Next let’s create a namespace for Argo CD to deploy all of its components in. To do this run:

kubectl create namespace argocd

Next we can install Argo CD into the new namespace we created. We will reference Argo CD’s GitHub repository for the latest Argo CD operator. Run the following:

kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

You should see the following:

You should end up with many objects in the Argo CD namespace.

By default, the Argo CD is not accessible externally. It is deployed with a service type of ClusterIP.

Leaving it at ClusterIP is fine but for the purposes of this blog/lab and getting started lets change this so we can easily access the ArgoCD Server website. # Change the argocd-server service type to LoadBalancer. To do this run the following:

kubectl patch svc argocd-server -n argocd -p ‘{“spec”: {“type”: “LoadBalancer”}}’

Now you will be able to see that the argocd-server service type has been changed to a LoadBalancer type. This means that it now has a public Azure load balancer attached to it with an external IP.

NOTE: This is not recommended in production environments. Only use in a lab or dev environment. In production environments, it is recommended to use an ingress for the Argo CD API server that is secured.

Argo CD auto generated a password during the deployment. We need to get the Argo CD password so we can log into it. To get the password run the following:

kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath=”{.data.password}” | base64 -d && echo

You will see the password in readable format so you can copy it such as shown in the screenshot.

Note the default Argo CD username is admin.

To access the Argo CD web portal you need to access the Argo CD API Server. To do this you can either do this from the external IP of the argo-cd object or the via the Argo CD CLI using the following:

 argocd login <ARGOCD_SERVER>

The Argo CD web portal will look like:

That’s it! You have Argo CD deployed on your AKS cluster. In the next post, I will walk through deploying a simple app to your Kubernetes cluster via Argo CD.

Read more

Pluralsight Author Talk – GitOps: The Big Picture

Coming up soon I will be giving my first Pluralsight Author Talk. This live session will be on GitOps. I will cover a fundamental understanding of GitOps, the need for GitOps, GitOps architecture, GitOps workflow, GitOps principles, practices, & tooling such as Flux, Argo CD, AND Jenkins X.

More information here: https://www.pluralsight.com/events/2021/delta-author-talk-q4

Read more

Azure Arc enabled Kubernetes: Getting Started – Pluralsight Course

Today Pluralsight published 1 of 2 Azure Arc courses I am building. This marks my 10th course on Pluralsight! This first course is Azure Arc enabled Kubernetes: Getting Started. Azure Arc is a key service in the Azure story extending Azure to the on-premises data center and or multiple clouds outside of Azure.

This course is just under 2 hours and packed full of information & demos to help you get started with the topic. I go into a deeper understanding of the multi-cloud market, Kubernetes in the enterprise, and Azure Arc enabled Kubernetes. Azure Arc enabled Kubernetes architecture. Setting up and using Azure Arc enabled Kubernetes. And using GitOps with Azure Arc enabled Kubernetes.

NEW 
RELEASES 
(-18 _ 
Google Cloud 
Understanding Google Cloud 
Security and Operations 
Google Cloud 
lh 17m 
Beginner 
Jun 2021 
Azure Arc Enabled 
Kubernetes: Getting Started 
Steve Buchanan 
Intermediate 
lh 57m 
Jun 2021 
Using the Xamarin.Forms 
Shell 
Christopher Miller 
Intermediate 
Jun 2021 
(30) 
lh 33m 
Demystifying the AWS 
Certified Developer: Associ... 
Ryan Lewis 
Intermediate 
Jun 2021 
57m 
(32)

I am excited about releasing this course for several reasons: #1 Azure Arc is a newer technology from Microsoft and I am happy to share my knowledge about it, #2 This one is a combination of Azure Arc, Kubernetes, & GitOps all technologies I have been working with regularly, #3 Azure is growing, Kubernetes is growing, and multi-cloud is growing all in the enterprise and this course covers all three of these.

Azure Arc Enabled Kubernetes: Getting Started 
by 
w i u how end A«

Here is the description of the course:

Managing Kubernetes clusters across on-premises and multiple clouds can be disjointed and overly complicated. In this course, Azure Arc Enabled Kubernetes: Getting Started, you’ll learn to how to manage external Kubernetes clusters with Azure Arc. First, you’ll explore what Azure Arc k8s is and how to use it. Next, you’ll discover the features of Azure Arc K8s and how to use them. Finally, you’ll learn how to how to use Azure Arc K8s and GitOps to deploy applications. When you’re finished with this course, you’ll have the skills and knowledge of Azure Arc enabled Kubernetes needed to manage Kubernetes clusters across on-premises and multiple clouds.

Here are the areas and topics of the course:

Understanding Azure Arc Enabled Kubernetes                

Intro and Topics                

Understanding Azure Arc Enabled Kubernetes                

Understanding Azure Arc Enabled Kubernetes Use Cases                

Understanding Azure Arc Enabled Kubernetes Architecture                

Summary

 

Working with Kubernetes Clusters Using Azure Arc                

Intro and Topics                

Connecting Kubernetes Clusters to Azure Arc                

Demo: Connecting a Kubernetes Cluster to Azure Arc                

Monitoring projected Kubernetes Clusters with Azure Monitor and Azure Arc                

Demo: Setup Azure Monitor of Projected Kubernetes Cluster in Azure Arc                

Defining Authorization on Azure Arc Projected Kubernetes with Azure RBAC                

Protecting Azure Arc Projected Kubernetes Clusters with the Azure Defender                

Administering Projected Kubernetes Clusters with Azure Policy and Azure Arc                

Summary

 

Deploying Applications to Projected Kubernetes Clusters Using Azure Arc and GitOps                

Intro and Topics                

Understanding How GitOps Works with Azure Arc Enabled Kubernetes                

Deploying Applications and Configurations to Projected Kubernetes Clusters with Azure Arc and GitOps                

Demo: Deployment of an Application to a Projected Kubernetes Cluster via GitOps and Azure Arc K8s                

Deploying Helm Charts to Projected Kubernetes Clusters with Azure Arc and GitOps                

Deploying IoT Workloads with Azure Arc and GitOps                

Understanding the Future of Azure Arc Enabled Kubernetes                

Summary               

Here is a preview of the Azure Arc enabled Kubernetes course:

Here is a link to the full Azure Arc enabled Kubernetes course on Pluralsight: https://app.pluralsight.com/library/courses/azure-arc-enabled-kubernetes-getting-started

This course is also a part of the all new Azure Arc path titled “Managing Environments with Azure Arc” on Pluralsight. There are other courses in the path already such as Azure Arc: The Big Picture, Azure Arc-enabled Data Services: The Big Picture, and Azure Arc and Azure Lighthouse: First Look and many more Azure Arc courses on the way.

Managing Environments with Azure Arc 
This learning path introduces you to Microsoft's latest Azure offering: 
Anne Arc. With able to and 
deployed within of Azwe through the same 
control plane. As pu watch the courses in this path, first gain a 
of Arc and its befMe rm•mg 
to leaming product's specific including 
Azure Arc-enabled servers, Azure Arc-enabled data services, Azure Arc- 
«labled SOL 
leaming. Azure Arc and Amre At end of 
this learning path, you'll have a good foundation of knowledge to get 
started with Anne 
Related Topics 
Azure. SOL Server. Kubernetes. Machine Learning. Azure 
Light o 
Prerequisites 
Learners interested in path have experience With Azure administration and be familiar With the 
fund—Ital coruepts of data SOL 
What You Will Learn 
F u nd—-vtal Äpts 
AA' re with 
AA' re Arc's vah_æ and off«ings 
HOW Azure Arc enabled Se•rvetS work 
How Azure Arc enabled SOL Server works 
HOW Azure Arc enabled machine learning works 
How Azure Arc and Azure 
Authors 
Add ed to channel 
New questions are jn_theworks. 
with the courses below. 
Ben a Data MVP. with SOL siQ SOL 
Sewer mainly in the al/Datawarehousing held, Besides his muralsi9ht courses, Ben is_ 
Steve Buchanan is a Director, & Midwest Containers Services Lead on a Cloud 
Trarßfotmati•M/DevOps team with a large He is a 9-time '&rOSOft 
Niraj is a AWS/Azure Specialist with over a decade of work experience "Ito 
Data Modeling with Databases like Cassarudra, M0090Da, SparkSG•L ElasticSearch and SOL

Here is a link to check out the new Azure Arc path: https://app.pluralsight.com/paths/skills/managing-environments-with-azure-arc

I hope you find value in this new Azure Arc enabled Kubernetes: Getting Started course. Be sure to follow my profile on Pluralsight so you will be notified as I release new courses including my second Azure Arc related course! 

Here is the link to my Pluralsight profile to follow mehttps://app.pluralsight.com/profile/author/steve-buchanan

Read more

30 Minutes of Azure Kubernetes Services (AKS)

Today I went on “Tech Talk Wednesday” a podcast and radio show with Kazeem Adegboyega The topic was “30 Minutes of Azure Kubernetes Services (AKS)“. It streamed online via Microsoft Teams and aired in Lagos, Nigeria on Lagos State University (LASU) radio (95.7).

I had a great time talking with Kazeem! Even Sam Erskine made a guest appearance. If you missed the live show you can watch it on YouTube:

Or go directly to the video here: https://www.youtube.com/watch?v=q7GCHQudCWg

Read more

Tech Talk Wednesday Guest – Topic: Azure Kubernetes Service (AKS)

I am honored to be a guest next Wednesday, August 26th on the “Tech Talk Wednesday” podcast and radio show with Kazeem Adegboyega (@KazeemCanTeach)! We will be chatting about Azure Kubernetes Service (AKS).

This show will be streamed online via Microsoft Teams and will air in Lagos, Nigeria on Lagos State University (LASU) radio (95.7)!

One of my goals is to help spread knowledge about tech in Africa and showcase African technologists in the US. This is the first step in that journey.

Be sure to tune in. Go here to register: http://kazeem.com.ng/index.php/tech-talk

Don’t worry if you can’t make the live stream. It will be recorded and posted later on Kazeem’s website.

Read more

Passed – Docker Certified Associate (Study guide)

Today I passed the Docker Certified Associate exam! In this post, I will share some details about the exam and the resources I used to study for it.

The certification is good for 2 years after passing the exam. It demonstrates that you have foundational real-world Docker skills. It is multiple choice with 55 questions, you have 1.5 hours to finish it and costs $195 USD. It is recommended that you have 6 to 12 months of hands-on experience with Docker before taking the exam. You can read about and sign up for the exam here: https://success.docker.com/certification.

I have had some folks ask me why I would waste my time taking the Docker exam. They say to focus on Kubernetes and Open Shift instead of Docker. Lets talk about why I chose to pursue the Docker certification. First off you have to run containers on those orchestration platforms mentioned before and chances are you will run Docker containers on them. Therefore before diving into an orchestration platform it is important to be knowledgable on containers. Also, I have seen many scenarios in the cloud where it makes sense to run containers directly on the cloud platform itself and again chances are those will be Docker containers. Docker is still a leader in the container space. There are several reports and articles that point to this. Here are some of the reports and articles backing this up:

Docker listed as the leader in the “Container Tools Used” section of the RightScale 2019 State of the Cloud Report” here:

Docker is listed as #2  with 31.35% market share on Datanyze 2020 “Containerization Market Share Competitor Analysis Report” here:

Source: https://www.datanyze.com/market-share/containerization–321

In “Sysdiags 2018 Docker usage report” they show “What container runtimes are in use?” showing Docker as the leader. 

Source: https://sysdig.com/blog/2018-docker-usage-report/

And finally this article arguing that Docker is better than LXC here:

https://www.upguard.com/articles/docker-vs-lxc

I will call out that the Docker exam covers Swarm mode orchestration platform that is included with Docker. Swarm mode is a lot easier to learn and use compared to Kubernetes however, Kubernetes has won the orchestration platform war. It would be nice if Docker would revamp the exam reducing or removing Swarm and replace with some Kubernetes objectives. This would make more sense because there is a strong chance Swarm will not be used in the real world. 

The Docker exam was not an easy exam and you definitely want to have some hands-on with Docker before taking it. There are a ton of resources out there that you can leverage beyond hands-on to assist in your study for this certification. There are many books available. You can do a quick search on Amazon and check the reviews for one that would be a good fit for you. I have read a couple of books on Docker and have co-authored a book on AKS with a chapter dedicated to Docker in it.

Here is the list of what I used to study.

Free Hands on Docker labs (This resource was huge for me. It gave me environments to use and scenarios for training with Docker and Docker Swarm mode.):

I attended a “Docker JumpStart Virtual Workshop” by Microsoft MVP Mike Pfeiffer and Microsoft MVP/Docker Captain Dan Wahlin (This workshop ocurred in the past but I beleive you can sign up and watch the recordings from the workshop.):

Free Docker Certification review questions here (This blogger has a bunch of review questions to help you get in the right mindset. They cover all the exam areas.): 

Docker courses and learning checks on Pluralsight (The courses are great. I found the learning checks very useful becuase it was a good way to check my knowledge in all of the exam areas.):

Spent time working with Docker on some projects (self explanatory).

Overall the Docker certification is a good move for your career as an IT Pro, developer, if you work in DevOps, and with cloud. I definitely recommend getting this certification. If you decide to go after it good luck!

Stay tuned for more blog posts with insights on certifications in the future.

Read more