Running Cron Jobs in Kubernetes Cluster

Running Cron Jobs in Kubernetes Cluster

In this guide , Lets see how we can deploy and run Cron Jobs in the Kubernetes Cluster in a Scheduled and Parallel manner. 
Cron-Jobs-(1).png

CronJob runs the jobs at the scheduled time 

It's like configuring a command or the script in the linux crontab to execute once in a while. 
For example : To create backup at a specific time  , Sending Emails etc. 

A Job will create a pod / pods which will make sure the pods are deleted after the completion of the job
Deleting the Job will also delete the pod created for it. 

In case of node failure or manual deletion of a Job, The Job will be rescheduled.
Job ensures successful completion by its Pod. 

Prerequisites 

  1. Kubectl should be installed on the local machine to communicate with the kubernetes cluster 
  2. Minikube or the Kubernetes cluster running on any cloud will work for this tutorial. 

Types of Jobs in Kubernetes 

  1. Schedulers (CronJob) – Its like scheduling tasks in crontab in Linux. 
  2. Run to Completion – It runs the Job in parallel by creating one or more pods for the successful completion 

Installing Kubectl 

Download kubectl using the below commands. 
curl -o kubectl https://amazon-eks.s3.us-west-2.amazonaws.com/1.16.8/2020-04-16/bin/linux/amd64/kubectl 

Add execute permission to the downloaded binary 
chmod +x ./kubectl 

Copy or Move it to the /usr/local/bin directory so that we don't have to type the full path of the kubectl. 

Run the below command to check the version of kubectl installed,
kubectl version 
 

You may also like:How can Cloud Services Transform your Business?

version-of-kubectl-installed-(1).png

Creating a CronJob 

Lets deploy a job which can run for every 5 minutes and output as “Hello Kubernetes Cron Job”. 

Kind : CronJob will be created for this. 

Create a file named job.yaml and enter the below contents and then Save and close the file. 

apiVersion: batch/v1
kind: CronJob
metadata:
   name: kube-job
spec:
   schedule: "*/5 * * * *" 
   jobTemplate:
     spec:
       template:
          spec:
            containers:
            - name: hello
              image: busybox
              command : ["echo", "Hello Kubernetes Cron Job"]
            restartPolicy: OnFailure 


Creating-a-CronJob.png

Run the below command to create a job
kubectl create -f job.yaml 

It will give the below response.
 
command-to-create-a-job.png
Using the following command , Check the status of the Job
kubectl get cronjob -o wide 

command-Check-the-status.png

Monitor the job using the command. 
kubectl get jobs --watch 

command-Monitor-the-job.png

The Job will create a Pod.  

Run the below command to get the pod details. 
kubectl get pods 

Check the logs for a pod by executing the below command. 
kubectl logs podname 

create-a-Pod.png

As per the configured job.yaml file , The Job is successfully created and displays the required response. 

Deleting Cron Job

First Get the name of the job using this command
kubectl get cronjob 

command-name-of-the-job.png

We can delete the Cron Job using the below command. 
kubectl delete cronjob sample-job 

The above command will delete all the job and the pods. 

command-delete-all-the-job-and-the-pods-(1).png

The additional parameters can be used while creating a Cton job as follows. 

apiVersion: batch/v1beta1
kind: CronJob
metadata:
   name: test-job
spec:
   schedule: "*/5 * * * *" 
   concurrencyPolicy: Forbid
   successfulJobsHistoryLimit: 5
   failedJobsHistoryLimit: 5
   jobTemplate:
     spec:
       template:
         spec:
           containers:
           - name: hello
             image: busybox
             command : ["echo", "Hello Kubernetes Job"]
           restartPolicy: OnFailure 


Concurrency Policy – is responsible for parallel jobs. 

The following concurrency policy can be used as follows , 

Allow - which by Default allow the The cron job to run concurrently 
Forbid – Doesn’t allow concurrent jobs. 

Replace – The old job will be replaced with the new job if the old job is not completed on time. 

Also Read: How to Trigger Lambda Functions Automatically

Job History Limits – such as successfulJobHistoryLimit =3 and failedJobsHistoryLimit=1 are optional fields. 

It refers how many successful and failed job history can be present on the cluster 

Set the limit to 0 , There won't be any job history 

Running Multiple Job Pods in Parallel 

When a Job is deployed , It will create multiple pods and run the jobs in parallel. 

We will create a  file named multiple-jobs.yaml and add the below config. 

apiVersion: batch/v1
kind: Job
metadata:
    generateName: kube-jobs-
    name: kube-parallel-job
    labels:
      jobgroup: kubecron-group
spec:
    completions: 3
    parallelism: 2
    template:
      metadata:
         name: kube-parallel-job
         labels:
           jobgroup: kubecron-group
      spec:
        containers:
        - name: busybox
          image: busybox
          command: ["echo" , "kubernetes jobs parallel"]
        restartPolicy: OnFailure 


Multiple-Jobs-(1).png

Understanding the parameters used in multiple-jobs.yaml file. 

Running multiple jobs with the same name will cause an error reporting that the job with the same name already exists. 

To fix this issue , We should add the generateName field in the metadata section. 

So when the Job is executed it will create the pods with prefix kube-jobs– and with numbers. 

completions – the no. of pods that can be used for the successful completion. 

restartPolicy – accepts always , Never , OnFailure.  

As the jobs are intended to run pods till completion , We should use never and onFailure for restartPolicy. 

Lets deploy the Cron job using the below command. 
kubectl create -f multiple-jobs.yaml 

deploy-the-Cron-job-(1).png

Check the available jobs,
kubectl get jobs -o wide

Available-Jobs-(1).png

Get the list of pods created by the Job,
kubectl get po 

You will get the similar response. 

list-of-pods-created-by-the-Job.png

You can check the logs for one of the pod using the below command
kubectl logs -f parallel-job-jbfpb 

command-check-the-logs.png

Execute the below command to delete the job
kubectl delete -f multiple-jobs.yaml 

delete-the-job.png

We have learned to run Cron Jobs in a Scheduled and Parallelistic way on the Kubernetes Cluster. 

Recent Comments:-

There are no comments yet.

Leave message