Running Cron Jobs in Kubernetes Cluster
September 13, 2021
In this guide , Lets see how we can deploy and run Cron Jobs in the Kubernetes Cluster in a Scheduled and Parallel manner.
CronJob runs the jobs at the scheduled time
It's like configuring a command or the script in the linux crontab to execute once in a while.
For example : To create backup at a specific time , Sending Emails etc.
A Job will create a pod / pods which will make sure the pods are deleted after the completion of the job
Deleting the Job will also delete the pod created for it.
In case of node failure or manual deletion of a Job, The Job will be rescheduled.
Job ensures successful completion by its Pod.
- Kubectl should be installed on the local machine to communicate with the kubernetes cluster
- Minikube or the Kubernetes cluster running on any cloud will work for this tutorial.
Types of Jobs in Kubernetes
- Schedulers (CronJob) – Its like scheduling tasks in crontab in Linux.
- Run to Completion – It runs the Job in parallel by creating one or more pods for the successful completion
Download kubectl using the below commands.
curl -o kubectl https://amazon-eks.s3.us-west-2.amazonaws.com/1.16.8/2020-04-16/bin/linux/amd64/kubectl
Add execute permission to the downloaded binary
chmod +x ./kubectl
Copy or Move it to the /usr/local/bin directory so that we don't have to type the full path of the kubectl.
Run the below command to check the version of kubectl installed,
You may also like:How can Cloud Services Transform your Business?
Creating a CronJob
Lets deploy a job which can run for every 5 minutes and output as “Hello Kubernetes Cron Job”.
Kind : CronJob will be created for this.
Create a file named job.yaml and enter the below contents and then Save and close the file.
schedule: "*/5 * * * *"
- name: hello
command : ["echo", "Hello Kubernetes Cron Job"]
Run the below command to create a job
kubectl create -f job.yaml
It will give the below response.
Using the following command , Check the status of the Job
kubectl get cronjob -o wide
Monitor the job using the command.
kubectl get jobs --watch
The Job will create a Pod.
Run the below command to get the pod details.
kubectl get pods
Check the logs for a pod by executing the below command.
kubectl logs podname
As per the configured job.yaml file , The Job is successfully created and displays the required response.
Deleting Cron Job
First Get the name of the job using this command
kubectl get cronjob
We can delete the Cron Job using the below command.
kubectl delete cronjob sample-job
The above command will delete all the job and the pods.
The additional parameters can be used while creating a Cton job as follows.
schedule: "*/5 * * * *"
- name: hello
command : ["echo", "Hello Kubernetes Job"]
Concurrency Policy – is responsible for parallel jobs.
The following concurrency policy can be used as follows ,
Allow - which by Default allow the The cron job to run concurrently
Forbid – Doesn’t allow concurrent jobs.
Replace – The old job will be replaced with the new job if the old job is not completed on time.
Also Read: How to Trigger Lambda Functions Automatically
Job History Limits – such as successfulJobHistoryLimit =3 and failedJobsHistoryLimit=1 are optional fields.
It refers how many successful and failed job history can be present on the cluster
Set the limit to 0 , There won't be any job history
Running Multiple Job Pods in Parallel
When a Job is deployed , It will create multiple pods and run the jobs in parallel.
We will create a file named multiple-jobs.yaml and add the below config.
- name: busybox
command: ["echo" , "kubernetes jobs parallel"]
Understanding the parameters used in multiple-jobs.yaml file.
Running multiple jobs with the same name will cause an error reporting that the job with the same name already exists.
To fix this issue , We should add the generateName field in the metadata section.
So when the Job is executed it will create the pods with prefix kube-jobs– and with numbers.
completions – the no. of pods that can be used for the successful completion.
restartPolicy – accepts always , Never , OnFailure.
As the jobs are intended to run pods till completion , We should use never and onFailure for restartPolicy.
Lets deploy the Cron job using the below command.
kubectl create -f multiple-jobs.yaml
Check the available jobs,
kubectl get jobs -o wide
Get the list of pods created by the Job,
kubectl get po
You will get the similar response.
You can check the logs for one of the pod using the below command
kubectl logs -f parallel-job-jbfpb
Execute the below command to delete the job
kubectl delete -f multiple-jobs.yaml
We have learned to run Cron Jobs in a Scheduled and Parallelistic way on the Kubernetes Cluster.