This blog outlines a streamlined approach to automating your website's regression tests using Cypress. We'll harness Jenkins pipelines orchestrated with Kubernetes and Docker containers to manage dependencies seamlessly and deliver clear, comprehensive test reports.
If you're new to Cypress or Jenkins, our previous blogs, Visual Testing with Cypress and Setting Up Cypress with Jenkins Pipeline, provide foundational insights to help you get started.
Docker Containers for Cypress Testing
We utilize Kubernetes Pods orchestrated by Jenkins to run Docker containers, which encapsulate our testing environments. We configure our Jenkinsfile to define Kubernetes Pods and Docker images, set working directories and storage capacities.
- Cypress Container: Includes Cypress latest Docker image.
- Alpine Container: Lightweight container used for auxiliary tasks, such as report generation and dependency management.
Docker Images
Setting up the Jenkins Pipeline:
Defining the Jenkins file
Below is the Jenkinsfile with Kubernetes Pod and Docker Image defined:
pipeline {
agent {
kubernetes {
label 'cypress-initialyze-qa'
yaml """
kind: Pod
metadata:
name: cypress-initialyze-qa
spec:
containers:
- name: cypress-initialyze-qa
image: cypress/included:latest
imagePullPolicy: Always
command: ['cat']
tty: true
workingDir: /home/jenkins/agent/
resources:
requests:
memory: "2Gi"
limits:
memory: "4Gi"
- name: cypress-allure-reports
image: alpine:latest
imagePullPolicy: Always
command: ['cat']
tty: true
workingDir: /home/jenkins/agent/
resources:
requests:
memory: "2Gi"
limits:
memory: "4Gi"
"""
}
}
Automated Triggers with pollSCM
Jenkins pipeline uses the pollSCM trigger to regularly monitor code changes:
triggers {
pollSCM 'H/3 * * * *' // Checks source code repository every 3 hours
}
What is pollSCM?
pollSCM is a Jenkins trigger mechanism that periodically polls the Source Code Management (SCM) system, such as Git or SVN, to check for new commits or updates. The provided cron-like expression 'H/3 * * * *' indicates that Jenkins polls the repository every three hours.
Pipeline Stages for Regression Testing
Jenkins pipelines comprise distinct stages, each executing specific testing processes:
Build Stage
In this initial stage, dependencies and necessary configurations are established:
- Dependencies are installed in the cypress-regression-test container.
- JDK installation occurs in the secondary cypress-allure-reports container due to limitations in the Cypress image.
stages {
stage ('Build') {
steps {
container('cypress-initialyze-qa') {
script {
sh 'echo build'
// Cache node_modules directory
if (fileExists('node_modules')) {
echo 'Using cached node_modules directory'
} else {
sh 'npm i'
}
sh 'curl --version'
sh 'ls -a'
}
} //container
container('cypress-allure-reports') {
script {
sh '''
apk update
apk add openjdk11
apk add nodejs npm
'''
}
} //container
}//steps
} //End: Build
2. Run Tests Stage
Here, Cypress executes automated regression tests:
- Base images for visual comparisons are downloaded and prepared.
- Tests are executed using
npm run test - Post-steps store test artifacts for analysis.
stage ('Run Tests') {
steps {
container('cypress-initialyze-aem-qa') {
script {
sh 'echo start tests'
// Fetch the baseline images from shared drive
sh 'curl -L -o baseline.zip "https://drive.google.com/"'
sh 'ls -lh baseline.zip'
// Create a baseline directory and unzip the content on it.
sh 'mkdir -p cypress-image-diff-html-report/visual-test/screenshots'
sh 'unzip baseline.zip -d cypress-image-diff-html-report/visual-test/screenshots'
sh 'ls -a cypress-image-diff-html-report/visual-test/screenshots'
// Execute the test.
sh 'npm run test'
}
} //container
}//steps
post {
// To display artifacts of the build
always {
archiveArtifacts artifacts: 'allure-results/**'
}
}//post
} //End: Run Tests
3. Post Stage
The Post stage runs irrespective of the build outcome, generating clear, informative reports using the PublishHTML plugin:
- Allure Reports: Detailed, step-by-step test results, highlighting failures clearly.
- Cypress Image Diff Reports: Visual comparison between expected (baseline) and actual images, allowing easy updates and clear insights into UI changes.
post {
success {
cleanWs()
}
always {
container('cypress-allure-reports') {
script {
sh 'echo Generating reports'
// Allure command to run report
sh 'npm run allure:report'
// Allure command to generate report
sh 'npm run generate-test'
}
publishHTML([
allowMissing: false,
alwaysLinkToLastBuild: true,
keepAll: true,
reportDir: 'cypress-image-diff-html-report/',
reportFiles: 'allure-report/index.html,index.html',
reportName: 'Combined Report',
reportTitles: 'Allure Report, Screenshot Comparison Report',
useWrapperFileDirectly: true])
// To display artifacts of the build
archiveArtifacts artifacts: 'allure-report/**, cypress-image-diff-html-report/**, visual-test/**'
} //container
} //End: Always
} //post
} // pipeline
Jenkins Dashboard and Reporting
The Jenkins dashboard provides a centralized view of your entire testing pipeline. With every execution:
- You get detailed build logs, test results, and status updates.
- HTML reports (including Allure and visual regression reports) are directly accessible from the Jenkins interface.
- Report links persist across builds, making it easy to compare test runs and identify regressions.
- All archived artifacts can be reviewed or downloaded for offline analysis.
This visibility and traceability make Jenkins a powerful tool for monitoring the health of your application through automated testing.
Final Thoughts:
By integrating Cypress with Jenkins CI/CT, and using docker and kubernetes for environment orchestration, you can build a robust and scalable automated regression testing pipeline. This setup not only improves testing efficiency but also provides detailed insights through advanced reporting, enabling teams to identify and fix issues early in the development lifecycle.
Need help setting up or scaling your automated regression testing pipeline? Reach out to us, we specialize in building end-to-end testing solutions that deliver quality at speed.