Mastering CI/CD with Docker, Jenkins Agents, and Dockerhub Integration
Introduction:
On the 28th day of my 90DaysOfDevOps challenge, I achieved a breakthrough in CI/CD by constructing a robust pipeline for my Node.js application. This blog post reveals the step-by-step process, from setting up prerequisites and creating Jenkins agents to integrating Docker and Dockerhub into the pipeline.
Jenkins Master (Server)
Jenkins’s server or master node holds all key configurations. Jenkins master server is like a control server that orchestrates all the workflow defined in the pipelines. For example, scheduling a job, monitoring the jobs, etc.
Jenkins Agent
An agent is typically a machine or container that connects to a Jenkins master and this agent that actually execute all the steps mentioned in a Job. When you create a Jenkins job, you have to assign an agent to it. Every agent has a label as a unique identifier.
Benefits of Using Agents:
Resource Isolation: Agents help isolate and distribute the workload, allowing different stages of the pipeline to run on different machines or environments.
Parallel Execution: When stages are assigned to different agents, they can run in parallel, improving overall pipeline efficiency.
Optimized Resource Utilization: By specifying agents with specific labels, you can optimize resource utilization based on the requirements of each stage.
Task: Crafting the CI/CD Pipeline
Prerequisite Setup:
Before diving into the CI/CD pipeline, let's ensure everything is set up correctly:
EC2 Instance:
Ensure you have an EC2 instance serving as the agent server for Jenkins.
Software Installations:
Docker and Docker Compose: Installed on the agent server.
Java: Must be installed on the agent server.
Application Deployment:
Deploy your Node.js application on the Jenkins agent.
i. SSH Credentials (Step-by-Step Process):
Certainly! Below is a step-by-step process for creating SSH credentials in Jenkins:
Step 1: Log in to Jenkins
Access your Jenkins dashboard by navigating to the Jenkins URL in your web browser.
Step 2: Navigate to "Manage Jenkins"
On the Jenkins dashboard, click on "Manage Jenkins" on the left-hand side.
Step 3: Access "Manage Credentials"
Under "Manage Jenkins," find and click on the "Manage Credentials" link.
Step 4: Choose "Jenkins" (or another domain)
Inside the "Manage Credentials" page, you'll see different domains. If you're setting up SSH credentials for general use, you can choose "(global)" or "Jenkins" (or another domain depending on your setup).
Step 5: Click "Add Credentials"
Within the selected domain, click on the "Add Credentials" link.
Step 6: Select "Kind" as "SSH Username with private key"
In the "Kind" dropdown, choose "SSH Username with private key" since you are setting up SSH credentials.
Step 7: Enter SSH Username and Private Key
Fill in the required fields:
Username: Your SSH username for connecting to the remote server.
Private Key: Paste your private key. You can either directly paste it or use the "From a file" option if the private key is in a file.
on EC2 Agent or Main Server:
COPY & PASTE in Jenkins Private Key:
COPY Public Key :
Goto Agent Server And PASTE in authorized_Keys:
Step 8: (Optional) Add a Description
Optionally, you can add a description to help identify these credentials later.
Step 9: Click "OK" to Save
Click the "OK" button to save your SSH credentials.
Step 10: Verify Credentials
After saving, you'll see the newly added SSH credentials in the list. You can click on them to verify that they are correctly configured.
Your Jenkins instance is now equipped with the SSH credentials needed for interactions with remote servers during the CI/CD pipeline setup.
This SSH credential will be useful when launching agents via SSH, as seen in the example mentioned in the previous blog. It ensures secure communication between Jenkins and the remote agent server.
ii. Jenkins Agent Creation:
Let's walk through the process of creating a Jenkins agent:
Dashboard: Navigate to Jenkins' dashboard.
Manage Jenkins: Click on "Manage Jenkins."
Nodes: Choose "Nodes" to create a new agent.
Agent Configuration:
Name: Assign a name to your agent.
Description: Add a brief description.
Number of Executors: Set the desired number of executors.
Remote Root Directory: Specify the remote root directory for your agent.
Labels: Define labels for usage in the pipeline.
Launch Method: Opt for "Launch agents via SSH."
Host: Enter the host (IP or DNS).
Credentials: Provide the necessary SSH credentials.
Host Key Verification Strategy: Choose the preferred strategy.
Availability: Set the agent's availability preferences.
iii. Dockerhub Credentials (Step-by-Step Process):
Before diving into the CI/CD pipeline, let's set up Dockerhub credentials:
Login to Jenkins: Access Jenkins and navigate to the dashboard.
Credentials: In the dashboard, go to "Manage Jenkins" > "Manage Credentials."
(Optional) System Configuration: If needed, add Dockerhub server credentials globally by going to "Manage Jenkins" > "Configure System" and scrolling down to "Global Docker Hub Settings."
Add Credentials:
Click on "(Jenkins)" under "Stores scoped to Jenkins."
Click "Add Credentials."
Choose "Username with password" for Dockerhub.
Enter your Dockerhub username and password.
iV. Docker-Integrated Jenkins Declarative Pipeline with Console:
Proceed with creating a declarative pipeline within Jenkins:
pipeline {
agent any
stages {
stage('Checkout') {
steps {
checkout scm
}
}
stage('Build') {
steps {
echo 'Building the Node.js application...'
// Your build steps for Node.js application
}
}
stage('Test') {
steps {
echo 'Running tests...'
// Your testing steps
}
}
stage('Deploy') {
steps {
echo 'Deploying the application...'
// Your deployment steps
}
}
}
}
V. GitHub Integration with Webhook:
Finally, integrate the Jenkins pipeline with GitHub using a webhook. In your GitHub repository settings, add a webhook pointing to your Jenkins instance.
Conclusion:
Day 28 marked a significant stride in CI/CD with a fortified pipeline for my Node.js application. The detailed guide covers prerequisites, Jenkins agent creation, Dockerhub credential setup, and the creation of a Docker-integrated Jenkins declarative pipeline. Check out the complete tutorial on Hashnode and stay tuned as I continue unraveling insights on this DevOps journey! 🚀🛠️ #DevOps #CI/CD #Jenkins #Docker #NodeJS 🚀