Continuous Integration and Continuous Deployment (CI/CD) pipelines have become critical in software development for delivering reliable, efficient, and automated workflows. In this blog, we’ll explore the process of building a robust CI/CD pipeline leveraging Git, Jenkins, Maven, and Tomcat, divided into three comprehensive steps.

Step 1: Setting Up Essential Tools and Environments

1.1 Process to Create EC2 Instances and Connect via SSH

Log in to AWS : Access the AWS Management Console

Navigate to EC2 Dashboard : In the Services menu, select EC2 under Compute.

Launch Instance

Click on Launch Instance.

Name and Tags: Assign a name to your instance (e.g., "Jenkins-Server").

Application and OS Images (Amazon Machine Image): Choose Amazon Linux 2 AMI (HVM), SSD Volume Type.

Instance Type: Select t2.micro (eligible for free tier).

Key Pair (Login)
- Create a new key pair or select an existing one.
- If creating new, download the .pem file and store it securely; you'll need it to access your instance.
Network Settings: Create a new security group with the following inbound rules:
- SSH (port 22): Allows secure shell access.
- HTTP (port 80): Enables web traffic.
- Custom TCP (port 8080): Required for Jenkins & tomcat access.
Configure Storage: The default storage configuration is typically sufficient.

Launch: Review all settings and click Launch Instance.

Verify Instance Status

Return to the EC2 Dashboard.
Ensure your instance's Instance State is running.

Set Key Permissions

Open your terminal.
Modify the permissions of your key pair file to ensure it's not publicly viewable:

chmod 400 /path/to/DEV_OPS_KEY.pem

Retrieve Public IP

In the EC2 Dashboard, select your instance.
Note the Public IPv4 address listed in the instance details.

Establish SSH Connection

Use the following command, replacing /path/to/your-key-pair.pem with the path to your key pair file and ec2-user@your-public-ip with the appropriate username and IP address:

ssh -i /path/to/your-key-pair.pem ec2-user@your-public-ip

For example:

ssh -i DEV_OPS_KEY.pem ec2-user@65.2.57.151

1.2 Process to Install Java

Jenkins requires Java to run. Follow these steps to install Java 17:

# Switch to the root user  
sudo su -

# Update all packages to the latest version
yum update -y   

# Install Amazon Corretto 17 (Java 17)
yum install java-17-amazon-corretto -y 

# Check the installed Java version
java -version

Expected output:

1.3 Process to Install Jenkins

# Download and add the Jenkins repository configuration  
wget -O /etc/yum.repos.d/jenkins.repo https://pkg.jenkins.io/redhat-stable/jenkins.repo  

# Import the Jenkins repository GPG key for package verification  
rpm --import https://pkg.jenkins.io/redhat-stable/jenkins.io-2023.key  

# Install Jenkins  
yum install jenkins -y  

# Check the current status of the Jenkins service
systemctl status jenkins  

# Enable Jenkins to start on system boot 
systemctl enable jenkins  

# Start the Jenkins service 
systemctl start jenkins

Access Jenkins Web Interface

Open a browser and navigate to:

http://<public-ip-address>:8080

Replace <public-ip-address> with your EC2 instance's public IP.

Retrieve the initial admin password:

cat /var/lib/jenkins/secrets/initialAdminPassword

Copy the password and paste it into the Jenkins setup page.

[!WARNING]

If Can’t access username or password, forgot these credentials after installing Jenkins

Other Necessary Commands

# Stop the Jenkins service if running  
systemctl stop jenkins  

# Check Jenkins service logs to debug any issues 
journalctl -u jenkins -b  

# Display the installed Jenkins version (if Jenkins is in the system PATH)
jenkins --version

Expected output:

1.4 Process to Install Git

# Install Git from the default Amazon Linux repository
yum install git -y    

# Verify the installed Git version
git --version          

# Find the path of the Git binary
which git

Configure Git in Jenkins:

Install the GitHub plugin:

Navigate to Manage Jenkins > Plugins > Available Plugins tab.
Search for GitHub and install it.

Configure Git Path

Navigate to Manage Jenkins > Tools tab.

1.5 Process to Install Maven

Go to the /opt directory, which is typically used for storing applications on Linux.

cd /opt

Visit maven.apache.org to get the latest Maven tar.gz file. Alternatively, use wget to download directly:

wget https://dlcdn.apache.org/maven/maven-3/3.9.9/binaries/apache-maven-3.9.9-bin.tar.gz

Extract the Archive:

tar -xvzf apache-maven-3.9.9-bin.tar.gz
mv apache-maven-3.9.9 maven

Verify Maven Installation

Before proceeding with setting up the environment variables, let's verify that Maven is correctly installed. Navigate to the Maven folder and check the version:

cd maven
./bin/mvn -v

This will show the Maven version, Java version, and operating system details.

Note: Running mvn -v will only work if you're inside the /opt/maven/ folder. For access it globally we have to set environment variable.

Configure Environment Variables

Now that Maven is installed, we need to set up environment variables (JAVA_HOME, M2_HOME, and M2) to make Maven accessible globally on the system.

Find the Java Home Path If you don't already know where Java is installed, you can use the find command to locate it:

find / -name java-17*

Once you find the Java installation path, note it down. For example, the path might be /usr/lib/jvm/java-17-openjdk

Set Environment Variables

Edit the .bash_profile file to add the necessary environment variables.

nano ~/.bash_profile

Add the following lines at the end of the file, replacing the paths with the correct ones for your setup:

# `JAVA_HOME` points to the Java installation directory.
JAVA_HOME=/usr/lib/jvm/java-17-amazon-corretto.x86_64

#`M2_HOME` points to the directory where Maven is installed.
M2_HOME=/opt/maven

#`M2` points to the Maven `bin` directory, which contains the `mvn` command.
M2=/opt/maven/bin

#We update the `PATH` variable to include `Maven` and `Java` locations, ensuring that Maven can be accessed globally.
PATH=$PATH:$HOME/bin:$JAVA_HOME:$M2_HOME:$M2

Save the file and exit the editor.
Reload the profile to apply the changes:

source ~/.bash_profile

Verify that Maven is now globally accessible by running:

mvn -v

This should show Maven's version along with the Java version.

Configure Maven in Jenkins:

Global Path Configure Maven in Jenkins

Navigate to Manage Jenkins > Tools.
Add a new Maven installation by providing the name and path.

Install the Maven Integration plugin

Go to Manage Jenkins > Plugins > Available tab.
Search for Maven Integration and install it.

1.6 Process to Install Tomcat

Download Apache Tomcat

Navigate to the

  /opt

directory:

  cd /opt

Download the Apache Tomcat package:

  wget https://dlcdn.apache.org/tomcat/tomcat-9/v9.0.98/bin/apache-tomcat-9.0.98.tar.gz

Extract the package:

  tar -xvzf apache-tomcat-9.0.98.tar.gz
  mv apache-tomcat-9.0.98 tomcat

Set Permissions

Grant execute permissions to startup.sh and shutdown.sh

  chmod +x /opt/tomcat/bin/startup.sh
  chmod +x /opt/tomcat/bin/shutdown.sh

Create Startup and Shutdown Links

Create symbolic links for easier management:

  ln -s /opt/tomcat/bin/startup.sh /usr/local/bin/tomcatup
  ln -s /opt/tomcat/bin/shutdown.sh /usr/local/bin/tomcatdown

Start Tomcat:

  tomcatup

Access the Tomcat Application

Open a browser and navigate to:

  http://<Public_IP>:8080

Enable Browser Login

Locate the

  context.xml

files:

  find / -name context.xml

Edit both /opt/tomcat/webapps/host-manager/META-INF/context.xml and /opt/tomcat/webapps/manager/META-INF/context.xml files
Comment the <Value ClassName> field in the context.xml files located under the webapps directory.

/opt/tomcat/webapps/host-manager/META-INF/context.xml

/opt/tomcat/webapps/manager/META-INF/context.xml

Restart Tomcat:

  tomcatdown
  tomcatup

now

Configure User Access

Edit the

  tomcat-users.xml

file:

  cd /opt/tomcat/conf

Add the following user configurations:

  <role rolename="manager-gui"/>
  <role rolename="manager-script"/>
  <role rolename="manager-jmx"/>
  <role rolename="manager-status"/>
  <user username="admin" password="admin" roles="manager-gui,manager-script,manager-jmx,manager-status"/>
  <user username="deployer" password="deployer" roles="manager-script"/>
  <user username="tomcat" password="s3cret" roles="manager-gui"/>

Restart Tomcat:

  tomcatdown
  tomcatup

Verify Login

Access the Tomcat application in your browser:

  http://<Public_IP>:8080 OR http://<Public_IP>:8090

Log in using the credentials configured in tomcat-users.xml. You should now have successful access.

Change Default Port ( OPTIONAL )

By default, Tomcat runs on port 8080, which may conflict with other applications like Jenkins. To change the port:

Edit the

  server.xml

file:

  cd /opt/tomcat/conf

Locate and update the

  <Connector port>

field to 8090 in

  server.xml

Restart Tomcat for changes to take effect:

  tomcatdown
  tomcatup

Verify access on the new port:

  http://<Public_IP>:8090

Configure Necessary Settings in Jenkins for Tomcat:

Install the "Deploy to Container" plugin in Jenkins:

Go to Manage Jenkins > Plugins > Available tab.
Search for Deploy to Container and install it.

Add Tomcat Credentials in Jenkins:

Navigate to Manage Jenkins > Credentials > System > Global credentials (unrestricted).
Click Add Credentials.
Choose Username and password as the credential type.
Enter the Tomcat manager username and password (as configured in conf/tomcat-users.xml).

Save the credentials for use in Jenkins jobs.

Step 2: Setting Up the CI/CD Pipeline Environment

1. Create the First EC2 Instance
Follow 1.1 Process to Create EC2 Instances and Connect via SSH to launch an EC2 instance and connect to it securely. Ensure the instance is properly configured with the required security group rules (e.g., open ports for SSH and Jenkins).

2. Install Java
Complete 1.2 Process to Install Java to install the Java Development Kit (JDK) on your EC2 instance. Java is required for running Jenkins and other tools in the pipeline.

3. Install Jenkins
Follow 1.3 Process to Install Jenkins to set up Jenkins on the EC2 instance. Jenkins is the core automation server that will orchestrate the CI/CD pipeline. Ensure Jenkins is up and running, and accessible through its web interface.

4. Install Git
Complete 1.4 Process to Install Git to install Git on the instance. Git will be used to manage source code repositories and enable Jenkins to pull the latest code for building and deployment.

5. Install Maven
Proceed with 1.5 Process to Install Maven to set up Maven on the EC2 instance. Maven will handle the build and dependency management for the applications deployed in the CI/CD pipeline.

Step 3: Setting Up the Deployment Server

1. Create the Second EC2 Instance

Refer to 1.1 Process to Create EC2 Instances and Connect via SSH to launch a new EC2 instance.
Ensure the security group is configured to allow traffic on ports required for Tomcat (e.g., port 8080 for HTTP).

2. Install Java

Complete 1.2 Process to Install Java on the new EC2 instance to set up the Java Development Kit (JDK).
Java is required to run Tomcat, which is a Java-based application server.

3. Install Tomcat

Follow 1.6 Process to Install Tomcat to set up Apache Tomcat on the EC2 instance.
Tomcat will act as the deployment server, hosting the applications built and managed by the CI/CD pipeline.

Step 4: Configuring Jenkins to Build and Deploy the Project

In this step, we will configure Jenkins to automate the build and deployment process for a Java project using Maven and Tomcat. Follow the detailed steps below:

1. Log in to Jenkins

Access the Jenkins web interface using the URL: http://<Jenkins-Server-IP>:8080.
Use the credentials retrieved during Jenkins setup to log in.

2. Create a New Maven Project

Click on New Item in Jenkins.
Enter a name for the project (e.g., Java-App-Build).
Select Maven Project as the project type and click OK.

3. Configure Git Repository

In the Source Code Management section, select Git.
Enter the Git repository URL (e.g., https://github.com/devops-vsc/hello-java).

Add Git credentials if the repository is private.

4. Set Up Build Triggers

Select Poll SCM as the build trigger.
In the schedule field, enter

  * * * * *

to poll the repository every minute for changes.

Why? This ensures Jenkins automatically triggers a build whenever there are changes in the repository.

5. Configure Build Steps

In the Build section, specify the Root POM (e.g., pom.xml).
In Goals and options, enter

  clean install

Why clean install?
- clean: Removes any previous build files to ensure a fresh build.
- install: Compiles the code, runs tests, and packages the project as a .war file for deployment.

6. Add Post-Build Action

Select Deploy war/ear to a container from the Post-Build Actions dropdown.

7. Configure Deployment to Tomcat

In the WAR/EAR files field, enter

  **/*.war

Why? This ensures Jenkins deploys all .war files generated during the build.
- Under Containers, select Tomcat 9.x Remote.
- Provide the following details:
Credentials: Select the credentials added in ( 1.6 Process to Install Tomcat > Add Tomcat Credentials in Jenkins)
Tomcat URL: Enter the URL and port where Tomcat is running (e.g., http://<Tomcat-Server-IP>:8080).

8. Save and Build the Job

Click Apply and then Save.
Trigger a build by clicking Build Now.
Jenkins will:

Pull the latest code from the Git repository.
Build the project using Maven.
Deploy the .war artifact to the Tomcat server.

9. Verify Deployment

Open a browser and navigate to the Tomcat server URL (e.g., http://<Tomcat-Server-IP>:8080/<project-name>).
Confirm that the application is successfully deployed and running.

With this setup, Jenkins automates the entire CI/CD process, ensuring code changes are built, tested, and deployed seamlessly to the Tomcat server.