Gold Site E-Commerce - Complete Project Overview | Sagar Kakkala's World





Requirements Before We Begin Gold Site E-Commerce Project

It is recommended to have New Email Account and create other accounts like GitHub, Docker hub, GoDaddy..... using this mail account




GoDaddy Domain or any Domain 


SSL Certificate  - Recommended but not mandatory

Slack Account - Recommended to create only when you are doing project with slack

Tools to download





Tech Stack Involved in the Project:

  • Version Control: Git, GitHub
  • Cloud Platform: AWS
  • Scripting Tool: Bash
  • Containerization: Docker
  • Orchestration: Kubernetes
  • Repository Management: Helm Charts
  • Code Quality: SonarQube
  • Image Vulnerability Scanning: Trivy
  • Configuration Management: Ansible
  • Continuous Integration/Continuous Deployment (CI/CD): Jenkins, ArgoCD
  • Infrastructure as Code (IaC): Terraform
  • Monitoring and Alerting: Prometheus, Grafana, AlertManager

Evaluating Gold Site E-Commerce Project: Pros and Cons

Pro's of Project:

  • Comprehensive Coverage: Covers the basics of all tools involved, ensuring a solid foundation.
  • Hands-On Learning: Includes installation, configuration, and practical usage of tools in the DevOps process
  • Enhanced Adaptability: Empowers you to easily understand any related projects after completion.
  • Holistic Approach: Explains the "Why," "What," and "How" of each tool for deeper understanding.
  • Interconnected Tools: Clearly demonstrates how various tools are integrated and work together in a seamless DevOps workflow

What´s missing in Project:

  • Not in Advanced Topics: The project does not delve into advanced aspects of each tool, focusing primarily on the basics
  • Not Focused on Networking Concepts: Networking aspects are not deeply covered, which might limit understanding in that area

Where will you find the Project:

If you know Telugu, You can check Youtube Playlist of Sagar Kakkala's World - DevOps with Sagar Kakkala , watch till Day-20 , as that is when our project gets concluded

If you are Non-Telugu, start with Blog and Practise with code snippets present - DevOps with Sagar Kakkala Blog

If you want to develop project in one-go, instead of developing bit by bit as designed in project, you can build in one go till sonarqube and Trivy using unlisted links and later you need to follow from Day-15

20 Days Plan for Project 



Gold Site E-Commerce: Comprehensive Infrastructure Overview


Final Infrastructure of Project


Let us understand how the flow works,

Before that understand SSH connections and tools that are installed in each server

We have three servers here namely Jenkins Server, Build Server , Deploy Server

SSH Connetion

Jenkins Server has SSH connection only to Build Server, Build Server has SSH connection to Deploy Server

Jenkins Server ---ssh---Build Server---ssh---Deploy Server

Tools installed in each server

Jenkins Server - Jenkins, SonarQube

Build Server - Ansible, Docker, Trivy

Deploy Server - Kubernetes(microk8s), Helm Charts, Prometheus, Grafana, ArgoCD

Important Things to Know

GitHub Repository is setup with Jenkins Webhook, that means whenever there is a change in code, Jenkins Pipeline gets triggered

Ansible from Build Server is connected to Deploy Server via SSH ,that means you can stay in Build Server but still do the tasks in Deploy server using Ansible

Both the Build Server and Deploy Server has AWS CLI installed

Build Server and Deploy Server also has IAM roles of s3fullbucketaccess which means they can modify, read, write or delete data

Docker Tags are saved with GitHub Commit ID's for the particular code change

Jenkins Pipeline Workflow

Let us know understand, how the flow has been setup in this Project

When Developer commits the code, Jenkins Pipeline gets triggered and 

Stage-1 

It uses SonarQube to check the code quality and proceeds to stage-2

Stage-2

Here Jenkins uses Ansible in  Build Server to run build.sh script in within same Build Server

Stage-3

Here Jenkins uses Ansible in Build Server to trigger deploy.sh script in Deploy Server

About Build.sh and Deploy.sh

Here both the build.sh and deploy.sh are main scripts which runs our complete automation, let us understand what happens in each script - You can check scripts here - Build.sh and Deploy.sh

build.sh - This Script has soon as we get code quality, 

Steps involved in build.sh

step-1: It runs npm package for our code to get into Build file

step2: These build files are then stored into Build Files Bucket in DDMMYYYY format

step3: And next step it uses same build files, and containerizes it using docker

step4: And new image generated would be given tagged with git commit id that generated with commit due to code change

step5: New docker image with tag is scanned by Trivy for image vulnerability

step6: Once it scans and sends image vulnerability, it pushes our image to be updated in our docker hub account

step7: It also gets values of new_value.txt and old_value.txt from Helm_values bucket, deletes new_value.txt, creates a new_value.txt file and updates the content of it with new github commit id, and pushes it back to s3 bucket

Steps involved in deploy.sh

step1: it copies values of Helm values s3 bucket, it reads values of old_value.txt and new_value.txt, here old_value.txt is previous git commit id, new_value.txt is the value that build server updated with new commit id

Here, we use helm and helm by default used docker tags to update latest image, We need to note that we are using GitHub commit ID's as our Tags here

step2: script used sed command to replace old Git commit ID with New commit ID in values.yaml file of helm

step3: once the values are changes, it commits code and updates in Helm Chart Repo

ArgoCD Workflow

Since we are using ArgoCD as CD tool here and Our Helm chart repo is connected to ArgoCD, as soon as the Repo gets updated by deploy.sh, ArgoCD detects the changes and notifies us that the application is in OutofSync state

since we have choosen, sync options as manual, we will be clicking sync Manually, - Manual Sync is recommended for Production Environments

Prometheus and Grafana

We do have Prometheus and Grafana integrated in our project, while both the tools do not come under our automation workflow, Prometheus helps us in monitoring our pods health and alerts us using Prometheus Rules, And Alert Manager helps us to get alerts into our slack channel

Grafana helps in visualizing metrics of our application through user friendly dashboards
This Concludes Our Blog here


🔹 Important NoteAlso, before proceeding to the next session, please do the homework to understand the session better - DevOps Homework



I Post most of my content in Telugu related to contrafactums(changing lyrics to original songs),fun vlogs, Travel stories and much more to explore, You can use this link as single point of link to access - Sagar Kakkala One Stop

🖊feedback,queries and suggestions about blog are welcome in the comments.






Comments

Post a Comment