CI/CD: A Perspective

This is a series of articles planned to explain what is Continuous Integration and Continuous Distribution. It also discusses CI best practices, features and types. Further it tries to find out what is the value-add CI/CD brings to the software development, and what points should be consider while deciding upon a CI/CD engine. So, without further delay let's start with the CI first.

 Disclaimer: The diagrams/images used in my articles are "inherited" from the Internet. I am willing to give proper credit to the author if he/she contacts me. Also, if anybody has any objections about me using these diagrams/images, please let me know and I will remove them from my articles.

Continuous integration




As more and more projects are embracing Agile development methodology, CI is becoming an integral part of development process. CI is always there in one form or other, right from the beginning days of
software development.

In the initial days, having a build script and a build machine is sufficient. As the development activity and bug fix rate increases, it is necessary to have scheduled builds and nightly builds. It also makes sense to preserve build artifacts for later analysis.

With the increased pace of development cycle, it is necessary to start the build as soon as code is committed. It also helps to maintain required dependencies into a repository instead of getting and building them from source every time. There is also a requirement for different types of builds for
different audience, like debug build, QA build etc. Having build configuration files helps to create different types of builds automatically.

As the volume of builds increases, clusters are deployed to make builds faster. With even faster pace of development cycle, the commit volume increases with decrease in individual commit size. This is in line with the "build fast, fail fast" rule, as build failures are more acceptable and expected in agile. To
cope up with this, two-stage or multi-staged commits are introduced. This also increases the total frequency and number of builds. Virtualisation is also used to lessen the burden of having and maintaining hardware required for development cycle.

The Wikipedia defines CI as "Continuous integration is a practice of merging changes from different developer working copies to a shared mainline as early as possible several times a day."

For CI to be effective, it prescribes a few best practices. They are:
  • Maintain a code repository
    This advocates use of a revision control system to maintain commits and roll backs. It also advises to have all the dependencies available in the revision control system, so that once the code is checked out, it should build without any problem.
  • Automate builds
    Builds are automated using scripts which pull-in dependencies and compiles the source code to produce the target.
  • Make builds self-testing
    Automated tests should follows a successful build to ensure that there are no regressions and everything works as expected.
  • Frequent commits on same day by multiple developers
    Frequent commits facilitate smaller changes, which in turn help catch bugs or regressions easily and early. Since the changes are small and change-set is known, time to fix the issue also tends to be relatively smaller.
  • Every commit to baseline is built
    This ensures that there are no issues with integration of changes. Generally, every commit made to the baseline in the revision control system automatically triggers a build.
  • Fast builds (and fast failures)
    This is the essence of continuous integration. The build should get completed quickly so that any failures are identified in a shorter time.
  • Testing in cloned production environment
    Exact clone of production environment is used to test the latest build, than test environment. This way, there are less chances of failure when the software is deployed in the field.
  • Ease of getting latest deliverables
    Build artifacts are made available to stakeholders, such as QA, integration etc. as soon as builds are successful.
  • Easy access to latest build results
    Easy access to build status and logs facilitates early detection and repair of the build failures.
  • Automated delivery/deployment
    The build artifacts are automatically uploaded to repositories from where they can be downloaded by interested parties for further utilization. In case of service modules, they can be deployed to cloud so that consumers can utilize the services with their updated features immediately.
Implementing continuous integration offers many benefits, such as,
  • Reduced development and deployment process overhead
  • Reduced integration time and effort (early detection, easy tracking, small chunks of changes)
  • Quick feedback for every change
  • Reduced debugging time
  • Minimized divergence between developers
  • Avoiding last minute chaos at release date
We will discuss Continuous Distribution in the next part.

Comments

Post a Comment

Popular posts from this blog

Security in Linux Kernel - Part 2

Image
  In the previous part of this series, we saw role of LSMs in kernel security. As mentioned earlier, there are eight LSMs available today in the modern Linux kernel. In this part, let's have a look at different main stream LSMs. SMACK Simplified Mandatory Access Control Kernel ( SMACK ) is designed primarily for embedded linux systems with an intention to make it easier for administrators. This was the second LSM (after SELinux) to be accepted in the linux kernel. It appeared in the 2.6.25 kernel release. This is an attribute (label) based simple LSM, and is the default for Linux implementations tuned for Automotive industry. Yama Yama collects system-wide DAC security restrictions that are not handled by the core kernel itself. It offers control over scope of ptrace() system call to control ptrace attachment by processes. Build time configuration option CONFIG_SECURITY_YAMA and runtime option through sysctls can be used to enable this LSM. The ptrace restrictions can be controll
Read more

Trusted Platform Module

Image
In the previous article, we discussed end device security and various factors affecting it. There was a fleeting mention about TPM - the Trusted Platform Module. In this installment we will discuss mode about the TPM. Generally, for any device, the safest way to ensure it's integrity is to make sure it always boots from a secured, verified and unalterable source. This forms the logical "root" of the entire boot process. Once the device boots from a trusted source, it further uses it to verify each and every software component it utilizes. Every module checks and verifies integrity of the next module to be loaded thus forming a "chain of trust" which if broken can be easily detected. This chain of trust then can be extended to encompass other devices across and upstream the network hierarchy. In this installment, let's have a deeper look at TPM. What is TPM? TPM is (generally) a chip designed to provide security related functions. It provides functiona
Read more

Common Git Tips

Image
Obtain list of commits git reflog --oneline git log --oneline These commands will also print commit ID - a.k.a. commit hash. Obtain list of branches created by a user git for-each-ref --format=' %(authorname) %09 %(refname)' --sort=authorname | grep <username> Obtain branch hash git show-ref branch_name Delete a local/remote branch Assume that a branch "bugfix/114355" is created to fix a bug. To delete the local branch: git branch -d bugfix/114355 git branch -D bugfix/114355 '-D' is same as specifying "--delete --force" . To remove the remote branch as well: git push origin --delete bugfix/114355 Undo last commit git reset --soft HEAD~1 Roll back repo to a commit To roll-back the repository to a particular commit, follow these steps. find the hash corresponding to the particular commit by checking the commit logs: git log --oneline roll back the repository by resetting the HEAD pointer: git reset --hard <commit_hash> Remove the latest co
Read more