About Jenkins

Jenkins

Jenkins - Features

• Installation is more flexible than Go (Go is tied to Jetty and comes with it, but can be fronted with some other web server. Jenkins is not tied to any web server).
• Extensibility. Plugins - About 600 of them.
• Slightly easier to get started since it can work in “master-only mode”, where there is no slave node (“Agent” installation is a must in Go). 
• Integration with VCS: Almost everything you’ve heard of and some you haven’t, because of plugins.
• First class and often deep support for many tools: Maven (reads POM to figure out dependencies), Rake, Vagrant, Debian package building
• Well-known. Community is pretty good, especially plugin writers.

Jenkins - Missing features (compared to Go)

• Modelling capability is limited. While there are options to model in parallel-serial-parallel-serial approach using plugins, it is not very convincing. Support for multiple VCS for a job is supported through plugins which makes implementing a feature like fan-in difficult (plugin for fan-in will need to talk to VCS plugins)
• Visualization and VSM - End-to-end visualization and auditability not at the level of Go. [how-to-use-jenkins-for-job-chaining-and-visualizations]
• Fan-in: Jenkins does build incompatible builds resulting in lots of wasted build time. All the literature around the build flow plugin is still not convincing enough to say that it supports fan-in.
• Go security seems more fine-grained than Jenkins’. Go allows per-user/group and per-pipeline authorization. Jenkins seems to allow this only at a global, per-user/group level.

Go CD VS Jenkins

1. Go CD is a best-of-breed tool for Continuous Delivery (CD).

    Jenkins is a general purpose automation tool and is built for Continuous Integration(CI).

2. Go CD aims to support the most common CD scenarios out of the box without any plugin installation where as Extensibility is core to Jenkins. Its flexibility attributes itself to plugins being critical to Jenkins’ functionality. So it depends heavily on plugins for pretty much any use case.

3. Go CD was built on the principles of [Continuous Delivery](https://continuousdelivery.com/). This is visible in its abstractions as pipeline is a first class concept. Go CD also encourages that there be only one way to implement the fundamental CD patterns. When you search for help on how to implement the various deployment pipeline patterns, you will generally find a single, well-known, well-tested answer.

With Jenkins 2.0, CD is implemented by the installation of a variety of plugins. Many common CD patterns (build an artifact only once, full traceability up and down stream, and more) are either impossible to implement or can only be cobbled together with fragile   combinations of plugins.

4. Although Go CD is built specifically with CD in mind, it has sophisticated features for continuous integration.

Jenkins is built for CI. Anything beyond that, requires plugins.

About Go Continuous Delivery

About GoCD

GoCD is an open source tool which is used in software development to achieve continuous delivery of software. It supports automating the entire build-test-release process from code check-in to deployment. It helps to keep producing valuable software in short cycles and ensure that the software can be reliably released at any time.

·        A Pipeline consists of multiple stages, each of which will be run in order.

·        A Stage consists of multiple jobs, each of which can run independently of the others.

·        A Job consists of multiple tasks, each of which will be run in order.

·        A Task is an action that needs to be performed. Usually, it is a single command.

·        A Material is a cause for a pipeline to run. This could be a commit made to a source code repository or a timer trigger.

·        Agents run the jobs assigned to them.

Go CD benefits

• Visualization with features like Value Stream Map 
• Fan-in support - Go does not build incompatible revisions reducing lot of wasted build time
• Good modelling ability (pipelines, stages, jobs, tasks).
• Easy parallelizability (jobs across agents, can use resources to tie jobs to a set of agents)
• Traceability - from deployed build all the way back to source code commits
• Security - Secure communication between agents and server, possible to configure Go to be https-only.
• Fine-grained control of permissions
• Collaboration - Go can be used across the organisation. Here teams can use it to do automated deployments. Dev teams can see where there commit has reached (unit tests, acceptance tests, QA env etc.), QA teams can see what build is deployed on what env. and other stakeholders can compare revisions to see what features / bugs are going into a release. 
• Ability to segregate pipelines using groups, agents using resources and trying both together using environments. Using the “Environments” feature of Go to separate build pipelines from deployment pipelines.
• One-click backup

Go CD - Missing features

• Only a couple of extension points (plugins) - Go team is working to make more extension points available. Currently external Package Repository as material (for yum, rpm, etc. post Go 13.3) and Task Plugin (plugin support coming from Go 14.1, to be released soon) are available.
• Custom email notifications not configurable enough.
• Multiple server support

Facts about CI & CD

CI & CD

Continuous Integration (CI)  VS Continuous delivery (CD)  VS Continuous Deployment

Continuous Integration (CI) is a software engineering practice in which developers integrate code into a shared repository several times a day in order to obtain rapid feedback of the feasibility of that code. CI enables automated build and testing so that teams can rapidly work on a single project together.

Continuous integration is the process by which a developer’s code is integrated with the main code branch as quickly as possible. Continuous integration puts a great emphasis on testing automation to check that the application is not broken whenever new commits are integrated into the main branch.

Continuous delivery (CD) is a software engineering practice in which teams develop, build, test, and release software in short cycles. It depends on automation at every stage so that cycles can be both quick and reliable.

Continuous delivery takes CI one step further, with an emphasis not just on integrating code, but also making the code shippable at any time. Continuous delivery usually requires automated testing, so that the developers are confident that the code can be shipped Or we can say that Continuous delivery is an extension of continuous integration to make sure that you can release new changes to your customers quickly in a sustainable way. This means that on top of having automated your testing, you also have automated your release process and you can deploy your application at any point of time by clicking on a button.

Continuous Deployment is the process by which qualified changes in software code or architecture are deployed to production as soon as they are ready and without human intervention.

Continuous deployment is a term that is usually applied to cloud software systems, where software code changes are automatically deployed to production, in a safe way. Continuous deployment strategies usually involve incremental deployments that allow changes to be tested incrementally. Continuous deployment goes one step further than continuous delivery. With this practice, every change that passes all stages of your production pipeline is released to your customers. There’s no human intervention

Continuous Deployment is the act of deploying every addition to the mainline branch to the consumer. Continuous Deployment would happen after CI passes and the feature/bug fixes made were merged into the mainline/trunk branch. All the changes would be sent to the distribution/production environment.

Finally, we can conclude that

Continuous integration (CI) can mean one of two things:

1.     The work done by a continuous integration server, such as Jenkins or Bamboo

2.     The process of integrating changes into software at all stages of the delivery pipeline

Continuous Delivery is the frequent shipping of code to a given environment (such as test or production) via manual release.

Continuous Deployment is the automated release of code to a production environment.

Composition v/s Inheritance : Which should be preferable?

Composition V/S Inheritance

Inheritance is an "is-a" relationship where as Composition is a "has-a".

Composition allows reuse of code without extending it but in case of Inheritance we must extend the class for any reuse of code or functionality.

• With Inheritance we are defining which class we are going to extend means static binding and this can’t be changed at runtime but with Composition we just define an object type which we want to use, which can hold its different implementation during execution means dynamic binding. So we can say that using Composition is more flexible than Inheritance.

• In case of Inheritance we can extend only one class but if we want to take advantage of multiple class functionality then we can use Composition

• At some point of time, Inheritance breaks encapsulation because in case of Inheritance, sub class is dependent upon super class behavior and if parent classes are changed than child class will also get affected and as a result it breaks sub class functionality.

• Composition also allows code reuse for final class and this is not possible in Inheritance because we can’t extend final class.

•  When we are using Composition in our application then during unit testing it is easy to test because we are able to provide mock implementation of required classes but when we are using Inheritance then we required parent classes in order to test child classes and we can’t mock parent classes. 

Example of Composition and Inheritance-

Car.java

package com.inheritancecomposition.relationship;

public class Car {
    
/** Class/Instance members and Methods implementation */
    private int maxSpeed;
    private String color;

    public void setColor(String color) {
        this.color = color;
    }
    public void setMaxSpeed(int maxSpeed) {
        this.maxSpeed = maxSpeed;
    }

    public void carDescription(){
        System.out.println("The "+color + " color car is running at max Speed of " + maxSpeed);
    }
}

Engine.java

package com.inheritancecomposition.relationship;

public class Engine {
    public void start(){
        System.out.println("** Engine Started **");
    }
    public void stop(){
        System.out.println("** Engine Stopped **");
    }
}

Hyundai.java

package com.inheritancecomposition.relationship;

public class Hyundai extends Car{  /** Inheritance is used here */
    /**
     * Hyundai extends Car and thus inherits all methods from Car (except final
     * and static) Hyundai can also define all its specific functionality
     */
    public void hyundaiStartDemo(){
        Engine hyundaiEngine = new Engine(); /** Composition is used here */
        hyundaiEngine.start();
        }
}


RelationsDemo.java

package com.inheritancecomposition.relationship;

public class RelationsDemo {
    public static void main(String[] args) {
        Hyundai myHyundai = new Hyundai();
        myHyundai.hyundaiStartDemo();
        myHyundai.setColor("Carbon Grey");
        myHyundai.setMaxSpeed(200);
        myHyundai.carDescription();
    }
}

Result:-

** Engine Started **
The Carbon Grey color car is running at max Speed of 200

Java Memory Management

Memory Management Part-II

Question : - What a Garbage Collector will do?

Answer:- Let's talk about Garbage Collector jobs

                Things to Consider

• Stop the application events :- Garbage Collector pauses the entire application and at that            point it collects garbage.

• Memory Fragmentation :- Memory fragmentation is when most of your memory is allocated in a large number of non-contiguous blocks and leaving a good percentage of our total memory unallocated, but that is unusable for most typical scenarios. This results in out of memory exceptions,When garbage collector runs does it defrag the memory fragment            to once or leave it to the latest state. 
• ThroughPut:– how quickly can it run and how quickly can it collect?
• Multi Core:- Now we have multiple processors or multiple threads Pauses are the times when an application appears unresponsive because garbage collection is occurring.
• Promptness is the time between when an object becomes dead and memory becomes available, which is very important for distributed systems, including RMI.

 Question:- What is the way of Direct memory access in java?

Answer:- Java HotSpot VM contains a “backdoor” that provides a number of low-level operations to control threads and memory directly. This backdoor class sun.misc.Unsafe which is widely used by JDK itself in packages like java.nio or java.util.concurrent. This class provides an easy way to look into HotSpot JVM internals and this class can also be used for profiling and development tools.

Division of  java memory pool

The heap memory is the runtime data area from which the Java VM allocates memory for all class instances and arrays. The heap may be of a fixed or variable size. JVM Heap memory is physically divided into two parts –Young Generation and Old Generation.

• Eden Space: The pool from which memory is initially allocated for most objects. Most initial objects allocated in Eden space.
•  Survivor Space: The pool containing objects that have survived the garbage collection of the Eden space.
•  Tenured Generation: The pool containing objects that have existed for some time in the survivor space.
•  Non-heap memory: Non-heap memory includes a method area shared among all threads and memory required for the internal processing or optimization for the Java VM. It stores per-class structures such as a runtime constant pool, field and method data, and the code for methods and constructors. The method area is logically part of the heap but, depending on the implementation, a Java VM may not garbage collect or compact it. Like the heap memory, the method area may be of a fixed or variable size.
•  Permanent Generation: The pool containing all the reflective data of the virtual machine itself, such as class and method objects. With Java VMs that use class data sharing, this generation is divided into read-only and read-write areas.
• Code Cache: The HotSpot Java VM also includes a code cache, containing memory that is used for compilation and storage of native code.

For demonstartion how Garbage Collection works in Java

GarbageCollectionExample.java

package com.gaurav.memorymanagement;

import java.lang.reflect.Field;

import sun.misc.Unsafe;

public class GarbageCollectionExample {
private static Unsafe unsafe;
static {
try {
Field field = Unsafe.class.getDeclaredField("theUnsafe");
field.setAccessible(true);
unsafe = (Unsafe) field.get(null);
} catch (Exception e) {
e.printStackTrace();
}
}

public static long addressOf(Object o) throws Exception {
Object array[] = new Object[] { o };
long baseOffset = unsafe.arrayBaseOffset(Object[].class);
int addressSize = unsafe.addressSize();
long objectAddress;
switch (addressSize) {
case 4:
objectAddress = unsafe.getInt(array, baseOffset);
break;
case 8:
objectAddress = unsafe.getLong(array, baseOffset);
break;
default:
throw new Error("unsupported address size: " + addressSize);
}
return (objectAddress);
}

public static void main(String args[]){
try{
for(int i=0; i< 40000; i++){
Object mine = new GCObjects();
long address = addressOf(mine);
System.out.println(address);
}
}catch(Exception e){
e.printStackTrace();
}
}
}

class GCObjects {
long data;
long a;
long aa;
long aaa;
long aaaa;
long aaaaa;
long aaaaaa;
long aaaaaaa;
long aaaaaaaa;
long aaaaaaaaa;
long aaaaaaaaaa;
long aaaaaaaaaaa;
long aaaaaaaaaaaa;
long aaaaaaaaaaaaa;
long aaaaaaaaaaaaaa;
long aaaaaaaaaaaaaaa;
long aaaaaaaaaaaaaaaa;
long aaaaaaaaaaaaaaaaa;

}

Execution process - For demonstartion how Garbage Collection works in Java

            Java -cp . com.gaurav.memorymanagement.GarbageCollectionExample > GarbageCollectionExampleOutput.csv

This command will create a CSV file, which we can open with Microsoft Excel and represent those data using line chart. The representation will look like follows :

Java Memory Management

Memory Management - PART - 1

Memory Management in java is responsibility of garbage collector.Garbage Collection is not the only form of Memory Management in Java.Real-time Specification for Java (RTSJ)is also being used for Memory Management. These efforts were mainly dedicated to real-time and embedded programming in Java for which GC was not suitable - due to performance overhead.

Understanding JVM Memory Model is very important if we want to know the working process of Java Garbage Collection.

Types of Garbage Collector : -

• Do nothing : - It might just decide never to run and never to do anything, no memory gets free but it do stills gurantee to not collecting live objects.

• Reference Counting garbage collector : - COM programming environment is the best example of Reference Counting Garbage collector. COM application may call 2 functions. First is Add Ref and second is Release. Add Ref increments the count of the object and Release decrease the count. When count goes to zero then ref is no longer been used or we can say that Areference count is maintained for each object on the heap.When an object is first created and a reference to it is assigned to a variable, the object's reference count is set to one. When any other variable is assigned a reference to that object, the object's count is incremented. When a reference to an object goes out of scope or is assigned a new value, the object's count is decremented. Any object with a reference count of zero can be garbage collected. When an object is garbage collected, any objects that it refers to have their reference counts decremented. In this way the garbage collection of one object may lead to the subsequent garbage collection of other objects.

• Mark and Sweep : - To determine which objects are no longer in use, the JVM intermittently runs mark-and-sweep algorithm. Garbage collector runs in 2 phases, In mark phase it is marking that memory is still alive or thisalgorithm traverses all object references, starting with the GC roots, and marks every object found as alive and in sweep phase all of the heap memory that is not occupied by marked objects is reclaimed. It is simply marked as free, essentially swept free of unusedobjects.

• Copying - Copying garbage collectors move all live objects to a new area and the old area is known to be all free space. This is not following any separate Mark and Sweep phases Objects are copied(these objects are discovered by the traversal from the root nodes) to the new area on the fly and forwarding ponters are left in their old locations and these pointers allows the garbage collector to detect references to objects that have already been moved. The garbage collector can then assign the value of the forwarding pointer to the references so they point to the object's new location.

• Generational - Copying collectors spend much of their time for copying the same long-lived objects again and again. In order to address this inefficiency Generational collectors work with grouping objects by age and garbage collecting younger objects more often than older objects. This approach works by dividing the heap into two or more sub-heaps, each of which serves one "generation" of objects. The youngest generation is garbage collected most often. As most objects are short-lived, only a small percentage of young objects are likely to survive their first collection. Once an object has survived a few garbage collections as a member of the youngest generation, the object is promoted to the next generation: it is moved to another sub-heap.

• Incremental - Rather than attempting to find and discard all unreachable objects at each invocation an incremental garbage collector just attempts to find and discard a portion of the unreachable objects. Because only a portion of the heap is garbage collected at each invocation, each invocation should in theory run in less time. A garbage collector that can perform incremental collections, each of which is guaranteed to require less than a certain maximum amount of time, can help make a Java virtual machine suitable for real-time environments.  

Reference taken from other sources

How to Compare two Arraylists which contain objects of the same class?

Compare two Arraylists which contain objects of the same class

I have two lists of different sizes which belong to the same class Student which is having attributes like:

• Integer id;
• String name;
• Integer age;

We will compare these two lists and remove the elements which are common
in both of them. Below is the example available to compare two lists:-

Student.java

public class Student {

/**

  * @author Gaurav

  *

  */

private Integer id;

private String name;

private Integer age;

public Student(Integer id, String name, Integer age){

    this.id = id;

    this.name = name;

    this.age = age;

}

public Integer getId() {

return id;

}

public void setId(Integer id) {

this.id = id;

}

public Integer getAge() {

return age;

}

public void setAge(Integer age) {

this.age = age;

}

@Override

public String toString() {

    return "Id : " + this.id +" Name : "+this.name+ " age : "+this.age;

}

public String getName() {

return name;

}

public void setName(String name) {

this.name = name;

}

@Override

public boolean equals(Object obj) {

if (obj == this) {

return true;

}

if (obj == null || obj.getClass() != this.getClass()) {

return false;

}

Student student = (Student) obj;

return id == student.id

&& (name == student.name || (name != null && name

.equals(student.getName())))

&& (age == student.age || (age != null && age

.equals(student.getAge())));

}

}

CompareLists.java

import java.util.ArrayList;

import java.util.Collection;

import java.util.List;

import org.apache.commons.collections.CollectionUtils;

/**

 * @author Gaurav

 *

 */

public class CompareLists {

public static void main( String[] args )

{

  List<Student> l1 = new ArrayList<Student>();

  l1.add(new Student(1,"Kumar",33));

  l1.add(new Student(2,"Hiten",35));

  l1.add(new Student(3,"Varun",46));

  l1.add(new Student(4,"Gaurav",35));

  l1.add(new Student(5,"Vishal",40));

  List<Student> l2 = new ArrayList<Student>();

  l2.add(new Student(4,"Gaurav",35));

  l2.add(new Student(5,"Vishal",40));

  

 

  //This is the first way to compare two list containing objects of same classes and removing the common item.

  

  List<Student> firstList = new ArrayList<Student>(l1);

  System.out.println("First list before removing common elements");

  System.out.println(firstList);

  firstList.removeAll(l2);

  System.out.println("\nFirst list after removing common elements");

  System.out.println(firstList);

  

 //This is the Second way to compare two list containing objects of same classes  and removing the common item.

  List<Student> secondList = new ArrayList<Student>();

  System.out.println("\nSecond list before removing common elements");

  System.out.println(l1);

  for (int i = 0; i < l1.size(); i++){

       if (!l2.contains(l1.get(i))){

           secondList.add(l1.get(i));

       }

   }

  System.out.println("\nSecond list after removing common elements");

  System.out.println(secondList);

  

  //This is the Third way to compare two list containing objects of same classes  and removing the common item.

  List<Student> thirdList = new ArrayList<Student>(l1);

  System.out.println("\nThird list before removing common elements");

  System.out.println(thirdList);

  for(Student apv : l1){

for(Student pv : l2){

if(apv.getId() == pv.getId() && apv.getName().equalsIgnoreCase(pv.getName())){

thirdList.remove(pv);

}

}

}

  System.out.println("\nThird list after removing common elements");

  System.out.println(thirdList);

   

  

  //This is the Fourth way to compare two list containing objects of same classes  and removing the common item.

 

 /**org.apache.commons.collections.CollectionUtils - with the help of this class below call is possible and for this explicitly, we need to add the jar named commons-collections-3.2.1.jar*/

  

  System.out.println("\nFourth list after removing common elements");

  @SuppressWarnings("unchecked")

  Collection<Student> fourthList= CollectionUtils.subtract(l1, l2);

  System.out.println(fourthList);

 

}

}

Result:-

First list before removing common elements

[Id : 1 Name : Kumar age : 33, Id : 2 Name : Hiten age : 35, Id : 3 Name : Varun age : 46, Id : 4 Name : Gaurav age : 35, Id : 5 Name : Vishal age : 40]

First list after removing common elements

[Id : 1 Name : Kumar age : 33, Id : 2 Name : Hiten age : 35, Id : 3 Name : Varun age : 46]

Second list before removing common elements

[Id : 1 Name : Kumar age : 33, Id : 2 Name : Hiten age : 35, Id : 3 Name : Varun age : 46, Id : 4 Name : Gaurav age : 35, Id : 5 Name : Vishal age : 40]

Second list after removing common elements

[Id : 1 Name : Kumar age : 33, Id : 2 Name : Hiten age : 35, Id : 3 Name : Varun age : 46]

Third list before removing common elements

[Id : 1 Name : Kumar age : 33, Id : 2 Name : Hiten age : 35, Id : 3 Name : Varun age : 46, Id : 4 Name : Gaurav age : 35, Id : 5 Name : Vishal age : 40]

Third list after removing common elements

[Id : 1 Name : Kumar age : 33, Id : 2 Name : Hiten age : 35, Id : 3 Name : Varun age : 46]

Fourth list after removing common elements

[Id : 1 Name : Kumar age : 33, Id : 2 Name : Hiten age : 35, Id : 3 Name : Varun age : 46]