Docker Basics understanding



Docker basics


In DevOps era, Every organization wants to streamline their deployment strategy. To streamline deployments, the center of attraction is IT infrastructure to be precise different kind of servers to maintain application and managed by Operation teams.


Every Organization needs their application should be scalable. But  term scalable may be easy to say but believe me, it is very hard to implement.

What do we mean by scalable?
Suppose a project maintain images, on a normal day, it receives 10,000 requests. The project maintains a load balancer under this balancer project maintains two pools.

Pool A and Pool B.  Pools are acted as Active/Passive mode as Project supports Blue/Green deployment. For a given moment one pool is active.
Furthermore, each pool contains 5 web servers.

Now if Each webserver can take maximum 10000 requests.

So Total request handling capacity of this project is
5*10,000=50,000 requests ,at a given moment.

I can say this project can handle 50,000 requests at a time.

But consider a situation, Let say for Olympics there will be more traffic than a normal day . After all this project is made for hosting Images.

Say in  Olympic this projects got 1,000000 (lakhs) requests, what happens then?
Obviously, servers are gone down as it can maximum handles 50000 requests for a moment so every moment there will 50000 extra requests in a pool and another 100000 comes so next moment it has to serve 150000 requests. So requests in queue increase in exponential order so the system will fall down eventually.
 How to manage such situation?

There are two ways to handle such situation
1.      Vertical scaling: By meaning of Vertical scaling, make your server a  super computer so it can serve many more requests. But Cost of Super computer is Very high ,most of the organization can’t effort it.

2.      Horizontal scaling: Another way is to add many more commodity servers under load balancer so load balancer can distribute requests among them.

Second one look promising right…

But in Current Infrastructure the problem is to add a new server to the pool. To add new servers you need to configure those servers, so you need to a. install OS , b. add necessary software’s,c.  add network configuration, d. add a firewall, update load balancer etc.

Which takes much more times even if you have golden image still it takes time.

So the Solution is Docker.

 Docker is nothing but a slick container. It runs just above host Operating systems and access host m/c hardware but it can spawn multiple containers and each container act as separate m/c and each has separate address spaces, each has separate guest OS with minimum requirement to hosting a software, so If to host a software needs one WebLogic server so a container only contains a Guest OS with WebLogic server.

Docker is faster than a Virtual machine (VM) as Virtual machine contains one or more guest OS which runs on top of host machine OS and in between OS there is a Hypervisor layer which manages guest OS calls or it’s requirements and changes that requirement into host machine specific OS call. Mainly hypervisor responsible for  host OS and Guest OS communication.

Unlike VM, Docker just runs on top of your OS it shares your OS network, hardware everything but Docker container maintains a separate OS which has its own address space and only contains require software to work with. So Docker are very light-weight and easy to spawn, later you can see, by a command you can spawn a container so Horizontal Scale up is very easy







Difference between Docker and VM





Docker Architecture:


 Docker maintains client-server architecture where Docker client talks with Docker daemon via a socket or Restful API. Docker daemon and client can be in the same host or Docker daemon can be hosted on a different machine. If so Docker client has to communicate with Remote Docker daemon via socket.

Docker contains Three main parts
1.       Docker Daemon:  Docker Daemon does most of the work upon  Docker client or Restful API commands/instruction. According to command, it builds an image, spawn a new container, can run a container, update an Image, can push an image etc.
2.       Restful API:  Docker publish Restful API so if you want to control Docker daemon through the program you can call this API and controls Docker daemon.
3.       Docker client: Docker client is a CLI (command Line Interface) where you can fire command then Docker client  talks to Docker daemon visual Rest API or Socket and Docker daemon perform the task for you. You can consider it as Linux terminal which talks to kernel.
 




Docker Architecture – Picture Courtesy:  Docker Site






To understand How docker works you need to know some Docker terms

1.       Docker images: Docker images is nothing but a logical template based on this a new Docker container can spawn.  So according to Image hosting Project states above ,it needs an Image which has an OS say Ubuntu 13.04 with a WebLogic server. So an image can be created with this requirement.
The image can be created locally using Docker file or can be pulled from a global repository (If it exists) which is managed by Docker. we call it Docker Hub or registry.


2.       Docker Hub/Registry: Docker hub contains images which can be pulled to satisfy your requirements. In Docker hub, there can be just OS images and can be Hybrid images such as Solaris with Tomcat. One can push its own image to Docker Hub, to do it you need Docker account.

Docker Hub is a central repository, you can have thought it as a maven Repository. And like GIT you can push your image into it.

3.       Docker Containers: Docker containers is an actual runtime environment spawn from an image. It acts as a separate machine.



Some important commands:

To download an Image from Docker Hub:
In Docker client type
Docker pull <image-name> //that is name of the image

To spawn a container and execute a command:
Docker run -i -t <image-name> <command name>
Docker run -I -t ubuntu13.03 ls

To Push an image in Docker Hub:
Docker push <image-name>

I will discuss more command in next Docker section where I will guide you to set up a Docker container.


MicroServices with Devops Example



MicroServices with Devops  Example






In this post I show you How to create Micro service application and Manage it through Dev-ops.

To understand MicroServices please look at my blog post 

To understand Devops please look at my blog post


Tools are Used to implement Microservice
1.       Eclipse as Editor
2.       Java as Language
3.       Using REST Service to communicate between Microservices
4.       Maven as build tool
5.       Tomcat as Web server.


Tool are Used to Manage Microservices using Devops
1.       Jenkins as CI (Continous integration server).
2.       GitHub as SCM(Source control Management).


My intent is creating two Micro services


1.       An Order Service

2.       A Billing service

If client order for a product, Order Service project will communicate with Billing service project by REST API. Billing Service returns Product price if exists in the Product Catalog Otherwise It return a Message Product not found.

Order service accept this response and generate a new response to Client.


To Manage these Microservice, I using Devops

So that Two can be deployed independently, and Two projects are hosted in
GitHub. If someone do a change the code and push it in Github a build has been triggered by Jenkins and Once build is completed, Jenkins install the artifact that is the war file in Local Repo (maven Local REPO) then deploy it to Tomcat Server.

No Manual intervention is needed.


I have made it very simple but in real time there will be much more complexity than two simple services and we need Monitoring tool for health check for different projects which is absent in this example.

I have used Java for Two services but real time one could be in java another in PHP or Node JS any language.


Installation:

Java:  Install java from
Set JAVA_HOME in environment variables to java installation path.


Download Tomcat 6 Zip Version and extract it in your local drive that will your Tomcat installation directory.
Maven:
Extract the zip in your local file Set M2_Home as maven Installation path in your local directory.


Go to Git Command prompt after installation
Change Directory to C: Using Cd ../../
Now apply command (Need Internet connection)

after completing

again hit

Import two project in to Eclipse by import from local option.

If you want to create your new repositories, please go to Jenkins setup section and follow the step.

I prefer you create your new repositories to work independently. Unless if you clone it you will be dependent on me. If I gave you permission, then only you can push change on my project.







Jenkins :
 http://mirrors.jenkins-ci.org/war-stable/  (Take stable release. I use 1.65). Download war and deploy it to <Tomcat_installation_path>/webapps. In tomcat console when a message shows
“Jenkins full up and running” go to browser hit


First thing you need to require add plugins in to your local Jenkins Server

So Follow the steps
Go to Manage Jenkins from left panel
Go to manage Plugin
In Available Tab Search  with “git” and check gitplugin and github plugin
Hit install without restart button
Wait until installation finished.


Go to configure system from manage plugin
 






Create an account in GitHub and then add your username and your email Id.


Then Go to new Item create a Maven project.

Create a Job Name as “BuildandDeploy MicroOrderService”

Please see the following Screen shot




 
In the Source code management section set your Git repository URL
If you download source code from Git by download Zip and create your new repository and push the code

Steps are
After Download project from

and
using “ CloneorDownload”  button on GitHub
Extract Zip files

Create a project on Github using “+” button (Create new)
You will get a url like https://github.com/shami83/<projectname>.git


Execute following commands for two projects in your local computer

Go to  Git cmd
Go to the homefolder of the Extract project
a.       git init
b.      git add .
c.       git commit -m “demo”
d.      git push origin master

Please do the steps for two projects that will create two repositories in Git (require internet connection).

In jenkins on repository url put the url


If you clone it then you need my permission to push code to my stream, please send me a mail the at
mitrashamik@gmail.com or add a comment in blog.


Then put your credentials in credential section your username and password of Git repository.


Follow next step



In Build triggers section Check ” BuildWhen a change pushed to Git hub”

And poll SCM

Put value as * * * * * in schedule text-area

This will detect if any changes pushed on GitHub repository. If so this Jenkins target will run

Now Go to Build section
Set Root POM as pom.xml as you can see a workspace folder create under your Jenkins target where Jenkins pull the codebase from Git server and build it locally.

Set Goals and Option as clean install.
Install step is requires as it maintains your artifacts in your local maven repository.


Last I want to deploy it on Tomcat server

To do that follow the following steps

First go to Tomcat installation directory under conf folder
Open tomcat-users.xml in edit mode
Add following lines under <tomcat-users> tag.

<role rolename="tomcat"/>
  <role rolename="role1"/>
  <user username="tomcat" password="tomcat" roles="admin,manager-script,manager-gui"/>
  <user username="both" password="tomcat" roles="tomcat,role1"/>
  <user username="role1" password="tomcat" roles="role1"/>


Then go to Jenkins job configure section

 





In War/EAR files put target\MicroOrderService.war as maven generates artifacts in target directory.


After that Set user name as tomcat in Tomcat6.x section password is tomcat
Tomcat Url is http://localhost:8080

Click save button that will create MicroOrderService job.


To check hit BuildNow button and go to console output, you will see
Jenkins pull code from GitHub then build it using maven install the artifacts then deploy it in your Tomcat Server



Perform Same thing for Billingservice, create a new Job and configure that.


Now to Check just make a change in Source code
Push it to Git using Git Cmd
Steps
git add .
git commit -m ”change”
git push origin master

go to Jenkins dashboard you will see a build is initiated and deployed.






Main Classes in Micro Order Source Code


OrderService.java


package com.microservice.order;
 import java.io.IOException;
import java.io.StringReader;

import javax.ws.rs.GET;
import javax.ws.rs.Path;
import javax.ws.rs.PathParam;
import javax.ws.rs.Produces;
import javax.ws.rs.core.MediaType;
import javax.xml.bind.JAXBContext;
import javax.xml.bind.JAXBException;
import javax.xml.bind.Unmarshaller;

import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.util.EntityUtils;

import com.microservice.model.BillingStatus;
import com.microservice.model.OrderStatus;

@Path("/order")
public class OrderService {

    @GET
    @Path("{name}")
    @Produces(MediaType.TEXT_XML)
    public OrderStatus getOrderStatus(@PathParam("name") String name)
            throws ClientProtocolException, IOException {
        System.out.println("Orerd Item name is ==>" + name);
        DefaultHttpClient httpClient = new DefaultHttpClient();
        HttpGet getRequest = new HttpGet(
                "http://localhost:8080/billingservice/billingservice/billing/"
                        + name);

        String response = getResponse(httpClient, getRequest);
        BillingStatus stat = new BillingStatus();
        stat = this.converStringToObject(stat, response);
        System.out.println("Return Object from JAXB "  + stat);

        OrderStatus status = new OrderStatus();
        status.setOrderName(stat.getItem());
        status.setOrderCost(stat.getPrice());
        return status;
    }

    public String getResponse(DefaultHttpClient client, HttpGet request) {

        HttpResponse response = null;
        String apiOutput = null;

        try {

            response = client.execute(request);

            // verify the valid error code first
            int statusCode = response.getStatusLine().getStatusCode();
            if (statusCode != 200) {
                throw new RuntimeException("Failed with HTTP error code : "
                        + statusCode);
            }

            HttpEntity httpEntity = response.getEntity();
            apiOutput = EntityUtils.toString(httpEntity);
           
           

            System.out.println(apiOutput);

        } catch (Exception ex) {
            ex.printStackTrace();
        }

        return apiOutput;

    }

    public <T> T converStringToObject(T t, String res) {

        try {
            JAXBContext jaxbContext = JAXBContext.newInstance(t.getClass());
            Unmarshaller jaxbUnmarshaller = jaxbContext.createUnmarshaller();

            t = (T) jaxbUnmarshaller.unmarshal(new StringReader(res));
        } catch (JAXBException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }

        return t;

    }

}




Main Class in Billing service Module


package com.microservice.billing.service;

import javax.ws.rs.GET;
import javax.ws.rs.Path;
import javax.ws.rs.PathParam;
import javax.ws.rs.Produces;
import javax.ws.rs.core.MediaType;

import com.microservice.billing.model.BillStatus;
import com.microservice.billing.utility.PriceCatalog;



@Path("/billing")
public class BillingService {
   
   
    @GET
    @Path("{name}")
    @Produces(MediaType.TEXT_XML)
    public BillStatus getOrderStatus(@PathParam("name")String name)
    {
       
        System.out.println("Check Item in Inventory ==>" + name);
        return PriceCatalog.getCatalog().getStatus(name.toLowerCase());
    }

}


PriceCatalog.java

package com.microservice.billing.utility;

import java.util.HashMap;   
import java.util.Map;

import com.microservice.billing.model.BillStatus;

public class PriceCatalog {
   
    private static PriceCatalog calalog=new PriceCatalog();
    private Map<String,String> priceMap=new HashMap<String,String>();
    private PriceCatalog()
    {
        init();
    }
   
    private void init()
    {
        priceMap.put("pen", "10");
        priceMap.put("shirt", "1000");
        priceMap.put("televison", "20000");
        priceMap.put("ac", "30000");
        priceMap.put("smartphone", "14000");
        priceMap.put("radio", "2000");
        priceMap.put("musicbox", "20000");
        priceMap.put("bike", "66000");
        priceMap.put("scooter", "26000");
        priceMap.put("pant", "100");
        priceMap.put("harddisc", "3000");
        // end of map
       
       
    }
   
    public static PriceCatalog getCatalog()
    {
        return calalog;
    }
   
    public BillStatus getStatus(String key)
    {
        BillStatus status= new BillStatus();
        if(key == null || priceMap.get(key)==null)
        {
            status.setItem("Sorry" + key + " Not found in Inventory!!");
            status.setPrice("Not Applicable");
            return status;
        }
        status.setItem(key.toUpperCase());
        String price =  priceMap.get(key) + " Rupees only.";
        status.setPrice(price);
        return status;
    }
   
   
   

}
Please Dowload Code base from


https://github.com/shami83/MicroOrderService
and 
https://github.com/shami83/BillingService