The joy of pushing your changes to git and few minutes later seeing the changes automatically pushed on the production servers is the most happy feeling for any developer, so i created this guide to help other developers and devops folks out there who might need help getting setup with CI and CD for docker based applications.

Managing CI and CD for containerized applications is very frequently asked question to me when i am taking to other engineers or folks from the startup community. I personally don't like spending a lot of time doing devops activities so I rely on automation to manage most of things and deployments is also part of this. I created this guide to help others interested in Continuous integration and Continuous Delivery for containerized based application with rancher and ecr using drone ci.

I will also address issues related to version of drone ci in the rancher public catalog not working out of the box and how to fix this.

cd-ci

Lets get started

There are three major components to involved

  1. version control system, example git , mercurial
  2. continuous integration, example jenkins, drone, travis
  3. deployment tool, example amazon beanstalk, docker cloud, rancher

In this guide i will be using GitHub or BitBucket for version control and drone for Integrations and rancher for Deployments. We will also create a standalone ninjaframework webapp, commit our changes in github and drone will automatically run the compile the code, run the test, push our docker images to ecr and deploy to our rancher environment.

Prerequisites for fully understanding what i am doing require some knowledge of github/bitbucket account, rancher and amazon ecr (docker repository).

Create oauth app in github account/bitbucket

If you are using a personal account you will find the option to add oauth app under developer options in setting on github https://github.com/settings/developers. For adding the same to your bitbucket account the url will look something like https://bitbucket.org/account/user/*YOUR_USERNAME*/api. For those who want to limit access to a CI server to there organization, I will strongly recommends that you create the oauth app under your organization setting in github.
Also , bitbucket users need be sure to check the following permissions:

Account:Email
Account:Read
Team Membership:Read
Repositories:Read
Webhooks:Read and Write

Once you have added the oauth application, please make a note of the client id and client secret. this will be used later when setting up drone server.

Launch the drone stack

rancher-launch-drone-1

Go to rancher community catalog and add select drone ci stack by clicking view details. Make sure you fill out all the require configurations like Drone Host URL , Drone Host PORT, Drone Host PORT etc. Here are the recommended configurations that you should set.

Drone Host URL : drone-ci.<YOUR_DOMAIN>.com
Server and Agent secret : <ANY RANDOM SECRET TEXT>
Open Registration : False
Remote Driver : GitHub or BitBucket
Remote Driver Client : <OAUTH APP CLIENT ID FROM GITHUB/BITBUCKET>
Remote Driver SECRET : <OAUTH APP CLIENT SECRET FROM GITHUB/BITBUCKET>

Apart from the above , you don't need to change anything in the configurations. Launch the stack, this will take you to the screen where you will see the services getting launched. Everything should be fine but you will see that the lb (load balancer) is unhealthy, this is because the load balancer expects that at least one host to have label drone_lb = true. We have two choices here, add the label to host in the rancher cluster or just remove the Scheduling Rule by upgrading the lb to solve this issue. While you upgrade , also change the request port from 8000 to 80 and then point the sub-domain drone-ci.<YOUR_DOMAIN>.com to the rancher host running the load balancer.
We will also upgrade the drone server and agent version to the latest 0.8.1 so that everything is stable.

Now Visit drone-ci.<YOUR_DOMAIN>.com and you should see the login screen. time to login.After you login it will ask to activate a repository that you would like to build with drone.

Create a docker sample webapp

Lets create a a web application and add it to git , also we will create a repository on your amazon elastic container registry.

this will create java based webapp

mvn archetype:generate -DarchetypeGroupId=org.ninjaframework -DarchetypeArtifactId=ninja-servlet-archetype-simple -DartifactId=SampleApp -DgroupId=com.sesmetric -Dpackage=jar -Dversion=0.01

Push your code to git repository

cd SampleApp
git init
git add .
git commit -m "first commit"
git remote add origin git@github.com:visarya/SampleApp.git
git push -u origin master

lets add a Dockerfile in the folder so we can run this standalone app

FROM        maven
MAINTAINER Vishal Arya <vishal@finketech.com>
WORKDIR /usr/src/app
ADD target/SampleApp-0.01.jar /usr/src/app/SampleApp-0.01.jar
CMD ["java", "-jar", "/usr/src/app/SampleApp-0.01.jar"]

Adding app to Drone

This is the moment where we put all the pieces together so that every time we do a build the drone will pick the changes and make docker image and push it to ecr.

Lets add our drone file to compile the project first, we are going to create the .drone.yml file in the root of the project with below contents

pipeline:
  test:
    image: maven:alpine
    commands:
      - mvn --quiet --batch-mode test

  build_jar:
    image: maven:alpine
    commands:
    - mvn --batch-mode clean package

Now that we have added our first drone configurations, lets activate the repository on our drone ci server

Dron-Ci-Activate

In case you see a error "Error response from daemon: client is newer than server (client API version: 1.26, server API version: 1.24)" , just upgrade the drone agent on rancher and add property DOCKER_API_VERSION=1.24

Lets commit commit the drone yml file and push it to repository to trigger a build

change the ssl port in the app configuration file src/main/java/conf/application.conf to -1

ninja.ssl.port=-1

lets push the changes

git add src/main/java/conf/application.conf
git commit -m "config changes"
git push origin master

Push the images to ECR

Now its time to add configuration to push the docker image to ecr repository. We will add the below lines to the .drone.yml to push the docker image to ecr. you will also need to add you aws access key and secret key in drone secrets as ecr_access_key and ecr_secret_key respectively.

  ecr:
    image: plugins/ecr
    registry: 00000000.dkr.ecr.ap-southeast-1.amazonaws.com
    repo: 00000000.dkr.ecr.ap-southeast-1.amazonaws.com/sample-web-app
    secrets: [ ecr_access_key, ecr_secret_key ]
    tags: latest
    file: Dockerfile

now commit the file and push your changes to git

Auto deploy on rancher

the final piece of the puzzle is deploy the build to production server, this is fairly simple with rancher. lets add the rancher configurations to our drone yml file. In order for this to work seamlessly, make sure you have the amazon ecr service deployed from rancher catalog in your rancher environment.

  rancher:
    image: peloton/drone-rancher
    url: <RANCHER_URL>
    access_key: <RANCHER_ACCESS_KEY>
    secret_key: <RANCHER_SECRET_KEY>
    service: webapp/sample-web-app
    docker_image: 00000000.dkr.ecr.ap-southeast-1.amazonaws.com/sample-web-app:latest
    start_first: true
    confirm: true
    timeout: 180