Agile Software Development 

Agile software development has evolved in computing industry gaining everyone's attention. In this article I am going to explain few important facts about its components. 

What is a User Story?

Basically it should describe a functionality that will be valuable to a user / purchaser of a software.

It should comprised of :

1. Description of the story 

2. Conversations we had regarding the user story.

3. Tests to determine when the user story is completed. 

Importantly ,

* User stories should represents the functionality that will be valued by users. Users will not be interested in technical details of the system instead the output they expect.

Eg: The system will use mysql as the database. 

- This does not add a value to user unless the system is going to be interface for using DB's. 

* If the story it self is very large you can break them into several stories. 

* Ideally a user story shall be coded & tested spanning from a day or perhaps half a day to one or two weeks.

* Remember the user stories should be written in the language in business rather than set of technical jargon. Story cards are not contracts its more of a reminder to dev & customer to start the conversation where it was left. 

* Characteristics of a good story are : value added to customers , negotiable , testable , small , estimable , independent. (try to avoid dependencies between stories since this will lead to need of prioritization)

What is velocity? 

After the team decide what  is the iteration length of the project ( i.e each project will be divided into similar size iterations ) the developers will estimate the work they can cover per a iteration. 

AWS - Terraform Short Notes

1. Authenticating & Authorising AWS User for using terraform 

 AWS User - > Give Policy -> Credentials download -> Pass credentials in terraform

* You can do it in two ways
    a) As a provider
    b) Directly export AWS_ACCESS_KEY_ID & AWS_SECRET_ACCESS_KEY

2. Creating EC2 instance: -
     terraform init - Should run when created a new terraform file / cloning one from git. Safe to run multiple times. Will make the current terraform WD up-todate.
     terraform plan  - will create the AWS instance plan
     terraform apply - will create the AWS instance ( if you make a change once you execute plan & apply commands now you want to change an existing iname of the EC2 instance do it and just use "terraform apply ")

3. Removing the EC2 instance:-
     terraform destroy

4. Printing Output Variable

      output "my_public_id"{
      value="${aws_instance.dimuthu.public_ip}"
     }

5. Creating security groups
    * When creating security groups point to ponder is to you need to :
            1. Create the security group
            2. Add a policy to security group

So lets create a Bastion Host Security Group - The only host expose to outside world which filters malicious attacks. Places probably between one / two firewalls.

Ingress traffic - Data from outside is transferred to local network.
Egress traffic - Data inside the local network is transferred  to outside world.

     resource "aws_security_group" "default" {
     name= "My_Security_Group1"
     }

Containerised CI Deployment 

Today's article is about is a discussion about how to achive an end-to-end CI &CD pipeline solution for Test Automation. I am writing this blog today since I have not seen many articles written on how to integrate your UI Automation tests with modern technologies such as "Docker" , "Kubernetes" & CI Sever such as "Jenkins". For this article I am going to present you a complete solution using Docker, K8S & Jenkins built on top of CentOS.

So lets first get some idea on how Docker works on Linux (In our case CentOS) environment. 

As you can see there is a CentOS machine which runs Docker Server (which is the host machine) and also Docker client & the communication between these two are happening via REST API. 

So the Docker server deficits as the Docker daemons which is responsive for create , manage Docker objects. 

Docker clients use CLI to communicate with one or many Docker daemons. 

You can build your Dokcer file to create Docker images and run Docker images to transform them into Docker containers. 

                                     buid                                    run

               Dockerfile --------------> Docker image -------> Docker containers

Following is an example Dockerfile written for creating and running of Docker image which is capable of executing UI tests which uses "Cucumber" & "Selenium" tests based on "Java" & "Maven". 

Additionally if you are running on an old version of  forefox as with a different profile you canl run Mozilla Firefox for this test scenario & you how to run UI Firefox tests using Firefox profile manager (i.e creating a Firefox profile called eg: "P2"). You can see the profile information in CentOS by navigating to ~/.mozilla/firefox/profile.ini file.

Inside your docker container you can run Firefox either in Headless mode or with real Firefox browser loading. If you need to run with real Firefox browser use VNCServer. For this article I will use running Firefox in headless mode.

First I will describe what the actual docker file does here.

The architecture I am going to describe would consists of several layers of docker image.

FROM centos:centos7   will pull the centos7 docker image (image1) from the docker hub to your local machine. Next our Docker file will also create a seperate Docker image (image2)  called ":latest" which built on top of centos7 image. Next we will use another Dockerfile which uses "" as the base image for its implementation of the newest Docker image (image3).

                Dockerfile1                       ----------->                      Dockerfile2   
(  uses Dockerimage1 - centos7 ---> Dockerimage2 ) ---> (   Dockerimage3 ---> )

We will trigger a build using Jenkins server to create the Dockerimage3 and to start the respective Docker container by running Dockerimage3.

Running the Docker container from Docker image3 can be achived using two ways.

1. Allowing Jenkins to use the Agent machines workspace to clone github repo and start executing your tests. (For this approach you only need to configure a Jenkins Agent with your master Jenkins machine.)

2. Allowing Jenkins to create a separate workspace for each Docker container you are going to spawn using Dockerimage3. (For this approach you need to configure Jenkins Agent with master Jenkins machine and a separate Docker NodeLabel parameter plugin.)

[1] https://wiki.jenkins.io/display/JENKINS/NodeLabel+Parameter+Plugin

The drawback of the first approach is all Docker containers will have to use the same workspace of the Agent machine which will drive you into many problems. So we recommend to use the second approach as it will help us to spawn several; Docker containers which uses its own workspace to execute tests, this way we can spawn many Docker containers using the same Dockerimage3 and inside each Docker container we can run distinct test cases.

Now then back to technical stuff :) ....

So lets list down two Dockerfiles.

Dockerfile 1



Dockerfile 2


2 nd Approach
===========

I am sure that you would have experience when ever we run maven tests with docker , every time , every container will start downloading maven dependencies to its own m2 repo. Well that's not nice if the dependency list is very long since the container will consume a considerable time on downloading dependencies. One solution to prevent this is a mapping between m2 repo of the Agent machine to Docker container, so that every time a Docker container spin up instead of downloading maven dependencies to container from the beginning we can tell the container to use Agent machine's m2 repo for running tests which will all Docker containers use parallelly. But then how to do that? Below is a sample Dockerfile to be used for that purpose.

FROM centos:centos7
MAINTAINER Dimuthu De Lanerolle

# Proxy Settings And Configurations
#==================================
# Your Proxy Settings Goes Here

#Install git
#===========

RUN echo yes | yum install -y git

#Install Firefox Latest Version
#======================================

RUN yum install firefox -y

#For Installing Firefox Older Version
#====================================

#RUN mkdir -p /opt/firefox/50.1.0
#RUN curl -SL https://ftp.mozilla.org/pub/firefox/releases/50.1.0/linux-x86_64/en-US/firefox-50.1.0.tar.bz2
#RUN tar -xjC /opt/firefox/50.1.0
#COPY firefox.sh /usr/bin/firefox
#RUN chmod +x /usr/bin/firefox


#Install From Firefox Distribution Directly
#===========================================

#COPY firefox-50.1.0.tar.gz /tmp/
#workdir /tmp/
#RUN tar -xzf firefox-50.1.0.tar.gz
#RUN cp -a firefox /opt
#RUN cp -a /opt/firefox /usr/bin
#RUN mv /usr/bin/firefox /usr/bin/firefox-old
#RUN cp -a /opt/firefox /usr/bin
#RUN rm -rf /usr/bin/firefox
#RUN ln -s /opt/firefox/firefox /usr/bin/firefox

#Install openjdk
#===============

RUN yum install -y \
       java-1.8.0-openjdk \
       java-1.8.0-openjdk-devel

ENV JAVA_HOME /usr/lib/jvm/java-1.8.0-openjdk
ENV PATH $JAVA_HOME/bin:$PATH

RUN echo 'export PATH=/usr/lib/jvm/java-1.8.0-openjdk/bin:$PATH' >>~/.bash_profile
RUN echo ~/.bash_profile

#Install Maven
#=============

ARG MAVEN_VERSION=3.5.0
ARG USER_HOME_DIR="/root"
ARG SHA1=878b8b93a8f9685aefba5c21a17b46eb141b1122
ARG BASE_URL=http://archive.apache.org/dist/maven/maven-3/$MAVEN_VERSION/binaries/apache-maven-$MAVEN_VERSION-bin.tar.gz

RUN mkdir -p /usr/share/maven /usr/share/maven/ref \
  && curl -fsSL -o /tmp/apache-maven.tar.gz ${BASE_URL} \
  && echo "${SHA1}  /tmp/apache-maven.tar.gz" | sha1sum -c - \
  && tar -xzf /tmp/apache-maven.tar.gz -C /usr/share/maven --strip-components=1 \
  && rm -f /tmp/apache-maven.tar.gz \
  && ln -s /usr/share/maven/bin/mvn /usr/bin/mvn

ENV MAVEN_HOME /usr/share/maven
ENV MAVEN_CONFIG "$USER_HOME_DIR/.m2"

#Additional Configurations
#=========================

#Transfer m2 settings.xml file
#===========================

COPY /config/settings.xml /usr/share/maven/conf/

#Transfer certificates and Import certificates
#==============================================

COPY /cert/xx.der /tmp/
workdir /tmp/
RUN echo yes | keytool -importcert -keystore /etc/alternatives/jre/lib/security/cacerts -file /tmp/xx.der -storepass yyy
RUN rm -rf /tmp/

Important Docker Commands
------------------------------------

Docker :

Docker build from the DockerFile : docker build -t friendlyhello .
Docker image list : docker image list
Dockcer container list : docker ps -a
Docker container start : docker container start bd9038b3d022
Docker container start with even logs : docker events&    (Get the Event ID & run the docker image by run bd9038b3d022)
Docker container stop : docker stop bd9038b3d022
Docker container kill : docker kill bd9038b3d022

Docker remove image : docker rmi
Docker remove all images at once : docker rmi $(docker images -q)
Docker remove container : docker rm
Docker remove all containers at once : docker rm $(docker ps -a -q)

Docker remove all* : docker system prune –a

*WARNING! This will remove:
        - all stopped containers
        - all networks not used by at least one container
        - all images without at least one container associated to them
        - all build cache

Software Quality Assurance 

According to the ISO9126 standard we identify six (06) main characteristics of software quality.

1. Functionalty
2. Reliability
3. Usability
4. Efficiency
5. Maintainability
6. Portability

Software security testing has basic seven (07) main characteristics.

1. Authentication       
2. Authorization
3. Confidentiality
4. Availability
5. Integrity
6. Non-repudiation
7. Resilience

Different types of testing approaches

Penetration Testing
 - Here the tester should stimulate on how an attacker would deal with the system or in another way he should stimulate a malicious attack on the system it self for testing purposes.

Static Testing
 - Testing source code compilation , syntax errors, program structure etc. Its basically verification.

Dynamic Testing
- Executing program/ part of the code  with a set given set of test data in-order to perform validation.

Whitebox Testing
- White box testing is when a tester walk through the program code to derive the test cases executable against the program code snippets.
  White box testing can be perform at unit/integration/sub-system/system levels.
  In many flavors - Static testing, API testing, Code coverage, Fault injection etc.

Sandbox Testing
- Mirrored production environment exposed to mostly third party developers and testers to carryout testing.

Testing can be aligned with any level of software development. Even if you are developing a POC (Proof of concept) you have the freedom to carry out the testing in terms of code validation and Non functional & functional requirements.

Automation Frameworks

In the modern day of computing there are many test automation frameworks we can use to automate. A few are listed below.

1. JavaScript - Jasmine is JS testing framework.
2. Angular / AngularJS - Protrator (E2E Testing)
3. Unit Testing - Karma
4. Performance Testing - Gatling, Jmeter, Flidder

To be continued ....

Selenium + Cucumber Integration

Cucumber is nice way of achieving automation. Below is a sample automation steps on logging into Facebook app & logging out of the application itself.

Note: Below sample includes :
            Selenium and Cucumber integration with passing parameters
            Data driven testing sample
            All reports generation for Cucumber
       

You have to have following Jar list :

cucumber-reporting-0.1.14-sources.jar
junit-4.12-sources.jar
cobertura-2.1.1.jar
cucumber-jvm-deps-1.0.3.jar
cucumber-core-1.2.2.jar
cucumber-junit-1.2.2.jar
cucumber-java-1.2.2.jar
cucumber-html-0.2.2-sources.jar

Data driven testing sample
===================

import org.junit.runner.RunWith;

import cucumber.api.CucumberOptions;
import cucumber.api.junit.Cucumber;

@RunWith(Cucumber.class)
// glue - here you give the full qualified package name
@CucumberOptions(features="features",
glue="feature.description",
plugin={"html:target/cucumber-html-report",
"json:target/cucumber.json",
"pretty:target/cucumber-pretty.txt",
"usage:target/cucumber-usage.json",
"junit:target/cucumber-results.xml"})
public class TestRunner {

}


import java.util.concurrent.TimeUnit;

import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import cucumber.api.java.en.Given;
import cucumber.api.java.en.Then;
import cucumber.api.java.en.When;

public class SampleTest {

    private WebDriver driver;

@Given("^Open \"([^\"]*)\" and the \"([^\"]*)\" start \"([^\"]*)\" application$")
public void Open_chrome_and_start_application(String driverName ,String driverLocation, String applicationName) throws Throwable {

    System.setProperty(driverName, driverLocation);

            ChromeOptions chromeOptions = new ChromeOptions();
            chromeOptions.addArguments("--disable-notifications");

    driver = new ChromeDriver(chromeOptions);

    driver.manage().timeouts().implicitlyWait(30, TimeUnit.SECONDS);
            driver.manage().timeouts().pageLoadTimeout(40, TimeUnit.SECONDS);
         
    driver.manage().window().maximize();
            driver.get(applicationName);
}

@When("^Enter valid \"([^\"]*)\" and valid \"([^\"]*)\"$")
public void Enter_valid_username_and_valid_password(String userName, String password) throws Throwable {
    driver.findElement(By.id("email")).sendKeys(userName);
    driver.findElement(By.id("pass")).sendKeys(password);     
}

@Then("user should be able to login successfully")
public void user_should_be_able_to_login_successfully() throws Throwable {
    driver.findElement(By.id("loginbutton")).click();
}

@Then("user should be able to logout successfully")
public void user_should_be_able_to_logout_successfully() throws Throwable {
             driver.findElement(By.id("userNavigationLabel")).click();
     driver.findElement(By.linkText("Log Out")).click();
}
}

Feature: Test Facebook smoke scenario

    Scenario Outline: Test login with valid credentials

    Given Open "webdriver.chrome.driver" and the "C:\\xyz\\chromedriver.exe" start "http://www.facebook.com" application

    When Enter valid "<userName>" and valid "<password>"

    Then user should be able to login successfully

    And user should be able to logout successfully

    Examples: 

      | userName                 | password  |

      | abc_efgh@yahoo.com | xxxx |
      |xyz@yahoo.com | yyyy

Selenium ...

Selenium Grid

* You can automate upto 5 browsers.
* You can run both Web Driver & RC.
* Only one Hub for a grid.
* Before creating the grid make sure firewall settings are correctly configured.

Setting up the Hub

G:\TestAutomation\Selenium>java -jar selenium-server-standalone-2.48.2.jar -role hub
13:51:50.635 INFO - Launching Selenium Grid hub
2018-01-22 13:51:51.626:INFO::main: Logging initialized @1291ms
13:51:51.637 INFO - Will listen on 4444
13:51:51.672 INFO - Will listen on 4444
2018-01-22 13:51:51.675:INFO:osjs.Server:main: jetty-9.2.z-SNAPSHOT
2018-01-22 13:51:51.722:INFO:osjsh.ContextHandler:main: Started o.s.j.s.ServletContextHandler@2ddc9a9f{/,null,AVAILABLE}
2018-01-22 13:51:52.619:INFO:osjs.ServerConnector:main: Started ServerConnector@1a75e76a{HTTP/1.1}{0.0.0.0:4444}
2018-01-22 13:51:52.622:INFO:osjs.Server:main: Started @2287ms
13:51:52.624 INFO - Nodes should register to http://172.18.175.50:4444/grid/register/
13:51:52.625 INFO - Selenium Grid hub is up and running
14:00:46.294 INFO - Registered a node http://172.18.175.50:5555

Setting up/ Registering Node1

G:\TestAutomation\Selenium>java -Dwebdriver.chrome.driver=E:\Software\chromedriver.exe -jar selenium-server-standalone-2.48.2.jar -role node -hub  http://172.18.175.50:4444/grid/register/
14:00:45.605 INFO - Launching a Selenium Grid node
14:00:46.113 INFO - Java: Oracle Corporation 9.0.1+11
14:00:46.113 INFO - OS: Windows 10 10.0 amd64
14:00:46.119 INFO - v2.48.0, with Core v2.48.0. Built from revision 41bccdd
14:00:46.167 INFO - Driver class not found: com.opera.core.systems.OperaDriver
14:00:46.168 INFO - Driver provider com.opera.core.systems.OperaDriver is not registered
14:00:46.198 INFO - Version Jetty/5.1.x
14:00:46.200 INFO - Started HttpContext[/selenium-server,/selenium-server]
14:00:46.256 INFO - Started org.openqa.jetty.jetty.servlet.ServletHandler@45385f75
14:00:46.256 INFO - Started HttpContext[/wd,/wd]
14:00:46.257 INFO - Started HttpContext[/selenium-server/driver,/selenium-server/driver]
14:00:46.258 INFO - Started HttpContext[/,/]
14:00:46.263 INFO - Started SocketListener on 0.0.0.0:5555
14:00:46.263 INFO - Started org.openqa.jetty.jetty.Server@2c1b194a
14:00:46.264 INFO - Selenium Grid node is up and ready to register to the hub
14:00:46.285 INFO - Starting auto registration thread. Will try to register every 5000 ms.
14:00:46.285 INFO - Registering the node to the hub: http://172.18.175.50:4444/grid/register
14:00:46.294 INFO - The node is registered to the hub and ready to use 

Accessing grid hub from URL :

http://172.18.175.50:4444/grid/console

import org.openqa.selenium.Platform;

import org.openqa.selenium.WebDriver;

import org.openqa.selenium.remote.DesiredCapabilities;

import org.openqa.selenium.remote.RemoteWebDriver;

import java.net.URL;

import java.util.concurrent.TimeUnit;

public class SeleniumGrid {

    WebDriver driver;   // WD   

    String nodeUrl;  // node1

   

/**     

 * WD & node01 runs in two different machines      

 */    

private void invokeRemoteBrowser() {

        try {

            nodeUrl = "http://172.18.175.50:5555/wd/hub";

            DesiredCapabilities desiredCapabilities = DesiredCapabilities.chrome();

            desiredCapabilities.setBrowserName("chrome");

            desiredCapabilities.setPlatform(Platform.WINDOWS);

            driver = new RemoteWebDriver(new URL(nodeUrl), desiredCapabilities);

            driver.manage().deleteAllCookies();

            driver.manage().window().maximize();

            driver.manage().timeouts().implicitlyWait(30, TimeUnit.SECONDS);

            driver.manage().timeouts().pageLoadTimeout(40, TimeUnit.SECONDS);

            driver.get("https://www.yahoo.com");

        } catch (Exception e) {

            e.printStackTrace();

        }

    }

}

Identity Claim Management : 

This is a screen cast video done by me on claim management :

Web Link :
https://www.youtube.com/watch?v=lIcDvJzhvIE