Agile Software Development 

Agile software development has evolved in computing industry gaining everyone's attention. In this article I am going to explain few important facts about its components. 

What is a User Story?

Basically it should describe a functionality that will be valuable to a user / purchaser of a software.

It should comprised of :

1. Description of the story 

2. Conversations we had regarding the user story.

3. Tests to determine when the user story is completed. 

Importantly ,

* User stories should represents the functionality that will be valued by users. Users will not be interested in technical details of the system instead the output they expect.

Eg: The system will use mysql as the database. 

- This does not add a value to user unless the system is going to be interface for using DB's. 

* If the story it self is very large you can break them into several stories. 

* Ideally a user story shall be coded & tested spanning from a day or perhaps half a day to one or two weeks.

* Remember the user stories should be written in the language in business rather than set of technical jargon. Story cards are not contracts its more of a reminder to dev & customer to start the conversation where it was left. 

* Characteristics of a good story are : value added to customers , negotiable , testable , small , estimable , independent. (try to avoid dependencies between stories since this will lead to need of prioritization)

What is velocity? 

After the team decide what  is the iteration length of the project ( i.e each project will be divided into similar size iterations ) the developers will estimate the work they can cover per a iteration. 

AWS - Terraform Short Notes

1. Authenticating & Authorising AWS User for using terraform 

 AWS User - > Give Policy -> Credentials download -> Pass credentials in terraform

* You can do it in two ways
    a) As a provider
    b) Directly export AWS_ACCESS_KEY_ID & AWS_SECRET_ACCESS_KEY

2. Creating EC2 instance: -
     terraform init - Should run when created a new terraform file / cloning one from git. Safe to run multiple times. Will make the current terraform WD up-todate.
     terraform plan  - will create the AWS instance plan
     terraform apply - will create the AWS instance ( if you make a change once you execute plan & apply commands now you want to change an existing iname of the EC2 instance do it and just use "terraform apply ")

3. Removing the EC2 instance:-
     terraform destroy

4. Printing Output Variable

      output "my_public_id"{
      value="${aws_instance.dimuthu.public_ip}"
     }

5. Creating security groups
    * When creating security groups point to ponder is to you need to :
            1. Create the security group
            2. Add a policy to security group

So lets create a Bastion Host Security Group - The only host expose to outside world which filters malicious attacks. Places probably between one / two firewalls.

Ingress traffic - Data from outside is transferred to local network.
Egress traffic - Data inside the local network is transferred  to outside world.

     resource "aws_security_group" "default" {
     name= "My_Security_Group1"
     }

Containerised CI Deployment 

Today's article is about is a discussion about how to achive an end-to-end CI &CD pipeline solution for Test Automation. I am writing this blog today since I have not seen many articles written on how to integrate your UI Automation tests with modern technologies such as "Docker" , "Kubernetes" & CI Sever such as "Jenkins". For this article I am going to present you a complete solution using Docker, K8S & Jenkins built on top of CentOS.

So lets first get some idea on how Docker works on Linux (In our case CentOS) environment. 

As you can see there is a CentOS machine which runs Docker Server (which is the host machine) and also Docker client & the communication between these two are happening via REST API. 

So the Docker server deficits as the Docker daemons which is responsive for create , manage Docker objects. 

Docker clients use CLI to communicate with one or many Docker daemons. 

You can build your Dokcer file to create Docker images and run Docker images to transform them into Docker containers. 

                                     buid                                    run

               Dockerfile --------------> Docker image -------> Docker containers

Following is an example Dockerfile written for creating and running of Docker image which is capable of executing UI tests which uses "Cucumber" & "Selenium" tests based on "Java" & "Maven". 

Additionally if you are running on an old version of  forefox as with a different profile you canl run Mozilla Firefox for this test scenario & you how to run UI Firefox tests using Firefox profile manager (i.e creating a Firefox profile called eg: "P2"). You can see the profile information in CentOS by navigating to ~/.mozilla/firefox/profile.ini file.

Inside your docker container you can run Firefox either in Headless mode or with real Firefox browser loading. If you need to run with real Firefox browser use VNCServer. For this article I will use running Firefox in headless mode.

First I will describe what the actual docker file does here.

The architecture I am going to describe would consists of several layers of docker image.

FROM centos:centos7   will pull the centos7 docker image (image1) from the docker hub to your local machine. Next our Docker file will also create a seperate Docker image (image2)  called ":latest" which built on top of centos7 image. Next we will use another Dockerfile which uses "" as the base image for its implementation of the newest Docker image (image3).

                Dockerfile1                       ----------->                      Dockerfile2   
(  uses Dockerimage1 - centos7 ---> Dockerimage2 ) ---> (   Dockerimage3 ---> )

We will trigger a build using Jenkins server to create the Dockerimage3 and to start the respective Docker container by running Dockerimage3.

Running the Docker container from Docker image3 can be achived using two ways.

1. Allowing Jenkins to use the Agent machines workspace to clone github repo and start executing your tests. (For this approach you only need to configure a Jenkins Agent with your master Jenkins machine.)

2. Allowing Jenkins to create a separate workspace for each Docker container you are going to spawn using Dockerimage3. (For this approach you need to configure Jenkins Agent with master Jenkins machine and a separate Docker NodeLabel parameter plugin.)

[1] https://wiki.jenkins.io/display/JENKINS/NodeLabel+Parameter+Plugin

The drawback of the first approach is all Docker containers will have to use the same workspace of the Agent machine which will drive you into many problems. So we recommend to use the second approach as it will help us to spawn several; Docker containers which uses its own workspace to execute tests, this way we can spawn many Docker containers using the same Dockerimage3 and inside each Docker container we can run distinct test cases.

Now then back to technical stuff :) ....

So lets list down two Dockerfiles.

Dockerfile 1



Dockerfile 2


2 nd Approach
===========

I am sure that you would have experience when ever we run maven tests with docker , every time , every container will start downloading maven dependencies to its own m2 repo. Well that's not nice if the dependency list is very long since the container will consume a considerable time on downloading dependencies. One solution to prevent this is a mapping between m2 repo of the Agent machine to Docker container, so that every time a Docker container spin up instead of downloading maven dependencies to container from the beginning we can tell the container to use Agent machine's m2 repo for running tests which will all Docker containers use parallelly. But then how to do that? Below is a sample Dockerfile to be used for that purpose.

FROM centos:centos7
MAINTAINER Dimuthu De Lanerolle

# Proxy Settings And Configurations
#==================================
# Your Proxy Settings Goes Here

#Install git
#===========

RUN echo yes | yum install -y git

#Install Firefox Latest Version
#======================================

RUN yum install firefox -y

#For Installing Firefox Older Version
#====================================

#RUN mkdir -p /opt/firefox/50.1.0
#RUN curl -SL https://ftp.mozilla.org/pub/firefox/releases/50.1.0/linux-x86_64/en-US/firefox-50.1.0.tar.bz2
#RUN tar -xjC /opt/firefox/50.1.0
#COPY firefox.sh /usr/bin/firefox
#RUN chmod +x /usr/bin/firefox


#Install From Firefox Distribution Directly
#===========================================

#COPY firefox-50.1.0.tar.gz /tmp/
#workdir /tmp/
#RUN tar -xzf firefox-50.1.0.tar.gz
#RUN cp -a firefox /opt
#RUN cp -a /opt/firefox /usr/bin
#RUN mv /usr/bin/firefox /usr/bin/firefox-old
#RUN cp -a /opt/firefox /usr/bin
#RUN rm -rf /usr/bin/firefox
#RUN ln -s /opt/firefox/firefox /usr/bin/firefox

#Install openjdk
#===============

RUN yum install -y \
       java-1.8.0-openjdk \
       java-1.8.0-openjdk-devel

ENV JAVA_HOME /usr/lib/jvm/java-1.8.0-openjdk
ENV PATH $JAVA_HOME/bin:$PATH

RUN echo 'export PATH=/usr/lib/jvm/java-1.8.0-openjdk/bin:$PATH' >>~/.bash_profile
RUN echo ~/.bash_profile

#Install Maven
#=============

ARG MAVEN_VERSION=3.5.0
ARG USER_HOME_DIR="/root"
ARG SHA1=878b8b93a8f9685aefba5c21a17b46eb141b1122
ARG BASE_URL=http://archive.apache.org/dist/maven/maven-3/$MAVEN_VERSION/binaries/apache-maven-$MAVEN_VERSION-bin.tar.gz

RUN mkdir -p /usr/share/maven /usr/share/maven/ref \
  && curl -fsSL -o /tmp/apache-maven.tar.gz ${BASE_URL} \
  && echo "${SHA1}  /tmp/apache-maven.tar.gz" | sha1sum -c - \
  && tar -xzf /tmp/apache-maven.tar.gz -C /usr/share/maven --strip-components=1 \
  && rm -f /tmp/apache-maven.tar.gz \
  && ln -s /usr/share/maven/bin/mvn /usr/bin/mvn

ENV MAVEN_HOME /usr/share/maven
ENV MAVEN_CONFIG "$USER_HOME_DIR/.m2"

#Additional Configurations
#=========================

#Transfer m2 settings.xml file
#===========================

COPY /config/settings.xml /usr/share/maven/conf/

#Transfer certificates and Import certificates
#==============================================

COPY /cert/xx.der /tmp/
workdir /tmp/
RUN echo yes | keytool -importcert -keystore /etc/alternatives/jre/lib/security/cacerts -file /tmp/xx.der -storepass yyy
RUN rm -rf /tmp/

Important Docker Commands
------------------------------------

Docker :

Docker build from the DockerFile : docker build -t friendlyhello .
Docker image list : docker image list
Dockcer container list : docker ps -a
Docker container start : docker container start bd9038b3d022
Docker container start with even logs : docker events&    (Get the Event ID & run the docker image by run bd9038b3d022)
Docker container stop : docker stop bd9038b3d022
Docker container kill : docker kill bd9038b3d022

Docker remove image : docker rmi
Docker remove all images at once : docker rmi $(docker images -q)
Docker remove container : docker rm
Docker remove all containers at once : docker rm $(docker ps -a -q)

Docker remove all* : docker system prune –a

*WARNING! This will remove:
        - all stopped containers
        - all networks not used by at least one container
        - all images without at least one container associated to them
        - all build cache

Software Quality Assurance 

According to the ISO9126 standard we identify six (06) main characteristics of software quality.

1. Functionalty
2. Reliability
3. Usability
4. Efficiency
5. Maintainability
6. Portability

Software security testing has basic seven (07) main characteristics.

1. Authentication       
2. Authorization
3. Confidentiality
4. Availability
5. Integrity
6. Non-repudiation
7. Resilience

Different types of testing approaches

Penetration Testing
 - Here the tester should stimulate on how an attacker would deal with the system or in another way he should stimulate a malicious attack on the system it self for testing purposes.

Static Testing
 - Testing source code compilation , syntax errors, program structure etc. Its basically verification.

Dynamic Testing
- Executing program/ part of the code  with a set given set of test data in-order to perform validation.

Whitebox Testing
- White box testing is when a tester walk through the program code to derive the test cases executable against the program code snippets.
  White box testing can be perform at unit/integration/sub-system/system levels.
  In many flavors - Static testing, API testing, Code coverage, Fault injection etc.

Sandbox Testing
- Mirrored production environment exposed to mostly third party developers and testers to carryout testing.

Testing can be aligned with any level of software development. Even if you are developing a POC (Proof of concept) you have the freedom to carry out the testing in terms of code validation and Non functional & functional requirements.

Automation Frameworks

In the modern day of computing there are many test automation frameworks we can use to automate. A few are listed below.

1. JavaScript - Jasmine is JS testing framework.
2. Angular / AngularJS - Protrator (E2E Testing)
3. Unit Testing - Karma
4. Performance Testing - Gatling, Jmeter, Flidder

To be continued ....

Selenium + Cucumber Integration

Cucumber is nice way of achieving automation. Below is a sample automation steps on logging into Facebook app & logging out of the application itself.

Note: Below sample includes :
            Selenium and Cucumber integration with passing parameters
            Data driven testing sample
            All reports generation for Cucumber
       

You have to have following Jar list :

cucumber-reporting-0.1.14-sources.jar
junit-4.12-sources.jar
cobertura-2.1.1.jar
cucumber-jvm-deps-1.0.3.jar
cucumber-core-1.2.2.jar
cucumber-junit-1.2.2.jar
cucumber-java-1.2.2.jar
cucumber-html-0.2.2-sources.jar

Data driven testing sample
===================

import org.junit.runner.RunWith;

import cucumber.api.CucumberOptions;
import cucumber.api.junit.Cucumber;

@RunWith(Cucumber.class)
// glue - here you give the full qualified package name
@CucumberOptions(features="features",
glue="feature.description",
plugin={"html:target/cucumber-html-report",
"json:target/cucumber.json",
"pretty:target/cucumber-pretty.txt",
"usage:target/cucumber-usage.json",
"junit:target/cucumber-results.xml"})
public class TestRunner {

}


import java.util.concurrent.TimeUnit;

import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import cucumber.api.java.en.Given;
import cucumber.api.java.en.Then;
import cucumber.api.java.en.When;

public class SampleTest {

    private WebDriver driver;

@Given("^Open \"([^\"]*)\" and the \"([^\"]*)\" start \"([^\"]*)\" application$")
public void Open_chrome_and_start_application(String driverName ,String driverLocation, String applicationName) throws Throwable {

    System.setProperty(driverName, driverLocation);

            ChromeOptions chromeOptions = new ChromeOptions();
            chromeOptions.addArguments("--disable-notifications");

    driver = new ChromeDriver(chromeOptions);

    driver.manage().timeouts().implicitlyWait(30, TimeUnit.SECONDS);
            driver.manage().timeouts().pageLoadTimeout(40, TimeUnit.SECONDS);
         
    driver.manage().window().maximize();
            driver.get(applicationName);
}

@When("^Enter valid \"([^\"]*)\" and valid \"([^\"]*)\"$")
public void Enter_valid_username_and_valid_password(String userName, String password) throws Throwable {
    driver.findElement(By.id("email")).sendKeys(userName);
    driver.findElement(By.id("pass")).sendKeys(password);     
}

@Then("user should be able to login successfully")
public void user_should_be_able_to_login_successfully() throws Throwable {
    driver.findElement(By.id("loginbutton")).click();
}

@Then("user should be able to logout successfully")
public void user_should_be_able_to_logout_successfully() throws Throwable {
             driver.findElement(By.id("userNavigationLabel")).click();
     driver.findElement(By.linkText("Log Out")).click();
}
}

Feature: Test Facebook smoke scenario

    Scenario Outline: Test login with valid credentials

    Given Open "webdriver.chrome.driver" and the "C:\\xyz\\chromedriver.exe" start "http://www.facebook.com" application

    When Enter valid "<userName>" and valid "<password>"

    Then user should be able to login successfully

    And user should be able to logout successfully

    Examples: 

      | userName                 | password  |

      | abc_efgh@yahoo.com | xxxx |
      |xyz@yahoo.com | yyyy

Selenium ...

Selenium Grid

* You can automate upto 5 browsers.
* You can run both Web Driver & RC.
* Only one Hub for a grid.
* Before creating the grid make sure firewall settings are correctly configured.

Setting up the Hub

G:\TestAutomation\Selenium>java -jar selenium-server-standalone-2.48.2.jar -role hub
13:51:50.635 INFO - Launching Selenium Grid hub
2018-01-22 13:51:51.626:INFO::main: Logging initialized @1291ms
13:51:51.637 INFO - Will listen on 4444
13:51:51.672 INFO - Will listen on 4444
2018-01-22 13:51:51.675:INFO:osjs.Server:main: jetty-9.2.z-SNAPSHOT
2018-01-22 13:51:51.722:INFO:osjsh.ContextHandler:main: Started o.s.j.s.ServletContextHandler@2ddc9a9f{/,null,AVAILABLE}
2018-01-22 13:51:52.619:INFO:osjs.ServerConnector:main: Started ServerConnector@1a75e76a{HTTP/1.1}{0.0.0.0:4444}
2018-01-22 13:51:52.622:INFO:osjs.Server:main: Started @2287ms
13:51:52.624 INFO - Nodes should register to http://172.18.175.50:4444/grid/register/
13:51:52.625 INFO - Selenium Grid hub is up and running
14:00:46.294 INFO - Registered a node http://172.18.175.50:5555

Setting up/ Registering Node1

G:\TestAutomation\Selenium>java -Dwebdriver.chrome.driver=E:\Software\chromedriver.exe -jar selenium-server-standalone-2.48.2.jar -role node -hub  http://172.18.175.50:4444/grid/register/
14:00:45.605 INFO - Launching a Selenium Grid node
14:00:46.113 INFO - Java: Oracle Corporation 9.0.1+11
14:00:46.113 INFO - OS: Windows 10 10.0 amd64
14:00:46.119 INFO - v2.48.0, with Core v2.48.0. Built from revision 41bccdd
14:00:46.167 INFO - Driver class not found: com.opera.core.systems.OperaDriver
14:00:46.168 INFO - Driver provider com.opera.core.systems.OperaDriver is not registered
14:00:46.198 INFO - Version Jetty/5.1.x
14:00:46.200 INFO - Started HttpContext[/selenium-server,/selenium-server]
14:00:46.256 INFO - Started org.openqa.jetty.jetty.servlet.ServletHandler@45385f75
14:00:46.256 INFO - Started HttpContext[/wd,/wd]
14:00:46.257 INFO - Started HttpContext[/selenium-server/driver,/selenium-server/driver]
14:00:46.258 INFO - Started HttpContext[/,/]
14:00:46.263 INFO - Started SocketListener on 0.0.0.0:5555
14:00:46.263 INFO - Started org.openqa.jetty.jetty.Server@2c1b194a
14:00:46.264 INFO - Selenium Grid node is up and ready to register to the hub
14:00:46.285 INFO - Starting auto registration thread. Will try to register every 5000 ms.
14:00:46.285 INFO - Registering the node to the hub: http://172.18.175.50:4444/grid/register
14:00:46.294 INFO - The node is registered to the hub and ready to use 

Accessing grid hub from URL :

http://172.18.175.50:4444/grid/console

import org.openqa.selenium.Platform;

import org.openqa.selenium.WebDriver;

import org.openqa.selenium.remote.DesiredCapabilities;

import org.openqa.selenium.remote.RemoteWebDriver;

import java.net.URL;

import java.util.concurrent.TimeUnit;

public class SeleniumGrid {

    WebDriver driver;   // WD   

    String nodeUrl;  // node1

   

/**     

 * WD & node01 runs in two different machines      

 */    

private void invokeRemoteBrowser() {

        try {

            nodeUrl = "http://172.18.175.50:5555/wd/hub";

            DesiredCapabilities desiredCapabilities = DesiredCapabilities.chrome();

            desiredCapabilities.setBrowserName("chrome");

            desiredCapabilities.setPlatform(Platform.WINDOWS);

            driver = new RemoteWebDriver(new URL(nodeUrl), desiredCapabilities);

            driver.manage().deleteAllCookies();

            driver.manage().window().maximize();

            driver.manage().timeouts().implicitlyWait(30, TimeUnit.SECONDS);

            driver.manage().timeouts().pageLoadTimeout(40, TimeUnit.SECONDS);

            driver.get("https://www.yahoo.com");

        } catch (Exception e) {

            e.printStackTrace();

        }

    }

}

Identity Claim Management : 

This is a screen cast video done by me on claim management :

Web Link :
https://www.youtube.com/watch?v=lIcDvJzhvIE

XACML Architecture

1. Its an access control policy language.

2. The Identity Server supports XACML 3.0, which is based on Balana XACML implementation.

3. The XACML engine of the WSO2 Identity Server has two major components, i.e., PAP and PDP. 

Eg: 

MyPolicy.xml

==========

<Policy xmlns="urn:oasis:names:tc:xacml:3.0:core:schema:wd-17" PolicyId="MyPolicy" RuleCombiningAlgId="urn:oasis:names:tc:xacml:3.0:rule-combining-algorithm:deny-overrides" Version="1.0">

     <Target>

   <AnyOf>

            <AllOf>

  <Match MatchId="urn:oasis:names:tc:xacml:1.0:function:string-equal">

         <AttributeValue DataType="http://www.w3.org/2001/XMLSchema#string">read</AttributeValue>

         <AttributeDesignator AttributeId="urn:oasis:names:tc:xacml:1.0:action:action-id" Category="urn:oasis:names:tc:xacml:3.0:attribute-category:action" 

                                            DataType="http://www.w3.org/2001/XMLSchema#string" MustBePresent="true"/>

  </Match>

     </AllOf>

  </AnyOf>

      </Target>

          <Rule Effect="Permit" RuleId="permit"/>

</Policy>

Request 

=======

https://localhost:9443/entitlement/Decision/pdp

Authorization         Basic YWRtaW46YWRtaW4=

Accept                   application/xml

Content-Type        application/xml

<Request CombinedDecision="false" ReturnPolicyIdList="false" xmlns="urn:oasis:names:tc:xacml:3.0:core:schema:wd-17">

    <Attributes Category="urn:oasis:names:tc:xacml:3.0:attribute-category:action">

        <Attribute AttributeId="urn:oasis:names:tc:xacml:1.0:action:action-id" IncludeInResult="false">

            <AttributeValue DataType="http://www.w3.org/2001/XMLSchema#string">read</AttributeValue>

        </Attribute>

    </Attributes>

    <Attributes Category="urn:oasis:names:tc:xacml:3.0:attribute-category:resource">

        <Attribute AttributeId="urn:oasis:names:tc:xacml:1.0:resource:resource-id" IncludeInResult="false">

            <AttributeValue DataType="http://www.w3.org/2001/XMLSchema#string">http://127.0.0.1/service/very_secure/</AttributeValue>

        </Attribute>

    </Attributes>

</Request>

Response

========

<Response xmlns="urn:oasis:names:tc:xacml:3.0:core:schema:wd-17">

    <Result>

        <Decision>Permit</Decision>

        <Status>

            <StatusCode Value="urn:oasis:names:tc:xacml:1.0:status:ok"/>

        </Status>

    </Result>

</Response>

Java Script Basics 

[1] Calling a function of another file from your javascript file

file 01:  graphinventor.js
=================

 console.log("-----------------------------------------Time Unit ");
 console.log("Previous Time Stamp ");


var test123 = function (data){
console.log("-----------------------------------------inside test123 ");
console.log(data)
alert("This is an alert" +data);

}

[3] https://datatables.net/manual/ajax

file 02: gadgetconf.js
===============

processData: function(data) {

console.log('data '+JSON.stringify(data));
 // in console of the browser (Ctrl+c) you see the content now as json data

console.log("------------------------");
test123(data);
console.log("------------------------");

}

main.js file (Loading graphinventor.js file first)
==========

       <!-- Custom -->
          <script src="js/graphinventor.js"></script>
          <script src="js/gadgetconf.js"></script>
          <script src="js/main.js"></script>

[2] Callback function is a function passed into another function.

Java Script Basics 

[1] Calling a function of another file from your javascript file

file 01:  graphinventor.js
=================

 console.log("-----------------------------------------Time Unit ");
 console.log("Previous Time Stamp ");


var test123 = function (data){
console.log("-----------------------------------------inside test123 ");
console.log(data)
alert("This is an alert" +data);

}

file 02: gadgetconf.js
===============

processData: function(data) {

console.log('data '+JSON.stringify(data));
 // in console of the browser (Ctrl+c) you see the content now as json data

console.log("------------------------");
test123(data);
console.log("------------------------");

}

main.js file (Loading graphinventor.js file first)
==========

       <!-- Custom -->
          <script src="js/graphinventor.js"></script>
          <script src="js/gadgetconf.js"></script>
          <script src="js/main.js"></script>

[2] Callback function is a function passed into another function.

Json  Notes:  

Json Objects:

var me = { "age" : "25", "address" : "Kottawa", "gender" : "male" };

Json Arrays:

var family = [{    "name" : "Dimuthu",    "age" : "25",    "gender" : "male" }, {    "name" : "Nadeesha",    "age" : "25",    "gender" : "male" }];

JsonNeted Objects:

var family = {    "Dimuthu" : {        "sirname" : "Lanerolle",        "age" : "25",        "gender" : "male"    },    "Nadeesha" : {        "sirname" : "Lanerolle",        "age" : "25",        "gender" : "male"    } }

Wso2 Server Error Tips
----------------------------

1. Starting wso2server gives below error ...

ERROR {org.opensaml.xml.XMLConfigurator} -  Can not create instance of org.opensaml.xml.schema.impl.XSAnyMarshaller
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.opensaml.xml.XMLConfigurator.createClassInstance(XMLConfigurator.java:360)
at org.opensaml.xml.XMLConfigurator.initializeObjectProviders(XMLConfigurator.java:240)
at org.opensaml.xml.XMLConfigurator.load(XMLConfigurator.java:182)
at org.opensaml.xml.XMLConfigurator.load(XMLConfigurator.java:166)
at org.opensaml.xml.XMLConfigurator.load(XMLConfigurator.java:143)
at org.opensaml.DefaultBootstrap.initializeXMLTooling(DefaultBootstrap.java:224)
at org.opensaml.DefaultBootstrap.initializeXMLTooling(DefaultBootstrap.java:207)
at org.opensaml.DefaultBootstrap.bootstrap(DefaultBootstrap.java:100)
at org.apache.rahas.Rahas.init(Rahas.java:46)
at org.apache.axis2.context.ConfigurationContextFactory.initModules(ConfigurationContextFactory.java:252)
at org.apache.axis2.context.ConfigurationContextFactory.init(ConfigurationContextFactory.java:230)
at org.apache.axis2.context.ConfigurationContextFactory.createConfigurationContext(ConfigurationContextFactory.java:93)
at org.wso2.carbon.core.CarbonConfigurationContextFactory.createNewConfigurationContext(CarbonConfigurationContextFactory.java:65)
at org.wso2.carbon.core.init.CarbonServerManager.initializeCarbon(CarbonServerManager.java:398)
at org.wso2.carbon.core.init.CarbonServerManager.start(CarbonServerManager.java:219)
at org.wso2.carbon.core.internal.CarbonCoreServiceComponent.activate(CarbonCoreServiceComponent.java:94)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.eclipse.equinox.internal.ds.model.ServiceComponent.activate(ServiceComponent.java:260)
at org.eclipse.equinox.internal.ds.model.ServiceComponentProp.activate(ServiceComponentProp.java:146)
at org.eclipse.equinox.internal.ds.model.ServiceComponentProp.build(ServiceComponentProp.java:345)
at org.eclipse.equinox.internal.ds.InstanceProcess.buildComponent(InstanceProcess.java:620)
at org.eclipse.equinox.internal.ds.InstanceProcess.buildComponents(InstanceProcess.java:197)
at org.eclipse.equinox.internal.ds.Resolver.getEligible(Resolver.java:343)
at org.eclipse.equinox.internal.ds.SCRManager.serviceChanged(SCRManager.java:222)
at org.eclipse.osgi.internal.serviceregistry.FilteredServiceListener.serviceChanged(FilteredServiceListener.java:107)
at org.eclipse.osgi.framework.internal.core.BundleContextImpl.dispatchEvent(BundleContextImpl.java:861)
at org.eclipse.osgi.framework.eventmgr.EventManager.dispatchEvent(EventManager.java:230)
at org.eclipse.osgi.framework.eventmgr.ListenerQueue.dispatchEventSynchronous(ListenerQueue.java:148)
at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEventPrivileged(ServiceRegistry.java:819)
at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.publishServiceEvent(ServiceRegistry.java:771)
at org.eclipse.osgi.internal.serviceregistry.ServiceRegistrationImpl.register(ServiceRegistrationImpl.java:130)
at org.eclipse.osgi.internal.serviceregistry.ServiceRegistry.registerService(ServiceRegistry.java:214)
at


Solution :

Download these 2 jars from Oracle and place inside Java\jdk1.7.0_10\jre\lib\security

[1] http://www.oracle.com/technetwork/java/javase/downloads/jce-7-download-432124.html

Accessing H2 DB from the browser
----------------------------------------------

1. Navigate to  [Product_Home]/repository/conf/carbon.xml
2. Uncomment below.
 
  <H2DatabaseConfiguration>
        <property name="web" />
        <property name="webPort">8082</property>
        <property name="webAllowOthers" />
        <property name="webSSL" />
        <property name="tcp" />
        <property name="tcpPort">9092</property>
        <property name="tcpAllowOthers" />
        <property name="tcpSSL" />
        <property name="pg" />
        <property name="pgPort">5435</property>
        <property name="pgAllowOthers" />
        <property name="trace" />
        <property name="baseDir">${carbon.home}</property>
    </H2DatabaseConfiguration>

3. Navigate to your primary datasource which points to the H2 datasoure to identify username & password.

eg:

        <datasource>
            <name>WSO2_CARBON_DB</name>
            <description>The datasource used for registry and user manager</description>
            <jndiConfig>
                <name>jdbc/WSO2CarbonDB</name>
            </jndiConfig>
            <definition type="RDBMS">
                <configuration>
                    <url>jdbc:h2:./repository/database/WSO2CARBON_DB;DB_CLOSE_ON_EXIT=FALSE;LOCK_TIMEOUT=60000</url>
                    <username>wso2carbon</username>
                    <password>wso2carbon</password>
                    <driverClassName>org.h2.Driver</driverClassName>
                    <maxActive>50</maxActive>
                    <maxWait>60000</maxWait>
                    <testOnBorrow>true</testOnBorrow>
                    <validationQuery>SELECT 1</validationQuery>
                    <validationInterval>30000</validationInterval>
                    <defaultAutoCommit>false</defaultAutoCommit>
                </configuration>
            </definition>
        </datasource>

4. Type in the browser-url        :      https://localhost:8280

Publishing API Runtime Statistics Using WSO2 DAS & WSO2 APIM

-------------------------------------------------------------------------------------------

Please note that for this tutorial I am using APIM 1.10.0 and DAS 3.0.1 versions.

WSO2 API Manager is a complete solution for designing and publishing APIs, creating and managing a developer community, and for securing and routing API traffic in a scalable way. It leverages proven components from the WSO2 platform to secure, integrate and manage APIs. In addition, it integrates with the WSO2 Analytics Platform, and provides out of the box reports and alerts, giving you instant insight into APIs' behavior.

In order for downloading WSO2 APIM click here.

WSO2 Data Analytics Server is a comprehensive enterprise data analytics platform; it fuses batch and real-time analytics of any source of data with predictive analytics via machine learning. It supports the demands of not just business, but Internet of Things solutions, mobile and Web apps.

In order for downloading WSO2 DAS click here.

Configuring WSO2 APIM

===================

1. Navigate to [APIM_HOME]/repository/conf/api-manager.xml and enable the below section.

<!-- For APIM implemented Statistic client for RDBMS -->

    <StatisticClientProvider>org.wso2.carbon.apimgt.usage.client.impl.APIUsageStatisticsRdbmsClientImpl</StatisticClientProvider>

2. Start APIM server. 

3. Login to dash-board. eg: https://localhost:9443/admin-dashboard. Select "Configure Analytics" & select enable checkbox. Configure as shown below.

Configuring WSO2 DAS
==================

1. In-order to prevent server startup conflicts we will start DAS with port offset value of 1.
To do so navigate to [DAS_HOME]/repository/conf and open carbon.xml file.

           <Offset>1</Offset>

2. Navigate to [DAS_HOME]/repository/conf/datasources/master-datasources.xml and add the below.

<datasource>
  <name>WSO2AM_STATS_DB</name>
  <description>The datasource used for setting statistics to API Manager</description>
  <jndiConfig>
    <name>jdbc/WSO2AM_STATS_DB</name>
    </jndiConfig>
  <definition type="RDBMS">
    <configuration>
      <url>jdbc:mysql://localhost:3306/TestStatsDB?autoReconnect=true&amp;relaxAutoCommit=true</url>
      <username>admin</username>
      <password>admin</password>
      <driverClassName>com.mysql.jdbc.Driver</driverClassName>
      <maxActive>50</maxActive>
      <maxWait>60000</maxWait>
      <testOnBorrow>true</testOnBorrow>
      <validationQuery>SELECT 1</validationQuery>
      <validationInterval>30000</validationInterval>
      </configuration>
    </definition>

</datasource>


** IMPORTANT
--------------------

Prio-adding this config you need to create a DB called 'TestStatsDB' in your mysql database &  and create required tables. To do so follow below instructions.

[1] Navigate to [APIM_HOME]/dbscripts/stat/sql  from your ubuntu console and select mysql.sql script.

type :   source [APIM_HOME]/dbscripts/stat/sql/mysql.sql

- Which will create the required tables for our scenario.

[2] Now navigate to [APIM_HOME]/repository/components/lib and add the required mysql connector jar from here.


Create / Publish / Invoke API
======================

1. Navigate to APIM Publisher and create an API. Publish it.

2. Navigate to APIM Store and subscribe to the API.

3. In the Publisher instance select created API and navigate to 'Implement' section and select

            Destination-Based Usage Tracking: enable

4. Invoke the API several times.


View API Statistics
==============


Navigate to APIM publisher instance. Select "Statistics" section. Enjoy !!!



PS ::

If you wanna do a load test scenario or just to play with Jmeter for your easy reference I have attached herewith a sample Jmeter script for 'Calculator API'.

<?xml version="1.0" encoding="UTF-8"?>
<jmeterTestPlan version="1.2" properties="2.8" jmeter="2.13 r1665067">
  <hashTree>
    <HTTPSamplerProxy guiclass="HttpTestSampleGui" testclass="HTTPSamplerProxy" testname="HTTP Request" enabled="true">
      <elementProp name="HTTPsampler.Arguments" elementType="Arguments" guiclass="HTTPArgumentsPanel" testclass="Arguments" testname="User Defined Variables" enabled="true">
        <collectionProp name="Arguments.arguments"/>
      </elementProp>
      <stringProp name="HTTPSampler.domain">localhost</stringProp>
      <stringProp name="HTTPSampler.port">8280</stringProp>
      <stringProp name="HTTPSampler.connect_timeout"></stringProp>
      <stringProp name="HTTPSampler.response_timeout"></stringProp>
      <stringProp name="HTTPSampler.protocol"></stringProp>
      <stringProp name="HTTPSampler.contentEncoding"></stringProp>
      <stringProp name="HTTPSampler.path">/calc/1.0/subtract?x=33&amp;y=9</stringProp>
      <stringProp name="HTTPSampler.method">GET</stringProp>
      <boolProp name="HTTPSampler.follow_redirects">true</boolProp>
      <boolProp name="HTTPSampler.auto_redirects">false</boolProp>
      <boolProp name="HTTPSampler.use_keepalive">true</boolProp>
      <boolProp name="HTTPSampler.DO_MULTIPART_POST">false</boolProp>
      <stringProp name="HTTPSampler.implementation">HttpClient4</stringProp>
      <boolProp name="HTTPSampler.monitor">false</boolProp>
      <stringProp name="HTTPSampler.embedded_url_re"></stringProp>
    </HTTPSamplerProxy>
    <hashTree>
      <HeaderManager guiclass="HeaderPanel" testclass="HeaderManager" testname="HTTP Header Manager" enabled="true">
        <collectionProp name="HeaderManager.headers">
          <elementProp name="" elementType="Header">
            <stringProp name="Header.name">Authorization</stringProp>
            <stringProp name="Header.value">Bearer 2e431777ce280f385c30ac82c1e1f21c</stringProp>
          </elementProp>
        </collectionProp>
      </HeaderManager>
      <hashTree/>
    </hashTree>
  </hashTree>
</jmeterTestPlan>

Useful OSGI Commands

1. Finding a package inside which jar?
     -  p com.ibm.msg.client.commonservices.Log

     - Reply : com.ibm.msg.client.commonservices.Log; version="0.0.0"<com.ibm.mq_1.0.0 [24]>
                    com.ibm.msg.client.commonservices.Log; version="0.0.0"<jmqi_7.5.0.3_1.0.0 [82]>

2. Getting actual jar name
      - diag 24

Understanding different WSO2 ESB Scopes 

Scopes:

1. Synapse scope

When the scope of a property mediator is synapse, its value is available throughout both the in sequence and the out sequence

2. axis2 scope

When the scope of a property mediator is axis2, its value is available only throughout the sequence for which the property is defined (e.g., if you add the property to an in sequence, its value will be available only throughout the in sequence).

Testing WebsphereMQ 8.0 (Windows server 2012) together with WSO2 ESB 4.8.1 message stores and message processors - Full Synapse Configuration

For configuaring IBM WebsphreMQ with WSO2 ESB please click on this link :

WSO2 ESB Full Synapse Configuration
-------------------------------------------------
<?xml version="1.0" encoding="UTF-8"?>
<definitions xmlns="http://ws.apache.org/ns/synapse">
   <registry provider="org.wso2.carbon.mediation.registry.WSO2Registry">
      <parameter name="cachableDuration">15000</parameter>
   </registry>
   <proxy name="MySample2"
          transports="https http"
          startOnLoad="true"
          trace="disable">
      <description/>
      <target>
         <inSequence>
            <property name="FORCE_SC_ACCEPTED"
                      value="true"
                      scope="axis2"
                      type="STRING"/>
            <property name="OUT_ONLY" value="true" scope="default" type="STRING"/>
            <store messageStore="JMSStore2"/>
         </inSequence>
      </target>
   </proxy>
   <proxy name="MySample"
          transports="https http"
          startOnLoad="true"
          trace="disable">
      <description/>
      <target>
         <inSequence>
            <property name="FORCE_SC_ACCEPTED"
                      value="true"
                      scope="axis2"
                      type="STRING"/>
            <property name="OUT_ONLY" value="true" scope="default" type="STRING"/>
            <store messageStore="JMSStore1"/>
         </inSequence>
      </target>
   </proxy>
   <proxy name="MyMockProxy"
          transports="https http"
          startOnLoad="true"
          trace="disable">
      <description/>
      <target>
         <inSequence>
            <log level="custom">
               <property name="it" value="** Its Inline Sequence of MockProxy"/>
            </log>
            <payloadFactory media-type="xml">
               <format>
                  <Response xmlns="">
                     <status>OK</status>
                     <code>1</code>
                  </Response>
               </format>
               <args/>
            </payloadFactory>
            <header name="To" action="remove"/>
            <property name="RESPONSE" value="true" scope="default" type="STRING"/>
            <property name="NO_ENTITY_BODY" scope="axis2" action="remove"/>
            <property name="messageType" value="application/xml" scope="axis2"/>
            <send/>
         </inSequence>
      </target>
   </proxy>
   <endpoint name="MyMockProxy">
      <address uri="http://192.168.48.140:8280/services/MyMockProxy"/>
   </endpoint>
   <sequence name="fault">
      <log level="full">
         <property name="MESSAGE" value="Executing default 'fault' sequence"/>
         <property name="ERROR_CODE" expression="get-property('ERROR_CODE')"/>
         <property name="ERROR_MESSAGE" expression="get-property('ERROR_MESSAGE')"/>
      </log>
      <drop/>
   </sequence>
   <sequence name="main">
      <in>
         <log level="full"/>
         <filter source="get-property('To')" regex="http://localhost:9000.*">
            <send/>
         </filter>
      </in>
      <out>
         <send/>
      </out>
      <description>The main sequence for the message mediation</description>
   </sequence>
   <messageStore class="org.apache.synapse.message.store.impl.jms.JmsStore"
                 name="JMSStore1">
      <parameter name="java.naming.factory.initial">com.sun.jndi.fscontext.RefFSContextFactory</parameter>
      <parameter name="store.jms.password">wso2321#qa</parameter>
      <parameter name="java.naming.provider.url">file:\C:\Users\Administrator\Desktop\jndidirectory</parameter>
      <parameter name="store.jms.connection.factory">MyQueueConnectionFactory</parameter>
      <parameter name="store.jms.username">Administrator</parameter>
      <parameter name="store.jms.JMSSpecVersion">1.1</parameter>
      <parameter name="store.jms.destination">LocalQueue1</parameter>
   </messageStore>
   <messageStore class="org.apache.synapse.message.store.impl.jms.JmsStore"
                 name="JMSStore2">
      <parameter name="java.naming.factory.initial">com.sun.jndi.fscontext.RefFSContextFactory</parameter>
      <parameter name="store.jms.password">wso2321#qa</parameter>
      <parameter name="java.naming.provider.url">file:\C:\Users\Administrator\Desktop\jndidirectory</parameter>
      <parameter name="store.jms.connection.factory">MyQueueConnectionFactory5</parameter>
      <parameter name="store.jms.username">Administrator</parameter>
      <parameter name="store.jms.JMSSpecVersion">1.1</parameter>
      <parameter name="store.jms.destination">LocalQueue5</parameter>
   </messageStore>
   <messageProcessor class="org.apache.synapse.message.processor.impl.forwarder.ScheduledMessageForwardingProcessor"
                     name="JMSMessagePro2"
                     targetEndpoint="MyMockProxy"
                     messageStore="JMSStore2">
      <parameter name="client.retry.interval">1000</parameter>
      <parameter name="max.delivery.attempts">5</parameter>
      <parameter name="interval">10000</parameter>
      <parameter name="is.active">true</parameter>
   </messageProcessor>
   <messageProcessor class="org.apache.synapse.message.processor.impl.forwarder.ScheduledMessageForwardingProcessor"
                     name="JMSMessagePro"
                     targetEndpoint="MyMockProxy"
                     messageStore="JMSStore1">
      <parameter name="max.delivery.attempts">2</parameter>
      <parameter name="client.retry.interval">1000</parameter>
      <parameter name="interval">10000</parameter>
      <parameter name="is.active">true</parameter>
   </messageProcessor>
</definitions>

Important Points :

[1] Identify the message flow ...

 SoapRequest  --> MySampleProxy --> MessageStore -->MessageQueue --> MessageProcessor
     (SoapUI)                 (ESB)                        (ESB)           (WebsphereMQ)             (ESB)
                                                                                                                                        |
                                                                                                         MyMockProxy  <--|
                                                                                                                 (ESB)

[2] Remember to set additional MS / MP settings depending on your requirement.
      eg: Maximum delivery attempts , Retry period etc.

Proxy with enrich mediator sends HTTP 200 together with a custom message

<?xml version="1.0" encoding="UTF-8"?>
<proxy xmlns="http://ws.apache.org/ns/synapse"
       name="200Proxy"
       transports="https,http"
       statistics="disable"
       trace="disable"
       startOnLoad="true">
   <target>
      <inSequence>
         <log level="custom">
            <property name="it" value="** Its Inline Sequence ****"/>
         </log>
         <loopback/>
      </inSequence>
      <outSequence>
         <property name="HTTP_SC" value="200" scope="axis2"/>
         <enrich>
            <source type="inline" clone="true">
               <soapenv:Envelope xmlns:soapenv="http://www.w3.org/2003/05/soap-envelope">
                  <soapenv:Header/>
                  <soapenv:Body>DIMUTHUD</soapenv:Body>
               </soapenv:Envelope>
            </source>
            <target type="envelope"/>
         </enrich>
         <property name="messageType" value="application/soap+xml" scope="axis2"/>
         <header name="To" action="remove"/>
         <property name="RESPONSE" value="true" scope="default" type="STRING"/>
         <send/>
      </outSequence>
   </target>
   <description/>
</proxy>