How To Write Your First Greg Integration Test In New WSO2 Test Framework 4.3.0 .....

In this documentation I am going to explain how to write a simple GREG integration test.

Objective :

We will illustrate how to write a test to add schema to greg server and verify the existence of the added / uploaded schema  to the server.  

Note : you can start writing your tests in the following location according to your greg server version :

eg:

/home/xx/svn/platform/branches/turing/products/greg/x.x.x/modules/integration/registry/tests-new/src/test/java/org/wso2/carbon/registry

Steps to  follow :

Refer below table  where I have created SchemaUploadTestCase.java in a more common form for illustration purposes only.

SchemaUploadTestCase.java

package org.wso2.carbon.registry.addresources.test; import org.apache.commons.logging.Log; import org.apache.commons.logging.LogFactory; import org.testng.annotations.BeforeClass; import org.testng.annotations.Test; import org.wso2.carbon.authenticator.stub.LoginAuthenticationExceptionException; import org.wso2.carbon.automation.api.clients.registry.ResourceAdminServiceClient; import org.wso2.carbon.automation.engine.context.AutomationContext; import org.wso2.carbon.automation.utils.registry.RegistryProviderUtil; import org.wso2.carbon.governance.api.schema.SchemaManager; import org.wso2.carbon.governance.api.schema.dataobjects.Schema; import org.wso2.carbon.registry.app.RemoteRegistry; import org.wso2.carbon.registry.core.Registry; import org.wso2.carbon.registry.core.exceptions.RegistryException; import org.wso2.carbon.registry.handler.stub.ExceptionException; import org.wso2.carbon.registry.resource.stub.ResourceAdminServiceExceptionException; import javax.activation.DataHandler; import javax.xml.xpath.XPathExpressionException; import java.io.File; import java.io.IOException; import java.net.URL; import java.rmi.RemoteException; import static org.testng.Assert.assertTrue; public class SchemaUploadTestCase {    private static final Log log = LogFactory.getLog(SchemaUploadTestCase.class);    private AutomationContext automationContext;    private ResourceAdminServiceClient resourceAdminServiceClient;    private RegistryProviderUtil registryProviderUtil = new RegistryProviderUtil();    private  SchemaManager schemaManager;    private String schemaId;    private Registry governance;    @BeforeClass(groups = {"wso2.greg"})    public void init() throws Exception {     // 01        automationContext = new AutomationContext("GREG", "lbw001", "superTenant", "superAdmin");        int userId = 1;        RemoteRegistry registry = registryProviderUtil.getRemoteRegistry(userId, "GREG");        governance = registryProviderUtil.getGovernanceRegistry(registry, userId);        resourceAdminServiceClient = new ResourceAdminServiceClient(getBackendURL(),getSessionCookie());    }   @AfterClass    public void DeleteLCResources() throws GovernanceException {      schemaManager.removeSchema(schemaId); // 07    }    @Test(groups = {"wso2.greg"}, description = "Add new schema")    public void addSchema() throws IOException, ExceptionException, RegistryException, ResourceAdminServiceExceptionException {        String resourceLocation = "xxx/xxx/xxx/xxx";        String filePath = resourceLocation +                File.separator + "schema" + File.separator + "calculator.xsd";        DataHandler dh = new DataHandler(new URL("file:///" + filePath)); // 04        resourceAdminServiceClient.addSchema("Adding My Schema",dh);        SchemaManager schemaManager = new SchemaManager(governance); // 05        Schema[] schemas = schemaManager.getAllSchemas();        boolean resourceFound = false;        for (Schema schema : schemas) {            if (schema.getQName().getLocalPart().equals("calculator.xsd")) {                resourceFound = true;                schemaId = schema.getId();                log.info("Schema ID is : "+ schemaId);            }        }       assertTrue(resourceFound, "Schema Not Found"); // 06    }    private String getBackendURL() throws XPathExpressionException {   // 02        return automationContext.getContextUrls().getBackEndUrl();    }    // 03    private String getSessionCookie() throws LoginAuthenticationExceptionException, XPathExpressionException, RemoteException {        return automationContext.login();    } }

Description :

Prerequisites :

You need to place the schema to be uploaded to the server inside the resourceLocation.

eg:

resourceLocation = "/home/xxx/svn/platform/branches/turing/products/greg/x.x.x/modules/integration/registry/tests-new/src/test/resources/artifacts/GREG/”

01. Create a method called init() and initialize an automationContext object. Pass relevant parameters in accordance with the given values of the automation.xml.

02 & 03 .  Now you can derive the backendservice urls and session cookies as shown in  getBackendURL() & getSessionCookie() methods.

Create instances from ResourceAdminServiceClient & RegistryProviderUtil classes as you may need these instances to add the schema to the server. Note the parameters needs to pass when creating an instance from ResourceAdminServiceClient.

Now start writing your test scenario under @Test annotation. Create a DataHandler from the schema resource file as shown in step 04 and invoke the addResource(...) method of the resourceAdminServiceClient to add / upload the schema to the server.

05 shows the steps involved in verifying the schema added to the server. In here by calling an instance from the  SchemaManager class I can seek for the schema uploaded and if the schema was successfully uploaded assert statement shown in 06 will not encounter a failure.

@AfterClass you can remove the added schema as shown in 07 by invoking removeSchema(...) method using schemamanager instance.

Thats all. You can now start writing numerous testing scenarios using SchemaUploadTestCase .java as a template / guidance.

The main purpose of the DistributionValidationTest.java class is to validate contents inside a product distribution. This post will illustrate the main aspects of distribution validation test class and some related information.

Objective

There are seven major distribution validation aspects being covered from  DistributionValidationTest.java class. These are,

1.Validation of jar files inside the product distribution against the LICENSE.txt.

2.Recursive scan to find the occurrence of SNAPSHOT keyword in file names and contents of textual files( eg : .txt, .xsd, etc).

3.Validate duplicate .jar entries inside the  LICENSE.txt  file.

4.Check whether maven variables are properly replaced inside configuration files found in

repository/conf directory.

5.Validate product distribution size against the latest released distribution in the maven repo

6.Identification of duplicate jars inside the distribution pack.

7.Product specific checks - validate sample directory - versions, resources validation

** There are several distinct test cases being developed to cover above major aspects and based on the findings/outputs under each category a summarized report will be generated with all the information included.  You can navigate to .. /target/DistributionValidationTestReport.txt to find the report.

Prerequisites

• XSD (XML Schema Definition) files for configuration files (eg: carbon.xml , axis2.xml) inside repository/conf.  - You can generate these schemas using on-line XSD generators.
• Latest released version of the product distribution being hosted in maven-repo.

Description

1. Validation of jar files inside the product distribution against the LICENSE.txt

           - The implemented test case is capable of identifying the .jar files which resides inside the xxx/repository/components/plugins directory against .jar file names mentioned in the carbon_home/LICENSE.txt file.  Firstly it will derive all .jar file locations inside the plugins directory and match these against the LICENSE.txt file to identify missing jars.  

2. Recursive scan for occurrence of SNAPSHOT keyword in file names and contents in textual files (.txt, .xml, etc).

           - The test will perform a recursive scan in-order to identify occurrences of the keyword “SNAPSHOT” inside the textual files and in their names. If an occurrence/s found the test will encounter a failure at the relevant assert statement.

3. Validate duplicate entries in LICENSE.txt file.

           - Test case will search for duplicate  .jar file names (If the same name is appearing more than once) in the LICENSE.txt file. These file names will be stored using a collection interface and finally the test case will output these .jar file names to the report.

4. Checking whether Maven variables are properly replaced inside configuration files found inside repository/conf

          - The test case will check for maven variable replacements by performing xml schema validation techniques. The user of the test case need to provide the xsd (XML Schema Definition) files of corresponding xml’s need to be checked. Mainly the test case will focus on the ’ $ ’ marks (these are the tags expected to update at the build time) in the XSD files and validate the relevant XML file for updation of each tag attribute value.

5. Validate product distribution size against the latest released distribution in the maven repo

         - The intention of this test case is to validate the running distribution size against the latest released distribution pack’s size. The test case will access the relevant maven repo and seek for the latest distribution hosted in it. Next It will retrieve the size of the distribution hosted and evaluate it with the running distribution’s size.

(Note: you need to make sure once a released been done , the distribution pack should be hosted in the repo. Size comparison will be based on pre-accepted value range for all wso2 products.

6. Identification of duplicate jars inside the distribution pack.

        - Test case is designed using a special regular expressions to derive the version of every .jar file and identification of duplicate .jars inside the pack. Based on the findings of the test case .jar file locations will be printed in the report file under two categories namely,  duplicate jar files in the same directory and with different versions inside the distribution.

7. Product specific checks for whether samples/resources are properly packaged

           - For this purpose the test case will perform some specific scans inside each samples directory and verification will be carried out to confirm that top most level sample directories does not carry any versions with their names. In addition the test case is capable of identifying k whether samples directory (if exists) has a README.txt/pom.xml/build.xml at the same level where src directory exists and to check whether resources directory (inside the src) contains any files.   

In a Nutshell .... At a Glance .....

Enabling wire logs for wso2-apimgr 

1 . To enable wire logs you need to go to wso2am-1.x.0/repository/conf/log4j.properties file.

2. Uncomment the following entry.
log4j.logger.org.apache.synapse.transport.http.wire=DEBUG

3. Now restart the server.

4. Send your request.

5. Now go to wso2am-1.x.0/repository/logs/wso2carbon.log file.

6. Identify the message flow,

    Form the wire to api-mger

    DEBUG {org.apache.synapse.transport.http.wire} -  >> "POST /t/dimuthu.com/1.0 HTTP/1.1[\r
     DEBUG {org.apache.synapse.transport.http.wire} -  >> "Host: xx.xx.x.xx:8243[\r][\n]" {org.apache.synapse.transport.http.wire}  
     .................
     .................

    From api-mger to the wire

    DEBUG {org.apache.synapse.transport.http.wire} -  << "POST /user/ HTTP/1.1[\r][\n]" {org.apache.synapse.transport.http.wire}
     DEBUG {org.apache.synapse.transport.http.wire} - << "Host: yy.yy.yy.yy:9000[\r][\n]" {org.apache.synapse.transport.http.wire} 
    ..............
    ..............

Note : The best way to identify the message flow is to consider the host ip.

WSO2 ESB --> OUT_ONLY property

   <inSequence>

                <property action="set" name="OUT_ONLY" value="true"/>

   </inSequence>

 

 Now this means that you are not expecting any response.

Selenium 

A sample java-selenuim code to upload an artifact (Helloword.aar) on your desktop to wso2 application server instance. Note that we will use firefox browser for the demonstration.

package selenium;

import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.firefox.FirefoxDriver;

import static org.testng.Assert.assertEquals;

public class AppserverArtifactUpload {

    private static final Log log = LogFactory.getLog(AppserverArtifactUpload.class);

    public static void main(String[] args) {

        // Initializing the web driver
        WebDriver driver = new FirefoxDriver();

        // Setting the url
        driver.get("https://ip:port/carbon/"); 
        // your server ip & port (eg: ip-100.100.100.100 , port- 9443)

        // Creating username & password fields
        WebElement userName = driver.findElement(By.name(("username")));
        WebElement password = driver.findElement(By.name(("password")));

        // Setting values for username & password
        userName.sendKeys("admin");
        password.sendKeys("admin");

        // Now submit the form. WebDriver
        password.submit();

        // Check the title of the page
        log.info("Page Title : " + driver.getTitle());
        assertEquals("WSO2 Management Console", driver.getTitle(), "Page Title Mismatch");

        // Traveling to AAR upload page
        driver.findElement(By.xpath("/html/body/table/tbody/tr[2]/td[2]/table/tbody/tr/td/div/ul/li[5]/ul/li[4]/ul/li[3]/ul/li/a")).click();

        /*// if u want to add additional browser buttons
        driver.findElement(By.cssSelector("input[type='button']")).click();*/

        // browsing and selecting the .aar file
        driver.findElement(By.xpath("/html/body/table/tbody/tr[2]/td[3]/table/tbody" +
                "/tr[2]/td/div/div/form/table/tbody/tr[2]/td/input"))
                .sendKeys("/home/dimuthu/Desktop/HelloWorld.aar");

        // uploading the selected aar file
        WebElement upload = driver.findElement(By.name(("upload")));
        upload.submit();

        log.info("HelloWorld.aar uploaded successfully");
    }
}

Identifying OSGI components in server startup .......

For this post I will publish a code snippet for identifying unsatisfied osgi bundles. 

import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.apache.commons.net.telnet.TelnetClient;
import org.testng.annotations.AfterClass;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.Test;
import org.wso2.carbon.automation.core.MultipleServersManager;

import java.io.*;
import java.net.InetAddress;
import java.net.SocketTimeoutException;
import java.util.ArrayList;
import java.util.HashMap;

import static org.testng.Assert.assertTrue;

/*
  This class can be used to identify required osgi component service components (eg: unsatisfied) in server startup
 */
public class OSGIServerBundleStatusTest {

    private static final Log log = LogFactory.getLog(OSGIServerBundleStatusTest.class);

    private static int telnetPort = 2000;
    private TelnetClient telnet = new TelnetClient();
    private ArrayList<String> arrList = new ArrayList<String>();
    private ArrayList<String> unsatisfiedList = new ArrayList<String>();
    private HashMap<String, String> serverPropertyMap = new HashMap<String, String>();
    private MultipleServersManager manager = new MultipleServersManager();

    @BeforeClass(alwaysRun = true)
    public void init() throws Exception {

        serverPropertyMap.put("-DportOffset", "1"); // to start the server from a different port offset
        serverPropertyMap.put("-DosgiConsole", Integer.toString(telnetPort)); // start with osgicomponentservice
        CarbonTestServerManager server = new CarbonTestServerManager(System.getProperty("carbon.zip"),
                serverPropertyMap);
        manager.startServers(server);
    }

    @AfterClass(alwaysRun = true)
    public void stopServers() throws Exception {
        disconnect();  // telnet disconnection
        // manager.stopAllServers();
    }

    @Test(groups = "wso2.as", description = "Identifying and storing unsatisfied OSGI components")
    public void testOSGIUnsatisfiedComponents() throws Exception {

        telnet.connect(InetAddress.getLocalHost().getHostAddress(), telnetPort);
        telnet.setSoTimeout(10000);

        ArrayList<String> arr = retrieveUnsatisfiedComponentsList("ls");

        for (int x = 0; x < arr.size(); x++) {
            unsatisfiedList.add(arrList.get(x).split("\t")[3]);
            log.info(unsatisfiedList.get(x));
        }


        assertTrue(unsatisfiedList.size() > 0);  // TODO - Will vary according to the requirement
    }

    private ArrayList<String> retrieveUnsatisfiedComponentsList(String command) throws IOException {
        writeInputCommand(command);
        try {
            readResponse();
        } catch (SocketTimeoutException e) {
            log.error("Socket timeout Exception "  + e);
        }
        return arrList;
    }

    private void writeInputCommand(String value) {

        PrintStream out = new PrintStream(telnet.getOutputStream());
        out.println(value);
        out.flush();
        log.info(value);
    }

    private void readResponse() throws IOException {

        InputStream in = telnet.getInputStream();
        BufferedReader inBuff = new BufferedReader(new InputStreamReader(in));
        String inputLine = inBuff.readLine();

        while (!inputLine.equals("")) {
            if (inputLine.contains("Unsatisfied")) {  // filtering Unsatisfied components
                arrList.add(inputLine);
            }
            inputLine = inBuff.readLine();
            log.info(inputLine);
        }
    }

    private void disconnect() throws IOException {
        telnet.disconnect();
        log.info("Telnet connection is disconnected");
    }
}


import java.io.IOException;
import java.util.HashMap;

public class CarbonTestServerManager extends TestServerManager {

    public CarbonTestServerManager() {
    }

    public CarbonTestServerManager(String carbonZip, HashMap<String, String> startupParameterMap) {
        super(carbonZip, startupParameterMap);
    }

    public CarbonTestServerManager(int portOffset) {
        super(portOffset);
    }

    public String startServer() throws IOException {
        String carbonHome = super.startServer();
        System.setProperty("carbon.home", carbonHome);
        return carbonHome;
    }

    public void stopServer() throws Exception {
        super.stopServer();
    }

    protected void copyArtifacts(String carbonHome) throws IOException {
    }
}

Java Tips .....

To get directory names inside a particular directory ....

private String[] getDirectoryNames(String path) {

        File fileName = new File(path);
        String[] directoryNamesArr = fileName.list(new FilenameFilter() {
            @Override
            public boolean accept(File current, String name) {
                return new File(current, name).isDirectory();
            }
        });
        log.info("Directories inside " + path + " are " + Arrays.toString(directoryNamesArr));
        return directoryNamesArr;
    }


To retrieve links on a web page ......

 private List<String> getLinks(String url) throws ParserException {
        Parser htmlParser = new Parser(url);
        List<String> links = new LinkedList<String>();

        NodeList tagNodeList = htmlParser.extractAllNodesThatMatch(new NodeClassFilter(LinkTag.class));
        for (int x = 0; x < tagNodeList.size(); x++) {
            LinkTag loopLinks = (LinkTag) tagNodeList.elementAt(m);
            String linkName = loopLinks.getLink();
            links.add(linkName);
        }
        return links;
    }


To search for all files in a directory recursively from the file/s extension/s ......

private List<String> getFilesWithSpecificExtensions(String filePath) throws ParserException {

// extension list - Do not specify "." 
 List<File> files = (List<File>) FileUtils.listFiles(new File(filePath),
                new String[]{"txt"}, true);

        File[] extensionFiles = new File[files.size()];

        Iterator<File> itFileList = files.iterator();
        int count = 0;

        while (itFileList.hasNext()) {
            File filePath = itFileList.next();
            extensionFiles[count] = filePath;
            count++;
        }
        return extensionFiles;
} 

Reading files in a zip

     public static void main(String[] args) throws IOException {
        final ZipFile file = new ZipFile("Your zip file path goes here");
        try
        {
            final Enumeration<? extends ZipEntry> entries = file.entries();
            while (entries.hasMoreElements())
            {
                final ZipEntry entry = entries.nextElement();
                System.out.println( "Entry "+ entry.getName() );
                readInputStream( file.getInputStream( entry ) );
            }
        }
        finally
        {
            file.close();
        }
    }
        private static int readInputStream( final InputStream is ) throws IOException {
            final byte[] buf = new byte[8192];
            int read = 0;
            int cntRead;
            while ((cntRead = is.read(buf, 0, buf.length) ) >=0)
            {
                read += cntRead;
            }
            return read;
        } 


Converting Object A to Long[]

 long [] myLongArray = (long[])oo;
        Long myLongArray [] = new Long[myLongArray.length];
        int i=0;

        for(long temp:myLongArray){
            myLongArray[i++] = temp;
        } 

Getting cookie details on HTTP clients

import org.apache.http.impl.client.DefaultHttpClient;

HttpClient httpClient = new DefaultHttpClient();

((DefaultHttpClient) httpClient).getCookieStore().getCookies(); 

 HttpPost post = new HttpPost(URL);
        post.setHeader("User-Agent", USER_AGENT);
        post.addHeader("Referer",URL );
        List<NameValuePair> urlParameters = new ArrayList<NameValuePair>();
        urlParameters.add(new BasicNameValuePair("username", "admin"));
        urlParameters.add(new BasicNameValuePair("password", "admin"));
        urlParameters.add(new BasicNameValuePair("sessionDataKey", sessionKey));
        post.setEntity(new UrlEncodedFormEntity(urlParameters));
        return httpClient.execute(post);


Ubuntu Commands

1. Getting the process listening to a given port (eg: port 9000) 

sudo netstat -tapen | grep ":9000 "


Running  a bash script from python script

shell.py
-----------

import os

def main():

    os.system("sh hello.sh")

if __name__=="__main__":
 os.system("sh hello.sh")


hello.sh
-----------
#Linux shell Script


echo "Hello Python from Shell";

public void scriptExecutor() throws IOException {

    log.info("Start executing the script to trigger the docker build ... ");

    Process p = Runtime.getRuntime().exec(
            "python  /home/dimuthu/Desktop/Python/shell.py ");
    BufferedReader in = new BufferedReader(new InputStreamReader(
            p.getInputStream()));
    log.info(in.readLine());

    log.info("Finished executing the script to trigger the docker build ... ");

}