. . .

Integration Test Structure Recommendation Guide

On this page we want to share with you how we run automated integration tests for our modules on a buildserver. This article explains the current state of our structure and might be changed in future.
We're running the 2.x version of Jenkins on our buildserver, with the Pipeline and Bitbucket Branch Source Plugin and we're using Atlassian Bitbucket for our Source Code Management.

Project structure

Our repositories containing the modules have a specific structure, here exemplary for the SMTP module:

  • / (root directory of the checked out project)

    • /smtp/

    • /smtp-integrationtest/

    • /Jenkinsfile

As you can see, we use the Jenkinsfile feature to define the build pipeline.

Besides that file we have one folder containing the normal content of the module (such as build.xml, src folder, etc.) and one folder named by the module name postfixed with "-integrationtest".

The latter folder contains a simple Maven project, which has no files in the src/main folder and contains only the test classes in the src/test folder. The tests are simple JUnit tests which are doing HTTP-Requests against our ApiOmat REST API. We can provide you our integration-helper library with classes that help you to make common requests against our API (like creating a backend, changing a module's configuration etc.). Feel free to contact us for that. We're also currently planning to make this library available on Maven Central in the future.

To create such a project, you can either create a new Maven project in Eclipse and check the "Create simple project (Skip archetype selection)" box in the Dialog, or you create a project using Maven with the following command and remove the generated Java files afterwards.

Create a new simple project
mvn archetype:generate -DgroupId=com.apiomat.nativemodule.integrationtest -DartifactId=smtp-integrationtest -DarchetypeArtifactId=maven-archetype-quickstart -DinteractiveMode=false

Of course, you have to replace the values for the groupId and artifactId according to your project. After that you can start to write your tests in the src/test folder.

A typical maven dependency configuration would be the following, using our integration-test-helper package. Newset version can be found at Maven Central Repository.

pom.xml
...
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.11</version>
<scope>test</scope>
</dependency>
<!-- contains basic helper methods for interactions with ApiOmat-->
<dependency>
<groupId>com.apiomat.helper</groupId>
<artifactId>integration-test-helper</artifactId>
<version>latest</version>
</dependency>
</dependencies>
...

Build structure

For now, we have two different build jobs for the modules.
The first build job uses our maven integration and runs mvn install -Dant.target=package in the module folder and hence it will run all your simple unit tests contained in the test folder in the root of your module directory (in our example above: /SMTP/test) in the Maven phase test and then create the module jar in the dist folder (in the example /SMTP/dist/smtp.jar). We've set up the Jenkinsfile for that build to publish the created module jar in the end of the process.

The build job item is created as "Bitbucket Team/Project" item and will scan for Jenkinsfiles in all repositories of the specified project on our Bitbucket server. Therefore, every module with a Jenkinsfile will automatically be added to the builds.

Jenkinsfile for the module artifact generation:

Jenkinsfile for the module artifact generation
/* extract the module name from current buildjob */
@NonCPS
def extractModuleName() {
echo ("Extracting module name from job name: ${env.JOB_NAME}")
jobMatcher = "${env.JOB_NAME}".split('/')
ret = jobMatcher[1]
jobMatcher = null
return ret;
}
 
node {
env.PATH = "${tool 'Ant'}/bin:${env.PATH}"
stage ('Preparations') {
properties ([
/* define some build-specific things, like the number of builds to keep, a timer when the build should be scheduled and no concurrent builds */
buildDiscarder(logRotator(artifactDaysToKeepStr: '', artifactNumToKeepStr: '', daysToKeepStr: '', numToKeepStr: '6')),
pipelineTriggers([[ $class: 'hudson.triggers.TimerTrigger', spec : "1 3 * * *"]]),
disableConcurrentBuilds(),
/* we set the permission to copy the artifacts from the second buildjob here */
[$class: 'CopyArtifactPermissionProperty', projectNames: 'ModuleTests']
])
}
 
stage ('Package') {
/* check out the current module, define the modulename and execute the maven task */
checkout scm
def moduleName = extractModuleName()
sh "mvn -B -f ${moduleName}/pom.xml clean install -Dant.target=package"
}
stage ('Archive') {
/* archive the package */
archiveArtifacts artifacts: '**/dist/*.jar'
}
}

The second build copies the produced artifact of the first build to the workspace of the second build, uploads it to the running test instance of ApiOmat, releases it and then runs mvn test for the <modulename>-integrationtest Maven project.

If you're using the integration-helper library in your test cases and added it as a maven dependency to the pom.xml, you have to make sure that you've installed the helper library to your local repository.

The Jenkinsfile for that build is located in another repository and also contains a simple script to deploy the module (including: creating the customer and application if necessary, creating and uploading the module, adding the module to the application, releasing the module and activate the application).

Jenkinsfile for integrationtests:

Jenkinsfile for integrationtests
import jenkins.*
import jenkins.model.*
import hudson.*
import hudson.model.*
 
/* request the jenkins API to get all native module builds to process */
@NonCPS
def getNativeModulesBuilds() {
def moduleNames = [];
/* adapt the values to your needs */
def authString = "jenkinsusername:password".getBytes().encodeBase64().toString()
def url = "http://@buildserver/job/Modules/api/json";
def conn = url.toURL().openConnection();
conn.setRequestProperty( "Authorization", "Basic ${authString}" )
 
if( conn.responseCode == 200 ) {
modules = new groovy.json.JsonSlurperClassic().parseText(conn.content.text);
for (job in modules.jobs) {
moduleNames.add( job.name );
}
versions = null
}
else {
echo( "Something bad happened.")
echo( "url: ${url} - ${conn.responseCode}: ${conn.responseMessage}")
build.status = "FAILURE"
}
conn = null
return moduleNames
}
/* this method deploys the given module and runs the integration tests */
def testModule( moduleName, realModuleName ) {
echo ("testModule $moduleName")
// deploy
step([$class: 'CopyArtifact', filter: '**/*.jar', fingerprintArtifacts: true, flatten: true, target: 'dist',
projectName: "Modules/$moduleName", selector: [$class: 'StatusBuildSelector', stable: true]])
echo ("deploy")
/* you may have to adapt the url to your ApiOmat instance */
sh "./deployModule.sh 'http://localhost:8000' '../target' '$realModuleName'"
 
//test
echo ("folder")
def folderName = "Modules/${moduleName}/${moduleName}-integrationtest"
dir("$folderName") {
/* you may have to adapt the url to your ApiOmat instance */
sh '''mvn -B -Dmaven.test.failure.ignore=true -DyambasHost=http://localhost:8000 -DyambasTestHost=http://localhost:8000 clean integration-test'''
}
}
node {
try {
/* execute all module tests */
stage ('Module Tests') {
sh 'mkdir -p dist'
sh 'mkdir -p Modules'
def moduleNames = getNativeModulesBuilds()
sh 'chmod 777 getUsedModules.sh'
def dependingModuleRealNames = []
def dependingModuleNames = []
def oldSize = 0
while (dependingModuleRealNames.size() < moduleNames.size()) {
oldSize = dependingModuleRealNames.size()
for(moduleName in moduleNames) {
try {
// checkout module, for later tests and real module name
/* you may have to adapt the branch name and url to your repository here */
checkout([$class: 'GitSCM', branches: branches: [[name: '*/master']], extensions: [[$class: 'RelativeTargetDirectory', relativeTargetDir: "Modules/$moduleName"]], userRemoteConfigs: [[url: "ssh://yourrepository/nm/${moduleName}.git"]]])
// get real module name
/* you may have to adapt the path according to the workspace path for that build on your buildserver */
realModuleName = sh returnStdout: true, script: "cat Modules/$moduleName/$moduleName/sdk.properties | grep moduleName | awk -F'=' '{print \$2}'"
realModuleName = realModuleName.trim()
// get used modules
/* you may have to adapt the path according to the workspace path for that build on your buildserver */
def fileName = "Modules/$moduleName/$moduleName/src/com/apiomat/nativemodule/$moduleName/${realModuleName}.java"
usedModulesStr = sh returnStdout: true, script: "./getUsedModules.sh $fileName"
def usedModules = usedModulesStr.split(',')
echo ("Found used modules for module ${moduleName}: '$usedModulesStr'")
contained = true
for(um in usedModules) {
def depModule = um.replaceAll(',','')
if(depModule != "Mandrill" && depModule != "" && depModule.startsWith("documentationURL")==false ) {
contained &= dependingModuleRealNames.contains("$depModule")
}
}
echo ("Is used module contained in list: $contained")
if(contained) {
dependingModuleRealNames.add( "$realModuleName" )
dependingModuleNames.add( "$moduleName" )
}
}
catch(Exception e) {
echo ("Could not use module ${moduleName}: " + e.getMessage())
}
}
if(oldSize == dependingModuleRealNames.size()) {
echo("No further dependency found!")
break
}
}
for(moduleName in dependingModuleNames) {
try {
echo ("Testing Modules/$moduleName")
// get real module name
/* you may have to adapt the path according to the workspace path for that build on your buildserver */
realModuleName = sh returnStdout: true, script: "cat Modules/$moduleName/$moduleName/sdk.properties | grep moduleName | awk -F'=' '{print \$2}'"
realModuleName = realModuleName.trim()
testModule( moduleName, realModuleName, branch)
}
catch(Exception e) {
echo ("Error testing module $realModuleName in branch $branch: " + e.getMessage())
}
}
}
} catch (Exception e) {
echo ("Error while testing the modules: " + e.getMessage())
} finally {
stage ('Test results') {
junit '**/surefire-reports/**/*.xml'
}
}
}

deployModule.sh:

deployModule.sh
#!/bin/bash
echo "starting deployModule.sh..."
 
curlOpts="--write-out %{http_code} --silent --output /dev/null"
# adapt the values according to your needs
customer=yourtestcustomer
adminEmail="$customer@yourcompany.com"
adminPassword="yourpassword"
customerAuth="$adminEmail:$adminPassword"
superadminEmail=yoursuperadmin@yourcompany.com
superadminPassword=yoursuperadminpassword
appName=ModuleTestApplication
hostName=$1
if [ -z "$hostName" ]
then
hostName="http://localhost:8080"
fi
host="$hostName/yambas/rest"
 
outDir=$2
if [ -z "$outDir" ]
then
outDir="./"
fi
 
moduleName="$3"
if [ -z "$3" ]
then
echo -e "No modulename provided!"
exit 1
fi
 
 
echo -e "--- Ready to rumble ---\nHost: $host\nOutDir: $outDir\nModule:$moduleName\n-------------"
 
#check if customer (apps admin) ex
http_code=$(curl $curlOpts -X GET $host/customers/$customer -u $superadminEmail:$superadminPassword -H "Accept:application/json")
echo "Get Customer: $http_code"
if [ $http_code -ne 200 ]
then
#create customer
http_code=$(curl $curlOpts -X POST $host/customers -d name=$customer -d email=$adminEmail -d password=$adminPassword -u $superadminEmail:$superadminPassword)
echo "Create Customer Return: $http_code"
fi
#check if app exists and delete
http_code=$(curl $curlOpts -X GET $host/customers/$customer/apps/$appName -u $customerAuth -H "Accept:application/json")
echo "GET App: $http_code"
if [ $http_code -eq 200 ]
then
http_code=$(curl $curlOpts -X DELETE $host/customers/$customer/apps/$appName -u $customerAuth -H "Accept:application/json")
echo "Deleted App: $http_code"
fi
 
#create app with given name
app_href=$(curl --silent -i -X POST $host/customers/$customer/apps -d name=$appName -u $customerAuth -H "Accept:application/json" | awk '/^Location:/ { print $2 }' | sed -e 's/[[:cntrl:]]//')
if [ -z $app_href ]
then
echo "Can't find APP. abort"
exit 1
fi
#get APP JSON
echo "App Href: $app_href"
app_json=$(curl --silent -X GET $app_href -u $customerAuth -H "Accept:application/json")
#echo "$app_json"
app_name=$(expr "$app_json" : '.*"applicationName":"\([^"]*\)"')
appName=${app_name}
api_key_live=$(echo $app_json | jq -r '.apiKeys.liveApiKey')
echo "API KEY: $api_key_live"
rm -f /tmp/apikey
echo $api_key_live >> /tmp/apikey
 
# set app active
http_code=$(curl $curlOpts -i -X PUT -H "Content-Type: application/json" -H "X-apiomat-system: LIVE" -u $customerAuth -d "{\"applicationStatus\":{\"LIVE\": \"ACTIVE\"}}" $app_href)
if [ $http_code -eq 200 ]
then
echo "$moduleName module released"
else
echo "Can't release $moduleName module: $http_code"
exit 1;
fi
 
# deploy module
if [ ! -e "dist/$moduleName.jar" ]
then
echo "Module file does not exist at dist/$moduleName.jar!"
exit 1;
fi
 
http_code=$(curl $curlOpts -i -X POST -H "Content-Type: application/octet-stream" -u $customerAuth --data-binary @"dist/$moduleName.jar" $host/modules/$moduleName/sdk?update=OVERWRITE)
if [ $http_code -eq 200 ]
then
echo "$moduleName module deployed"
else
echo "Can't deploy $moduleName module: $http_code"
exit 1;
fi
 
# release module
http_code=$(curl $curlOpts -i -X PUT -H "Content-Type: application/json" -H "X-apiomat-system: LIVE" -u $customerAuth -d "{\"releaseState\": \"RELEASED\"}" $host/modules/$moduleName)
if [ $http_code -eq 200 ]
then
echo "$moduleName module released"
else
echo "Can't release $moduleName module: $http_code"
exit 1;
fi

getUsedModules.sh:

getUsedModules.sh
#!/bin/bash
# this script parses the usedModules of a nativemodule directly from the module main file, which path should be passed as argument for this script
cat $1| grep -A 1 'usedModules =' | awk -F 'usedModules =' '{print $2}' ORS=' ' | sed ':a;N;$!ba;s/\n/ /g' | sed s/}.*\$//gI | sed s/{//g | sed s/\)//g | sed s/\(//g | sed s/\"//g | sed s/[[:space:]]//g

Note on Scripts

Please note that these scripts should only give you an idea about how to set up automatic integration tests with a Jenkins buildserver.

They are highly dependant to our own infrastructure and configuration and probably won't work with your setup out of the box, but can save some time for thoughts about the general processes and some pitfalls (e.g. the used modules).