. . .

Integration Test Structure Recommendation Guide

On this page we want to share with you how we run automated integration tests for our modules on a buildserver. This article explains the current state of our structure and might be changed in future.
We're running the 2.x version of Jenkins on our buildserver, with the Pipeline and Bitbucket Branch Source Plugin and we're using Atlassian Bitbucket for our Source Code Management.

Project structure

Our repositories containing the modules have a specific structure, here exemplary for the SMTP module:

  • / (root directory of the checked out project)

    • /smtp/

    • /smtp-integrationtest/

    • /Jenkinsfile

As you can see, we use the Jenkinsfile feature to define the build pipeline.

Besides that file we have one folder containing the normal content of the module (such as src folder, pom.xml , etc.) and one folder named by the module name postfixed with "-integrationtest".

The latter folder contains a simple Maven project, which has no files in the src/main folder and contains only the test classes in the src/test folder. The tests are simple JUnit tests which are doing HTTP-Requests against our ApiOmat REST API. We can provide you our integration-helper library with classes that help you to make common requests against our API (like creating a backend, changing a module's configuration etc.). Feel free to contact us for that. The integration-helper is available on Maven Central.

If you're using the integration-helper library in your test cases and added it as a maven dependency to the pom.xml, you have to make sure that you've installed the helper library to your local repository.

To create such a project, you can either create a new Maven project in Eclipse and check the "Create simple project (Skip archetype selection)" box in the Dialog, or you create a project using Maven with the following command and remove the generated Java files afterwards.

Create a new simple project
mvn archetype:generate -DgroupId=com.apiomat.nativemodule.integrationtest -DartifactId=smtp-integrationtest -DarchetypeArtifactId=maven-archetype-quickstart -DinteractiveMode=false

Of course, you have to replace the values for the groupId and artifactId according to your project. After that you can start to write your tests in the src/test folder.

Build structure

For now, we have two different build jobs for the modules.

The Module Build Job

The first build job uses our maven integration and runs mvn clean package in the module folder and hence it will run all your simple unit tests contained in the test folder in the root of your module directory (in our example above: /SMTP/test) in the Maven phase test and then create the module jar in the target folder (in the example /SMTP/target/smtp-1.0.0-NM.jar). We've set up the Jenkinsfile for that build to publish the created module jar in the end of the process.

The build job item is created as "Bitbucket Team/Project" item and will scan for Jenkinsfiles in all repositories of the specified project on our Bitbucket server. Therefore, every module with a Jenkinsfile will automatically be added to the builds.

Our default Jenkinsfile downloads a template, which is publicly available on github, and loads it afterwards.

Jenkinsfile for the module artifact generation:

Jenkinsfile for the module artifact generation
node {
stage ( 'Checkout' ) {
checkout scm
}
sh 'wget -O Jenkinsfile.template https://raw.githubusercontent.com/ApinautenGmbH/ModuleJenkinsfileTemplate/master/Jenkinsfile.template'
load 'Jenkinsfile.template'
}

This template currently contains an additional Test stage where the implemeneted integrationtests of your project structure were used to run against an instance of ApiOmat Yambas.

The Modules Test Job

Another build copies the produced artifact of all modules (see module build job) to the workspace of the modules test build. After that all modules are uploaded and released to the running test instance of ApiOmat. Then the mvn test command will be executed for the <modulename>-integrationtest Maven project.

The Jenkinsfile for that build is located in another repository and also contains a simple script to deploy the module (including: creating the customer and application if necessary, creating and uploading the module, adding the module to the application, releasing the module and activate the application).

Jenkinsfile for integrationtests:

... main file that contains the definition of this build job and orchestrates all needed scripts and results.

Jenkinsfile for centralized integrationtests
import jenkins.*
import jenkins.model.*
import hudson.*
import hudson.model.*
 
/* request the jenkins API to get all native module builds to process */
@NonCPS
def getNativeModulesBuilds()
{
def moduleNames = [];
def authString = "jenkinsusername:password".getBytes().encodeBase64().toString()
def url = "http://@buildserver/job/Modules/api/json";
def conn = url.toURL().openConnection();
conn.setRequestProperty( "Authorization", "Basic ${authString}" )
if( conn.responseCode == 200 )
{
modules = new groovy.json.JsonSlurperClassic().parseText( conn.content.text );
for ( job in modules.jobs )
{
moduleNames.put( job.name, dir );
}
echo( "Found module names " + moduleNames )
versions = null
}
else
{
echo( "Something bad happened." )
echo( "url: ${url} - ${conn.responseCode}: ${conn.responseMessage}" )
build.status = "FAILURE"
}
conn = null
return moduleNames
}
 
/* this method deploys the given module and runs the integration tests */
def testModule( moduleName, realModuleName )
{
echo ( "testModule $moduleName" )
 
// deploy
step( [$class: 'CopyArtifact', filter: '**/*.jar', fingerprintArtifacts: true, flatten: true, target: 'dist',
projectName: "Modules/$moduleName", selector: [$class: 'StatusBuildSelector', stable: true]] )
 
/* you may have to adapt the url to your ApiOmat instance */
exitCode = sh returnStatus: true, script: "./deployModule.sh 'http://localhost:8000' '$realModuleName'"
if( exitCode != 0 )
{
throw new Exception( "Failed to deploy module $realModuleName" )
}
 
// test
def folderName = "Modules/${moduleName}/${moduleName}-integrationtest"
dir( "$folderName" )
{
/* you may have to adapt the url to your ApiOmat instance */
sh returnStatus: true, script: "mvn -B -Dmaven.test.failure.ignore=true -DyambasHost=http://localhost:8000 -DyambasTestHost=http://localhost:8000 clean integration-test"
}
}
 
stage 'Selector'
properties ([
buildDiscarder(logRotator(artifactDaysToKeepStr: '', artifactNumToKeepStr: '', daysToKeepStr: '', numToKeepStr: '20')),
pipelineTriggers([]),
parameters(
[
/* define your build parameters here */
]
)
])
 
node
{
try
{
stage ( 'Setup VM' )
{
/* do your VM setup here where you boot up your yambas testing instance */
}
 
stage ( 'Module Tests' )
{
sh 'mkdir -p dist'
sh 'mkdir -p Modules'
def moduleNames = getNativeModulesBuilds()
sh 'chmod 777 getUsedModulesFromPom.sh'
sh 'chmod 777 getRealModuleName.sh'
 
def dependingModuleRealNames = []
def dependingModuleNames = []
def oldSize = 0
 
while ( dependingModuleRealNames.size() < moduleNames.size() )
{
oldSize = dependingModuleRealNames.size()
 
for( moduleName in moduleNames )
{
if( dependingModuleNames.contains( "${moduleName}" ) )
{
echo ( "Module already in list: ${moduleName}" )
continue;
}
try
{
// checkout module, for later tests and real module name
/* you may have to adapt the branch name and url to your repository here */
checkout([$class: 'GitSCM', branches: branches: [[name: '*/master']],
extensions: [[$class: 'RelativeTargetDirectory', relativeTargetDir: "Modules/$moduleName"]],
userRemoteConfigs: [[url: "ssh://yourrepository/nm/${moduleName}.git"]]])
// get real module name
/* you may have to adapt the path according to the workspace path for that build on your buildserver */
realModuleName = sh returnStdout: true, script: "./getRealModuleName.sh 'Modules/${moduleName}/${moduleName}/src/com/apiomat/nativemodule/${moduleName}' '${moduleName}'"
 
// get used modules
def fileName = ""
def fileName = "Modules/${moduleName}/${moduleName}/pom.xml"
usedModulesStr = sh returnStdout: true, script: "./getUsedModulesFromPom.sh $fileName"
echo ( "Found used modules for module ${moduleName}: '$usedModulesStr'" )
def usedModules = usedModulesStr.split( ',' )
 
// look if depended modules are already added to the list
contained = true
for( um in usedModules )
{
def depModule = um.replaceAll( ',', '' )
if( depModule != "Mandrill" && depModule != "" && depModule.startsWith( "documentationURL" ) == false )
{
contained &= dependingModuleRealNames.any{it.toLowerCase().contains( "$depModule".toLowerCase() )}
}
}
 
echo ( "Are all used modules already contained in list? Result: $contained" )
if( contained )
{
dependingModuleRealNames.add( "$realModuleName" )
dependingModuleNames.add( "${moduleName}" )
}
}
catch( Exception e )
{
echo ( "Could not use module ${moduleName}: " + e.getMessage() )
}
}
 
if( oldSize == dependingModuleRealNames.size() )
{
echo( "No further dependency found!" )
break
}
}
 
echo ("=================================================================================");
echo ("Testing modules: ${dependingModuleNames}")
echo ("=================================================================================");
for( moduleName in dependingModuleNames )
{
try
{
echo ( "Testing Modules/$moduleName" )
// get real module name
/* you may have to adapt the path according to the workspace path for that build on your buildserver */
realModuleName = sh returnStdout: true, script: "./getRealModuleName.sh 'Modules/$moduleName/$moduleName/src/com/apiomat/nativemodule/$moduleName' '$moduleName'"
 
testModule( moduleName, realModuleName )
}
catch( Exception e )
{
currentBuild.result = 'UNSTABLE'
echo ( "Error testing module $realModuleName: " + e.getMessage() )
}
}
}
}
catch ( Exception e )
{
echo ( "Error while testing the modules: " + e.getMessage() )
}
finally
{
stage ( 'Cleanup VM' )
{
/* clean up your created vm */
}
stage ( 'Test results' )
{
junit '**/surefire-reports/**/*.xml'
}
}
}

deployModule.sh:

... is used to create customer and application such aus to deploy and to release the module

deployModule.sh
#!/bin/bash
echo "starting deployModule.sh..."
 
curlOpts="--write-out %{http_code} --silent --output /dev/null"
# adapt the values according to your needs
customer=yourtestcustomer
adminEmail="$customer@yourcompany.com"
adminPassword="yourpassword"
customerAuth="$adminEmail:$adminPassword"
superadminEmail=yoursuperadmin@yourcompany.com
superadminPassword=yoursuperadminpassword
appName=ModuleTestApplication
hostName=$1
if [ -z "$hostName" ]
then
hostName="https://localhost:8080"
fi
host="$hostName/yambas/rest"
 
outDir=$2
if [ -z "$outDir" ]
then
outDir="./"
fi
 
moduleName="$3"
if [ -z "$3" ]
then
echo -e "No modulename provided!"
exit 1
fi
 
 
echo -e "--- Ready to rumble ---\nHost: $host\nOutDir: $outDir\nModule:$moduleName\n-------------"
 
#check if customer (apps admin) ex
http_code=$(curl $curlOpts -X GET $host/customers/$customer -u $superadminEmail:$superadminPassword -H "Accept:application/json")
echo "Get Customer: $http_code"
if [ $http_code -ne 200 ]
then
#create customer
http_code=$(curl $curlOpts -X POST $host/customers -d name=$customer -d email=$adminEmail -d password=$adminPassword -u $superadminEmail:$superadminPassword)
echo "Create Customer Return: $http_code"
fi
#check if app exists and delete
http_code=$(curl $curlOpts -X GET $host/customers/$customer/apps/$appName -u $customerAuth -H "Accept:application/json")
echo "GET App: $http_code"
if [ $http_code -eq 200 ]
then
http_code=$(curl $curlOpts -X DELETE $host/customers/$customer/apps/$appName -u $customerAuth -H "Accept:application/json")
echo "Deleted App: $http_code"
fi
 
#create app with given name
app_href=$(curl --silent -i -X POST $host/customers/$customer/apps -d name=$appName -u $customerAuth -H "Accept:application/json" | awk '/^Location:/ { print $2 }' | sed -e 's/[[:cntrl:]]//')
if [ -z $app_href ]
then
echo "Can't find APP. abort"
exit 1
fi
#get APP JSON
echo "App Href: $app_href"
app_json=$(curl --silent -X GET $app_href -u $customerAuth -H "Accept:application/json")
#echo "$app_json"
app_name=$(expr "$app_json" : '.*"applicationName":"\([^"]*\)"')
appName=${app_name}
api_key_live=$(echo $app_json | jq -r '.apiKeys.liveApiKey')
echo "API KEY: $api_key_live"
rm -f /tmp/apikey
echo $api_key_live >> /tmp/apikey
 
# set app active
http_code=$(curl $curlOpts -i -X PUT -H "Content-Type: application/json" -H "X-apiomat-system: LIVE" -u $customerAuth -d "{\"applicationStatus\":{\"LIVE\": \"ACTIVE\"}}" $app_href)
if [ $http_code -eq 200 ]
then
echo "$moduleName module released"
else
echo "Can't release $moduleName module: $http_code"
exit 1;
fi
 
# deploy module
if [ ! -e "dist/$moduleName.jar" ]
then
echo "Module file does not exist at dist/$moduleName.jar!"
exit 1;
fi
 
http_code=$(curl $curlOpts -i -X POST -H "Content-Type: application/octet-stream" -u $customerAuth --data-binary @"dist/$moduleName.jar" $host/modules/$moduleName/sdk?update=OVERWRITE)
if [ $http_code -eq 200 ]
then
echo "$moduleName module deployed"
else
echo "Can't deploy $moduleName module: $http_code"
exit 1;
fi
 
# release module
http_code=$(curl $curlOpts -i -X PUT -H "Content-Type: application/json" -H "X-apiomat-system: LIVE" -u $customerAuth -d "{\"releaseState\": \"RELEASED\"}" $host/modules/$moduleName)
if [ $http_code -eq 200 ]
then
echo "$moduleName module released"
else
echo "Can't release $moduleName module: $http_code"
exit 1;
fi

getRealModuleName.sh:

... is used to retrieve the correct case-sensitive name of the module

getRealModuleName.sh
#!/bin/bash
 
# 1. read file list from directory ($1=directory)
# 2. find module class by ignoring case ($2=moduleName)
# 3. remove trailing file ending
# 4. delete last linebreak
ls "$1" | grep -i "${2}.java" | sed s/\.java//g | tr -d '\n'

getUsedModulesFromPom.sh:

... is used to retrieve all used module relations from pom.xml of the module

getUsedModulesFromPom.sh
#!/bin/bash
 
# 1. read in pom.xml
# 2. search for all groupid com.apiomat.nativemodule and get next line with artifact
# 3. trim trailing an leading spaces
# 4. remove artifact tags
# 5. remove first line because it is the parent module itself
# 6. replace linebreaks by commas
# 7. delete last linebreak
cat "$1" | awk '/<groupId>com.apiomat.nativemodule<\/groupId>/{getline; print}' | awk '{$1=$1};1' | sed s/\<artifactId\>//g | sed s+\</artifactId\>++g | tail -n +2 | sed 'H;1h;$!d;x;y/\n/,/' | tr -d '\n'

Note on Scripts

Please note that these scripts should only give you an idea about how to set up automatic integration tests with a Jenkins buildserver.

They are highly dependant to our own infrastructure and configuration and probably won't work with your setup out of the box, but can save some time for thoughts about the general processes and some pitfalls (e.g. the used modules).