Axis C++ Ant Test Framework

Introduction
Conventions
Running the Tests
Creating new tests
How the Framework Works
Test Properties Files
Conversions
Handlers
Dynamic Clients
TCP Monitor Utility
Files Specific to Ant Test Framework
Files Required by Ant Test Framework

Introduction

This document describes the Apache Ant based test framework used for testing the Apache Axis C++ Web Services project. Please refer to the antbuild-guide.html for information about Ant versions and pre-requisites.

The entire Axis C++ source tree is not required to run the test suite, but all the files in ws-axis/c/build and ws-axis/c/tests/auto_build/testcases directories are required. The product must be installed on the test machine. Please refer to the lininstall-guide.html for Linux install instructions or the wininstall-guide.html for Windows install instructions.

Conventions

• The AXISCPP_DEPLOY directory, the directory where the product is installed into, will be referred to as the PACKAGE directory.
• The AXISCPP_HOME/ws-axis/c will be referred to as the BASEDIR.
• The XML parser library location will be referred to as XMLPARSER_HOME. This is the directory with the libraries, e.g. $XERCES_HOME/lib.

Running the Tests

Each test has a property file and all the property files are located in the BASEDIR/tests/auto_build/testcases/tests directory. By default all the tests which have a property file in this directory will be run, but it is possible to define a subset of tests to run or to run an individual test.

In order for the tests to run, the location of the XML parser libraries, XMLPARSER_HOME needs to be specified, and the install location of the Axis C++ package needs to be specified. These can either be specified on the command line or added to the platform specific properties files.

To run all tests type:

ant -f test.xml -Ddir.xmlParser=XMLPARSER_HOME -Ddir.release=PACKAGE

To run a sub-set of tests a file with the name of the test property files to be used for the test run. The location of the file needs to be passed to the Ant script. If a relative path is specified then it must be relevant to the BASEDIR directory. e.g. to run the subset of tests specified in a file called tests_to_run in the BASEDIR directory run:

ant -f test.xml -Ddir.xmlParser=XMLPARSER_HOME -Ddir.release=PACKAGE -Dtest.list=tests_to_run

To run an individual test specify the name of the test property file. e.g. to run only the CalculatorDoc test run:

ant -f test.xml -Ddir.xmlParser=XMLPARSER_HOME -Ddir.release=PACKAGE -Dtest.name=CalculatorDoc

Creating new tests

When creating a new test at least 3 files must be added to the test framework.

1. test client implementation code
• This is the client program that uses the generated stubs and has the main() function. This must be located in the BASEDIR/tests/auto_build/testcases/client/<lang> directory.
2. expected output file
• The output from the client is compared against this file to determine success or failure. This must be located in the BASEDIR/tests/auto_build/testcases/output directory.
3. test property file
• This is the XML property file that defines the location of the other required files. This must be located in the BASEDIR/tests/auto_build/testcases/tests directory.

Additionally the following files may also be added

4. test service WSDL
• If a new service has been created for the test then the WSDL for the new service will need to be added to the framework. This must be located in the BASEDIR/tests/auto_build/testcases/wsdls directory.
5. expected request file
• If a test modifies the HTTP or SOAP header, then the request message should be checked on-the-wire to make sure that it has been modified as expected. The TCP monitor utility captures the message on the wire and writes it to a file which will be compared to the expected request file. This must be located in the BASEDIR/tests/auto_build/testcases/output directory.
6. handler directory structure
• If a test requires 1 or more handlers, then the code for the handlers must be located under the BASEDIR/tests/auto_build/testcases/handlers directory. (See Handlers).

How the Framework Works

When the tests are run the each test is compiled into the test output directory. This directory is created in the directory structure pointed to by the dir.obj property in the build.xx.properties files. Under the dir.obj directory the following directory structure is created.

	
	test/generated*
	test/generated/etc*
	test/generated/<lang>
	test/generated/<lang>/<test name>
	
	

test/generated

The results.log file for the test run is created in this directory. For Windows the axiscpp.conf file is also created in this directory.

test/generated/etc

For non-windows platforms the axiscpp.conf file is created in this directory.

test/generated/<lang>

To easily differentiate between the possible languages of the test client a directory is created to indicate the language of the test. Valid values are c or cpp.

test/generated/<lang>/<test name>

Each test has its own output directory.

• The generated stubs are created in this directory.
• The client test implementation code is copied here.
• The test is run from here
• The output from the test is redirected to files in this directory, <test name>.out and <test name>.err
• The TCP monitor utility output files tcpm.req and tcpm.res are created here.
• The trace output file for the current test is created here.

After initializing and compiling the TCP monitor utility, for each test property file in the list of tests to run (See Running Tests the framework will:

• generated the stubs form the specified WSDL
• copy the client implementation code
• compile the test client
• generate the axiscpp.conf configuration file
• compile any handlers required, generate the client.wsdd file and update the axiscpp.conf configuration file
• copy the expected output and expected request files
• modify the test endpoint
• start the monitor and write a copy of the request and response
• execute the test, capturing stdout and stderr
• stop the monitor
• compare the expected and actual outputs
• compare the expected and actual requests
• write the results to the results.log file

Test Properties Files

The test property files are pseudo XML documents that define all the required elements to run a test (See Creating new tests). A basic example of the properties file can be seen in example 1 which has all the mandatory elements.

    <test>
       <name>CalculatorDoc</name>
       <description>CalculatorDoc</description>
       <clientLang>cpp</clientLang>
       <clientCode>CalculatorDocClient.cpp</clientCode>
       <wsdl>CalculatorDoc.wsdl</wsdl>
       <expected>
           <output>
                CalculatorDoc.cpp.out
           </output>
       </expected>
       <endpoint>http://localhost:80/Calculator/services/Calculator</endpoint>
    </test>
    

example 1

Each of the XML tags in example 1 are detailed below.

<name>

This is the name of the test being run. This should be unique within all the test properties files listed in the BASEDIR/tests/auto_build/testcases/tests directory. The name is used as the name of the test client that is built, the name of the test output directory, and the name of the output files created as part of the test run.

<description>

This is for information only, but could be used for documentation purposes.

<clientLang>

This is the language of the client. Valid values are c and cpp. This is used to determine which directory the client implementation code is located in.

<clientCode>

This is the name of the source code of the client test implementation.

<wsdl>

The WSDL file for the service to be used for the test. The stubs are generated from this WSDL.

<expected>

This is a parent tag which has 1 mandatory and 1 optional child elements.

<output>

The name of the expected output file to be used for comparison to determine if the test has passed. This file must exist in the ws-axis/c/tests/auto_build/testcases/output/ directory, and is copied to the test output directory and converted as required (See Conversions), and compared against the output from the client test program.

<endpoint>

This is the endpoint for the service used in the test. The server or server:port portion of the endpoint is changed to point to the TCP monitor utility, which will forward the request to the testHost:testPort combination specified in the build.xx.properties file. The monitor will be started on localhost on the port specified by the monitorPort property in the properties file. e.g. if the web service is running on a server called testserver which is listening on port 9080, and the monitor is configured to listen on port 12345 then the endpoint in example 1 will be changed to http://localhost:12345/Calculator/services/Calculator and the monitor will forward the request to testserver:9080

The example 2 shows all the optional tags that are available for handler tests.

     <test>
        <name>HandlerTest02</name>
        <description>CalculatorDoc with Handler Chain</description>
        <clientLang>cpp</clientLang>
        <clientCode>CalculatorDocClient.cpp</clientCode>
        <wsdl>CalculatorDoc.wsdl</wsdl>
        <expected>
            <output>
                CalculatorDoc.cpp.out
            </output>
            <request>
                HandlerTest2Request.out
            </request>
        </expected>
        <handler>
            <directory>
                handler_test2
            </directory>
            <service>
                Calculator
            </service>
        </handler>
        <endpoint>http://localhost:80/Calculator/services/Calculator</endpoint>
    </test>
    

example 2

The optional tags shown in example 2 are detailed below.

<request>

This is the name of the expected output of the request message sent to the service. The request message is captured by the TCP monitor utility and written to a file in the test output directory called tcpm.req. The expected output file specified by the <request> tag is copied to the test output directory and converted as required (See Conversions) and compared to the tcpm.req file to check that the request message is correct. This is used for tests which alter the HTTP or SOAP headers.

<handler>

This is a parent tag which has 2 mandatory child elements.

<directory>

This is the name of the parent directory which has a sub-directory for each of the handlers required by a client. Each of the sub-directories will be the name of the handler library, and contains all the source code required to compile the handler, excluding the Axis C++ include files.

<service>

This is the name of the web service being called. It should match the service portion of the SOAPAction of the request being sent to the service.

The example 3 shows all the additional tags that are available for tests.

     <test>
        <name>CalculatorDocSSL</name>
		  <description>CalculatorDoc using SSL</description>
        <clientLang>cpp</clientLang>
        <clientCode>CalculatorDocSSLClient.cpp</clientCode>
        <wsdl>CalculatorDoc.wsdl</wsdl>
        <expected>
            <output>
                CalculatorDoc.cpp.out
            </output>
        </expected>
        <endpoint>https://localhost:80/Calculator/services/Calculator</endpoint>
        <nomonitor>true</nomonitor>
        <port>9443</port>
    </test>
    

example 3

The optional tags shown in example 3 are detailed below.

<nomonitor>

This tells the framework not to invoke the TCP monitor utility when sending a request. This means that the message on-the-wire will not be checked. This is required for SSL tests as the monitor cannot operate over SSL

<port>

This is the port to send the request to. This will override the testPort property specified in the main Ant properties files or on the command line. This is used so that SSL tests can specify the SSL port of the service without having to do 2 tests runs.

The example 4 shows all the optional tags that are available for dynamic client tests (See Dynamic Clients).

     <test>
        <name>DynamicTest01</name>
        <description>CalculatorDoc implemented as a dynamic client</description>
        <dynamicCode>true</dynamicCode>
        <dynamicCodeDirectory>Test01</dynamicCodeDirectory>
        <clientLang>cpp</clientLang>
        <wsdl>CalculatorDoc.wsdl</wsdl>
        <expected>
            <output>
                CalculatorDoc.cpp.out
            </output>
        </expected>
        <endpoint>http://localhost:80/Calculator/services/Calculator</endpoint>
    </test>
    

example 4

The optional tags shown in example 4 are detailed below.

<dynamicCode>

This is a flag to indicate that the client is a dynamic client. The value has no relevance but must be set to something. WARNING setting it to false will have no effect. This property should only be used for dynamic client tests. Having this property will cause the framework to get the source from a different location and not to generate any stubs from the WSDL.

<dynamicCodeDirectory>

This is the name of the directory where all the dynamic code is copied from. All .cpp and .hpp files will be copied from this directory to the generated output directory.

Conversions

The expected output files, the request/response messages written by the TCP monitor utility and the output files (redirected stdout/stderr) written by the test client may be in different formats, e.g. DOS format or UNIX format or a combination of both. All the files used for comparison are first converted to the platform specific format of the machine the tests are being run on.

Additionally the expected request files will have host:port combination of the original request in the HTTP header information from when it was created. This information is altered to match the location of the TCP monitor utility.

Handlers

If handlers have been specified in the test properties file for a test (See Test Properties Files), then the directory specified will have sub-directories matching the names of the libraries to be created. This will be the name of the library without any prefix or suffix. A prefix of lib is usual for UNIX systems, and a suffix could be .DLL, .so, or .sl depending on platform. If the sub-directory is called testhandler then this will create a library called testhandler.DLL on windows and libtesthandler.so on Linux. All the source code for the library will be in this sub-directory.
e.g.
In example 2 above a directory of handler_chain was specified. This directory is the parent of the library directories, but a sub directory of the ws-axis/c/tests/auto_build/testcases/handlers/ directory. The handler_chain directory structure is shown in example 5

	
        handler_chain/testhandler
        handler_chain/testhandler/TestHandler.cpp
        handler_chain/testhandler/THandler.cpp
        handler_chain/testhandler/THandler.h
        handler_chain/testhandler2
        handler_chain/testhandler2/TestHandler2.cpp
        handler_chain/testhandler2/THandler2.h
        handler_chain/testhandler2/THandler2.cpp
	
	

example 5

When built this will produce 2 libraries in the test output directory called libtesthandler.so and libtesthandler2.so on Linux and testhandler.dll and testhandler2.dll on Windows.
The framework also creates the client.wsdd file required to invoke the handlers when the test is run and also updated the axiscpp.conf used by the test.
At present only <requestFlow ..> handlers are supported.

Dynamic Clients

A dynamic client is a client that uses the Call object directly and all initialization must be done manually. A dynamic client does NOT use stubs generated from the WSDL. Of course the dynamic client may have been based originally on a generated Stub. Using the dynamic client approach gives the developer greater control over when the APIs are called and may provide access to data protected via the Stub interface. The big disadvtage of a dynamic client is that a new release of AxisC++ may break the client more easily than if generated Stubs were used. All the code for dynamic client needs to be placed into a sub-directory of the dynamic directory ws-axis/c/tests/auto_build/testcases/dynamic.
e.g.
In example 5 above a dynamicCodeDirectory of Test01 was specified. This directory is a sub directory of the ws-axis/c/tests/auto_build/testcases/dynamic directory. This directory structure is shown in example 6.

	
        Test01/Calculator.cpp
        Test01/Calculator.hpp
	
	

example 6

TCP Monitor Utility

usage: TCPMonitor <-l listen port> <-p forward port> <-h forward host> <-o request output file> [-r response output file]

-l listen port

The port the monitor listens on for incoming requests. i.e. the port the client should send the request to.

-p forward port

The port the actual service is listening on.

-h forward host

The host name of the machine the actual service is running on.

-o request file

The filename the request is written to.

-r response file

Optional. The filename the response is written to.

The program will listen for all requests on the listen port and write them to a file. The program stays alive until a stop message is received so many requests for many services will be written to the same file if they are all sent to the same port.

usage: StopTCPMonitor <-p listen port> <-h host>

-p listen port

The port the monitor is listening on.

-h host

The host the listener is running on

e.g. to run CalculatorDoc through the monitor

• start the monitor by typing 'java -jar /some/where/monitor.jar -l 13260 -p 80 -h localhost -o /tmp/request.out -r /tmp/response.out', either in the background (&) or in a separate window.
• run the client CalculatorDoc http://localhost:13260/axis/Calculator.
• stop the monitor by typing 'java -cp /some/where/monitor.jar org.apache.test.StopTCPMonitor -p 13260 -h localhost'.

This will create 2 files in /tmp called request.out and response.out.

Files Specific to Ant Test Framework

ws-axis/c/test.xml

The main Ant script to run the tests. This script imports other Ant scripts from the ws-axis/c/build directory

ws-axis/c/build/testInitialize.xml

Sets or derives test specific properties and has tests initialization targets

ws-axis/c/build/buildHandler.xml

Has targets to build any number of handlers as specified in the test properties xml file. This also creates the client.wsdd file for the test and updates the axiscpp.conf file with the location of the client.wsdd file

ws-axis/c/build/validateTest.xml

Has targets to validate if client and handlers compiled and whether the client output is as expected and if the request message is as expected

ws-axis/c/build/buildTest.xml

Has targets to generate the stubs, copy the client code and compile the client into a test directory

ws-axis/c/build/executeTest.xml

Has targets to copy the expected output and request files to the test directory, performing any conversions as required (see Conversions), manipulates the test endpoint to run the test through the TCP monitor utility, creates the axiscpp.conf configuration file, starts the monitor, executes the test and stops the monitor

ws-axis/c/build/buildMonitor.xml

Has targets to build the TCP monitor utility program

ws-axis/c/tests/auto_build/testcases/tests/

directory containing the xml property files for each test (See Test Property Files)

ws-axis/c/tests/auto_build/testcases/handlers/

directory containing the source for any handlers used by the tests. Each sub-directory will be for a test and will have sub-directories of the handler name which contain the source for each handler. (See Handlers)

ws-axis/c/tests/auto_build/testcases/output/

directory containing the expected output files specified in the test properties xml files. (See Test Property Files)

ws-axis/c/tests/auto_build/testcases/wsdls/

directory containing the WSDL files for each service used by the tests

ws-axis/c/tests/auto_build/testcases/client/

directory has sub-directories for each supported language, e.g. c/c++ within each sub-directory is the test client implementation code.

Files Required by Ant Test Framework

buildInitialize.xml

Loads all the external task/type definitions and has the compiler/linker definitions. Also set platform specific properties and load one of the platform specific properties files listed below

ws-axis/c/build/build.AIX.properties

ws-axis/c/build/build.Linux.properties

ws-axis/c/build/build.OS400.properties

ws-axis/c/build/build.SunOS.properties

ws-axis/c/build/build.Win32.properties