TestRunner – soapUI project execution from command line

Posted: May 17, 2011 in soapUI, testrunner
Tags: ,

Folks, in this post i will be covering the use of test runner – by which you can execute your soapUI projects (from command line) without opening soapUI application.

This feature will by very useful when you want to perform the regression test on the newly deployed code & to store the response data in some files. And execution can be done by anyone (including people who don’t know “how to use soapUI”).

testrunner.bat (soapui_install_dir/bin folder) is a command line utility which is getting called whenever we execute our testcases with some predefined (or userdefined) conditions. In soapUI application, right click on any selected testcase/test suite and then select “launch test runner” from the context menu to call the test runner.bat file. This will open a new window wherein you can provide the required configuration data like, testcase/testsuite to be executed, endpoint url, folder to store the response, ignore the error results and so on.

After providing all the details, start the testrunner.bat by clicking launch button. In the window it will display the command line arguments being passed for the selected combination of input data.

Similarly, you can directly call the testrunner.bat from the command line utitlity and provide those parameters to perform the test execution. This will save lot of time to perform the test execution and also it can be executed by anyone (in your absence).

I have created a simple batch file (my_soapUI.bat) to perform this set of activity. So to perform the regression test, i need to just execute the “my_soapUI.bat” file from the command line and rest would be taken care.

cd C:\Program Files\eviware\soapUI-Pro-3.6\bin
testrunner.bat -ehttp://127.0.0.1/soapUI/Service.svc -sLoad_Test_Apr2011_TestSuite -r -a -fD:\Automate\store_response -I “C:\Documents and Settings\pradeep.bishnoi\My Documents\my_soapui-project.xml”

:: To override the endpoint for the teststeps use
:: -e(Endpoint URL)
:: -ehttp://x.x.x.x/WebService.svc

:: TestSuite to run, used to narrow down the tests to run
:: -s(TestSuite Name under the project)
:: -sTestSuiteSoapUI

:: TestCase to run, used to narrow down the tests to run
:: -c(TestCase Name under the project/testSuite)
:: -cTestCaseSoapUI

:: r : Turns on printing of a small summary report. To be used when you want to save generated reports
:: -r

:: f : Specifies the root folder to which test results should be exported
:: -f(Complete path of the folder where to save the response)
:: -fD:\soapUI\responseReport

:: a : Turns on exporting of all test results, not only errors. To be used when you want to export both passed and failed test step reports
:: -a

:: I : Do not stop if error occurs, ignore them. Execution will continue even when any test step fails/error comes.
:: -I

:: After providing all the parameters, pass the complete soapUI project file location in double quotes “” which contains the above mentioned testSuite & testCases for execution.
:: example      “C:\soapui-project.xml”

To use the same, just copy the code lines (in blue colored font) and save them into a “my_soapUI.bat” file and edit the required parameters like endpoint, project file name, and folder to store the response etc.

EndNote : Biggest advantage of using the command line approach is #Faster execution since we are directly executing the project without opening the soapUI.

Advertisements
Comments
  1. has anyone used the custom arguments before from within an application?

    The one that are passed using testrun -P myVar=xxxx

    I’d like to be able to use that “myVar” from within a groovy script on the run test case.

    • Yes & that can be done using the following :

      context.testCase.testSuite.project.getPropertyValue(“projectPropName”)

      in the testrunner while passing argument following should be passed

      -PprojectPropName=myPropValue

      I hope this will help.
      Best Regards,
      /Pradeep Bishnoi

  2. Abel says:

    Is it possible to use testrunner.sh without X?

  3. DblD says:

    What are the dependencies for calling testruner.sh/bat, does this post assume that entire bin and lib directory need to be copied to location from which script is executed?

    What are the implications for licence in case of the pro version?

    • Here the testrunner.bat file is called from it’s default location [%SOAPUI_INSTALL_DIR%/bin] and the test were executed. I didn’t understand what is referred when you say “script”.
      The testrunner.bat file can be executed from any location on local machine (just make sure you have soapUI installed) with a litte tweak in the testrunner.bat file.

      set SOAPUI_HOME=%~dp0
      be replaced with the path of your soapUI installation bin folder. c:\program files\eviware\soapui 3.6\bin

      And nothing special is offered (in terms of .bat file) with PRO version since .bat file with just call the soapui exe itself. So if it is open source version the features developed in PRO version won’t be executed.

      P.S. : %~dp0 is a Batch file parameter to tell, pick current drive & directory.

      regards,
      {pradeep bishnoi}

      • DblD says:

        Sorry for ambiguity in my initial post.

        Context of my question was distributed testing and automation. For instance in case of the CI build project one would need to move both bin and lib folders to the node executing the CI build.
        “script” was referring to any sh/py/groovy code that would wrap around testrunner.sh to set the stage if you will, fetch all dependencies for the test, checkout test project, select what tests to run etc..

        thx for the blog

  4. Camilo Lopes says:

    I am using open source version. this code will work in this version? I tried to run, but some information is not clear in the post, like url to web services. My web services is https. Here my config soapteste.bat:
    cd C:\Program Files\SmartBear\soapUI-4.0.1\bin
    testrunner.bat -ehttps://172.16.101.10:55443/camilo?wsdl -sTestSuiteMetering -r -a -fC:\Users\lopesca\Desktop\batsoap -I “C:\Users\lopesca\Desktop\batsoap\soapuiproject.xml”

    thanks. good post :).

  5. Rashmi says:

    How to pass the testcase and testsuite properties to testrunner.bat through CLI? Project properties are passed using -P. How abt test case and testsuite properties?

  6. Saurabh Gupta says:

    Hi,

    Is there any way to call a single test step of our test case with CLI execution?

  7. andres de la barra says:

    Hey,

    Is there a way to specify which TestSuites to Run?

    For example something like

    ./testrunner.bat -sSuite1 -sSuite2 …

    OR

    ./testrunner.bat -sSuite* … (as a regular expression)

    Thanks

  8. mitz says:

    How do we do LoadTest from command line?

  9. Omer says:

    Great Post!! I do have one question though, is it possible to generate a singl-page JUnit style report? when i use -j it generates a multi-html report..?

  10. Chinmay Dash says:

    Is it possible to run a project developed in soapUI (windows version) using command line in linux?

  11. Ajay says:

    Hi Pradeep, Is it possible to read command line arguments in the project load script? We use web interface for developers to select certain parameters like environment and soapUI project they would like to run. I pass in those values in the command line in my script file. However my soapUI project should know those values since based on that I need to read appropriate test data files.

    Many thanks

    • Hi Ajay,

      Never tried loading the command line arguments in project load script. However what you want to achieve can be done by using the following approach.
      Command line arguments do take Global/Project level properties as input.
      So create couple of Global Variable like [environment=’DEV’, url=’learnsoapui.wordpress.com’]
      and in the command line call this..
      testrunner.bat ..other details… -Genvironment=DEV -Gurl=learnsoapui.wordpress.com

      and while executing the load script just read the value of the Global Variable. Done.

      {Pradeep}

  12. Sree says:

    Good One Mite,

    Have a small query to make this work at my end. How can we pass the proxy details in command line which we pass in preferences of soapUI-Pro-4.5.0?

    Regards,
    Sree

  13. abbasdgr8 says:

    For people who want to execute their testrunner.bat from an automated batch file where the values are not hardcoded but read from a ‘monitor.properties’ file that can be changed by the user.
    I have shared the code :

    -x-x-x-x-x-x-x-x-Start of batch file-x-x-x-x-x-x-x-x-x-x-x-

    FOR /F “eol=; tokens=2,2 delims==” %%i IN (‘findstr /i “soapUITestReportsDumpFolder” monitor.properties’) DO set soapUITestReportsDumpFolder=%%i
    FOR /F “eol=; tokens=2,2 delims==” %%i IN (‘findstr /i “soapUIInstallationDirectory” monitor.properties’) DO set soapUIInstallationDirectory=%%i
    FOR /F “eol=; tokens=2,2 delims==” %%i IN (‘findstr /i “soapUIProjectFullyQualifiedPath” monitor.properties’) DO set soapUIProjectFullyQualifiedPath=%%i
    FOR /F “eol=; tokens=2,2 delims==” %%i IN (‘findstr /i “soapUITestSuite” monitor.properties’) DO set soapUITestSuite=%%i

    cd %soapUITestReportsDumpFolder%
    set TIMESTAMP=%DATE:~7,2%-%DATE:~4,2%-%DATE:~10,4%
    mkdir “%TIMESTAMP%”

    C:

    cd %soapUIInstallationDirectory%

    testrunner.bat -s”%soapUITestSuite%” -r -f”%soapUITestReportsDumpFolder%\%TIMESTAMP%” -a -I “%soapUIProjectFullyQualifiedPath%”

    -x-x-x-x-x-x-x-x-End of batch file-x-x-x-x-x-x-x-x-x-x-x-
    ““““““““““““““““““““““““““““““““`
    ““““““““““““““““““““““““““““““““`
    -x-x-x-x-x-x-x-x-Start of monitor.properties-x-x-x-x-x-x-x-x-x-x-x-

    soapUIInstallationDirectory=C:\Program Files\eviware\soapUI-3.5.1\bin
    monitoringScriptsLocation=D:\Cordys\MonitoringScripts
    soapUITestReportsDumpFolder=D:\Cordys\AutomatedMonitoring
    soapUIProjectFullyQualifiedPath=D:\Users\aattarwa\Desktop\PublishOrders&FileWriters-soapui-project.xml
    soapUITestSuite=POLMIChangesTestSuite

    -x-x-x-x-x-x-x-x-End of monitor.properties-x-x-x-x-x-x-x-x-x-x-x-

  14. t_ca says:

    hi,
    can include socket timeout at global level in commandline in form of argument?, something like
    -GHttpSettings.SOCKET_TIMEOUT=20000

  15. Preet says:

    Hi,
    I have tried the above mentioned way to run SOAP tests from command line but I am getting the following error –
    log4j:ERROR setFile(null,true) call failed.
    java.io.FileNotFoundException: soapui-errors.log (Access is denied)

    Can you suggest me what might be the problem?

  16. Satya says:

    Hi Pradeep,

    Thanks for your blog. I have learnt may things here.

    I’m wondering if there is a way to re-direct the script log generated in command line using the above statement.

    I have tried
    testrunner.ba -S(ProjSuiteName) -c(TestCaseName) -a -i -(proj xml path) > “c:\log.txt”

    but its not going through. Can you please suggest

  17. First of all thanks for sharing this useful info.
    while I am able to succefully execute this from command line I am not able to save the response –
    Here is my cmd
    testrunner.bat -ehttps://: -c”SoapUI_cmd” -r -a -fD:\SoapUI_response\ “D:\soapui-project.xml”

    Could you please help with this?
    Also can we pass arguments to test cases through command line and get test result?

  18. Sarath S Pillai says:

    Hi Pradeep,
    Thanks for the blog,
    We have done a CI integration of SoapUI with TFS for the project and have written the logic in C# code to run the testrunner on our build server,
    The problem we are facing now is, the Project level properties have few properties like Session ID or Authentication token, which gets updated using one of the test steps during execution.
    This works perfectly fine when I use our SoapUI GUI to run the test in project.
    But when it is run using the testrunner.bat file using C#, the properties does not get updated and my test cases start failing.
    From what I understand is the soapUi-settings.xml does not get updated with the latest sessionID or authentication token.
    Can you please let me know if you have resolution for this ???

    Thanks,
    Sarath

  19. When you run a SoapUI load test, is it possible to export all of the raw data using the command line? I.e.

    – LoadTestLog
    – LoadTestStepHistory
    – LoadTestStatisticsHistory
    – FailedTestSteps

  20. Tauseef says:

    Hi Pradeep,

    Please could you let me know if there’s any way to set up SOAPUI project and create request dynamically using some command / Script.

    I am looking for creating a tool on top of soapui which should be able to set up a project & create request on SOAPUI tool.

    Please help me know if there’s a way to do it.

    Many thanks

    Cheers
    Tauseef

  21. Robert says:

    Hello Pradeep ,

    Nice blog you have here 🙂
    Still there is one thing that is absolutely “killing me” lately.
    In my project I use some Global parameters so I keep things easy to maintain at one place and minimizing number of parameters needed.
    Many scripts use simple “com.eviware.soapui.SoapUI.globalProperties.setPropertyValue( “Global_param”, “someValue” )” calls and write calculated data in Global property variable which works just fine inside the SoapUI UI 🙂
    But if I run these scripts/testcases via testrunner.bat/.sh outside the GUI via command line then write of properties with call above ,simply doesnt work it seems. Global properties are not updated nor created ,as it is the case if I run these inside the SoapUI . Any ideas if this is known limitation ?

  22. Tejas says:

    Hi…I’m not getting complete information at step level result when I execute through testrunner.bat.

    testrunner.bat “D:\test_soapuIproj.xml” -c”Test1″ -M -a -f”D:\Results”

    Step result is:
    ————————————

    Status: OK
    Time Taken: 4712
    Size: 631
    Timestamp: Fri Oct 17 02:12:28 IST 2014
    TestStep: JDBCRequest

    —————– Messages ——————————

    ————————————————————————————————————

    But When I execute through SoapUI, I could see lot of information in the response for the same step.

    For example
    ————————————-

    5796
    [ {
    .
    .
    .
    .
    Can you please help me to get this information as well in the step level result

    Thanks,
    Tejas

  23. pratik says:

    Hi guys,

    I have added -I still my execution stops. any idea what wrong might i have done?

  24. Sam says:

    Hi Pradeep,
    I can generate a pdf report (test case report) using UI. How can I generate same report using command line on Windows?
    Please help.

    Thanks
    Sam

  25. nishant nalin says:

    I am using following scripts for parallel execution for projects:
    def project1 = [//batch file command].execute()
    def project2 = [//batch file command].execute()
    Is this going to cancel the license of SoapUI Pro?

  26. Smitha says:

    Hi, I am trying to set project custom properties file through commandline:
    testrunner.bat -Dsoapui.properties.project=D:\testproperties.txt “test.xml”
    However, this does not load the properties from external file. Can you tell me the right way of specifying and loading external properties file? Thanks

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s