Folks, in this post i will be covering the use of test runner – by which you can execute your soapUI projects (from command line) without opening soapUI application.
This feature will by very useful when you want to perform the regression test on the newly deployed code & to store the response data in some files. And execution can be done by anyone (including people who don’t know “how to use soapUI”).
testrunner.bat (soapui_install_dir/bin folder) is a command line utility which is getting called whenever we execute our testcases with some predefined (or userdefined) conditions. In soapUI application, right click on any selected testcase/test suite and then select “launch test runner” from the context menu to call the test runner.bat file. This will open a new window wherein you can provide the required configuration data like, testcase/testsuite to be executed, endpoint url, folder to store the response, ignore the error results and so on.
After providing all the details, start the testrunner.bat by clicking launch button. In the window it will display the command line arguments being passed for the selected combination of input data.
Similarly, you can directly call the testrunner.bat from the command line utitlity and provide those parameters to perform the test execution. This will save lot of time to perform the test execution and also it can be executed by anyone (in your absence).
I have created a simple batch file (my_soapUI.bat) to perform this set of activity. So to perform the regression test, i need to just execute the “my_soapUI.bat” file from the command line and rest would be taken care.
cd C:\Program Files\eviware\soapUI-Pro-3.6\bin
testrunner.bat -ehttp://127.0.0.1/soapUI/Service.svc -sLoad_Test_Apr2011_TestSuite -r -a -fD:\Automate\store_response -I “C:\Documents and Settings\pradeep.bishnoi\My Documents\my_soapui-project.xml”:: To override the endpoint for the teststeps use
:: -e(Endpoint URL)
:: -ehttp://x.x.x.x/WebService.svc:: TestSuite to run, used to narrow down the tests to run
:: -s(TestSuite Name under the project)
:: -sTestSuiteSoapUI:: TestCase to run, used to narrow down the tests to run
:: -c(TestCase Name under the project/testSuite)
:: -cTestCaseSoapUI:: r : Turns on printing of a small summary report. To be used when you want to save generated reports
:: -r:: f : Specifies the root folder to which test results should be exported
:: -f(Complete path of the folder where to save the response)
:: -fD:\soapUI\responseReport:: a : Turns on exporting of all test results, not only errors. To be used when you want to export both passed and failed test step reports
:: -a:: I : Do not stop if error occurs, ignore them. Execution will continue even when any test step fails/error comes.
:: -I:: After providing all the parameters, pass the complete soapUI project file location in double quotes “” which contains the above mentioned testSuite & testCases for execution.
:: example “C:\soapui-project.xml”
To use the same, just copy the code lines (in blue colored font) and save them into a “my_soapUI.bat” file and edit the required parameters like endpoint, project file name, and folder to store the response etc.
EndNote : Biggest advantage of using the command line approach is #Faster execution since we are directly executing the project without opening the soapUI.
has anyone used the custom arguments before from within an application?
The one that are passed using testrun -P myVar=xxxx
I’d like to be able to use that “myVar” from within a groovy script on the run test case.
Yes & that can be done using the following :
context.testCase.testSuite.project.getPropertyValue(“projectPropName”)
in the testrunner while passing argument following should be passed
-PprojectPropName=myPropValue
I hope this will help.
Best Regards,
/Pradeep Bishnoi
Is it possible to use testrunner.sh without X?
Not quite sure what exactly X stands in your comment above. testrunner.sh would also take the same set of parameters as shown for the .bat file.
What are the dependencies for calling testruner.sh/bat, does this post assume that entire bin and lib directory need to be copied to location from which script is executed?
What are the implications for licence in case of the pro version?
Here the testrunner.bat file is called from it’s default location [%SOAPUI_INSTALL_DIR%/bin] and the test were executed. I didn’t understand what is referred when you say “script”.
The testrunner.bat file can be executed from any location on local machine (just make sure you have soapUI installed) with a litte tweak in the testrunner.bat file.
set SOAPUI_HOME=%~dp0
be replaced with the path of your soapUI installation bin folder. c:\program files\eviware\soapui 3.6\bin
And nothing special is offered (in terms of .bat file) with PRO version since .bat file with just call the soapui exe itself. So if it is open source version the features developed in PRO version won’t be executed.
P.S. : %~dp0 is a Batch file parameter to tell, pick current drive & directory.
regards,
{pradeep bishnoi}
Sorry for ambiguity in my initial post.
Context of my question was distributed testing and automation. For instance in case of the CI build project one would need to move both bin and lib folders to the node executing the CI build.
“script” was referring to any sh/py/groovy code that would wrap around testrunner.sh to set the stage if you will, fetch all dependencies for the test, checkout test project, select what tests to run etc..
thx for the blog
I am using open source version. this code will work in this version? I tried to run, but some information is not clear in the post, like url to web services. My web services is https. Here my config soapteste.bat:
cd C:\Program Files\SmartBear\soapUI-4.0.1\bin
testrunner.bat -ehttps://172.16.101.10:55443/camilo?wsdl -sTestSuiteMetering -r -a -fC:\Users\lopesca\Desktop\batsoap -I “C:\Users\lopesca\Desktop\batsoap\soapuiproject.xml”
thanks. good post :).
Awesome information mate. Your information was helpful in running it using command line. woohoo. 🙂
Thank you. Keep tracking this blog many more useful snippets/post to come 😉
How to pass the testcase and testsuite properties to testrunner.bat through CLI? Project properties are passed using -P. How abt test case and testsuite properties?
Hi,
Is there any way to call a single test step of our test case with CLI execution?
Hi,
Nope. It is not possible to execute a single teststep with CLI execution.
However work arounds are possible.
{Pradeep Bishnoi}
Hey,
Is there a way to specify which TestSuites to Run?
For example something like
./testrunner.bat -sSuite1 -sSuite2 …
OR
./testrunner.bat -sSuite* … (as a regular expression)
Thanks
Hi,
Now that’s a tricky question. I doubt this is possible.
soapUI Forum would be good place to get valid answer 🙂
Regards,
{Pradeep Bishnoi}
How do we do LoadTest from command line?
Pelase refer http://soapui.org/Load-Testing/command-line-execution.html
{Pradeep Bishnoi}
Great Post!! I do have one question though, is it possible to generate a singl-page JUnit style report? when i use -j it generates a multi-html report..?
Yes, that is possible. You have to extend the library using Jasper Report API.
Is it possible to run a project developed in soapUI (windows version) using command line in linux?
Yes, there shouldn’t be any issue. Make sure that the .bat file when executed using the CMD prompt works fine. Same should run fine in linux when called with .sh.
Hi Pradeep, Is it possible to read command line arguments in the project load script? We use web interface for developers to select certain parameters like environment and soapUI project they would like to run. I pass in those values in the command line in my script file. However my soapUI project should know those values since based on that I need to read appropriate test data files.
Many thanks
Hi Ajay,
Never tried loading the command line arguments in project load script. However what you want to achieve can be done by using the following approach.
Command line arguments do take Global/Project level properties as input.
So create couple of Global Variable like [environment=’DEV’, url=’learnsoapui.wordpress.com’]
and in the command line call this..
testrunner.bat ..other details… -Genvironment=DEV -Gurl=learnsoapui.wordpress.com
and while executing the load script just read the value of the Global Variable. Done.
{Pradeep}
Good One Mite,
Have a small query to make this work at my end. How can we pass the proxy details in command line which we pass in preferences of soapUI-Pro-4.5.0?
Regards,
Sree
For people who want to execute their testrunner.bat from an automated batch file where the values are not hardcoded but read from a ‘monitor.properties’ file that can be changed by the user.
I have shared the code :
-x-x-x-x-x-x-x-x-Start of batch file-x-x-x-x-x-x-x-x-x-x-x-
FOR /F “eol=; tokens=2,2 delims==” %%i IN (‘findstr /i “soapUITestReportsDumpFolder” monitor.properties’) DO set soapUITestReportsDumpFolder=%%i
FOR /F “eol=; tokens=2,2 delims==” %%i IN (‘findstr /i “soapUIInstallationDirectory” monitor.properties’) DO set soapUIInstallationDirectory=%%i
FOR /F “eol=; tokens=2,2 delims==” %%i IN (‘findstr /i “soapUIProjectFullyQualifiedPath” monitor.properties’) DO set soapUIProjectFullyQualifiedPath=%%i
FOR /F “eol=; tokens=2,2 delims==” %%i IN (‘findstr /i “soapUITestSuite” monitor.properties’) DO set soapUITestSuite=%%i
cd %soapUITestReportsDumpFolder%
set TIMESTAMP=%DATE:~7,2%-%DATE:~4,2%-%DATE:~10,4%
mkdir “%TIMESTAMP%”
C:
cd %soapUIInstallationDirectory%
testrunner.bat -s”%soapUITestSuite%” -r -f”%soapUITestReportsDumpFolder%\%TIMESTAMP%” -a -I “%soapUIProjectFullyQualifiedPath%”
-x-x-x-x-x-x-x-x-End of batch file-x-x-x-x-x-x-x-x-x-x-x-
““““““““““““““““““““““““““““““““`
““““““““““““““““““““““““““““““““`
-x-x-x-x-x-x-x-x-Start of monitor.properties-x-x-x-x-x-x-x-x-x-x-x-
soapUIInstallationDirectory=C:\Program Files\eviware\soapUI-3.5.1\bin
monitoringScriptsLocation=D:\Cordys\MonitoringScripts
soapUITestReportsDumpFolder=D:\Cordys\AutomatedMonitoring
soapUIProjectFullyQualifiedPath=D:\Users\aattarwa\Desktop\PublishOrders&FileWriters-soapui-project.xml
soapUITestSuite=POLMIChangesTestSuite
-x-x-x-x-x-x-x-x-End of monitor.properties-x-x-x-x-x-x-x-x-x-x-x-
hi,
can include socket timeout at global level in commandline in form of argument?, something like
-GHttpSettings.SOCKET_TIMEOUT=20000
Hi,
I have tried the above mentioned way to run SOAP tests from command line but I am getting the following error –
log4j:ERROR setFile(null,true) call failed.
java.io.FileNotFoundException: soapui-errors.log (Access is denied)
Can you suggest me what might be the problem?
Hi Pradeep,
Thanks for your blog. I have learnt may things here.
I’m wondering if there is a way to re-direct the script log generated in command line using the above statement.
I have tried
testrunner.ba -S(ProjSuiteName) -c(TestCaseName) -a -i -(proj xml path) > “c:\log.txt”
but its not going through. Can you please suggest
First of all thanks for sharing this useful info.
while I am able to succefully execute this from command line I am not able to save the response –
Here is my cmd
testrunner.bat -ehttps://: -c”SoapUI_cmd” -r -a -fD:\SoapUI_response\ “D:\soapui-project.xml”
Could you please help with this?
Also can we pass arguments to test cases through command line and get test result?
Hi Pradeep,
Thanks for the blog,
We have done a CI integration of SoapUI with TFS for the project and have written the logic in C# code to run the testrunner on our build server,
The problem we are facing now is, the Project level properties have few properties like Session ID or Authentication token, which gets updated using one of the test steps during execution.
This works perfectly fine when I use our SoapUI GUI to run the test in project.
But when it is run using the testrunner.bat file using C#, the properties does not get updated and my test cases start failing.
From what I understand is the soapUi-settings.xml does not get updated with the latest sessionID or authentication token.
Can you please let me know if you have resolution for this ???
Thanks,
Sarath
Did you found any workaround for this besides changing all scripts as I am facing the same problem?
When you run a SoapUI load test, is it possible to export all of the raw data using the command line? I.e.
– LoadTestLog
– LoadTestStepHistory
– LoadTestStatisticsHistory
– FailedTestSteps
Hi Pradeep,
Please could you let me know if there’s any way to set up SOAPUI project and create request dynamically using some command / Script.
I am looking for creating a tool on top of soapui which should be able to set up a project & create request on SOAPUI tool.
Please help me know if there’s a way to do it.
Many thanks
Cheers
Tauseef
Hello Pradeep ,
Nice blog you have here 🙂
Still there is one thing that is absolutely “killing me” lately.
In my project I use some Global parameters so I keep things easy to maintain at one place and minimizing number of parameters needed.
Many scripts use simple “com.eviware.soapui.SoapUI.globalProperties.setPropertyValue( “Global_param”, “someValue” )” calls and write calculated data in Global property variable which works just fine inside the SoapUI UI 🙂
But if I run these scripts/testcases via testrunner.bat/.sh outside the GUI via command line then write of properties with call above ,simply doesnt work it seems. Global properties are not updated nor created ,as it is the case if I run these inside the SoapUI . Any ideas if this is known limitation ?
Hi…I’m not getting complete information at step level result when I execute through testrunner.bat.
testrunner.bat “D:\test_soapuIproj.xml” -c”Test1″ -M -a -f”D:\Results”
Step result is:
————————————
Status: OK
Time Taken: 4712
Size: 631
Timestamp: Fri Oct 17 02:12:28 IST 2014
TestStep: JDBCRequest
—————– Messages ——————————
————————————————————————————————————
But When I execute through SoapUI, I could see lot of information in the response for the same step.
For example
————————————-
5796
[ {
.
.
.
.
Can you please help me to get this information as well in the step level result
Thanks,
Tejas
Hi guys,
I have added -I still my execution stops. any idea what wrong might i have done?
Hi Pradeep,
I can generate a pdf report (test case report) using UI. How can I generate same report using command line on Windows?
Please help.
Thanks
Sam
I am using following scripts for parallel execution for projects:
def project1 = [//batch file command].execute()
def project2 = [//batch file command].execute()
Is this going to cancel the license of SoapUI Pro?
Hi, I am trying to set project custom properties file through commandline:
testrunner.bat -Dsoapui.properties.project=D:\testproperties.txt “test.xml”
However, this does not load the properties from external file. Can you tell me the right way of specifying and loading external properties file? Thanks