The integrated "Test Script Runner" (TSR) was created to automate test sequences on one or more RCE standalone installations. These installations are automatically set up using an RCE feature called "Instance Management" (IM), which is still under development, and therefore not fully documented yet. However, the instructions below should be sufficient to configure this feature as needed for the TSR.
As of RCE 9.0.0, the Test Script Runner is included in the standard application release, as well as the standard Eclipse checkout. Therefore, very little configuration is required to use it.
The only configuration that is required for using the TSR is adding an Instance
Management configuration to the RCE instance that will execute the test scripts. This
defines the root directory where work files and directories of managed RCE test
installations will be stored. Locate the profile directory that is being launched and
edit its configuration.json file. In this file, add this configuration
block on the root JSON level, and adjust the settings as necessary:
"instanceManagement": {
"dataRootDirectory": "/tmp/rce-im-dat"
}
Note that there are additional Instance Management configuration parameters available; however, these are not usually needed for using the TSR.
It is strongly recommended to use a short full filesystem
path as the dataRootDirectory, as RCE installations will be placed
inside of it, and long filesystem paths are known to cause problems with these. The
exact maximum lenght will be determined, documented, and maybe also automatically
checked in the future.
The TSR is invoked by a single RCE console command (run-test), with an
alias for readability (run-tests). The general syntax is:
run-test[s] <comma-separated list of test ids>|--all <build
id>
There are three typical scenarios for calling this command:
from within an RCE instance launched from Eclipse during development, usually using the GUI workflow console
from within a standalone RCE instance, also usually using the GUI workflow console
as a CLI batch run (rce --batch "...") using a standalone
instance.
The RCE installation to be tested is defined by the <build
id> parameter in the above command. One important aspect to understand
is that this installation is generally independent of the installation being used to
execute the TSR command. The latter is, in a sense, only the "host" of test scripts.
There are three ways of specifying the build to test:
A build download id, which corresponds to a certain part of the standard
download URL, for example snapshots/trunk or
releases/9.0.0. The structure should be self-explanatory.
(The major release tree to use for snapshot builds is one of the optional
Instance Management settings mentioned above; the default is to use the
current major version, ie 9.x.)
A path to an unpacked local standalone (product)
build, which can, for example, result from a local build run or from
unpacking a downloaded product zip file. The syntax for this is
local:. This directory can be either writable or
read-only. For example, it is also possible to test a (read-only) .deb or
.rpm package installation this way.<local installation
path>
Note that this path must point to an already-unpacked RCE build, unlike the first approach, which downloads zipped release packages and unpacks them automatically.
As it is a frequent use case when testing standalone builds to execute the
test command the installation itself, there is a convenient shortcut for
this. By specifying :self as the build id, the test scripts are
executed on the installation of the instance used to run the test
command.
Note that due to technical limitations, however, this shortcut is not possible when launching RCE from Eclipse, as the test scripts require a standard product build to execute.
The actual test scripts are located in *.feature files in the
/de.rcenvironment.supplemental.testscriptrunner.scripts/resources/scripts
directory. The "test ids" in the command description are the annotation-like tags above
the indivual test scenarios, e.g. "@Test01". These can be specified in the command with
or without the "@" character. "--all" executes all available test
scenarios. The reserved "@Disabled" and "@disabled" tags can
be used to mark tests as excluded in the *.feature files.
run-test Test02,Test04 snapshots/trunk - runs two specific
tests on the latest snapshot build
run-test DefaultTestSuite :self - runs the default collection
of tests on the current installation
run-test --all local:/tmp/local-rce-build - runs all
available tests on a local build
rce --batch "run-test DefaultTestSuite :self" - the full
command line for the standard self-test of an installation