Functional Testing¶
There are at least three different sets of tests for the GDP that use Python, Node and Java.
NOTE: This not unit testing, this is functional testing. By unit testing I mean test jigs to exercise individual modules from the inside. For example, see Wikipedia and StackOverflow. There's nothing wrong with functional testing; we should have that too. —eric
The Python tests use Pytest, which is a unit testing framework. The JavaScript tests use Mocha and the Java tests use the Tcl-testing framework. Some of the tests below do also do functional testing. Writing unit tests using Python would be fairly easy, just update gdp/test/run_tests.py, - Christopher
Action Items:¶
- Further develop the tests (Python or Node preferred)
- If the tests can generate JUnit xml output, then it is easy to view the results in Jenkins
- Use code coverage to verify that key functions are being invoked
Test Harnesses¶
gdp/test¶
The gdp/test directory contains a test harness that uses Python, see README.txt
The Python tests in use pytest, which is a unit testing framework.
Sample run using Pytest¶
The test/run.sh script runs the tests
bash-3.2$ ./run.sh
#### ./setupAndRun.sh: Running git pull in /Users/cxh/ptII/vendors/gdp/gdp_router
Already up-to-date.
#### ./setupAndRun.sh: Removing /Users/cxh/ptII/vendors/gdp/gcls and then recreating it.
#### ./setupAndRun.sh: Set up /Users/cxh/ptII/vendors/gdp/ep_adm_params
To run with these settings from the command line, use:
export EP_PARAM_PATH=/Users/cxh/ptII/vendors/gdp/ep_adm_params
#### ./setupAndRun.sh: Starting gdp_router
Command to start gdp_router (cd /Users/cxh/ptII/vendors/gdp/gdp_router; python ./src/gdp_r\
outer.py -l routerLog.txt) &
#### ./setupAndRun.sh: Starting gdplogd
Command to start gdplogd: /Users/cxh/ptII/vendors/gdp/gdp/gdplogd/gdplogd -F -N ealmac23.l\
ocal &
My GDP routing name = ealmac23.local
#### Creating log gdp.runTests.31803
Command to create a log: /Users/cxh/ptII/vendors/gdp/gdp/apps/gcl-create -k none -s ealmac\
23.local gdp.runTests.31803
Created new GCL irUh5vFzjd8DdbKoCtA5Qc8sgYPjr4djEfF0_D694lI
on log server ealmac23.local
exiting with status OK
./setupAndRun.sh: Invoking ./_internalRunPythonTests.sh run_tests.py gdp.runTests.31803
Command: py.test run_tests.py --logName=gdp.runTests.31803
#### py.test -v run_tests.py --logName=gdp.runTests.31803
================================== test session starts ===================================
platform darwin -- Python 2.7.13, pytest-3.0.5, py-1.4.32, pluggy-0.4.0 -- /opt/local/Libr\
ary/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python
cachedir: .cache
rootdir: /Users/cxh/ptII/vendors/gdp/gdp/test, inifile:
collected 4 items
run_tests.py::test_t_fwd_append PASSED
run_tests.py::test_gcl_create PASSED
run_tests.py::test_t_multimultiread PASSED
run_tests.py::test_t_sub_and_append PASSED
=============================== 4 passed in 10.49 seconds ================================
#### ./setupAndRun.sh: Stopping gdplogd and gdp_router
2017-01-22 14:25:22.608300 -0800 gdplogd: Terminating on signal 15
bash-3.2$
run.sh will download and start up the gdp_router and invoke the tests. How this works is a bit complex.
run.sh contains:
!/bin/sh ./setupAndRun.sh ./_internalRunPythonTests.sh run_tests.py $@
setupAndRun.sh does the following:
- If necessary, clone the gdp_router repo, otherwise, does a gdp pull.
- Sets up ep_adm_params and sets the EP_PARAM_PATH variable
- Starts up the gdp_router
- Starts up gdplogd
- Creates a log
- Invokes the script, in this case _internalRunPythonTests.sh
- When the script returns, gdplogd is terminated.
internalRunPythonTests.sh invokes py.test on runtests.py
py.test is part of the pytest testing infrastructure, in this case, py.test is run on gdp/test/run_tests.py, which defines tests that are invoked by pytest.
In theory, if the GDP daemons are running and you hold your mouth right, running the following might work:
py.test -v run_tests.py
Jenkins gdp job on terra¶
http://terra.eecs.berkeley.edu:8080/job/gdp - Nightly Jenkins build. To get access, email Christopher
The Jenkins gdp job configuration runs the following:
cd $WORKSPACE/test ./run.sh --junitxml=../reports/junit/gdpTest.xml || true cd $WORKSPACE/lang/python/test ./run.sh --junitxml=../../../reports/junit/gdpLangPythonTest.xml || true echo "Done running tests"
Accessors GDP tests¶
Accessors test the GDP using Node and Java
Accessors GDP Node Tests¶
Accessors test the GDP via the @terraswarm/gdp npm module using the Node Accessor Host
To run the tests, install Apache Ant, install Node, check out the repo and run ant:
To check out the Accessors repo read/write:
svn co https://repo.eecs.berkeley.edu/svn-anon/projects/terraswarm/accessors/trunk/accessors
For read-write access, you need a repo.eecs.berkeley.edu account, in which case you can check out a tree with:
svn co https://repo.eecs.berkeley.edu/svn/projects/terraswarm/accessors/trunk/accessors
Run ant:
cd accessors/web/ ant tests.mocha.composites
The output will include text like:
[exec] NodeHost
[exec] ✓ NodeHost./accessors/web/gdp/test/auto/GDPLogCreateAppendReadJS (1041ms)
[exec] ✓ NodeHost./accessors/web/gdp/test/auto/GDPLogSubscribeJS (1005ms)
To run just the Node GDP Accessor tests:
bash-4.1$ cd gdp/test bash-4.1$ ls auto GDPLogSubscribeOnlyJS.js runNodeAutoTests TestGdp.js bash-4.1$ ./runNodeAutoTests testNodeAuto.js: testNodeAuto(gdp/test/auto) NodeHost ^[[2K^[[0G ✓ NodeHost./accessors/webgdp/test/auto/GDPLogCreateAppendReadJS (1044ms) ^[[2K^[[0G ✓ NodeHost./accessors/webgdp/test/auto/GDPLogSubscribeJS (1007ms) 2 passing (2s) bash-4.1$
The tests in web/gdp/test/auto are functional tests that were created from corresponding Ptolemy models, see below for details.
The Accessor Node framework uses Mocha to run the functional tests.
Mocha is a JavaScript Unit Testing Framework.
It would not be difficult to add Mocha Unit tests to the gdp distribution, the key issue would be to ensure that gdplogd and the gdp_router were started.
The Jenkins Accessor Build Tests shows the test results.
Accessors GDP Java Tests¶
The Java-based Cape Code Accessor Host has a Java interface to the GDP.
See Ptolemy II for how to install Cape Code.
The GDP tests are in ptolemy/actor/lib/jjs/modules/gdp/test
To run the tests after installing Ptolemy II
cd $PTII/ptolemy/actor/lib/jjs/modules/gdp/test make
Note that the test builds a copy of the GDP in $PTII/vendors/gdp
These tests consist of several types of tests:
* Unit tests like ptolemy/actor/lib/jjs/modules/gdp/test/GDPManager.tcl. These tests use the Tcl test() framework, see http://ptolemy.eecs.berkeley.edu/ptolemyII/ptIIlatest/ptII/doc/coding/testing.htm
* Functional tests like ptolemy/actor/lib/jjs/modules/gdp/test/auto/GDPLogSubscribe.xml. These tests are Ptolemy models that use the Java-based Cape Code Accessor Host
* Functional tests like ptolemy/actor/lib/jjs/modules/gdp/test/auto/GDPLogSubscribeJS.xml. This is a Java-based Cape Code Accessor Host test that can generate a Composite Accessor that can be run by the Node Host, see $PTII/org/terraswarm/accessor/accessors/web/gdp/test/auto/GDPLogSubscribeJS.js
The tests will end with text like
Failed: 0 Total Tests: 9 ((Passed: 9, Newly Passed: 0) Known Failed: 0) /Users/cxh/ptII/ptolemy/actor/lib/jjs/modules/gdp/test