8/29/2004
Remote Testing View is an
eclipse plug-in developed to display the results of the testcases
run on a remote server server. The basic idea is
this, students not only check-in their source code but also the testcases into CVS. Every couple hours, the remote server
will checkout the testcases and run them against both
the student's project and the reference solution. The result will then be
checked in to the students' CVS repository. When the students synchronize with
CVS next time, they will get two data files that contain the results of the testcases. Remote Testing View will look into the tests/
directory and display the results in a view as part of eclipse. The frequency
that the remote testing runs may vary as needed.
Below is a screen-shot of
the Remote Testing view. This
screen-shot will be used as an example as we walk through different features of
the plug-in.
a.
arithmetic.decaf
i.
This testcase exercised 32.14% of the branches in the reference
solution.
ii.
Both the student
solution and the reference solution FINISHED
execution.
iii.
The "No
Input" row says it failed.
iv.
That is because the
student's output and the reference's output don't match.
b.
dividebyzero.decaf
v.
This testcase exercised 29.76% of the branches in the reference
solution.
vi.
Just as above,
both solutions FINISHED execution.
vii.
Since the
student's output and the reference's output match.
viii.
This testcase is considered passed.
c.
illegal.decaf
ix.
8.33% of the
branches are executed
x.
Both student
solution and the reference solution recognized the decaf program contains SYNTAX ERROR
xi.
This testcase is considered passed
regardless whether the output match or not.
You can right click in the
plug-in to do some manipulation with the view. "Show All Testcase Results" will expand the tree to the level
where all testcase files are shown but not the input
files. This gives you a concise view to see how many of them passed. "Show
All Input Results" will expand the tree to the input files level including
the testcase files. This representation lets you see
the output of both of your project and the reference solution. On the top right
corner of the view, there is a "Refresh" button that you can use to
refresh the view.