# README

INTRODUCTION

This is the cons-test package, a test suite for the Cons software
construction system.  Cons was originally written by Bob Sidebotham,
and is currently (February 2000) maintained by Rajesh Vaidheeswarran.
The home page for Cons itself can be found at:

	http://www.dsmit.com/cons/

The cons-test suite consists of a number of portable Perl scripts that
test most of the features in Cons, some supporting modules, and a wrapper
Perl script for executing the tests.



CONTENTS

The contents of this release:

	ChangeLog	List of changes.

	README		What you're looking at right now.

	Test/		Subdirectory containing the Test::Cmd
			and Test::Cmd::Cons Perl modules, which
			are used by the tests.

	Tests.txt	Summary documenting what each test does
			(often in too much detail).

	ToDo		A list of additional tests that we'd like to
			have some day.

	cons-test.pl	Wrapper script to execute tests.

	cons.diff	A diff -c of cons.orig (the original version of
			Cons) and cons.pl (the version used for these
			tests).

	cons.orig	The original, released version of Cons that
			was used as the basis for the version of Cons
			that these tests were written for.  See the
			file itself for Version information.

	cons.pl		The version of Cons used for these tests.
			May contain some fixes or enhancements from
			cons.orig (the original, released version;
			see above).

	t/t*.t		Tests.

	t/template	Template for creating additional tests.



THE cons-test.pl SCRIPT

By default, the cons-test.pl script will test the script 'cons' or
'cons.pl' (in that order) in the current directory.

An alternate 'cons' script name may be specified via the -x flag:

	$ perl cons-test.pl -x cons.NEW

	$ perl cons-test.pl -x /usr/foo/cons.experiment/cons

The cons-test.pl script will arrange for each test to use the specified
'cons' script.  The tests will feed the 'cons' script under test to Perl,
so the script need not have execute permission set.

By default, the cons-test.pl script executes all the tests in the
subdirectory 't' if it exists, or the local directory if there is no
't' subdirectory.  Each test is executed with the specified 'cons'
script passed in via the CONS environment variable.  After all the
tests have executed, the cons-test.pl script reports a summary of the
pass/fail score:

	cons-test:  'cons' passed all 35 tests.

	cons-test:  'cons' passed 33 tests, failed 2:
			t/t0003.t
			t/t0026.t

The cons-test.pl script can be given one or more tests as arguments,
in which case it will only execute the specified tests:

	$ perl cons-test.pl t/t0007.t t/t0023.t
	cons-test.pl:  perl -w t/t0007.t
	PASSED
	cons-test.pl:  perl -w t/t0023.t
	PASSED

	cons-test.pl:  'cons' passed all 2 tests.
	$

The cons-test.pl script may also be given environment variable assignments
as arguments.  These will be evaluated in order with the test arguments.
This allows, for example, executing the same test(s) with different
compilers in the same cons-test.pl invocation:

	$ perl cons-test.pl CC=gcc t/t0001.t CC=no_compiler t/t0001.t
	cons-test.pl:  perl -w t/t0001.t
	PASSED
	cons-test.pl:  perl -w t/t0001.t
	FAILED test #1 of cons [single-module Program]:
	no_compiler -c foo.c -o foo.o
	cons: failed to execute "no_compiler" (No such file or directory). Is this an executable on path "/bin:/usr/bin"?
	cons: *** [foo.o] Error 2
	cons: errors constructing foo.o
	FAILED test #1 of cons [single-module Program]

	cons-test.pl:  'cons' passed 1 tests, failed 1:
			t/t0001.t
	$

By default, the cons-test.pl script expects that the environment variables
$CC, $AR, and $RANLIB are set, and will generate warnings if they are not.
A -q option may be used to suppress these warnings.



TESTS

Tests conform to requirements of the Aegis project change supervisor,
which integrates creation and execution of regression tests into the
software development process.  Information about Aegis can be found at:

	http://www.tip.net.au/~millerp/aegis.html

Each test is a completely self-contained Perl script, and may be directly
executed by explicitly passing it to perl (-w flag preferred):

	$ perl -w t/t0001.t

The cryptic names are by Aegis convention only, and could have been named
to reflect the functionality being tested.  I've preserved numbering
them to give order to the tests:  Simpler, underlying functionality
is tested before more complicated features that rely on functionality
tested by prior tests.

Each test creates a temporary work directory under $TMPDIR (under /tmp
by default) and populates it with files from in-line here documents.

Each test checks the return value from every subroutine it calls and ever
command it executes, expecting a successful return value or exit code
from each.  When it's important, each test compares actual command output
against expected output.  Note that, in the usual case, this is actually
output of the executables generated by Cons, not the "cc" command lines
reported as the actions Cons executes to build the test executables.
There are some exceptions, tests which do examine the build actions;
these are noted in the Tests.txt file and in the commentary at the top
of each test.

By default, the tests rely on having normal C compilation, linking and
archiving tools available through the environment's PATH, and expect
that printf() is available through whatever library is linked into a
program via the default Cons build environment.  These tests use the
following environment variables for specification of alternate paths to
various utilities:

	AR	(default:  'ar')	library archiver
	CC	(default:  'cc')	C compiler
	RANLIB	(default:  'ranlib')	ranlib

As described previously, the cons-test.pl script accepts environment
variable assignments in its command-line arguments.  This may be
used, for example, to specify different C compilers for different
test invocations:

	$ perl cons-test.pl CC=gcc t/t0001.t CC=cc t/t0001.t

Each test reports one of three results:

	PASSED
		The Cons script being tested passed this test.

	FAILED test of [functionality]
		The Cons script being tested failed this test.

	NO RESULT for test of [functionality]
		The Cons script could not be tested due to
		some non-Cons problem (e.g., unable to
		create temporary directory or file).

Each test removes its temporary work directory upon completion.
For post facto debugging, the work directory can be preserved (and its
name reported) by setting environment variables:

	PRESERVE
		Don't remove the work directory.

	PRESERVE_PASS
		Don't remove the work directory if the test passes.

	PRESERVE_FAIL
		Don't remove the work directory if the test fails.

	PRESERVE_NO_RESULT
		Don't remove the work directory if there is no
		valid test result.


Changes, enhancements, portability fixes are welcome.

	Steven Knight
	knight@baldmt.com