openssh/regress
Darren Tucker f899e6a526 20050312
- (dtucker) [regress/test-exec.sh] DEBUG can cause problems where debug
   output ends up in the client's output, causing regress failures.  Found
   by Corinna Vinschen.

(got 4.0 branch and HEAD slightly askew, this is to resync)
2005-03-14 23:02:46 +11:00
..
agent-getpeereid.sh - [Makefile.in regress/agent-getpeereid.sh] config.h is now in $BUILDDIR 2003-09-04 18:04:16 +10:00
agent-ptrace.sh - (tim) [regress/agent-ptrace.sh] add another possible gdb error. 2005-03-02 21:49:56 -08:00
agent-timeout.sh - (dtucker) [regress/agent-timeout.sh] Timeout of 5 sec is borderline for 2003-09-13 09:46:03 +10:00
agent.sh - [regress/agent.sh regress/proto-version.sh regress/ssh-com.sh 2003-09-04 15:16:56 +10:00
banner.sh - dtucker@cvs.openbsd.org 2003/10/11 11:49:49 2004-01-30 12:58:51 +11:00
broken-pipe.sh
brokenkeys.sh - djm@cvs.openbsd.org 2004/10/29 23:59:22 2004-12-06 23:07:37 +11:00
bsd.regress.mk
connect-privsep.sh
connect.sh
dsa_ssh2.prv
dsa_ssh2.pub
dynamic-forward.sh Resync Ids 2004-12-06 22:58:11 +11:00
envpass.sh - djm@cvs.openbsd.org 2005/03/04 08:48:46 2005-03-07 18:35:34 +11:00
exit-status.sh
forwarding.sh - (dtucker) [dynamic-forward.sh forwarding.sh sftp-batch.sh (all regress/)] 2003-09-07 09:28:03 +10:00
keygen-change.sh
keyscan.sh
login-timeout.sh - djm@cvs.openbsd.org 2005/02/27 23:13:36 2005-03-07 18:34:04 +11:00
Makefile - djm@cvs.openbsd.org 2005/03/04 08:48:46 2005-03-07 18:35:34 +11:00
multiplex.sh - dtucker@cvs.openbsd.org 2005/02/27 11:33:30 2005-03-07 18:33:02 +11:00
proto-mismatch.sh
proto-version.sh - [regress/agent.sh regress/proto-version.sh regress/ssh-com.sh 2003-09-04 15:16:56 +10:00
proxy-connect.sh
README.regress - (dtucker) [regress/README.regress] Note compatibility issues with GNU head. 2004-08-17 22:31:32 +10:00
reconfigure.sh - [regress/reconfigure/sh] Make path to sshd fully qualified if required. 2003-09-04 13:59:48 +10:00
reexec.sh - djm@cvs.openbsd.org 2004/10/08 02:01:50 2004-12-06 23:05:52 +11:00
rekey.sh - (dtucker) [regress/rekey.sh] Touch datafile before filling with dd, since 2004-12-20 12:35:42 +11:00
rsa_openssh.prv
rsa_openssh.pub
rsa_ssh2.prv
runtests.sh
scp-ssh-wrapper.sh - dtucker@cvs.openbsd.org 2004/06/13 13:51:02 2004-06-16 20:15:59 +10:00
scp.sh - (dtucker) [regress/scp.sh] Use portable-friendly $DIFFOPTs in new test. 2004-12-07 11:26:15 +11:00
sftp-badcmds.sh [Makefile.in regress/sftp-badcmds.sh regress/test-exec.sh] 2004-02-12 07:17:10 -08:00
sftp-batch.sh - djm@cvs.openbsd.org 2004/01/13 09:49:06 2004-01-30 13:02:55 +11:00
sftp-cmds.sh - (dtucker) [regress/sftp-cmds.sh] Skip quoting test on Cygwin, since 2004-04-16 17:58:28 +10:00
sftp-glob.sh - fgsch@cvs.openbsd.org 2004/12/10 01:31:30 2005-03-07 18:25:53 +11:00
sftp.sh - [regress/sftp.sh] Remove dependency on /dev/stdin. 2003-09-04 15:00:55 +10:00
ssh-com-client.sh - markus@cvs.openbsd.org 2004/02/24 17:06:52 2004-02-29 20:33:51 +11:00
ssh-com-keygen.sh - markus@cvs.openbsd.org 2004/02/24 17:06:52 2004-02-29 20:33:51 +11:00
ssh-com-sftp.sh - markus@cvs.openbsd.org 2004/02/24 17:06:52 2004-02-29 20:33:51 +11:00
ssh-com.sh - markus@cvs.openbsd.org 2004/02/24 17:06:52 2004-02-29 20:33:51 +11:00
sshd-log-wrapper.sh - dtucker@cvs.openbsd.org 2005/02/27 11:33:30 2005-03-07 18:33:02 +11:00
stderr-after-eof.sh [regress/agent-ptrace.sh regress/dynamic-forward.sh 2003-09-11 22:32:17 -07:00
stderr-data.sh - [regress/sftp-badcmds.sh regress/sftp-cmds.sh regress/sftp.sh 2003-09-04 14:54:39 +10:00
t4.ok
t5.ok
test-exec.sh 20050312 2005-03-14 23:02:46 +11:00
transfer.sh - [regress/sftp-badcmds.sh regress/sftp-cmds.sh regress/sftp.sh 2003-09-04 14:54:39 +10:00
try-ciphers.sh - (tim) [regress/try-ciphers.sh] "if ! some_command" is not portable. 2004-06-22 20:53:02 -07:00
yes-head.sh [regress/yes-head.sh] shell portability fix. 2003-09-12 13:21:14 -07:00

Overview.

$ ./configure && make tests

You'll see some progress info. A failure will cause either the make to
abort or the driver script to report a "FATAL" failure.

The test consists of 2 parts. The first is the file-based tests which is
driven by the Makefile, and the second is a set of network or proxycommand
based tests, which are driven by a driver script (test-exec.sh) which is
called multiple times by the Makefile.

Failures in the first part will cause the Makefile to return an error.
Failures in the second part will print a "FATAL" message for the failed
test and continue.

OpenBSD has a system-wide regression test suite. OpenSSH Portable's test
suite is based on OpenBSD's with modifications.


Environment variables.

SUDO: path to sudo command, if desired. Note that some systems (notably
	systems using PAM) require sudo to execute some tests.
TEST_SSH_TRACE: set to "yes" for verbose output from tests 
TEST_SSH_QUIET: set to "yes" to suppress non-fatal output.
TEST_SSH_x: path to "ssh" command under test, where x=SSH,SSHD,SSHAGENT,SSHADD
	SSHKEYGEN,SSHKEYSCAN,SFTP,SFTPSERVER
OBJ: used by test scripts to access build dir.
TEST_SHELL: shell used for running the test scripts.
TEST_SSH_PORT: TCP port to be used for the listening tests.
TEST_SSH_SSH_CONFOTPS: Configuration directives to be added to ssh_config
	before running each test.
TEST_SSH_SSHD_CONFOTPS: Configuration directives to be added to sshd_config
	before running each test.


Individual tests.

You can run an individual test from the top-level Makefile, eg:
$ make tests LTESTS=agent-timeout

If you need to manipulate the environment more you can invoke test-exec.sh
directly if you set up the path to find the binaries under test and the
test scripts themselves, for example:

$ cd regress
$ PATH=`pwd`/..:$PATH:. TEST_SHELL=/bin/sh sh test-exec.sh `pwd` \
    agent-timeout.sh
ok agent timeout test


Files.

test-exec.sh: the main test driver. Sets environment, creates config files
and keys and runs the specified test.

At the time of writing, the individual tests are:
agent-timeout.sh:	agent timeout test
agent.sh:		simple agent test
broken-pipe.sh:		broken pipe test
connect-privsep.sh:	proxy connect with privsep
connect.sh:		simple connect
exit-status.sh:		remote exit status
forwarding.sh:		local and remote forwarding
keygen-change.sh:	change passphrase for key
keyscan.sh:		keyscan
proto-mismatch.sh:	protocol version mismatch
proto-version.sh:	sshd version with different protocol combinations
proxy-connect.sh:	proxy connect
sftp.sh:		basic sftp put/get
ssh-com-client.sh:	connect with ssh.com client
ssh-com-keygen.sh:	ssh.com key import
ssh-com-sftp.sh:	basic sftp put/get with ssh.com server
ssh-com.sh:		connect to ssh.com server
stderr-after-eof.sh:	stderr data after eof
stderr-data.sh:		stderr data transfer
transfer.sh:		transfer data
try-ciphers.sh:		try ciphers
yes-head.sh:		yes pipe head


Problems?

Run the failing test with shell tracing (-x) turned on:
$ PATH=`pwd`/..:$PATH:. sh -x test-exec.sh `pwd` agent-timeout.sh

Failed tests can be difficult to diagnose. Suggestions:
- run the individual test via ./test-exec.sh `pwd` [testname]
- set LogLevel to VERBOSE in test-exec.sh and enable syslogging of
  auth.debug (eg to /var/log/authlog).


Known Issues.

- If your build requires ssh-rand-helper regress tests will fail
  unless ssh-rand-helper is in pre-installed (the path to
  ssh-rand-helper is hard coded).

- Recent GNU coreutils deprecate "head -[n]": this will cause the yes-head
  test to fail.  The old behaviour can be restored by setting (and
  exporting) _POSIX2_VERSION=199209 before running the tests.

$Id: README.regress,v 1.9 2004/08/17 12:31:33 dtucker Exp $