package test

Most package component nmake(1) makefiles contain a test action that runs the regression tests for the component. Although the components may have different test styles, all tests conform to the rule that each failed test emits a line containing the substring FAIL or fail, and these substrings appear in no other output. i.e., grep -ic fail produces the number of failed tests.

All components are tested by running package test and individual components are tested by running nmake test in the corresponding component build directory.


General Problems

The tests are not perfect. There may be false positives and/or negatives due to:
  • locale specific variations -- some of the tests are locale sensitive but do not compensate. e.g., the tests may rely on a specific instantiation of the <ctype.h> macros.
  • version number and/or timestamp inconsistencies -- some software may emit version numbers or timestamps during testing, and these may have changed since the tests were written. The fix is to add a --regress (or equivalent) option to omit such output during testing.


Known Test Failures by Architecture

See the links below for regression test results for the binary packages posted on this site. The test failure output was generated by:
package results failed test

cygwin.i386
darwin.i386
freebsd6.i386
hp.ia64
linux.i386
linux.i386-64
linux.ia64
linux.ppc64
linux.ppc64-64
sgi.mips3
sgi.mips4
sol10.i386
sol10.sun4


August 08, 2012