diff options
| author | Jacob McDonnell <jacob@jacobmcdonnell.com> | 2026-04-25 19:55:43 -0400 |
|---|---|---|
| committer | Jacob McDonnell <jacob@jacobmcdonnell.com> | 2026-04-25 19:55:43 -0400 |
| commit | ac5e55f5f2af5b92794c2aded46c6bae85b5f5ed (patch) | |
| tree | 9367490586c84cba28652e443e3166d66c33b0d9 /static/freebsd/man7/tests.7 3.html | |
| parent | 253e67c8b3a72b3a4757fdbc5845297628db0a4a (diff) | |
docs: Added All FreeBSD Manuals
Diffstat (limited to 'static/freebsd/man7/tests.7 3.html')
| -rw-r--r-- | static/freebsd/man7/tests.7 3.html | 258 |
1 files changed, 258 insertions, 0 deletions
diff --git a/static/freebsd/man7/tests.7 3.html b/static/freebsd/man7/tests.7 3.html new file mode 100644 index 00000000..cd0d4f0e --- /dev/null +++ b/static/freebsd/man7/tests.7 3.html @@ -0,0 +1,258 @@ +<table class="head"> + <tr> + <td class="head-ltitle">TESTS(7)</td> + <td class="head-vol">Miscellaneous Information Manual</td> + <td class="head-rtitle">TESTS(7)</td> + </tr> +</table> +<div class="manual-text"> +<section class="Sh"> +<h1 class="Sh" id="NAME"><a class="permalink" href="#NAME">NAME</a></h1> +<p class="Pp"><code class="Nm">tests</code> — + <span class="Nd">introduction to the <span class="Ux">FreeBSD</span> Test + Suite</span></p> +</section> +<section class="Sh"> +<h1 class="Sh" id="DESCRIPTION"><a class="permalink" href="#DESCRIPTION">DESCRIPTION</a></h1> +<p class="Pp">The <span class="Ux">FreeBSD</span> Test Suite provides a + collection of automated tests for two major purposes. On one hand, the test + suite aids + <a class="permalink" href="#developers"><i class="Em" id="developers">developers</i></a> + to detect bugs and regressions when they modify the source tree. On the + other hand, it allows + <a class="permalink" href="#end"><i class="Em" id="end">end users</i></a> + (and, in particular, system administrators) to verify that fresh + installations of the <span class="Ux">FreeBSD</span> operating system behave + correctly on their hardware platform and also to ensure that the system does + not suffer from regressions during regular operation and maintenance.</p> +<p class="Pp">The <span class="Ux">FreeBSD</span> Test Suite can be found in the + <span class="Pa">/usr/tests</span> hierarchy.</p> +<p class="Pp">This manual page describes how to run the test suite and how to + configure some of its optional features. For information on writing the + tests, see <a class="Xr">atf(7)</a>.</p> +<section class="Ss"> +<h2 class="Ss" id="Installing_the_test_suite"><a class="permalink" href="#Installing_the_test_suite">Installing + the test suite</a></h2> +<p class="Pp">If the <span class="Pa">/usr/tests</span> directory is missing, + then you will have to enable the build of the test suite, rebuild your + system and install the results. You can do so by setting + ‘WITH_TESTS=yes’ in your <span class="Pa">/etc/src.conf</span> + file (see <a class="Xr">src.conf(5)</a> for details) and rebuilding the + system as described in <a class="Xr">build(7)</a>.</p> +</section> +<section class="Ss"> +<h2 class="Ss" id="When_to_run_the_tests?"><a class="permalink" href="#When_to_run_the_tests?">When + to run the tests?</a></h2> +<p class="Pp">Before diving into the details of how to run the test suite, here + are some scenarios in which you should run it:</p> +<ul class="Bl-bullet Bd-indent"> + <li>After a fresh installation of <span class="Ux">FreeBSD</span> to ensure + that the system works correctly on your hardware platform.</li> + <li>After an upgrade of <span class="Ux">FreeBSD</span> to a different version + to ensure that the new code works well on your hardware platform and that + the upgrade did not introduce regressions in your configuration.</li> + <li>After modifying the source tree to detect any new bugs and/or + regressions.</li> + <li>Periodically, maybe from a <a class="Xr">cron(8)</a> job, to ensure that + any changes to the system (such as the installation of third-party + packages or manual modifications to configuration files) do not introduce + unexpected failures.</li> +</ul> +</section> +<section class="Ss"> +<h2 class="Ss" id="Running_the_tests"><a class="permalink" href="#Running_the_tests">Running + the tests</a></h2> +<p class="Pp">By default, Kyua looks for tests in the current directory. To run + the whole test suite, either use the <code class="Fl">-k</code> option to + specify the top-level <span class="Pa">Kyuafile</span>:</p> +<div class="Bd Pp Bd-indent Li"> +<pre>$ kyua test -k /usr/tests/Kyuafile</pre> +</div> +<p class="Pp">or just change to the test suite root before running Kyua:</p> +<div class="Bd Pp Bd-indent Li"> +<pre>$ cd /usr/tests +$ kyua test</pre> +</div> +<p class="Pp">The above will iterate through all test programs in + <span class="Pa">/usr/tests</span> recursively, execute them, store their + results and debugging data in Kyua's database (by default in + <span class="Pa">~/.kyua/store/</span>), and print a summary of the results. + This summary includes a brief count of all total tests run and how many of + them failed.</p> +<p class="Pp">It is possible to restrict which tests to run by providing their + names, or a portion of their path, on the command line. For example, this + would execute all of the tests provided for the <a class="Xr">cp(1)</a> and + <a class="Xr">stat(1)</a> utilities:</p> +<div class="Bd Pp Bd-indent Li"> +<pre>$ cd /usr/tests +$ kyua test bin/cp usr.bin/stat</pre> +</div> +<p class="Pp">This would execute only one of the two test programs provided for + <a class="Xr">stat(1)</a>:</p> +<div class="Bd Pp Bd-indent Li"> +<pre>$ cd /usr/tests +$ kyua test usr.bin/stat/stat_test</pre> +</div> +<p class="Pp">This would execute just a single test case:</p> +<div class="Bd Pp Bd-indent Li"> +<pre>$ cd /usr/tests +$ kyua test usr.bin/stat/stat_test:t_flag</pre> +</div> +<p class="Pp">Finally, this would execute that test case in debug mode:</p> +<div class="Bd Pp Bd-indent Li"> +<pre>$ cd /usr/tests +$ kyua debug -p usr.bin/stat/stat_test:t_flag</pre> +</div> +<p class="Pp">The <code class="Fl">-p</code> option tells Kyua to pause before + cleanup so you can inspect the temporary directory to better understand why + the test failed.</p> +<p class="Pp">Note that some tests may require root privileges to execute:</p> +<div class="Bd Pp Bd-indent Li"> +<pre>$ cd /usr/tests +$ kyua debug usr.bin/stat/stat_test:h_flag +usr.bin/stat/stat_test:h_flag -> skipped: Requires root privileges +$ sudo kyua debug usr.bin/stat/stat_test:h_flag +[...] +usr.bin/stat/stat_test:h_flag -> passed</pre> +</div> +<p class="Pp">Conversely, some tests will only work correctly if run as an + unprivileged user. This will normally be noted in the test's metadata, + causing Kyua to automatically drop privileges before running the test.</p> +</section> +<section class="Ss"> +<h2 class="Ss" id="Obtaining_reports_of_the_tests_execution"><a class="permalink" href="#Obtaining_reports_of_the_tests_execution">Obtaining + reports of the tests execution</a></h2> +<p class="Pp">Additional information about the test results can be retrieved by + using Kyua's various reporting commands. For example, the following would + print a plain-text report of the tests executed in the latest test run and + show which ones failed:</p> +<div class="Bd Pp Bd-indent Li"> +<pre>$ kyua report --verbose</pre> +</div> +<p class="Pp">To show the results of an arbitrary test run, use the + <code class="Fl">-r</code> option to specify which results file to read:</p> +<div class="Bd Pp Bd-indent Li"> +<pre>$ kyua report --verbose \ + -r ~/.kyua/store/results.usr_tests.20260417-173009-335060.db</pre> +</div> +<p class="Pp">Keep in mind that if the tests were run as root, the results will + have been stored in root's <span class="Pa">kyua</span> directory, and the + easiest way to access them will be to run the <code class="Cm">report</code> + command as root as well:</p> +<div class="Bd Pp Bd-indent Li"> +<pre>$ cd /usr/tests +$ sudo kyua test usr.bin/stat +$ sudo kyua report --verbose</pre> +</div> +<p class="Pp">This example would generate an HTML report ready to be published + on a web server:</p> +<div class="Bd Pp Bd-indent Li"> +<pre>$ kyua report-html --output ~/public_html/tests</pre> +</div> +<p class="Pp">For further details on the command-line interface of Kyua, please + refer to its manual page <a class="Xr">kyua(1)</a>.</p> +</section> +<section class="Ss"> +<h2 class="Ss" id="Configuring_the_tests"><a class="permalink" href="#Configuring_the_tests">Configuring + the tests</a></h2> +<p class="Pp">Some test cases in the <span class="Ux">FreeBSD</span> Test Suite + require manual configuration by the administrator before they can be run. + Unless certain properties are defined, the tests that require them will be + skipped.</p> +<p class="Pp">Test suites are configured by defining their configuration + variables in <span class="Pa">/etc/kyua/kyua.conf</span>. The format of this + file is detailed in <a class="Xr">kyua.conf(5)</a>.</p> +<p class="Pp">The following configuration variables are available in the + <span class="Ux">FreeBSD</span> Test Suite:</p> +<dl class="Bl-tag"> + <dt id="allow_devfs_side_effects"><var class="Va">allow_devfs_side_effects</var></dt> + <dd>If defined, enables tests that may destroy and recreate semipermanent + device nodes, like disk devices. Without this variable, tests may still + create and destroy devices nodes that are normally transient, like + /dev/tap* and /dev/pts*, as long as they clean them up afterwards. + However, tests that require this variable have a relaxed cleanup + requirement; they must recreate any devices that they destroyed, but not + necessarily with the same devnames.</dd> + <dt id="allow_sysctl_side_effects"><var class="Va">allow_sysctl_side_effects</var></dt> + <dd>Enables tests that change globally significant <a class="Xr">sysctl(8)</a> + variables. The tests will undo any changes in their cleanup phases.</dd> + <dt id="allow_network_access"><var class="Va">allow_network_access</var></dt> + <dd>Enables tests that need to access the network the test host is connected + to. Such tests may require properly configured Internet access.</dd> + <dt id="disks"><var class="Va">disks</var></dt> + <dd>Must be set to a space delimited list of disk device nodes. Tests that + need destructive access to disks must use these devices. Tests are not + required to preserve any data present on these disks.</dd> + <dt id="fibs"><var class="Va">fibs</var></dt> + <dd>Must be set to a space delimited list of FIBs (routing tables). Tests that + need to modify a routing table may use any of these. Tests will cleanup + any new routes that they create.</dd> +</dl> +</section> +<section class="Ss"> +<h2 class="Ss" id="What_to_do_if_something_fails?"><a class="permalink" href="#What_to_do_if_something_fails?">What + to do if something fails?</a></h2> +<p class="Pp">If there is + <a class="permalink" href="#any"><i class="Em" id="any">any failure</i></a> + during the execution of the test suite, please consider reporting it to the + <span class="Ux">FreeBSD</span> developers so that the failure can be + analyzed and fixed. To do so, either send a message to the appropriate + mailing list or file a problem report. For more details please refer to:</p> +<ul class="Bl-bullet Bd-indent Bl-compact"> + <li><a class="Lk" href="https://lists.freebsd.org/">FreeBSD Mailing + Lists</a></li> + <li><a class="Lk" href="https://www.freebsd.org/support/">Problem + Reporting</a></li> +</ul> +</section> +</section> +<section class="Sh"> +<h1 class="Sh" id="FILES"><a class="permalink" href="#FILES">FILES</a></h1> +<dl class="Bl-tag Bl-compact"> + <dt><span class="Pa">/etc/kyua/kyua.conf</span></dt> + <dd>System-wide configuration file for <a class="Xr">kyua(1)</a>.</dd> + <dt><span class="Pa">~/.kyua/kyua.conf</span></dt> + <dd>User-specific configuration file for <a class="Xr">kyua(1)</a>; overrides + the system file.</dd> + <dt><span class="Pa">~/.kyua/logs/</span></dt> + <dd>Default location of Kyua debug logs (one <span class="Pa">.log</span> file + per test run).</dd> + <dt><span class="Pa">~/.kyua/store/</span></dt> + <dd>Default location of Kyua test results (one <span class="Pa">.db</span> + file per test run).</dd> + <dt><span class="Pa">/usr/tests/</span></dt> + <dd>Location of the <span class="Ux">FreeBSD</span> Test Suite.</dd> + <dt><span class="Pa">/usr/tests/Kyuafile</span></dt> + <dd>Top-level test suite definition file.</dd> +</dl> +</section> +<section class="Sh"> +<h1 class="Sh" id="SEE_ALSO"><a class="permalink" href="#SEE_ALSO">SEE + ALSO</a></h1> +<p class="Pp"><a class="Xr">kyua(1)</a>, <a class="Xr">atf(7)</a>, + <a class="Xr">build(7)</a>, <a class="Xr">development(7)</a></p> +</section> +<section class="Sh"> +<h1 class="Sh" id="HISTORY"><a class="permalink" href="#HISTORY">HISTORY</a></h1> +<p class="Pp">The <span class="Ux">FreeBSD</span> Test Suite first appeared in + <span class="Ux">FreeBSD 10.1</span> and was installed by default in + <span class="Ux">FreeBSD 11.0</span>.</p> +<p class="Pp">The <code class="Nm">tests</code> manual page first appeared in + <span class="Ux">NetBSD 6.0</span> and was later ported to + <span class="Ux">FreeBSD 10.1</span>.</p> +<p class="Pp">The test driver, <a class="Xr">kyua(1)</a>, was imported as part + of the base system in <span class="Ux">FreeBSD 13.0</span>, previously being + available only in <a class="Xr">ports(7)</a>.</p> +</section> +<section class="Sh"> +<h1 class="Sh" id="AUTHORS"><a class="permalink" href="#AUTHORS">AUTHORS</a></h1> +<p class="Pp"><span class="An">Julio Merino</span> + <<a class="Mt" href="mailto:jmmv@FreeBSD.org">jmmv@FreeBSD.org</a>></p> +</section> +</div> +<table class="foot"> + <tr> + <td class="foot-date">April 17, 2026</td> + <td class="foot-os">FreeBSD 15.0</td> + </tr> +</table> |
