summaryrefslogtreecommitdiff
path: root/static/freebsd/man7/tests.7 3.html
diff options
context:
space:
mode:
authorJacob McDonnell <jacob@jacobmcdonnell.com>2026-04-25 19:55:43 -0400
committerJacob McDonnell <jacob@jacobmcdonnell.com>2026-04-25 19:55:43 -0400
commitac5e55f5f2af5b92794c2aded46c6bae85b5f5ed (patch)
tree9367490586c84cba28652e443e3166d66c33b0d9 /static/freebsd/man7/tests.7 3.html
parent253e67c8b3a72b3a4757fdbc5845297628db0a4a (diff)
docs: Added All FreeBSD Manuals
Diffstat (limited to 'static/freebsd/man7/tests.7 3.html')
-rw-r--r--static/freebsd/man7/tests.7 3.html258
1 files changed, 258 insertions, 0 deletions
diff --git a/static/freebsd/man7/tests.7 3.html b/static/freebsd/man7/tests.7 3.html
new file mode 100644
index 00000000..cd0d4f0e
--- /dev/null
+++ b/static/freebsd/man7/tests.7 3.html
@@ -0,0 +1,258 @@
+<table class="head">
+ <tr>
+ <td class="head-ltitle">TESTS(7)</td>
+ <td class="head-vol">Miscellaneous Information Manual</td>
+ <td class="head-rtitle">TESTS(7)</td>
+ </tr>
+</table>
+<div class="manual-text">
+<section class="Sh">
+<h1 class="Sh" id="NAME"><a class="permalink" href="#NAME">NAME</a></h1>
+<p class="Pp"><code class="Nm">tests</code> &#x2014;
+ <span class="Nd">introduction to the <span class="Ux">FreeBSD</span> Test
+ Suite</span></p>
+</section>
+<section class="Sh">
+<h1 class="Sh" id="DESCRIPTION"><a class="permalink" href="#DESCRIPTION">DESCRIPTION</a></h1>
+<p class="Pp">The <span class="Ux">FreeBSD</span> Test Suite provides a
+ collection of automated tests for two major purposes. On one hand, the test
+ suite aids
+ <a class="permalink" href="#developers"><i class="Em" id="developers">developers</i></a>
+ to detect bugs and regressions when they modify the source tree. On the
+ other hand, it allows
+ <a class="permalink" href="#end"><i class="Em" id="end">end users</i></a>
+ (and, in particular, system administrators) to verify that fresh
+ installations of the <span class="Ux">FreeBSD</span> operating system behave
+ correctly on their hardware platform and also to ensure that the system does
+ not suffer from regressions during regular operation and maintenance.</p>
+<p class="Pp">The <span class="Ux">FreeBSD</span> Test Suite can be found in the
+ <span class="Pa">/usr/tests</span> hierarchy.</p>
+<p class="Pp">This manual page describes how to run the test suite and how to
+ configure some of its optional features. For information on writing the
+ tests, see <a class="Xr">atf(7)</a>.</p>
+<section class="Ss">
+<h2 class="Ss" id="Installing_the_test_suite"><a class="permalink" href="#Installing_the_test_suite">Installing
+ the test suite</a></h2>
+<p class="Pp">If the <span class="Pa">/usr/tests</span> directory is missing,
+ then you will have to enable the build of the test suite, rebuild your
+ system and install the results. You can do so by setting
+ &#x2018;WITH_TESTS=yes&#x2019; in your <span class="Pa">/etc/src.conf</span>
+ file (see <a class="Xr">src.conf(5)</a> for details) and rebuilding the
+ system as described in <a class="Xr">build(7)</a>.</p>
+</section>
+<section class="Ss">
+<h2 class="Ss" id="When_to_run_the_tests?"><a class="permalink" href="#When_to_run_the_tests?">When
+ to run the tests?</a></h2>
+<p class="Pp">Before diving into the details of how to run the test suite, here
+ are some scenarios in which you should run it:</p>
+<ul class="Bl-bullet Bd-indent">
+ <li>After a fresh installation of <span class="Ux">FreeBSD</span> to ensure
+ that the system works correctly on your hardware platform.</li>
+ <li>After an upgrade of <span class="Ux">FreeBSD</span> to a different version
+ to ensure that the new code works well on your hardware platform and that
+ the upgrade did not introduce regressions in your configuration.</li>
+ <li>After modifying the source tree to detect any new bugs and/or
+ regressions.</li>
+ <li>Periodically, maybe from a <a class="Xr">cron(8)</a> job, to ensure that
+ any changes to the system (such as the installation of third-party
+ packages or manual modifications to configuration files) do not introduce
+ unexpected failures.</li>
+</ul>
+</section>
+<section class="Ss">
+<h2 class="Ss" id="Running_the_tests"><a class="permalink" href="#Running_the_tests">Running
+ the tests</a></h2>
+<p class="Pp">By default, Kyua looks for tests in the current directory. To run
+ the whole test suite, either use the <code class="Fl">-k</code> option to
+ specify the top-level <span class="Pa">Kyuafile</span>:</p>
+<div class="Bd Pp Bd-indent Li">
+<pre>$ kyua test -k /usr/tests/Kyuafile</pre>
+</div>
+<p class="Pp">or just change to the test suite root before running Kyua:</p>
+<div class="Bd Pp Bd-indent Li">
+<pre>$ cd /usr/tests
+$ kyua test</pre>
+</div>
+<p class="Pp">The above will iterate through all test programs in
+ <span class="Pa">/usr/tests</span> recursively, execute them, store their
+ results and debugging data in Kyua's database (by default in
+ <span class="Pa">~/.kyua/store/</span>), and print a summary of the results.
+ This summary includes a brief count of all total tests run and how many of
+ them failed.</p>
+<p class="Pp">It is possible to restrict which tests to run by providing their
+ names, or a portion of their path, on the command line. For example, this
+ would execute all of the tests provided for the <a class="Xr">cp(1)</a> and
+ <a class="Xr">stat(1)</a> utilities:</p>
+<div class="Bd Pp Bd-indent Li">
+<pre>$ cd /usr/tests
+$ kyua test bin/cp usr.bin/stat</pre>
+</div>
+<p class="Pp">This would execute only one of the two test programs provided for
+ <a class="Xr">stat(1)</a>:</p>
+<div class="Bd Pp Bd-indent Li">
+<pre>$ cd /usr/tests
+$ kyua test usr.bin/stat/stat_test</pre>
+</div>
+<p class="Pp">This would execute just a single test case:</p>
+<div class="Bd Pp Bd-indent Li">
+<pre>$ cd /usr/tests
+$ kyua test usr.bin/stat/stat_test:t_flag</pre>
+</div>
+<p class="Pp">Finally, this would execute that test case in debug mode:</p>
+<div class="Bd Pp Bd-indent Li">
+<pre>$ cd /usr/tests
+$ kyua debug -p usr.bin/stat/stat_test:t_flag</pre>
+</div>
+<p class="Pp">The <code class="Fl">-p</code> option tells Kyua to pause before
+ cleanup so you can inspect the temporary directory to better understand why
+ the test failed.</p>
+<p class="Pp">Note that some tests may require root privileges to execute:</p>
+<div class="Bd Pp Bd-indent Li">
+<pre>$ cd /usr/tests
+$ kyua debug usr.bin/stat/stat_test:h_flag
+usr.bin/stat/stat_test:h_flag -&gt; skipped: Requires root privileges
+$ sudo kyua debug usr.bin/stat/stat_test:h_flag
+[...]
+usr.bin/stat/stat_test:h_flag -&gt; passed</pre>
+</div>
+<p class="Pp">Conversely, some tests will only work correctly if run as an
+ unprivileged user. This will normally be noted in the test's metadata,
+ causing Kyua to automatically drop privileges before running the test.</p>
+</section>
+<section class="Ss">
+<h2 class="Ss" id="Obtaining_reports_of_the_tests_execution"><a class="permalink" href="#Obtaining_reports_of_the_tests_execution">Obtaining
+ reports of the tests execution</a></h2>
+<p class="Pp">Additional information about the test results can be retrieved by
+ using Kyua's various reporting commands. For example, the following would
+ print a plain-text report of the tests executed in the latest test run and
+ show which ones failed:</p>
+<div class="Bd Pp Bd-indent Li">
+<pre>$ kyua report --verbose</pre>
+</div>
+<p class="Pp">To show the results of an arbitrary test run, use the
+ <code class="Fl">-r</code> option to specify which results file to read:</p>
+<div class="Bd Pp Bd-indent Li">
+<pre>$ kyua report --verbose \
+ -r ~/.kyua/store/results.usr_tests.20260417-173009-335060.db</pre>
+</div>
+<p class="Pp">Keep in mind that if the tests were run as root, the results will
+ have been stored in root's <span class="Pa">kyua</span> directory, and the
+ easiest way to access them will be to run the <code class="Cm">report</code>
+ command as root as well:</p>
+<div class="Bd Pp Bd-indent Li">
+<pre>$ cd /usr/tests
+$ sudo kyua test usr.bin/stat
+$ sudo kyua report --verbose</pre>
+</div>
+<p class="Pp">This example would generate an HTML report ready to be published
+ on a web server:</p>
+<div class="Bd Pp Bd-indent Li">
+<pre>$ kyua report-html --output ~/public_html/tests</pre>
+</div>
+<p class="Pp">For further details on the command-line interface of Kyua, please
+ refer to its manual page <a class="Xr">kyua(1)</a>.</p>
+</section>
+<section class="Ss">
+<h2 class="Ss" id="Configuring_the_tests"><a class="permalink" href="#Configuring_the_tests">Configuring
+ the tests</a></h2>
+<p class="Pp">Some test cases in the <span class="Ux">FreeBSD</span> Test Suite
+ require manual configuration by the administrator before they can be run.
+ Unless certain properties are defined, the tests that require them will be
+ skipped.</p>
+<p class="Pp">Test suites are configured by defining their configuration
+ variables in <span class="Pa">/etc/kyua/kyua.conf</span>. The format of this
+ file is detailed in <a class="Xr">kyua.conf(5)</a>.</p>
+<p class="Pp">The following configuration variables are available in the
+ <span class="Ux">FreeBSD</span> Test Suite:</p>
+<dl class="Bl-tag">
+ <dt id="allow_devfs_side_effects"><var class="Va">allow_devfs_side_effects</var></dt>
+ <dd>If defined, enables tests that may destroy and recreate semipermanent
+ device nodes, like disk devices. Without this variable, tests may still
+ create and destroy devices nodes that are normally transient, like
+ /dev/tap* and /dev/pts*, as long as they clean them up afterwards.
+ However, tests that require this variable have a relaxed cleanup
+ requirement; they must recreate any devices that they destroyed, but not
+ necessarily with the same devnames.</dd>
+ <dt id="allow_sysctl_side_effects"><var class="Va">allow_sysctl_side_effects</var></dt>
+ <dd>Enables tests that change globally significant <a class="Xr">sysctl(8)</a>
+ variables. The tests will undo any changes in their cleanup phases.</dd>
+ <dt id="allow_network_access"><var class="Va">allow_network_access</var></dt>
+ <dd>Enables tests that need to access the network the test host is connected
+ to. Such tests may require properly configured Internet access.</dd>
+ <dt id="disks"><var class="Va">disks</var></dt>
+ <dd>Must be set to a space delimited list of disk device nodes. Tests that
+ need destructive access to disks must use these devices. Tests are not
+ required to preserve any data present on these disks.</dd>
+ <dt id="fibs"><var class="Va">fibs</var></dt>
+ <dd>Must be set to a space delimited list of FIBs (routing tables). Tests that
+ need to modify a routing table may use any of these. Tests will cleanup
+ any new routes that they create.</dd>
+</dl>
+</section>
+<section class="Ss">
+<h2 class="Ss" id="What_to_do_if_something_fails?"><a class="permalink" href="#What_to_do_if_something_fails?">What
+ to do if something fails?</a></h2>
+<p class="Pp">If there is
+ <a class="permalink" href="#any"><i class="Em" id="any">any failure</i></a>
+ during the execution of the test suite, please consider reporting it to the
+ <span class="Ux">FreeBSD</span> developers so that the failure can be
+ analyzed and fixed. To do so, either send a message to the appropriate
+ mailing list or file a problem report. For more details please refer to:</p>
+<ul class="Bl-bullet Bd-indent Bl-compact">
+ <li><a class="Lk" href="https://lists.freebsd.org/">FreeBSD Mailing
+ Lists</a></li>
+ <li><a class="Lk" href="https://www.freebsd.org/support/">Problem
+ Reporting</a></li>
+</ul>
+</section>
+</section>
+<section class="Sh">
+<h1 class="Sh" id="FILES"><a class="permalink" href="#FILES">FILES</a></h1>
+<dl class="Bl-tag Bl-compact">
+ <dt><span class="Pa">/etc/kyua/kyua.conf</span></dt>
+ <dd>System-wide configuration file for <a class="Xr">kyua(1)</a>.</dd>
+ <dt><span class="Pa">~/.kyua/kyua.conf</span></dt>
+ <dd>User-specific configuration file for <a class="Xr">kyua(1)</a>; overrides
+ the system file.</dd>
+ <dt><span class="Pa">~/.kyua/logs/</span></dt>
+ <dd>Default location of Kyua debug logs (one <span class="Pa">.log</span> file
+ per test run).</dd>
+ <dt><span class="Pa">~/.kyua/store/</span></dt>
+ <dd>Default location of Kyua test results (one <span class="Pa">.db</span>
+ file per test run).</dd>
+ <dt><span class="Pa">/usr/tests/</span></dt>
+ <dd>Location of the <span class="Ux">FreeBSD</span> Test Suite.</dd>
+ <dt><span class="Pa">/usr/tests/Kyuafile</span></dt>
+ <dd>Top-level test suite definition file.</dd>
+</dl>
+</section>
+<section class="Sh">
+<h1 class="Sh" id="SEE_ALSO"><a class="permalink" href="#SEE_ALSO">SEE
+ ALSO</a></h1>
+<p class="Pp"><a class="Xr">kyua(1)</a>, <a class="Xr">atf(7)</a>,
+ <a class="Xr">build(7)</a>, <a class="Xr">development(7)</a></p>
+</section>
+<section class="Sh">
+<h1 class="Sh" id="HISTORY"><a class="permalink" href="#HISTORY">HISTORY</a></h1>
+<p class="Pp">The <span class="Ux">FreeBSD</span> Test Suite first appeared in
+ <span class="Ux">FreeBSD 10.1</span> and was installed by default in
+ <span class="Ux">FreeBSD 11.0</span>.</p>
+<p class="Pp">The <code class="Nm">tests</code> manual page first appeared in
+ <span class="Ux">NetBSD 6.0</span> and was later ported to
+ <span class="Ux">FreeBSD 10.1</span>.</p>
+<p class="Pp">The test driver, <a class="Xr">kyua(1)</a>, was imported as part
+ of the base system in <span class="Ux">FreeBSD 13.0</span>, previously being
+ available only in <a class="Xr">ports(7)</a>.</p>
+</section>
+<section class="Sh">
+<h1 class="Sh" id="AUTHORS"><a class="permalink" href="#AUTHORS">AUTHORS</a></h1>
+<p class="Pp"><span class="An">Julio Merino</span>
+ &lt;<a class="Mt" href="mailto:jmmv@FreeBSD.org">jmmv@FreeBSD.org</a>&gt;</p>
+</section>
+</div>
+<table class="foot">
+ <tr>
+ <td class="foot-date">April 17, 2026</td>
+ <td class="foot-os">FreeBSD 15.0</td>
+ </tr>
+</table>