1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
|
<table class="head">
<tr>
<td class="head-ltitle">DISKLESS(8)</td>
<td class="head-vol">System Manager's Manual</td>
<td class="head-rtitle">DISKLESS(8)</td>
</tr>
</table>
<div class="manual-text">
<section class="Sh">
<h1 class="Sh" id="NAME"><a class="permalink" href="#NAME">NAME</a></h1>
<p class="Pp"><code class="Nm">diskless</code> — <span class="Nd">booting
a system over the network with PXE</span></p>
</section>
<section class="Sh">
<h1 class="Sh" id="DESCRIPTION"><a class="permalink" href="#DESCRIPTION">DESCRIPTION</a></h1>
<p class="Pp">The ability to boot a machine over the network is useful for
<a class="permalink" href="#diskless"><i class="Em" id="diskless">diskless</i></a>
or
<a class="permalink" href="#dataless"><i class="Em" id="dataless">dataless</i></a>
machines, or as a temporary measure while repairing or re-installing file
systems on a local disk. This file provides a general description of the
interactions between a client and its server when a client is booting over
the network.</p>
</section>
<section class="Sh">
<h1 class="Sh" id="OPERATION"><a class="permalink" href="#OPERATION">OPERATION</a></h1>
<p class="Pp">When booting a system over the network, there are three phases of
interaction between client and server:</p>
<ol class="Bl-enum">
<li>The stage-1 bootstrap, typically PXE built into your Ethernet card, loads
a second-stage boot program.</li>
<li>The second-stage boot program, typically <a class="Xr">pxeboot(8)</a>,
loads modules and the kernel, and boots the kernel.</li>
<li>The kernel NFS mounts the root directory and continues from there.</li>
</ol>
<p class="Pp">Each of these phases are described in further detail below.</p>
<p class="Pp">First, the stage-1 bootstrap loads the stage-2 boot program over
the network. The stage-1 bootstrap typically uses BOOTP or DHCP to obtain
the filename to load, then uses TFTP to load the file. This file is
typically called <span class="Pa">pxeboot</span>, and should be copied from
<span class="Pa">/boot/pxeboot</span> into the TFTP directory on the server,
which is typically <span class="Pa">/tftpdir</span>.</p>
<p class="Pp">The stage-2 boot program then loads additional modules and the
kernel. These files may not exist on the DHCP or BOOTP server. You can use
the <code class="Ic">next-server</code> option available in DHCP
configurations to specify the server holding the second stage boot files and
kernel. The stage-2 program uses NFS or TFTP to obtain these files. By
default, NFS is used. If you are using <a class="Xr">pxeboot(8)</a>, you can
install a version that uses TFTP by setting
<code class="Li">LOADER_TFTP_SUPPORT=YES</code> in your
<a class="Xr">make.conf(5)</a>, then recompiling and reinstalling
<a class="Xr">pxeboot(8)</a> via the command listed below. It is often
necessary to use TFTP here so you can place a custom kernel in
<span class="Pa">/tftpdir/</span>. If you use NFS and do not have a custom
root file system for the <code class="Nm">diskless</code> client, the
stage-2 boot will load your server's kernel as the kernel for the
<code class="Nm">diskless</code> machine, which may not be what you want to
have happen.</p>
<div class="Bd Pp Bd-indent Li">
<pre>cd /usr/src/stand
make clean; make; make install
cp /boot/pxeboot /tftpdir/</pre>
</div>
<p class="Pp">In phase 3, the kernel acquires IP networking configuration in one
of two ways, and then proceeds to mount the root file system and start
operation. If the phase 2 loader supports passing network configuration to
the kernel using the kernel environment, then the kernel will configure the
network interface using that information. Otherwise, it must use DHCP or
BOOTP to acquire configuration information. The boot scripts recognize a
<code class="Nm">diskless</code> startup and perform the actions found in
<span class="Pa">/etc/rc.d/resolv</span>,
<span class="Pa">/etc/rc.d/tmp</span>,
<span class="Pa">/etc/rc.d/var</span>, and
<span class="Pa">/etc/rc.initdiskless</span>.</p>
</section>
<section class="Sh">
<h1 class="Sh" id="CONFIGURATION"><a class="permalink" href="#CONFIGURATION">CONFIGURATION</a></h1>
<p class="Pp">In order to run a <code class="Nm">diskless</code> client, you
need the following:</p>
<ul class="Bl-bullet">
<li>An NFS server which exports a root and <span class="Pa">/usr</span>
partitions with appropriate permissions. The
<code class="Nm">diskless</code> scripts work with read-only partitions,
as long as root is exported with <code class="Fl">-maproot</code>=0 so
that some system files can be accessed. As an example,
<span class="Pa">/etc/exports</span> can contain the following lines:
<div class="Bd Pp Bd-indent Li">
<pre><ROOT> -ro -maproot=0 -alldirs <list of diskless clients>
/usr -ro -alldirs <list of diskless clients></pre>
</div>
<p class="Pp">where ⟨ROOT⟩ is the mount point on the server of
the root partition. The script
<span class="Pa">/usr/share/examples/diskless/clone_root</span> can be
used to create a shared read-only root partition, but in many cases you
may decide to export (again as read-only) the root directory used by the
server itself.</p>
</li>
<li>A BOOTP or DHCP server. <a class="Xr">bootpd(8)</a> can be enabled by
uncommenting the “<code class="Li">bootps</code>” line in
<span class="Pa">/etc/inetd.conf</span>. A sample
<span class="Pa">/etc/bootptab</span> can be the following:
<div class="Bd Pp Bd-indent Li">
<pre> .default:\
hn:ht=1:vm=rfc1048:\
:sm=255.255.255.0:\
:sa=<SERVER>:\
:gw=<GATEWAY>:\
:rp="<SERVER>:<ROOT>":
<CLIENT>:ha=0123456789ab:tc=.default</pre>
</div>
<p class="Pp">where ⟨SERVER⟩, ⟨GATEWAY⟩ and
⟨ROOT⟩ have the obvious meanings.</p>
</li>
<li>A properly initialized root partition. The script
<span class="Pa">/usr/share/examples/diskless/clone_root</span> can help
in creating it, using the server's root partition as a reference. If you
are just starting out, you should simply use the server's own root
directory, <span class="Pa">/</span>, and not try to clone it.
<p class="Pp">You often do not want to use the same
<span class="Pa">rc.conf</span> or <span class="Pa">rc.local</span>
files for the <code class="Nm">diskless</code> boot as you do on the
server. The <code class="Nm">diskless</code> boot scripts provide a
mechanism through which you can override various files in
<span class="Pa">/etc</span> (as well as other subdirectories of
root).</p>
<p class="Pp">One difference that you should pay particular attention to is
the value of <var class="Va">local_startup</var> in
<span class="Pa">/etc/defaults/rc.conf</span>. A typical value for a
<code class="Nm">diskless</code> boot is
<var class="Va">mountcritremote</var>, however your needs may be
different.</p>
<p class="Pp">The scripts provide four overriding directories situated in
<span class="Pa">/conf/base</span>,
<span class="Pa">/conf/default</span>,
<span class="Pa">/conf/<broadcast-ip></span>, and
<span class="Pa">/conf/<machine-ip></span>. You should always
create <span class="Pa">/conf/base/etc</span>, which will entirely
replace the server's <span class="Pa">/etc</span> on the
<code class="Nm">diskless</code> machine. You can clone the server's
<span class="Pa">/etc</span> here or you can create a special file which
tells the <code class="Nm">diskless</code> boot scripts to remount the
server's <span class="Pa">/etc</span> onto
<span class="Pa">/conf/base/etc</span>. You do this by creating the file
<span class="Pa">/conf/base/etc/diskless_remount</span> containing the
mount point to use as a basis of the <code class="Nm">diskless</code>
machine's <span class="Pa">/etc</span>. For example, the file might
contain:</p>
<p class="Pp"></p>
<div class="Bd Bd-indent"><code class="Li">10.0.0.1:/etc</code></div>
<p class="Pp">Alternatively, if the server contains several independent
roots, the file might contain:</p>
<p class="Pp"></p>
<div class="Bd
Bd-indent"><code class="Li">10.0.0.1:/usr/diskless/4.7-RELEASE/etc</code></div>
<p class="Pp">This would work, but if you copied
<span class="Pa">/usr/diskless/4.7-RELEASE</span> to
<span class="Pa">/usr/diskless/4.8-RELEASE</span> and upgraded the
installation, you would need to modify the
<span class="Pa">diskless_remount</span> files to reflect that move. To
avoid that, paths in <span class="Pa">diskless_remount</span> files
beginning with <span class="Pa">/</span> have the actual path of the
client's root prepended to them so the file could instead contain:</p>
<p class="Pp"></p>
<div class="Bd Bd-indent"><code class="Li">/etc</code></div>
<p class="Pp">The <code class="Nm">diskless</code> scripts create memory
file systems to hold the overridden directories. Only a 5MB partition is
created by default, which may not be sufficient for your purposes. To
override this, you can create the file
<span class="Pa">/conf/base/etc/md_size</span> containing the size, in
512 byte sectors, of the memory disk to create for that directory.</p>
<p class="Pp">You then typically provide file-by-file overrides in the
<span class="Pa">/conf/default/etc</span> directory. At a minimum, you
must provide overrides for <span class="Pa">/etc/fstab</span>,
<span class="Pa">/etc/rc.conf</span>, and
<span class="Pa">/etc/rc.local</span> via
<span class="Pa">/conf/default/etc/fstab</span>,
<span class="Pa">/conf/default/etc/rc.conf</span>, and
<span class="Pa">/conf/default/etc/rc.local</span>.</p>
<p class="Pp">Overrides are hierarchical. You can supply network-specific
defaults in the
<span class="Pa">/conf/</span>⟨<var class="Ar">BROADCASTIP</var>⟩<span class="Pa">/etc</span>
directory, where ⟨<var class="Ar">BROADCASTIP</var>⟩
represents the broadcast IP address of the
<code class="Nm">diskless</code> system as given to it via BOOTP. The
<span class="Pa">diskless_remount</span> and
<span class="Pa">md_size</span> features work in any of these
directories. The configuration feature works on directories other then
<span class="Pa">/etc</span>, you simply create the directory you wish
to replace or override in
<span class="Pa">/conf/{base,default,<broadcast>,<ip>}/*</span>
and work it in the same way that you work
<span class="Pa">/etc</span>.</p>
<p class="Pp">Since you normally clone the server's
<span class="Pa">/etc</span> using the
<span class="Pa">/conf/base/etc/diskless_remount</span>, you might wish
to remove unneeded files from the memory file system. For example, if
the server has a firewall but you do not, you might wish to remove
<span class="Pa">/etc/ipfw.conf</span>. You can do this by creating a
<span class="Pa">/conf/base/</span>⟨<var class="Ar">DIRECTORY</var>⟩<span class="Pa">.remove</span>
file. For example, <span class="Pa">/conf/base/etc.remove</span>, which
contains a list of relative paths that the boot scripts should remove
from the memory file systems.</p>
<p class="Pp">As a minimum, you normally need to have the following in
<span class="Pa">/conf/default/etc/fstab</span></p>
<div class="Bd Pp Bd-indent Li">
<pre><SERVER>:<ROOT> / nfs ro 0 0
<SERVER>:/usr /usr nfs ro 0 0</pre>
</div>
<p class="Pp">You also need to create a customized version of
<span class="Pa">/conf/default/etc/rc.conf</span> which should contain
the startup options for the <code class="Nm">diskless</code> client, and
<span class="Pa">/conf/default/etc/rc.local</span> which could be empty
but prevents the server's own <span class="Pa">/etc/rc.local</span> from
leaking onto the <code class="Nm">diskless</code> system.</p>
<p class="Pp">In <span class="Pa">rc.conf</span>, most likely you will not
need to set <var class="Va">hostname</var> and
<var class="Va">ifconfig_*</var> because these will be already set by
the startup code. Finally, it might be convenient to use a
<code class="Ic">case</code> statement using
<code class="Li">`hostname`</code> as the switch variable to do
machine-specific configuration in case a number of
<code class="Nm">diskless</code> clients share the same configuration
files.</p>
</li>
<li>The kernel for the <code class="Nm">diskless</code> clients, which will be
loaded using NFS or TFTP, must include support for the NFS client:
<p class="Pp"></p>
<div class="Bd Bd-indent"><code class="Cd">options NFSCL</code></div>
<div class="Bd Bd-indent"><code class="Cd">options NFS_ROOT</code></div>
<p class="Pp">If you are using a boot mechanism that does not pass network
configuration to the kernel using the kernel environment, you will also
need to include the following options:</p>
<p class="Pp"></p>
<div class="Bd Bd-indent"><code class="Cd">options BOOTP</code></div>
<div class="Bd Bd-indent"><code class="Cd">options
BOOTP_NFSROOT</code></div>
<div class="Bd Bd-indent"><code class="Cd">options BOOTP_COMPAT</code></div>
<p class="Pp" id="Note"><a class="permalink" href="#Note"><i class="Em">Note</i></a>:
the PXE environment does not require these options.</p>
<p class="Pp">The <code class="Nm">diskless</code> booting environment
relies on memory-backed file systems to support temporary local storage
in the event that the root file system is mounted read-only; as such, it
is necessary to add the following to the device section of the kernel
configuration:</p>
<p class="Pp"></p>
<div class="Bd Bd-indent"><code class="Cd">device md</code></div>
<p class="Pp">If you use the firewall, remember to default to
“open”, or your kernel will not be able to send/receive
the BOOTP packets.</p>
</li>
</ul>
</section>
<section class="Sh">
<h1 class="Sh" id="SECURITY_ISSUES"><a class="permalink" href="#SECURITY_ISSUES">SECURITY
ISSUES</a></h1>
<p class="Pp">Be warned that using unencrypted NFS to mount root and user
partitions may expose information such as encryption keys.</p>
</section>
<section class="Sh">
<h1 class="Sh" id="SEE_ALSO"><a class="permalink" href="#SEE_ALSO">SEE
ALSO</a></h1>
<p class="Pp"><a class="Xr">ethers(5)</a>, <a class="Xr">exports(5)</a>,
<a class="Xr">make.conf(5)</a>, <a class="Xr">bootpd(8)</a>,
<a class="Xr">mountd(8)</a>, <a class="Xr">nfsd(8)</a>,
<a class="Xr">pxeboot(8)</a>, <a class="Xr">reboot(8)</a>,
<a class="Xr">tftpd(8)</a></p>
<p class="Pp"><span class="Pa">ports/net/etherboot</span></p>
</section>
<section class="Sh">
<h1 class="Sh" id="HISTORY"><a class="permalink" href="#HISTORY">HISTORY</a></h1>
<p class="Pp">The <code class="Nm">diskless</code> environment first appeared in
<span class="Ux">FreeBSD 2.2.5</span>.</p>
</section>
<section class="Sh">
<h1 class="Sh" id="BUGS"><a class="permalink" href="#BUGS">BUGS</a></h1>
<p class="Pp">This manpage is probably incomplete.</p>
<p class="Pp"><span class="Ux">FreeBSD</span> sometimes requires to write onto
the root partition, so the startup scripts mount MFS file systems on some
locations (e.g. <span class="Pa">/etc</span> and
<span class="Pa">/var</span>), while trying to preserve the original
content. The process might not handle all cases.</p>
</section>
</div>
<table class="foot">
<tr>
<td class="foot-date">August 11, 2024</td>
<td class="foot-os">FreeBSD 15.0</td>
</tr>
</table>
|