1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
|
<table class="head">
<tr>
<td class="head-ltitle">IAVF(4)</td>
<td class="head-vol">Device Drivers Manual</td>
<td class="head-rtitle">IAVF(4)</td>
</tr>
</table>
<div class="manual-text">
<section class="Sh">
<h1 class="Sh" id="NAME"><a class="permalink" href="#NAME">NAME</a></h1>
<p class="Pp"><code class="Nm">iavf</code> — <span class="Nd">Intel
Ethernet Adaptive Virtual Function Driver</span></p>
</section>
<section class="Sh">
<h1 class="Sh" id="SYNOPSIS"><a class="permalink" href="#SYNOPSIS">SYNOPSIS</a></h1>
<p class="Pp">To compile this driver into the kernel, place the following lines
in your kernel configuration file:</p>
<div class="Bd Pp Bd-indent"><code class="Cd">device iflib</code>
<br/>
<code class="Cd">device iavf</code></div>
<p class="Pp">To load the driver as a module at boot time, place the following
lines in <a class="Xr">loader.conf(5)</a>:</p>
<div class="Bd Pp Bd-indent Li">
<pre>if_iavf_load="YES"</pre>
</div>
</section>
<section class="Sh">
<h1 class="Sh" id="DESCRIPTION"><a class="permalink" href="#DESCRIPTION">DESCRIPTION</a></h1>
<p class="Pp">The <code class="Nm">iavf</code> driver provides support for any
PCI Virtual Function created from certain Intel Ethernet devices. This
driver is compatible with virtual functions bound to devices based on the
following:</p>
<p class="Pp"></p>
<ul class="Bl-bullet Bl-compact">
<li>Intel® Ethernet Controller E810-C</li>
<li>Intel® Ethernet Controller E810-XXV</li>
<li>Intel® Ethernet Connection E822-C</li>
<li>Intel® Ethernet Connection E822-L</li>
<li>Intel® Ethernet Connection E823-C</li>
<li>Intel® Ethernet Connection E823-L</li>
<li>Intel® Ethernet Controller I710</li>
<li>Intel® Ethernet Controller X710</li>
<li>Intel® Ethernet Controller XL710</li>
<li>Intel® Ethernet Network Connection X722</li>
<li>Intel® Ethernet Controller XXV710</li>
<li>Intel® Ethernet Controller V710</li>
</ul>
<p class="Pp">The associated Physical Function (PF) drivers for this VF driver
are:</p>
<p class="Pp"></p>
<ul class="Bl-bullet Bl-compact">
<li><a class="Xr">ice(4)</a></li>
<li><a class="Xr">ixl(4)</a></li>
</ul>
<p class="Pp">For questions related to hardware requirements, refer to the
documentation supplied with your Intel Ethernet Adapter. All hardware
requirements listed apply to use with <span class="Ux">FreeBSD</span>.</p>
<section class="Ss">
<h2 class="Ss" id="The_VF_Driver"><a class="permalink" href="#The_VF_Driver">The
VF Driver</a></h2>
<p class="Pp">The VF driver is normally used in a virtualized environment where
a host driver manages SR-IOV, and provides a VF device to the guest.</p>
<p class="Pp">In the <span class="Ux">FreeBSD</span> guest, the iavf driver
would be loaded and will function using the VF device assigned to it.</p>
<p class="Pp">The VF driver provides most of the same functionality as the core
driver, but is actually a subordinate to the host. Access to many controls
is accomplished by a request to the host via what is called the "Admin
queue." These are startup and initialization events, however; once in
operation, the device is self-contained and should achieve near native
performance.</p>
<p class="Pp">Some notable limitations of the VF environment:</p>
<ul class="Bl-bullet">
<li>The PF can configure the VF to allow promiscuous mode, using a
configuration parameter in <a class="Xr">iovctl.conf(5)</a>; otherwise,
promiscuous mode will not work</li>
<li>Media info is not available from the PF, so the active media will always
be displayed as auto in <a class="Xr">ifconfig(8)</a></li>
</ul>
</section>
<section class="Ss">
<h2 class="Ss" id="Adaptive_Virtual_Function"><a class="permalink" href="#Adaptive_Virtual_Function">Adaptive
Virtual Function</a></h2>
<p class="Pp">Adaptive Virtual Function (AVF) allows the virtual function
driver, or VF, to adapt to changing feature sets of the physical function
driver (PF) with which it is associated. This allows system administrators
to update a PF without having to update all the VFs associated with it. All
AVFs have a single common device ID and branding string.</p>
<p class="Pp">AVFs have a minimum set of features known as "base
mode," but may provide additional features depending on what features
are available in the PF with which the AVF is associated. The following are
base mode features:</p>
<ul class="Bl-bullet Bl-compact">
<li>4 Queue Pairs (QP) and associated Configuration Status Registers (CSRs)
for Tx/Rx</li>
<li>iavf descriptors and ring format</li>
<li>Descriptor write-back completion</li>
<li>1 control queue, with iavf descriptors, CSRs and ring format</li>
<li>5 MSI-X interrupt vectors and corresponding iavf CSRs</li>
<li>1 Interrupt Throttle Rate (ITR) index</li>
<li>1 Virtual Station Interface (VSI) per VF</li>
<li>1 Traffic Class (TC), TC0</li>
<li>Receive Side Scaling (RSS) with 64 entry indirection table and key,
configured through the PF</li>
<li>1 unicast MAC address reserved per VF</li>
<li>8 MAC address filters for each VF on an Intel® Ethernet 800 Series
device</li>
<li>16 MAC address filters for each VF on an Intel® Ethernet 700 Series
device</li>
<li>Stateless offloads - non-tunneled checksums</li>
<li>AVF device ID</li>
<li>HW mailbox is used for VF to PF communications</li>
</ul>
</section>
</section>
<section class="Sh">
<h1 class="Sh" id="CONFIGURATION_AND_TUNING"><a class="permalink" href="#CONFIGURATION_AND_TUNING">CONFIGURATION
AND TUNING</a></h1>
<section class="Ss">
<h2 class="Ss" id="Important_System_Configuration_Changes"><a class="permalink" href="#Important_System_Configuration_Changes">Important
System Configuration Changes</a></h2>
<p class="Pp">It is important to note that 100G operation can generate high
numbers of interrupts, often incorrectly being interpreted as a storm
condition in the kernel. It is suggested that this be resolved by setting
<var class="Va">hw.intr_storm_threshold</var> to 0.</p>
<p class="Pp">The default is 1000.</p>
<p class="Pp">Best throughput results are seen with a large MTU; use 9706 if
possible. The default number of descriptors per ring is 1024. Increasing
this may improve performance, depending on your use case.</p>
</section>
<section class="Ss">
<h2 class="Ss" id="Configuring_for_no_iflib"><a class="permalink" href="#Configuring_for_no_iflib">Configuring
for no iflib</a></h2>
<p class="Pp"><a class="Xr">iflib(4)</a> is a common framework for network
interface drivers for <span class="Ux">FreeBSD</span> that uses a shared set
of sysctl names.</p>
<p class="Pp">The default <code class="Nm">iavf</code> driver depends on it, but
it can be compiled without it.</p>
</section>
<section class="Ss">
<h2 class="Ss" id="Jumbo_Frames"><a class="permalink" href="#Jumbo_Frames">Jumbo
Frames</a></h2>
<p class="Pp">Jumbo Frames support is enabled by changing the Maximum
Transmission Unit (MTU) to a value larger than the default value of
1500.</p>
<p class="Pp">Use the <a class="Xr">ifconfig(8)</a> command to increase the MTU
size.</p>
<p class="Pp">To confirm the MTU used between two specific devices, use
<a class="Xr">route(8)</a>:</p>
<div class="Bd Pp Bd-indent Li">
<pre>route get <destination_IP_address></pre>
</div>
<p class="Pp">NOTE:</p>
<ul class="Bl-bullet">
<li>The maximum MTU setting for jumbo frames is 9706. This corresponds to the
maximum jumbo frame size of 9728 bytes.</li>
<li>This driver will attempt to use multiple page-sized buffers to receive
each jumbo packet. This should help to avoid buffer starvation issues when
allocating receive packets.</li>
<li>Packet loss may have a greater impact on throughput when you use jumbo
frames. If you observe a drop in performance after enabling jumbo frames,
enabling flow control may mitigate the issue.</li>
</ul>
</section>
<section class="Ss">
<h2 class="Ss" id="Checksum_Offload"><a class="permalink" href="#Checksum_Offload">Checksum
Offload</a></h2>
<p class="Pp">Checksum offloading supports both TCP and UDP packets and is
supported for both transmit and receive.</p>
<p class="Pp">TSO (TCP Segmentation Offload) supports both IPv4 and IPv6. Both
of these features are enabled and disabled via
<a class="Xr">ifconfig(8)</a>.</p>
<p class="Pp">NOTE:</p>
<ul class="Bl-bullet Bl-compact">
<li>TSO requires Tx checksum; if Tx checksum is disabled then TSO will also be
disabled.</li>
</ul>
</section>
<section class="Ss">
<h2 class="Ss" id="LRO"><a class="permalink" href="#LRO">LRO</a></h2>
<p class="Pp">LRO (Large Receive Offload) may provide Rx performance
improvement. However, it is incompatible with packet-forwarding workloads.
You should carefully evaluate the environment and enable LRO when
possible.</p>
</section>
<section class="Ss">
<h2 class="Ss" id="Rx_and_Tx_Descriptor_Rings"><a class="permalink" href="#Rx_and_Tx_Descriptor_Rings">Rx
and Tx Descriptor Rings</a></h2>
<p class="Pp">Allows you to set the Rx and Tx descriptor rings independently.
Set them via these <a class="Xr">iflib(4)</a> sysctls:</p>
<dl class="Bl-tag">
<dt>dev.iavf.#.iflib.override_nrxds</dt>
<dd style="width: auto;"> </dd>
<dt>dev.iavf.#.iflib.override_ntxds</dt>
<dd style="width: auto;"> </dd>
</dl>
</section>
<section class="Ss">
<h2 class="Ss" id="Link"><a class="permalink" href="#Link">Link-Level Flow
Control (LFC)</a></h2>
<p class="Pp">The VF driver does not have access to flow control settings. It
must be managed from the host side.</p>
</section>
</section>
<section class="Sh">
<h1 class="Sh" id="SEE_ALSO"><a class="permalink" href="#SEE_ALSO">SEE
ALSO</a></h1>
<p class="Pp"><a class="Xr">arp(4)</a>, <a class="Xr">ice(4)</a>,
<a class="Xr">iflib(4)</a>, <a class="Xr">ixl(4)</a>,
<a class="Xr">netintro(4)</a>, <a class="Xr">vlan(4)</a>,
<a class="Xr">ifconfig(8)</a></p>
<p class="Pp">See the “Intel® Ethernet Adapters and Devices User
Guide” for additional information on features. It is available on the
Intel website at either of the following:</p>
<ul class="Bl-bullet">
<li><a class="Lk" href="https://cdrdv2.intel.com/v1/dl/getContent/705831">https://cdrdv2.intel.com/v1/dl/getContent/705831</a></li>
<li><a class="Lk" href="https://www.intel.com/content/www/us/en/download/19373/adapter-user-guide-for-intel-ethernet-adapters.html">https://www.intel.com/content/www/us/en/download/19373/adapter-user-guide-for-intel-ethernet-adapters.html</a></li>
</ul>
<p class="Pp">For information on how to identify your adapter, and for the
latest Intel network drivers, refer to the Intel Support website:
⟨<a class="Lk" href="http://www.intel.com/support">http://www.intel.com/support</a>⟩</p>
</section>
<section class="Sh">
<h1 class="Sh" id="CAVEATS"><a class="permalink" href="#CAVEATS">CAVEATS</a></h1>
<section class="Ss">
<h2 class="Ss" id="Driver_Buffer_Overflow_Fix"><a class="permalink" href="#Driver_Buffer_Overflow_Fix">Driver
Buffer Overflow Fix</a></h2>
<p class="Pp">The fix to resolve CVE-2016-8105, referenced in Intel SA-00069
⟨<a class="Lk" href="https://www.intel.com/content/www/us/en/security-center/advisory/intel-sa-00069.html">https://www.intel.com/content/www/us/en/security-center/advisory/intel-sa-00069.html</a>⟩,
is included in this and future versions of the driver.</p>
</section>
<section class="Ss">
<h2 class="Ss" id="Network_Memory_Buffer_Allocation"><a class="permalink" href="#Network_Memory_Buffer_Allocation">Network
Memory Buffer Allocation</a></h2>
<p class="Pp"><span class="Ux">FreeBSD</span> may have a low number of network
memory buffers (mbufs) by default. If your mbuf value is too low, it may
cause the driver to fail to initialize and/or cause the system to become
unresponsive. You can check to see if the system is mbuf-starved by running
<code class="Li">netstat -m</code>. Increase the number of mbufs by editing
the lines below in <a class="Xr">sysctl.conf(5)</a>:</p>
<div class="Bd Pp Bd-indent Li">
<pre>kern.ipc.nmbclusters
kern.ipc.nmbjumbop
kern.ipc.nmbjumbo9
kern.ipc.nmbjumbo16
kern.ipc.nmbufs</pre>
</div>
<p class="Pp">The amount of memory that you allocate is system specific, and may
require some trial and error. Also, increasing the following in
<a class="Xr">sysctl.conf(5)</a> could help increase network
performance:</p>
<div class="Bd Pp Bd-indent Li">
<pre>kern.ipc.maxsockbuf
net.inet.tcp.sendspace
net.inet.tcp.recvspace
net.inet.udp.maxdgram
net.inet.udp.recvspace</pre>
</div>
</section>
<section class="Ss">
<h2 class="Ss" id="UDP_Stress_Test_Dropped_Packet_Issue"><a class="permalink" href="#UDP_Stress_Test_Dropped_Packet_Issue">UDP
Stress Test Dropped Packet Issue</a></h2>
<p class="Pp">Under small packet UDP stress with the
<code class="Nm">iavf</code> driver, the system may drop UDP packets due to
socket buffers being full. Setting the PF driver's Flow Control variables to
the minimum may resolve the issue.</p>
</section>
<section class="Ss">
<h2 class="Ss" id="Disable_LRO_when_routing/bridging"><a class="permalink" href="#Disable_LRO_when_routing/bridging">Disable
LRO when routing/bridging</a></h2>
<p class="Pp">LRO must be turned off when forwarding traffic.</p>
</section>
</section>
<section class="Sh">
<h1 class="Sh" id="SUPPORT"><a class="permalink" href="#SUPPORT">SUPPORT</a></h1>
<p class="Pp">For general information, go to the Intel support website at
⟨<a class="Lk" href="http://www.intel.com/support/">http://www.intel.com/support/</a>⟩.</p>
<p class="Pp">If an issue is identified with the released source code on a
supported kernel with a supported adapter, email the specific information
related to the issue to
<<a class="Mt" href="mailto:freebsd@intel.com">freebsd@intel.com</a>>.</p>
</section>
<section class="Sh">
<h1 class="Sh" id="LEGAL"><a class="permalink" href="#LEGAL">LEGAL</a></h1>
<p class="Pp">Intel® is a trademark or registered trademark of Intel
Corporation or its subsidiaries in the United States and / or other
countries.</p>
<p class="Pp">Other names and brands may be claimed as the property of
others.</p>
</section>
<section class="Sh">
<h1 class="Sh" id="HISTORY"><a class="permalink" href="#HISTORY">HISTORY</a></h1>
<p class="Pp">The <code class="Nm">iavf</code> device driver first appeared in
<span class="Ux">FreeBSD 10.1</span> under the name
<code class="Nm">ixlv</code>. It was converted to use
<a class="Xr">iflib(4)</a> and renamed in <span class="Ux">FreeBSD
12.4</span>.</p>
</section>
<section class="Sh">
<h1 class="Sh" id="AUTHORS"><a class="permalink" href="#AUTHORS">AUTHORS</a></h1>
<p class="Pp">The <code class="Nm">iavf</code> driver was written by the
<span class="An">Intel Corporation</span>
<<a class="Mt" href="mailto:freebsd@intel.com">freebsd@intel.com</a>></p>
</section>
</div>
<table class="foot">
<tr>
<td class="foot-date">May 21, 2024</td>
<td class="foot-os">FreeBSD 15.0</td>
</tr>
</table>
|