1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
|
<table class="head">
<tr>
<td class="head-ltitle">ICE(4)</td>
<td class="head-vol">Device Drivers Manual</td>
<td class="head-rtitle">ICE(4)</td>
</tr>
</table>
<div class="manual-text">
<section class="Sh">
<h1 class="Sh" id="NAME"><a class="permalink" href="#NAME">NAME</a></h1>
<p class="Pp"><code class="Nm">ice</code> — <span class="Nd">Intel
Ethernet 800 Series 1GbE to 200GbE driver</span></p>
</section>
<section class="Sh">
<h1 class="Sh" id="SYNOPSIS"><a class="permalink" href="#SYNOPSIS">SYNOPSIS</a></h1>
<p class="Pp"><code class="Cd">device iflib</code>
<br/>
<code class="Cd">device ice</code></p>
<p class="Pp">In <a class="Xr">loader.conf(5)</a>:
<br/>
<code class="Cd">if_ice_load</code>
<br/>
<code class="Cd">hw.ice.enable_health_events</code>
<br/>
<code class="Cd">hw.ice.irdma</code>
<br/>
<code class="Cd">hw.ice.irdma_max_msix</code>
<br/>
<code class="Cd">hw.ice.debug.enable_tx_fc_filter</code>
<br/>
<code class="Cd">hw.ice.debug.enable_tx_lldp_filter</code>
<br/>
<code class="Cd">hw.ice.debug.ice_tx_balance_en</code></p>
<p class="Pp">In <a class="Xr">sysctl.conf(5)</a> or
<a class="Xr">loader.conf(5)</a>:
<br/>
<code class="Cd">dev.ice.#.current_speed</code>
<br/>
<code class="Cd">dev.ice.#.fw_version</code>
<br/>
<code class="Cd">dev.ice.#.ddp_version</code>
<br/>
<code class="Cd">dev.ice.#.pba_number</code>
<br/>
<code class="Cd">dev.ice.#.hw.mac.*</code></p>
</section>
<section class="Sh">
<h1 class="Sh" id="DESCRIPTION"><a class="permalink" href="#DESCRIPTION">DESCRIPTION</a></h1>
<p class="Pp">The <code class="Nm">ice</code> driver provides support for any
PCI Express adapter or LOM (LAN On Motherboard) in the Intel Ethernet 800
Series.</p>
<p class="Pp">The following topics are covered in this manual:</p>
<p class="Pp"></p>
<ul class="Bl-bullet Bl-compact">
<li><a class="Sx" href="#Features">Features</a></li>
<li><a class="Sx" href="#Dynamic_Device_Personalization">Dynamic Device
Personalization</a></li>
<li><a class="Sx" href="#Jumbo_Frames">Jumbo Frames</a></li>
<li><a class="Sx" href="#Remote_Direct_Memory_Access">Remote Direct Memory
Access</a></li>
<li><a class="Sx" href="#RDMA_Monitoring">RDMA Monitoring</a></li>
<li><a class="Sx" href="#Data_Center_Bridging">Data Center Bridging</a></li>
<li><a class="Sx" href="#L3_QoS_Mode">L3 QoS Mode</a></li>
<li><a class="Sx" href="#Firmware_Link_Layer_Discovery_Protocol_Agent">Firmware
Link Layer Discovery Protocol Agent</a></li>
<li><a class="Sx" href="#Link_Level_Flow_Control">Link-Level Flow
Control</a></li>
<li><a class="Sx" href="#Forward_Error_Correction">Forward Error
Correction</a></li>
<li><a class="Sx" href="#Speed_and_Duplex_Configuration">Speed and Duplex
Configuration</a></li>
<li><a class="Sx" href="#Disabling_physical_link_when_the_interface_is_brought_down">Disabling
physical link when the interface is brought down</a></li>
<li><a class="Sx" href="#Firmware_Logging">Firmware Logging</a></li>
<li><a class="Sx" href="#Debug_Dump">Debug Dump</a></li>
<li><a class="Sx" href="#Debugging_PHY_Statistics">Debugging PHY
Statistics</a></li>
<li><a class="Sx" href="#Transmit_Balancing">Transmit Balancing</a></li>
<li><a class="Sx" href="#Thermal_Monitoring">Thermal Monitoring</a></li>
<li><a class="Sx" href="#Network_Memory_Buffer_Allocation">Network Memory
Buffer Allocation</a></li>
<li><a class="Sx" href="#Additional_Utilities">Additional Utilities</a></li>
<li><a class="Sx" href="#Optics_and_auto_negotiation">Optics and
auto-negotiation</a></li>
<li><a class="Sx" href="#PCI_Express_Slot_Bandwidth">PCI-Express Slot
Bandwidth</a></li>
<li><a class="Sx" href="#HARDWARE">HARDWARE</a></li>
<li><a class="Sx" href="#LOADER_TUNABLES">LOADER TUNABLES</a></li>
<li><a class="Sx" href="#SYSCTL_VARIABLES">SYSCTL VARIABLES</a></li>
<li><a class="Sx" href="#INTERRUPT_STORMS">INTERRUPT STORMS</a></li>
<li><a class="Sx" href="#IOVCTL_OPTIONS">IOVCTL OPTIONS</a></li>
<li><a class="Sx" href="#SUPPORT">SUPPORT</a></li>
<li><a class="Sx" href="#SEE_ALSO">SEE ALSO</a></li>
<li><a class="Sx" href="#HISTORY">HISTORY</a></li>
</ul>
<section class="Ss">
<h2 class="Ss" id="Features"><a class="permalink" href="#Features">Features</a></h2>
<p class="Pp">Support for Jumbo Frames is provided via the interface MTU
setting. Selecting an MTU larger than 1500 bytes with the
<a class="Xr">ifconfig(8)</a> utility configures the adapter to receive and
transmit Jumbo Frames. The maximum MTU size for Jumbo Frames is 9706. For
more information, see the <a class="Sx" href="#Jumbo_Frames">Jumbo
Frames</a> section.</p>
<p class="Pp">This driver version supports VLANs. For information on enabling
VLANs, see <a class="Xr">vlan(4)</a>. For additional information on
configuring VLANs, see <a class="Xr">ifconfig(8)</a>'s “VLAN
Parameters” section.</p>
<p class="Pp">Offloads are also controlled via the interface, for instance,
checksumming for both IPv4 and IPv6 can be set and unset, TSO4 and/or TSO6,
and finally LRO can be set and unset.</p>
<p class="Pp">For more information on configuring this device, see
<a class="Xr">ifconfig(8)</a>.</p>
<p class="Pp">The associated Virtual Function (VF) driver for this driver is
<a class="Xr">iavf(4)</a>.</p>
<p class="Pp">The associated RDMA driver for this driver is
<a class="Xr">irdma(4)</a>.</p>
</section>
<section class="Ss">
<h2 class="Ss" id="Dynamic_Device_Personalization"><a class="permalink" href="#Dynamic_Device_Personalization">Dynamic
Device Personalization</a></h2>
<p class="Pp">The DDP package loads during device initialization. The driver
looks for the <b class="Sy">ice_ddp</b> module and checks that it contains a
valid DDP package file.</p>
<p class="Pp">If the driver is unable to load the DDP package, the device will
enter Safe Mode. Safe Mode disables advanced and performance features and
supports only basic traffic and minimal functionality, such as updating the
NVM or downloading a new driver or DDP package. Safe Mode only applies to
the affected physical function and does not impact any other PFs. See the
“Intel Ethernet Adapters and Devices User Guide” for more
details on DDP and Safe Mode.</p>
<p class="Pp">If issues are encountered with the DDP package file, an updated
driver or <b class="Sy">ice_ddp</b> module may need to be downloaded. See
the log messages for more information.</p>
<p class="Pp">The DDP package cannot be updated if any PF drivers are already
loaded. To overwrite a package, unload all PFs and then reload the driver
with the new package.</p>
<p class="Pp">Only one DDP package can be used per driver, even if more than one
installed device uses the driver.</p>
<p class="Pp">Only the first loaded PF per device can download a package for
that device.</p>
</section>
<section class="Ss">
<h2 class="Ss" id="Jumbo_Frames"><a class="permalink" href="#Jumbo_Frames">Jumbo
Frames</a></h2>
<p class="Pp">Jumbo Frames support is enabled by changing the Maximum
Transmission Unit (MTU) to a value larger than the default value of
1500.</p>
<p class="Pp">Use <a class="Xr">ifconfig(8)</a> to increase the MTU size.</p>
<p class="Pp">The maximum MTU setting for jumbo frames is 9706. This corresponds
to the maximum jumbo frame size of 9728 bytes.</p>
<p class="Pp">This driver will attempt to use multiple page sized buffers to
receive each jumbo packet. This should help to avoid buffer starvation
issues when allocating receive packets.</p>
<p class="Pp">Packet loss may have a greater impact on throughput when jumbo
frames are in use. If a drop in performance is observed after enabling jumbo
frames, enabling flow control may mitigate the issue.</p>
</section>
<section class="Ss">
<h2 class="Ss" id="Remote_Direct_Memory_Access"><a class="permalink" href="#Remote_Direct_Memory_Access">Remote
Direct Memory Access</a></h2>
<p class="Pp">Remote Direct Memory Access, or RDMA, allows a network device to
transfer data directly to and from application memory on another system,
increasing throughput and lowering latency in certain networking
environments.</p>
<p class="Pp">The ice driver supports both the iWARP (Internet Wide Area RDMA
Protocol) and RoCEv2 (RDMA over Converged Ethernet) protocols. The major
difference is that iWARP performs RDMA over TCP, while RoCEv2 uses UDP.</p>
<p class="Pp">Devices based on the Intel Ethernet 800 Series do not support RDMA
when operating in multiport mode with more than 4 ports.</p>
<p class="Pp">For detailed installation and configuration information for RDMA,
see <a class="Xr">irdma(4)</a>.</p>
</section>
<section class="Ss">
<h2 class="Ss" id="RDMA_Monitoring"><a class="permalink" href="#RDMA_Monitoring">RDMA
Monitoring</a></h2>
<p class="Pp">For debugging/testing purposes, a sysctl can be used to set up a
mirroring interface on a port. The interface can receive mirrored RDMA
traffic for packet analysis tools like <a class="Xr">tcpdump(1)</a>. This
mirroring may impact performance.</p>
<p class="Pp">To use RDMA monitoring, more MSI-X interrupts may need to be
reserved. Before the <code class="Nm">ice</code> driver loads, configure the
following tunable provided by <a class="Xr">iflib(4)</a>:</p>
<div class="Bd Pp Bd-indent Li">
<pre>dev.ice.<interface #>.iflib.use_extra_msix_vectors=4</pre>
</div>
<p class="Pp">The number of extra MSI-X interrupt vectors may need to be
adjusted.</p>
<p class="Pp">To create/delete the interface:</p>
<div class="Bd Pp Bd-indent Li">
<pre>sysctl dev.ice.<interface #>.create_interface=1
sysctl dev.ice.<interface #>.delete_interface=1</pre>
</div>
<p class="Pp">The mirrored interface receives both LAN and RDMA traffic.
Additional filters can be configured in tcpdump.</p>
<p class="Pp">To differentiate the mirrored interface from the primary
interface, the network interface naming convention is:</p>
<div class="Bd Pp Bd-indent Li">
<pre><driver name><port number><modifier><modifier unit number></pre>
</div>
<p class="Pp">For example, “<code class="Li">ice0m0</code>” is the
first mirroring interface on
“<code class="Li">ice0</code>”.</p>
</section>
<section class="Ss">
<h2 class="Ss" id="Data_Center_Bridging"><a class="permalink" href="#Data_Center_Bridging">Data
Center Bridging</a></h2>
<p class="Pp">Data Center Bridging (DCB) is a configuration Quality of Service
implementation in hardware. It uses the VLAN priority tag (802.1p) to filter
traffic. That means that there are 8 different priorities that traffic can
be filtered into. It also enables priority flow control (802.1Qbb) which can
limit or eliminate the number of dropped packets during network stress.
Bandwidth can be allocated to each of these priorities, which is enforced at
the hardware level (802.1Qaz).</p>
<p class="Pp">DCB is normally configured on the network using the DCBX protocol
(802.1Qaz), a specialization of LLDP (802.1AB). The
<code class="Nm">ice</code> driver supports the following mutually exclusive
variants of DCBX support:</p>
<p class="Pp"></p>
<ul class="Bl-bullet Bl-compact">
<li>Firmware-based LLDP Agent</li>
<li>Software-based LLDP Agent</li>
</ul>
<p class="Pp">In firmware-based mode, firmware intercepts all LLDP traffic and
handles DCBX negotiation transparently for the user. In this mode, the
adapter operates in “willing” DCBX mode, receiving DCB
settings from the link partner (typically a switch). The local user can only
query the negotiated DCB configuration. For information on configuring DCBX
parameters on a switch, please consult the switch manufacturer'ss
documentation.</p>
<p class="Pp">In software-based mode, LLDP traffic is forwarded to the network
stack and user space, where a software agent can handle it. In this mode,
the adapter can operate in “nonwilling” DCBX mode and DCB
configuration can be both queried and set locally. This mode requires the
FW-based LLDP Agent to be disabled.</p>
<p class="Pp">Firmware-based mode and software-based mode are controlled by the
“fw_lldp_agent” sysctl. Refer to the Firmware Link Layer
Discovery Protocol Agent section for more information.</p>
<p class="Pp">Link-level flow control and priority flow control are mutually
exclusive. The ice driver will disable link flow control when priority flow
control is enabled on any traffic class (TC). It will disable priority flow
control when link flow control is enabled.</p>
<p class="Pp">To enable/disable priority flow control in software-based DCBX
mode:</p>
<div class="Bd Pp Bd-indent Li">
<pre>sysctl dev.ice.<interface #>.pfc=1 (or 0 to disable)</pre>
</div>
<p class="Pp">Enhanced Transmission Selection (ETS) allows bandwidth to be
assigned to certain TCs, to help ensure traffic reliability. To view the
assigned ETS configuration, use the following:</p>
<div class="Bd Pp Bd-indent Li">
<pre>sysctl dev.ice.<interface #>.ets_min_rate</pre>
</div>
<p class="Pp">To set the minimum ETS bandwidth per TC, separate the values by
commas. All values must add up to 100. For example, to set all TCs to a
minimum bandwidth of 10% and TC 7 to 30%, use the following:</p>
<div class="Bd Pp Bd-indent Li">
<pre>sysctl dev.ice.<interface #>.ets_min_rate=10,10,10,10,10,10,10,30</pre>
</div>
<p class="Pp">To set the User Priority (UP) to a TC mapping for a port, separate
the values by commas. For example, to map UP 0 and 1 to TC 0, UP 2 and 3 to
TC 1, UP 4 and 5 to TC 2, and UP 6 and 7 to TC 3, use the following:</p>
<div class="Bd Pp Bd-indent Li">
<pre>sysctl dev.ice.<interface #>.up2tc_map=0,0,1,1,2,2,3,3</pre>
</div>
</section>
<section class="Ss">
<h2 class="Ss" id="L3_QoS_Mode"><a class="permalink" href="#L3_QoS_Mode">L3 QoS
Mode</a></h2>
<p class="Pp">The <code class="Nm">ice</code> driver supports setting DSCP-based
Layer 3 Quality of Service (L3 QoS) in the PF driver. The driver initializes
in L2 QoS mode by default; L3 QoS is disabled by default. Use the following
sysctl to enable or disable L3 QoS:</p>
<div class="Bd Pp Bd-indent Li">
<pre>sysctl dev.ice.<interface #>.pfc_mode=1 (or 0 to disable)</pre>
</div>
<p class="Pp">If L3 QoS mode is disabled, it returns to L2 QoS mode.</p>
<p class="Pp">To map a DSCP value to a traffic class, separate the values by
commas. For example, to map DSCPs 0-3 and DSCP 8 to DCB TCs 0-3 and 4,
respectively:</p>
<div class="Bd Pp Bd-indent Li">
<pre>sysctl dev.ice.<interface #>.dscp2tc_map.0-7=0,1,2,3,0,0,0,0
sysctl dev.ice.<interface #>.dscp2tc_map.8-15=4,0,0,0,0,0,0,0</pre>
</div>
<p class="Pp">To change the DSCP mapping back to the default traffic class, set
all the values back to 0.</p>
<p class="Pp">To view the currently configured mappings, use the following:</p>
<div class="Bd Pp Bd-indent Li">
<pre>sysctl dev.ice.<interface #>.dscp2tc_map</pre>
</div>
<p class="Pp">L3 QoS mode is not available when FW-LLDP is enabled.</p>
<p class="Pp">FW-LLDP cannot be enabled if L3 QoS mode is active.</p>
<p class="Pp">Disable FW-LLDP before switching to L3 QoS mode.</p>
<p class="Pp">Refer to the
<a class="Sx" href="#Firmware_Link_Layer_Discovery_Protocol_Agent">Firmware
Link Layer Discovery Protocol Agent</a> section in this README for more
information on disabling FW-LLDP.</p>
</section>
<section class="Ss">
<h2 class="Ss" id="Firmware_Link_Layer_Discovery_Protocol_Agent"><a class="permalink" href="#Firmware_Link_Layer_Discovery_Protocol_Agent">Firmware
Link Layer Discovery Protocol Agent</a></h2>
<p class="Pp">Use sysctl to change FW-LLDP settings. The FW-LLDP setting is per
port and persists across boots.</p>
<p class="Pp">To enable the FW-LLDP Agent:</p>
<div class="Bd Pp Bd-indent Li">
<pre>sysctl dev.ice.<interface #>.fw_lldp_agent=1</pre>
</div>
<p class="Pp">To disable the FW-LLDP Agebt:</p>
<div class="Bd Pp Bd-indent Li">
<pre>sysctl dev.ice.<interface #>.fw_lldp_agent=0</pre>
</div>
<p class="Pp">To check the current LLDP setting:</p>
<div class="Bd Pp Bd-indent Li">
<pre>sysctl dev.ice.<interface #>.fw_lldp_agent</pre>
</div>
<p class="Pp">The UEFI HII LLDP Agent attribute must be enabled for this setting
to take effect. If the “LLDP AGENT” attribute is set to
disabled, the FW-LLDP Agent cannot be enabled from the driver.</p>
</section>
<section class="Ss">
<h2 class="Ss" id="Link-Level_Flow_Control"><a class="permalink" href="#Link-Level_Flow_Control">Link-Level
Flow Control</a></h2>
<p class="Pp">Ethernet Flow Control (IEEE 802.3x or LFC) can be configured with
<a class="Xr">sysctl(8)</a> to enable receiving and transmitting pause
frames for <code class="Nm">ice</code>. When transmit is enabled, pause
frames are generated when the receive packet buffer crosses a predefined
threshold. When receive is enabled, the transmit unit will halt for the time
delay specified in the firmware when a pause frame is received.</p>
<p class="Pp">Flow Control is disabled by default.</p>
<p class="Pp">Use sysctl to change the flow control settings for a single
interface without reloading the driver:</p>
<div class="Bd Pp Bd-indent Li">
<pre>sysctl dev.ice.<interface #>.fc</pre>
</div>
<p class="Pp">The available values for flow control are:</p>
<div class="Bd Pp Bd-indent Li">
<pre>0 = Disable flow control
1 = Enable Rx pause
2 = Enable Tx pause
3 = Enable Rx and Tx pause</pre>
</div>
<p class="Pp">Verify that link flow control was negotiated on the link by
checking the interface entry in <a class="Xr">ifconfig(8)</a> and looking
for the flags “txpause” and/or “rxpause” in the
“media” status.</p>
<p class="Pp">The <code class="Nm">ice</code> driver requires flow control on
both the port and link partner. If flow control is disabled on one of the
sides, the port may appear to hang on heavy traffic.</p>
<p class="Pp">For more information on priority flow control, refer to the
<a class="Sx" href="#Data_Center_Bridging">Data Center Bridging</a>
section.</p>
<p class="Pp">The VF driver does not have access to flow control. It must be
managed from the host side.</p>
</section>
<section class="Ss">
<h2 class="Ss" id="Forward_Error_Correction"><a class="permalink" href="#Forward_Error_Correction">Forward
Error Correction</a></h2>
<p class="Pp">Forward Error Correction (FEC) improves link stability but
increases latency. Many high quality optics, direct attach cables, and
backplane channels can provide a stable link without FEC.</p>
<p class="Pp">For devices to benefit from this feature, link partners must have
FEC enabled.</p>
<p class="Pp">If the <var class="Va">allow_no_fec_modules_in_auto</var> sysctl
is enabled Auto FEC negotiation will include
“<span class="No">FEC</span>” in case the link partner does
not have FEC enabled or is not FEC capable:</p>
<div class="Bd Pp Bd-indent Li">
<pre>sysctl dev.ice.<interface #>.allow_no_fec_modules_in_auto=1</pre>
</div>
<p class="Pp">NOTE: This flag is currently not supported on the Intel Ethernet
830 Series.</p>
<p class="Pp">To show the current FEC settings that are negotiated on the
link:</p>
<div class="Bd Pp Bd-indent Li">
<pre>sysctl dev.ice.<interface #>.negotiated_fec</pre>
</div>
<p class="Pp">To view or set the FEC setting that was requested on the link:</p>
<div class="Bd Pp Bd-indent Li">
<pre>sysctl dev.ice.<interface #>.requested_fec</pre>
</div>
<p class="Pp">To see the valid FEC modes for the link:</p>
<div class="Bd Pp Bd-indent Li">
<pre>sysctl -d dev.ice.<interface #>.requested_fec</pre>
</div>
</section>
<section class="Ss">
<h2 class="Ss" id="Speed_and_Duplex_Configuration"><a class="permalink" href="#Speed_and_Duplex_Configuration">Speed
and Duplex Configuration</a></h2>
<p class="Pp">The speed and duplex settings cannot be hard set.</p>
<p class="Pp">To have the device change the speeds it will use in
auto-negotiation or force link with:</p>
<div class="Bd Pp Bd-indent Li">
<pre>sysctl dev.ice.<interface #>.advertise_speed=<mask></pre>
</div>
<p class="Pp">Supported speeds will vary by device. Depending on the speeds the
device supports, valid bits used in a speed mask could include:</p>
<div class="Bd Pp Bd-indent Li">
<pre>0x0 - Auto
0x2 - 100 Mbps
0x4 - 1 Gbps
0x8 - 2.5 Gbps
0x10 - 5 Gbps
0x20 - 10 Gbps
0x80 - 25 Gbps
0x100 - 40 Gbps
0x200 - 50 Gbps
0x400 - 100 Gbps
0x800 - 200 Gbps</pre>
</div>
</section>
<section class="Ss">
<h2 class="Ss" id="Disabling_physical_link_when_the_interface_is_brought_down"><a class="permalink" href="#Disabling_physical_link_when_the_interface_is_brought_down">Disabling
physical link when the interface is brought down</a></h2>
<p class="Pp">When the <var class="Va">link_active_on_if_down</var> sysctl is
set to “0”, the port's link will go down when the interface is
brought down. By default, link will stay up.</p>
<p class="Pp">To disable link when the interface is down:</p>
<div class="Bd Pp Bd-indent Li">
<pre>sysctl dev.ice.<interface #>.link_active_on_if_down=0</pre>
</div>
</section>
<section class="Ss">
<h2 class="Ss" id="Firmware_Logging"><a class="permalink" href="#Firmware_Logging">Firmware
Logging</a></h2>
<p class="Pp">The <code class="Nm">ice</code> driver allows for the generation
of firmware logs for supported categories of events, to help debug issues
with Customer Support. Refer to the “Intel Ethernet Adapters and
Devices User Guide” for an overview of this feature and additional
tips.</p>
<p class="Pp">At a high level, to capture a firmware log:</p>
<ol class="Bl-enum Bl-compact">
<li>Set the configuration for the firmware log.</li>
<li>Perform the necessary steps to reproduce the issue.</li>
<li>Capture the firmware log.</li>
<li>Stop capturing the firmware log.</li>
<li>Reset the firmware log settings as needed.</li>
<li>Work with Customer Support to debug the issue.</li>
</ol>
<p class="Pp">NOTE: Firmware logs are generated in a binary format and must be
decoded by Customer Support. Information collected is related only to
firmware and hardware for debug purposes.</p>
<p class="Pp">Once the driver is loaded, it will create the
<var class="Va">fw_log</var> sysctl node under the debug section of the
driver's sysctl list. The driver groups these events into categories, called
“modules”. Supported modules include:</p>
<p class="Pp"></p>
<div class="Bd-indent">
<dl class="Bl-tag Bl-compact">
<dt id="general"><var class="Va">general</var></dt>
<dd>General (Bit 0)</dd>
<dt id="ctrl"><var class="Va">ctrl</var></dt>
<dd>Control (Bit 1)</dd>
<dt id="link"><var class="Va">link</var></dt>
<dd>Link Management (Bit 2)</dd>
<dt id="link_topo"><var class="Va">link_topo</var></dt>
<dd>Link Topology Detection (Bit 3)</dd>
<dt id="dnl"><var class="Va">dnl</var></dt>
<dd>Link Control Technology (Bit 4)</dd>
<dt id="i2c"><var class="Va">i2c</var></dt>
<dd>I2C (Bit 5)</dd>
<dt id="sdp"><var class="Va">sdp</var></dt>
<dd>SDP (Bit 6)</dd>
<dt id="mdio"><var class="Va">mdio</var></dt>
<dd>MDIO (Bit 7)</dd>
<dt id="adminq"><var class="Va">adminq</var></dt>
<dd>Admin Queue (Bit 8)</dd>
<dt id="hdma"><var class="Va">hdma</var></dt>
<dd>Host DMA (Bit 9)</dd>
<dt id="lldp"><var class="Va">lldp</var></dt>
<dd>LLDP (Bit 10)</dd>
<dt id="dcbx"><var class="Va">dcbx</var></dt>
<dd>DCBx (Bit 11)</dd>
<dt id="dcb"><var class="Va">dcb</var></dt>
<dd>DCB (Bit 12)</dd>
<dt id="xlr"><var class="Va">xlr</var></dt>
<dd>XLR (function-level resets; Bit 13)</dd>
<dt id="nvm"><var class="Va">nvm</var></dt>
<dd>NVM (Bit 14)</dd>
<dt id="auth"><var class="Va">auth</var></dt>
<dd>Authentication (Bit 15)</dd>
<dt id="vpd"><var class="Va">vpd</var></dt>
<dd>Vital Product Data (Bit 16)</dd>
<dt id="iosf"><var class="Va">iosf</var></dt>
<dd>Intel On-Chip System Fabric (Bit 17)</dd>
<dt id="parser"><var class="Va">parser</var></dt>
<dd>Parser (Bit 18)</dd>
<dt id="sw"><var class="Va">sw</var></dt>
<dd>Switch (Bit 19)</dd>
<dt id="scheduler"><var class="Va">scheduler</var></dt>
<dd>Scheduler (Bit 20)</dd>
<dt id="txq"><var class="Va">txq</var></dt>
<dd>TX Queue Management (Bit 21)</dd>
<dt id="acl"><var class="Va">acl</var></dt>
<dd>ACL (Access Control List; Bit 22)</dd>
<dt id="post"><var class="Va">post</var></dt>
<dd>Post (Bit 23)</dd>
<dt id="watchdog"><var class="Va">watchdog</var></dt>
<dd>Watchdog (Bit 24)</dd>
<dt id="task_dispatch"><var class="Va">task_dispatch</var></dt>
<dd>Task Dispatcher (Bit 25)</dd>
<dt id="mng"><var class="Va">mng</var></dt>
<dd>Manageability (Bit 26)</dd>
<dt id="synce"><var class="Va">synce</var></dt>
<dd>SyncE (Bit 27)</dd>
<dt id="health"><var class="Va">health</var></dt>
<dd>Health (Bit 28)</dd>
<dt id="tsdrv"><var class="Va">tsdrv</var></dt>
<dd>Time Sync (Bit 29)</dd>
<dt id="pfreg"><var class="Va">pfreg</var></dt>
<dd>PF Registration (Bit 30)</dd>
<dt id="mdlver"><var class="Va">mdlver</var></dt>
<dd>Module Version (Bit 31)</dd>
</dl>
</div>
<p class="Pp">The verbosity level of the firmware logs can be modified. It is
possible to set only one log level per module, and each level includes the
verbosity levels lower than it. For instance, setting the level to
“normal” will also log warning and error messages. Available
verbosity levels are:</p>
<p class="Pp"></p>
<ul class="Bl-item Bd-indent Bl-compact">
<li>0 = none</li>
<li>1 = error</li>
<li>2 = warning</li>
<li>3 = normal</li>
<li>4 = verbose</li>
</ul>
<p class="Pp">To set the desired verbosity level for a module, use the following
sysctl command and then register it:</p>
<div class="Bd Pp Bd-indent Li">
<pre>sysctl dev.ice.<interface #>.debug.fw_log.severity.<module>=<level></pre>
</div>
<p class="Pp">For example:</p>
<div class="Bd Pp Bd-indent Li">
<pre>sysctl dev.ice.0.debug.fw_log.severity.link=1
sysctl dev.ice.0.debug.fw_log.severity.link_topo=2
sysctl dev.ice.0.debug.fw_log.register=1</pre>
</div>
<p class="Pp">To log firmware messages after booting, but before the driver
initializes, use <a class="Xr">kenv(1)</a> to set the tunable. The
<var class="Va">on_load</var> setting tells the device to register the
variable as soon as possible during driver load. For example:</p>
<div class="Bd Pp Bd-indent Li">
<pre>kenv dev.ice.0.debug.fw_log.severity.link=1
kenv dev.ice.0.debug.fw_log.severity.link_topo=2
kenv dev.ice.0.debug.fw_log.on_load=1</pre>
</div>
<p class="Pp">To view the firmware logs and redirect them to a file, use the
following command:</p>
<div class="Bd Pp Bd-indent Li">
<pre>dmesg > log_output</pre>
</div>
<p class="Pp">NOTE: Logging a large number of modules or too high of a verbosity
level will add extraneous messages to dmesg and could hinder debug
efforts.</p>
</section>
<section class="Ss">
<h2 class="Ss" id="Debug_Dump"><a class="permalink" href="#Debug_Dump">Debug
Dump</a></h2>
<p class="Pp">Intel Ethernet 800 Series devices support debug dump, which allows
gathering of runtime register values from the firmware for
“clusters” of events and then write the results to a single
dump file, for debugging complicated issues in the field.</p>
<p class="Pp">This debug dump contains a snapshot of the device and its existing
hardware configuration, such as switch tables, transmit scheduler tables,
and other information. Debug dump captures the current state of the
specified cluster(s) and is a stateless snapshot of the whole device.</p>
<p class="Pp">NOTE: Like with firmware logs, the contents of the debug dump are
not human-readable. Work with Customer Support to decode the file.</p>
<p class="Pp">Debug dump is per device, not per PF.</p>
<p class="Pp">Debug dump writes all information to a single file.</p>
<p class="Pp">To generate a debug dump file in <span class="Ux">FreeBSD</span>
do the following:</p>
<p class="Pp">Specify the cluster(s) to include in the dump file, using a
bitmask and the following command:</p>
<div class="Bd Pp Bd-indent Li">
<pre>sysctl dev.ice.<interface #>.debug.dump.clusters=<bitmask></pre>
</div>
<p class="Pp">To print the complete cluster bitmask and parameter list to the
screen, pass the <code class="Fl">-d</code> argument. For example:</p>
<div class="Bd Pp Bd-indent Li">
<pre>sysctl -d dev.ice.0.debug.dump.clusters</pre>
</div>
<p class="Pp">Possible bitmask values for <var class="Va">clusters</var>
are:</p>
<ul class="Bl-bullet Bl-compact">
<li>0 - Dump all clusters (only supported on Intel Ethernet E810 Series and
Intel Ethernet E830 Series)</li>
<li>0x1 - Switch</li>
<li>0x2 - ACL</li>
<li>0x4 - Tx Scheduler</li>
<li>0x8 - Profile Configuration</li>
<li>0x20 - Link</li>
<li>0x80 - DCB</li>
<li>0x100 - L2P</li>
<li>0x400000 - Manageability Transactions (only supported on Intel Ethernet
E810 Series)</li>
</ul>
<p class="Pp">For example, to dump the Switch, DCB, and L2P clusters, use the
following:</p>
<div class="Bd Pp Bd-indent Li">
<pre>sysctl dev.ice.0.debug.dump.clusters=0x181</pre>
</div>
<p class="Pp">To dump all clusters, use the following:</p>
<div class="Bd Pp Bd-indent Li">
<pre>sysctl dev.ice.0.debug.dump.clusters=0</pre>
</div>
<p class="Pp">NOTE: Using 0 will skip Manageability Transactions data.</p>
<p class="Pp">If a single cluster is not specified, the driver will dump all
clusters to a single file. Issue the debug dump command, using the
following:</p>
<div class="Bd Pp Bd-indent Li">
<pre>sysctl -b dev.ice.<interface #>.debug.dump.dump=1 > dump.bin</pre>
</div>
<p class="Pp">NOTE: The driver will not receive the command if the sysctl is not
set to “1”.</p>
<p class="Pp">Replace “dump.bin” above with the preferred file
name.</p>
<p class="Pp">To clear the <var class="Va">clusters</var> mask before a
subsequent debug dump and then do the dump:</p>
<div class="Bd Pp Bd-indent Li">
<pre>sysctl dev.ice.0.debug.dump.clusters=0
sysctl dev.ice.0.debug.dump.dump=1</pre>
</div>
</section>
<section class="Ss">
<h2 class="Ss" id="Debugging_PHY_Statistics"><a class="permalink" href="#Debugging_PHY_Statistics">Debugging
PHY Statistics</a></h2>
<p class="Pp">The ice driver supports the ability to obtain the values of the
PHY registers from Intel(R) Ethernet 810 Series devices in order to debug
link and connection issues during runtime.</p>
<p class="Pp">The driver provides information about:</p>
<ul class="Bl-bullet">
<li>Rx and Tx Equalization parameters</li>
<li>RS FEC correctable and uncorrectable block counts</li>
</ul>
<p class="Pp">Use the following sysctl to read the PHY registers:</p>
<div class="Bd Pp Bd-indent Li">
<pre>sysctl dev.ice.<interface #>.debug.phy_statistics</pre>
</div>
<p class="Pp">NOTE: The contents of the registers are not human-readable. Like
with firmware logs and debug dump, work with Customer Support to decode the
file.</p>
</section>
<section class="Ss">
<h2 class="Ss" id="Transmit_Balancing"><a class="permalink" href="#Transmit_Balancing">Transmit
Balancing</a></h2>
<p class="Pp">Some Intel(R) Ethernet 800 Series devices allow for enabling a
transmit balancing feature to improve transmit performance under certain
conditions. When enabled, this feature should provide more consistent
transmit performance across queues and/or PFs and VFs.</p>
<p class="Pp">By default, transmit balancing is disabled in the NVM. To enable
this feature, use one of the following to persistently change the setting
for the device:</p>
<ul class="Bl-bullet">
<li>Use the Ethernet Port Configuration Tool (EPCT) to enable the
<var class="Va">tx_balancing</var> option. Refer to the EPCT readme for
more information.</li>
<li>Enable the Transmit Balancing device setting in UEFI HII.</li>
</ul>
<p class="Pp">When the driver loads, it reads the transmit balancing setting
from the NVM and configures the device accordingly.</p>
<p class="Pp">NOTE: The user selection for transmit balancing in EPCT or HII is
persistent across reboots. The system must be rebooted for the selected
setting to take effect.</p>
<p class="Pp">This setting is device wide.</p>
<p class="Pp">The driver, NVM, and DDP package must all support this
functionality to enable the feature.</p>
</section>
<section class="Ss">
<h2 class="Ss" id="Thermal_Monitoring"><a class="permalink" href="#Thermal_Monitoring">Thermal
Monitoring</a></h2>
<p class="Pp">Intel(R) Ethernet 810 Series and Intel(R) Ethernet 830 Series
devices can display temperature data (in degrees Celsius) via:</p>
<div class="Bd Pp Bd-indent Li">
<pre>sysctl dev.ice.<interface #>.temp</pre>
</div>
</section>
<section class="Ss">
<h2 class="Ss" id="Network_Memory_Buffer_Allocation"><a class="permalink" href="#Network_Memory_Buffer_Allocation">Network
Memory Buffer Allocation</a></h2>
<p class="Pp"><span class="Ux">FreeBSD</span> may have a low number of network
memory buffers (mbufs) by default. If the number of mbufs available is too
low, it may cause the driver to fail to initialize and/or cause the system
to become unresponsive. Check to see if the system is mbuf-starved by
running <code class="Ic">netstat</code> <code class="Fl">-m</code>. Increase
the number of mbufs by editing the lines below in
<span class="Pa">/etc/sysctl.conf</span>:</p>
<div class="Bd Pp Bd-indent Li">
<pre>kern.ipc.nmbclusters
kern.ipc.nmbjumbop
kern.ipc.nmbjumbo9
kern.ipc.nmbjumbo16
kern.ipc.nmbufs</pre>
</div>
<p class="Pp">The amount of memory that should be allocated is system specific,
and may require some trial and error. Also, increasing the following in
<span class="Pa">/etc/sysctl.conf</span> could help increase network
performance:</p>
<div class="Bd Pp Bd-indent Li">
<pre>kern.ipc.maxsockbuf
net.inet.tcp.sendspace
net.inet.tcp.recvspace
net.inet.udp.maxdgram
net.inet.udp.recvspace</pre>
</div>
</section>
<section class="Ss">
<h2 class="Ss" id="Additional_Utilities"><a class="permalink" href="#Additional_Utilities">Additional
Utilities</a></h2>
<p class="Pp">There are additional tools available from Intel to help configure
and update the adapters covered by this driver. These tools can be
downloaded directly from Intel at
<a class="Lk" href="https://downloadcenter.intel.com">https://downloadcenter.intel.com</a>,
by searching for their names:</p>
<ul class="Bl-bullet">
<li id="Ethernet">To change the behavior of the QSFP28 ports on E810-C
adapters, use the Intel
<a class="permalink" href="#Ethernet"><b class="Sy">Ethernet Port
Configuration Tool - FreeBSD</b></a>.</li>
<li id="Non-Volatile">To update the firmware on an adapter, use the Intel
<a class="permalink" href="#Non-Volatile"><b class="Sy">Non-Volatile
Memory (NVM) Update Utility for Intel Ethernet Network Adapters E810
series - FreeBSD</b></a></li>
</ul>
</section>
<section class="Ss">
<h2 class="Ss" id="Optics_and_auto-negotiation"><a class="permalink" href="#Optics_and_auto-negotiation">Optics
and auto-negotiation</a></h2>
<p class="Pp">Modules based on 100GBASE-SR4, active optical cable (AOC), and
active copper cable (ACC) do not support auto-negotiation per the IEEE
specification. To obtain link with these modules, auto-negotiation must be
turned off on the link partner's switch ports.</p>
<p class="Pp">Note that adapters also support all passive and active limiting
direct attach cables that comply with SFF-8431 v4.1 and SFF-8472 v10.4
specifications.</p>
</section>
<section class="Ss">
<h2 class="Ss" id="PCI-Express_Slot_Bandwidth"><a class="permalink" href="#PCI-Express_Slot_Bandwidth">PCI-Express
Slot Bandwidth</a></h2>
<p class="Pp">Some PCIe x8 slots are actually configured as x4 slots. These
slots have insufficient bandwidth for full line rate with dual port and quad
port devices. In addition, if a PCIe v4.0 or v3.0-capable adapter is placed
into into a PCIe v2.x slot, full bandwidth will not be possible.</p>
<p class="Pp">The driver detects this situation and writes the following message
in the system log:</p>
<div class="Bd Pp Bd-indent">PCI-Express bandwidth available for this device may
be insufficient for optimal performance. Please move the device to a different
PCI-e link with more lanes and/or higher transfer rate.</div>
<p class="Pp">If this error occurs, moving the adapter to a true PCIe x8 or x16
slot will resolve the issue. For best performance, install devices in the
following PCI slots:</p>
<ul class="Bl-bullet">
<li>Any 100Gbps-capable Intel(R) Ethernet 800 Series device: Install in a PCIe
v4.0 x8 or v3.0 x16 slot</li>
<li>A 200Gbps-capable Intel(R) Ethernet 830 Series device: Install in a PCIe
v5.0 x8 or v4.0 x16 slot</li>
</ul>
<p class="Pp">For questions related to hardware requirements, refer to the
documentation supplied with the adapter.</p>
</section>
</section>
<section class="Sh">
<h1 class="Sh" id="HARDWARE"><a class="permalink" href="#HARDWARE">HARDWARE</a></h1>
<p class="Pp">The <code class="Nm">ice</code> driver supports the following
Intel 800 series 1Gb to 200Gb Ethernet controllers:</p>
<p class="Pp"></p>
<ul class="Bl-bullet Bl-compact">
<li>Intel Ethernet Controller E810-C</li>
<li>Intel Ethernet Controller E810-XXV</li>
<li>Intel Ethernet Connection E822-C</li>
<li>Intel Ethernet Connection E822-L</li>
<li>Intel Ethernet Connection E823-C</li>
<li>Intel Ethernet Connection E823-L</li>
<li>Intel Ethernet Connection E825-C</li>
<li>Intel Ethernet Connection E830-C</li>
<li>Intel Ethernet Connection E830-CC</li>
<li>Intel Ethernet Connection E830-L</li>
<li>Intel Ethernet Connection E830-XXV</li>
<li>Intel Ethernet Connection E835-C</li>
<li>Intel Ethernet Connection E835-CC</li>
<li>Intel Ethernet Connection E835-L</li>
<li>Intel Ethernet Connection E835-XXV</li>
</ul>
<p class="Pp">The <code class="Nm">ice</code> driver supports some adapters in
this series with SFP28/QSFP28 cages which have firmware that requires that
Intel qualified modules are used; these qualified modules are listed below.
This qualification check cannot be disabled by the driver.</p>
<p class="Pp">The <code class="Nm">ice</code> driver supports 100Gb Ethernet
adapters with these QSFP28 modules:</p>
<p class="Pp"></p>
<ul class="Bl-bullet Bl-compact">
<li>Intel 100G QSFP28 100GBASE-SR4 E100GQSFPSR28SRX</li>
<li>Intel 100G QSFP28 100GBASE-SR4 SPTMBP1PMCDF</li>
<li>Intel 100G QSFP28 100GBASE-CWDM4 SPTSBP3CLCCO</li>
<li>Intel 100G QSFP28 100GBASE-DR SPTSLP2SLCDF</li>
</ul>
<p class="Pp">The <code class="Nm">ice</code> driver supports 25Gb and 10Gb
Ethernet adapters with these SFP28 modules:</p>
<p class="Pp"></p>
<ul class="Bl-bullet Bl-compact">
<li>Intel 10G/25G SFP28 25GBASE-SR E25GSFP28SR</li>
<li>Intel 25G SFP28 25GBASE-SR E25GSFP28SRX (Extended Temp)</li>
<li>Intel 25G SFP28 25GBASE-LR E25GSFP28LRX (Extended Temp)</li>
</ul>
<p class="Pp">The <code class="Nm">ice</code> driver supports 10Gb and 1Gb
Ethernet adapters with these SFP+ modules:</p>
<p class="Pp"></p>
<ul class="Bl-bullet Bl-compact">
<li>Intel 1G/10G SFP+ 10GBASE-SR E10GSFPSR</li>
<li>Intel 1G/10G SFP+ 10GBASE-SR E10GSFPSRG1P5</li>
<li>Intel 1G/10G SFP+ 10GBASE-SR E10GSFPSRG2P5</li>
<li>Intel 10G SFP+ 10GBASE-SR E10GSFPSRX (Extended Temp)</li>
<li>Intel 1G/10G SFP+ 10GBASE-LR E10GSFPLR</li>
</ul>
</section>
<section class="Sh">
<h1 class="Sh" id="LOADER_TUNABLES"><a class="permalink" href="#LOADER_TUNABLES">LOADER
TUNABLES</a></h1>
<p class="Pp">Tunables can be set at the <a class="Xr">loader(8)</a> prompt
before booting the kernel or stored in <a class="Xr">loader.conf(5)</a>. See
the <a class="Xr">iflib(4)</a> man page for more information on using iflib
sysctl variables as tunables.</p>
<dl class="Bl-tag">
<dt id="hw.ice.enable_health_events"><var class="Va">hw.ice.enable_health_events</var></dt>
<dd>Set to 1 to enable firmware health event reporting across all devices.
Enabled by default.
<p class="Pp">If enabled, when the driver receives a firmware health event
message, it will print out a description of the event to the kernel
message buffer and if applicable, possible actions to take to remedy
it.</p>
</dd>
<dt id="hw.ice.irdma"><var class="Va">hw.ice.irdma</var></dt>
<dd>Set to 1 to enable the RDMA client interface, required by the
<a class="Xr">irdma(4)</a> driver. Enabled by default.</dd>
<dt id="hw.ice.rdma_max_msix"><var class="Va">hw.ice.rdma_max_msix</var></dt>
<dd>Set the maximum number of per-device MSI-X vectors that are allocated for
use by the <a class="Xr">irdma(4)</a> driver. Set to 64 by default.</dd>
<dt id="hw.ice.debug.enable_tx_fc_filter"><var class="Va">hw.ice.debug.enable_tx_fc_filter</var></dt>
<dd>Set to 1 to enable the TX Flow Control filter across all devices. Enabled
by default.
<p class="Pp">If enabled, the hardware will drop any transmitted Ethertype
0x8808 control frames that do not originate from the hardware.</p>
</dd>
<dt id="hw.ice.debug.enable_tx_lldp_filter"><var class="Va">hw.ice.debug.enable_tx_lldp_filter</var></dt>
<dd>Set to 1 to enable the TX LLDP filter across all devices. Enabled by
default.
<p class="Pp">If enabled, the hardware will drop any transmitted Ethertype
0x88cc LLDP frames that do not originate from the hardware. This must be
disabled in order to use LLDP daemon software such as
<a class="Xr">lldpd(8)</a>.</p>
</dd>
<dt id="hw.ice.debug.ice_tx_balance_en"><var class="Va">hw.ice.debug.ice_tx_balance_en</var></dt>
<dd>Set to 1 to allow the driver to use the 5-layer Tx Scheduler tree topology
if configured by the DDP package.
<p class="Pp">Enabled by default.</p>
</dd>
</dl>
</section>
<section class="Sh">
<h1 class="Sh" id="SYSCTL_VARIABLES"><a class="permalink" href="#SYSCTL_VARIABLES">SYSCTL
VARIABLES</a></h1>
<dl class="Bl-tag">
<dt id="dev.ice._.current_speed"><var class="Va">dev.ice.#.current_speed</var></dt>
<dd>This is a display of the current link speed of the interface. This is
expected to match the speed of the media type in-use displayed by
<a class="Xr">ifconfig(8)</a>.</dd>
<dt id="dev.ice._.fw_version"><var class="Va">dev.ice.#.fw_version</var></dt>
<dd>Displays the current firmware and NVM versions of the adapter. This
information should be submitted along with any support requests.</dd>
<dt id="dev.ice._.ddp_version"><var class="Va">dev.ice.#.ddp_version</var></dt>
<dd>Displays the current DDP package version downloaded to the adapter. This
information should be submitted along with any support requests.</dd>
<dt id="dev.ice._.pba_number"><var class="Va">dev.ice.#.pba_number</var></dt>
<dd>Displays the Product Board Assembly Number. May be used to help identify
the type of adapter in use. This sysctl may not exist depending on the
adapter type.</dd>
<dt id="dev.ice._.hw.mac.*"><var class="Va">dev.ice.#.hw.mac.*</var></dt>
<dd>This sysctl tree contains statistics collected by the hardware for the
port.</dd>
</dl>
</section>
<section class="Sh">
<h1 class="Sh" id="INTERRUPT_STORMS"><a class="permalink" href="#INTERRUPT_STORMS">INTERRUPT
STORMS</a></h1>
<p class="Pp">It is important to note that 100G operation can generate high
numbers of interrupts, often incorrectly being interpreted as a storm
condition in the kernel. It is suggested that this be resolved by setting
<var class="Va">hw.intr_storm_threshold</var> to 0.</p>
</section>
<section class="Sh">
<h1 class="Sh" id="IOVCTL_OPTIONS"><a class="permalink" href="#IOVCTL_OPTIONS">IOVCTL
OPTIONS</a></h1>
<p class="Pp">The driver supports additional optional parameters for created VFs
(Virtual Functions) when using <a class="Xr">iovctl(8)</a>:</p>
<dl class="Bl-tag">
<dt>mac-addr (unicast-mac)</dt>
<dd>Set the Ethernet MAC address that the VF will use. If unspecified, the VF
will use a randomly generated MAC address and
“allow-set-mac” will be set to true.</dd>
<dt>mac-anti-spoof (bool)</dt>
<dd>Prevent the VF from sending Ethernet frames with a source address that
does not match its own. Enabled by default.</dd>
<dt>allow-set-mac (bool)</dt>
<dd>Allow the VF to set its own Ethernet MAC address. Disallowed by
default.</dd>
<dt>allow-promisc (bool)</dt>
<dd>Allow the VF to inspect all of the traffic sent to the port that it is
created on. Disabled by default.</dd>
<dt>num-queues (uint16_t)</dt>
<dd>Specify the number of queues the VF will have. By default, this is set to
the number of MSI-X vectors supported by the VF minus one.</dd>
<dt>mirror-src-vsi (uint16_t)</dt>
<dd>Specify which VSI the VF will mirror traffic from by setting this to a
value other than -1. All traffic from that VSI will be mirrored to this
VF. Can be used as an alternative method to mirror RDMA traffic to another
interface than the method described in the
<a class="Sx" href="#RDMA_Monitoring">RDMA Monitoring</a> section. Not
affected by the “allow-promisc” parameter.</dd>
<dt>max-vlan-allowed (uint16_t)</dt>
<dd>Specify maximum number of VLAN filters that the VF can use. Receiving
traffic on a VLAN requires a hardware filter which are a finite resource;
this is used to prevent a VF from starving other VFs or the PF of filter
resources. By default, this is set to 16.</dd>
<dt>max-mac-filters (uint16_t)</dt>
<dd>Specify maximum number of MAC address filters that the VF can use. Each
allowed MAC address requires a hardware filter which are a finite
resource; this is used to prevent a VF from starving other VFs or the PF
of filter resources. The VF's default mac address does not count towards
this limit. By default, this is set to 64.</dd>
</dl>
<p class="Pp">An up to date list of parameters and their defaults can be found
by using <a class="Xr">iovctl(8)</a> with the <code class="Fl">-S</code>
option.</p>
<p class="Pp">For more information on standard and mandatory parameters, see
<a class="Xr">iovctl.conf(5)</a>.</p>
</section>
<section class="Sh">
<h1 class="Sh" id="SUPPORT"><a class="permalink" href="#SUPPORT">SUPPORT</a></h1>
<p class="Pp">For general information and support, go to the Intel support
website at:
<a class="Lk" href="http://www.intel.com/support/">http://www.intel.com/support/</a>.</p>
<p class="Pp">If an issue is identified with this driver with a supported
adapter, email all the specific information related to the issue to
<<a class="Mt" href="mailto:freebsd@intel.com">freebsd@intel.com</a>>.</p>
</section>
<section class="Sh">
<h1 class="Sh" id="SEE_ALSO"><a class="permalink" href="#SEE_ALSO">SEE
ALSO</a></h1>
<p class="Pp"><a class="Xr">iflib(4)</a>, <a class="Xr">vlan(4)</a>,
<a class="Xr">ifconfig(8)</a>, <a class="Xr">sysctl(8)</a></p>
</section>
<section class="Sh">
<h1 class="Sh" id="HISTORY"><a class="permalink" href="#HISTORY">HISTORY</a></h1>
<p class="Pp">The <code class="Nm">ice</code> device driver first appeared in
<span class="Ux">FreeBSD 12.2</span>.</p>
</section>
<section class="Sh">
<h1 class="Sh" id="AUTHORS"><a class="permalink" href="#AUTHORS">AUTHORS</a></h1>
<p class="Pp">The <code class="Nm">ice</code> driver was written by
<span class="An">Intel Corporation</span>
<<a class="Mt" href="mailto:freebsd@intel.com">freebsd@intel.com</a>>.</p>
</section>
</div>
<table class="foot">
<tr>
<td class="foot-date">November 5, 2025</td>
<td class="foot-os">FreeBSD 15.0</td>
</tr>
</table>
|