diff options
Diffstat (limited to 'static/freebsd/man4/ice.4')
| -rw-r--r-- | static/freebsd/man4/ice.4 | 1171 |
1 files changed, 1171 insertions, 0 deletions
diff --git a/static/freebsd/man4/ice.4 b/static/freebsd/man4/ice.4 new file mode 100644 index 00000000..a54a6b3f --- /dev/null +++ b/static/freebsd/man4/ice.4 @@ -0,0 +1,1171 @@ +.\" +.\" SPDX-License-Identifier: BSD-3-Clause +.\" +.\" Copyright (c) 2019-2020, Intel Corporation +.\" All rights reserved. +.\" +.\" Redistribution and use in source and binary forms of the Software, with or +.\" without modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright notice, +.\" this list of conditions and the following disclaimer. +.\" +.\" 2. Redistributions in binary form must reproduce the above copyright notice, +.\" this list of conditions and the following disclaimer in the documentation +.\" and/or other materials provided with the distribution. +.\" +.\" 3. Neither the name of the Intel Corporation nor the names of its +.\" contributors may be used to endorse or promote products derived from +.\" this Software without specific prior written permission. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" +.\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE +.\" LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR +.\" CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +.\" SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +.\" INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN +.\" CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) +.\" ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE +.\" POSSIBILITY OF SUCH DAMAGE. +.\" +.\" * Other names and brands may be claimed as the property of others. +.\" +.Dd November 5, 2025 +.Dt ICE 4 +.Os +.Sh NAME +.Nm ice +.Nd Intel Ethernet 800 Series 1GbE to 200GbE driver +.Sh SYNOPSIS +.Cd device iflib +.Cd device ice +.Pp +In +.Xr loader.conf 5 : +.Cd if_ice_load +.Cd hw.ice.enable_health_events +.Cd hw.ice.irdma +.Cd hw.ice.irdma_max_msix +.Cd hw.ice.debug.enable_tx_fc_filter +.Cd hw.ice.debug.enable_tx_lldp_filter +.Cd hw.ice.debug.ice_tx_balance_en +.Pp +In +.Xr sysctl.conf 5 +or +.Xr loader.conf 5 : +.Cd dev.ice.#.current_speed +.Cd dev.ice.#.fw_version +.Cd dev.ice.#.ddp_version +.Cd dev.ice.#.pba_number +.Cd dev.ice.#.hw.mac.* +.Sh DESCRIPTION +The +.Nm +driver provides support for any PCI Express adapter or LOM +.Pq LAN On Motherboard +in the Intel Ethernet 800 Series. +.Pp +The following topics are covered in this manual: +.Pp +.Bl -bullet -compact +.It +.Sx Features +.It +.Sx Dynamic Device Personalization +.It +.Sx Jumbo Frames +.It +.Sx Remote Direct Memory Access +.It +.Sx RDMA Monitoring +.It +.Sx Data Center Bridging +.It +.Sx L3 QoS Mode +.It +.Sx Firmware Link Layer Discovery Protocol Agent +.It +.Sx Link-Level Flow Control +.It +.Sx Forward Error Correction +.It +.Sx Speed and Duplex Configuration +.It +.Sx Disabling physical link when the interface is brought down +.It +.Sx Firmware Logging +.It +.Sx Debug Dump +.It +.Sx Debugging PHY Statistics +.It +.Sx Transmit Balancing +.It +.Sx Thermal Monitoring +.It +.Sx Network Memory Buffer Allocation +.It +.Sx Additional Utilities +.It +.Sx Optics and auto-negotiation +.It +.Sx PCI-Express Slot Bandwidth +.It +.Sx HARDWARE +.It +.Sx LOADER TUNABLES +.It +.Sx SYSCTL VARIABLES +.It +.Sx INTERRUPT STORMS +.It +.Sx IOVCTL OPTIONS +.It +.Sx SUPPORT +.It +.Sx SEE ALSO +.It +.Sx HISTORY +.El +.Ss Features +Support for Jumbo Frames is provided via the interface MTU setting. +Selecting an MTU larger than 1500 bytes with the +.Xr ifconfig 8 +utility configures the adapter to receive and transmit Jumbo Frames. +The maximum MTU size for Jumbo Frames is 9706. +For more information, see the +.Sx Jumbo Frames +section. +.Pp +This driver version supports VLANs. +For information on enabling VLANs, see +.Xr vlan 4 . +For additional information on configuring VLANs, see +.Xr ifconfig 8 Ap s +.Dq VLAN Parameters +section. +.Pp +Offloads are also controlled via the interface, for instance, checksumming for +both IPv4 and IPv6 can be set and unset, TSO4 and/or TSO6, and finally LRO can +be set and unset. +.Pp +For more information on configuring this device, see +.Xr ifconfig 8 . +.Pp +The associated Virtual Function (VF) driver for this driver is +.Xr iavf 4 . +.Pp +The associated RDMA driver for this driver is +.Xr irdma 4 . +.Ss Dynamic Device Personalization +The DDP package loads during device initialization. +The driver looks for the +.Sy ice_ddp +module and checks that it contains a valid DDP package file. +.Pp +If the driver is unable to load the DDP package, the device will enter Safe +Mode. +Safe Mode disables advanced and performance features and supports only +basic traffic and minimal functionality, such as updating the NVM or +downloading a new driver or DDP package. +Safe Mode only applies to the affected physical function and does not impact +any other PFs. +See the +.Dq Intel Ethernet Adapters and Devices User Guide +for more details on DDP and Safe Mode. +.Pp +If issues are encountered with the DDP package file, an updated driver or +.Sy ice_ddp +module may need to be downloaded. +See the log messages for more information. +.Pp +The DDP package cannot be updated if any PF drivers are already loaded. +To overwrite a package, unload all PFs and then reload the driver with the +new package. +.Pp +Only one DDP package can be used per driver, +even if more than one installed device uses the driver. +.Pp +Only the first loaded PF per device can download a package for that device. +.Ss Jumbo Frames +Jumbo Frames support is enabled by changing the Maximum Transmission Unit (MTU) +to a value larger than the default value of 1500. +.Pp +Use +.Xr ifconfig 8 +to increase the MTU size. +.Pp +The maximum MTU setting for jumbo frames is 9706. +This corresponds to the maximum jumbo frame size of 9728 bytes. +.Pp +This driver will attempt to use multiple page sized buffers to receive +each jumbo packet. +This should help to avoid buffer starvation issues when allocating receive +packets. +.Pp +Packet loss may have a greater impact on throughput when jumbo frames are in +use. +If a drop in performance is observed after enabling jumbo frames, enabling +flow control may mitigate the issue. +.Ss Remote Direct Memory Access +Remote Direct Memory Access, or RDMA, allows a network device to transfer data +directly to and from application memory on another system, increasing +throughput and lowering latency in certain networking environments. +.Pp +The ice driver supports both the iWARP (Internet Wide Area RDMA Protocol) and +RoCEv2 (RDMA over Converged Ethernet) protocols. +The major difference is that iWARP performs RDMA over TCP, while RoCEv2 uses +UDP. +.Pp +Devices based on the Intel Ethernet 800 Series do not support RDMA when +operating in multiport mode with more than 4 ports. +.Pp +For detailed installation and configuration information for RDMA, see +.Xr irdma 4 . +.Ss RDMA Monitoring +For debugging/testing purposes, a sysctl can be used to set up a mirroring +interface on a port. +The interface can receive mirrored RDMA traffic for packet +analysis tools like +.Xr tcpdump 1 . +This mirroring may impact performance. +.Pp +To use RDMA monitoring, more MSI-X interrupts may need to be reserved. +Before the +.Nm +driver loads, configure the following tunable provided by +.Xr iflib 4 : +.Bd -literal -offset indent +dev.ice.<interface #>.iflib.use_extra_msix_vectors=4 +.Ed +.Pp +The number of extra MSI-X interrupt vectors may need to be adjusted. +.Pp +To create/delete the interface: +.Bd -literal -offset indent +sysctl dev.ice.<interface #>.create_interface=1 +sysctl dev.ice.<interface #>.delete_interface=1 +.Ed +.Pp +The mirrored interface receives both LAN and RDMA traffic. +Additional filters can be configured in tcpdump. +.Pp +To differentiate the mirrored interface from the primary interface, the network +interface naming convention is: +.Bd -literal -offset indent +<driver name><port number><modifier><modifier unit number> +.Ed +.Pp +For example, +.Dq Li ice0m0 +is the first mirroring interface on +.Dq Li ice0 . +.Ss Data Center Bridging +Data Center Bridging (DCB) is a configuration Quality of Service +implementation in hardware. +It uses the VLAN priority tag (802.1p) to filter traffic. +That means that there are 8 different priorities that traffic can be filtered +into. +It also enables priority flow control (802.1Qbb) which can limit or eliminate +the number of dropped packets during network stress. +Bandwidth can be allocated to each of these priorities, which is enforced at +the hardware level (802.1Qaz). +.Pp +DCB is normally configured on the network using the DCBX protocol (802.1Qaz), a +specialization of LLDP (802.1AB). The +.Nm +driver supports the following mutually exclusive variants of DCBX support: +.Pp +.Bl -bullet -compact +.It +Firmware-based LLDP Agent +.It +Software-based LLDP Agent +.El +.Pp +In firmware-based mode, firmware intercepts all LLDP traffic and handles DCBX +negotiation transparently for the user. +In this mode, the adapter operates in +.Dq willing +DCBX mode, receiving DCB settings from the link partner (typically a +switch). +The local user can only query the negotiated DCB configuration. +For information on configuring DCBX parameters on a switch, please consult the +switch manufacturer'ss documentation. +.Pp +In software-based mode, LLDP traffic is forwarded to the network stack and user +space, where a software agent can handle it. +In this mode, the adapter can operate in +.Dq nonwilling +DCBX mode and DCB configuration can be both queried and set locally. +This mode requires the FW-based LLDP Agent to be disabled. +.Pp +Firmware-based mode and software-based mode are controlled by the +.Dq fw_lldp_agent +sysctl. +Refer to the Firmware Link Layer Discovery Protocol Agent section for more +information. +.Pp +Link-level flow control and priority flow control are mutually exclusive. +The ice driver will disable link flow control when priority flow control +is enabled on any traffic class (TC). +It will disable priority flow control when link flow control is enabled. +.Pp +To enable/disable priority flow control in software-based DCBX mode: +.Bd -literal -offset indent +sysctl dev.ice.<interface #>.pfc=1 (or 0 to disable) +.Ed +.Pp +Enhanced Transmission Selection (ETS) allows bandwidth to be assigned to certain +TCs, to help ensure traffic reliability. +To view the assigned ETS configuration, use the following: +.Bd -literal -offset indent +sysctl dev.ice.<interface #>.ets_min_rate +.Ed +.Pp +To set the minimum ETS bandwidth per TC, separate the values by commas. +All values must add up to 100. +For example, to set all TCs to a minimum bandwidth of 10% and TC 7 to 30%, +use the following: +.Bd -literal -offset indent +sysctl dev.ice.<interface #>.ets_min_rate=10,10,10,10,10,10,10,30 +.Ed +.Pp +To set the User Priority (UP) to a TC mapping for a port, separate the values +by commas. +For example, to map UP 0 and 1 to TC 0, UP 2 and 3 to TC 1, UP 4 and +5 to TC 2, and UP 6 and 7 to TC 3, use the following: +.Bd -literal -offset indent +sysctl dev.ice.<interface #>.up2tc_map=0,0,1,1,2,2,3,3 +.Ed +.Ss L3 QoS Mode +The +.Nm +driver supports setting DSCP-based Layer 3 Quality of Service (L3 QoS) +in the PF driver. +The driver initializes in L2 QoS mode by default; L3 QoS is disabled by +default. +Use the following sysctl to enable or disable L3 QoS: +.Bd -literal -offset indent +sysctl dev.ice.<interface #>.pfc_mode=1 (or 0 to disable) +.Ed +.Pp +If L3 QoS mode is disabled, it returns to L2 QoS mode. +.Pp +To map a DSCP value to a traffic class, separate the values by commas. +For example, to map DSCPs 0-3 and DSCP 8 to DCB TCs 0-3 and 4, respectively: +.Bd -literal -offset indent +sysctl dev.ice.<interface #>.dscp2tc_map.0-7=0,1,2,3,0,0,0,0 +sysctl dev.ice.<interface #>.dscp2tc_map.8-15=4,0,0,0,0,0,0,0 +.Ed +.Pp +To change the DSCP mapping back to the default traffic class, set all the +values back to 0. +.Pp +To view the currently configured mappings, use the following: +.Bd -literal -offset indent +sysctl dev.ice.<interface #>.dscp2tc_map +.Ed +.Pp +L3 QoS mode is not available when FW-LLDP is enabled. +.Pp +FW-LLDP cannot be enabled if L3 QoS mode is active. +.Pp +Disable FW-LLDP before switching to L3 QoS mode. +.Pp +Refer to the +.Sx Firmware Link Layer Discovery Protocol Agent +section in this README for more information on disabling FW-LLDP. +.Ss Firmware Link Layer Discovery Protocol Agent +Use sysctl to change FW-LLDP settings. +The FW-LLDP setting is per port and persists across boots. +.Pp +To enable the FW-LLDP Agent: +.Bd -literal -offset indent +sysctl dev.ice.<interface #>.fw_lldp_agent=1 +.Ed +.Pp +To disable the FW-LLDP Agebt: +.Bd -literal -offset indent +sysctl dev.ice.<interface #>.fw_lldp_agent=0 +.Ed +.Pp +To check the current LLDP setting: +.Bd -literal -offset indent +sysctl dev.ice.<interface #>.fw_lldp_agent +.Ed +.Pp +The UEFI HII LLDP Agent attribute must be enabled for this setting +to take effect. +If the +.Dq LLDP AGENT +attribute is set to disabled, the FW-LLDP Agent cannot be enabled from the +driver. +.Ss Link-Level Flow Control +Ethernet Flow Control +.Pq IEEE 802.3x or LFC +can be configured with +.Xr sysctl 8 +to enable receiving and transmitting pause frames for +.Nm . +When transmit is enabled, pause frames are generated when the receive packet +buffer crosses a predefined threshold. +When receive is enabled, the transmit unit will halt for the time delay +specified in the firmware when a pause frame is received. +.Pp +Flow Control is disabled by default. +.Pp +Use sysctl to change the flow control settings for a single interface without +reloading the driver: +.Bd -literal -offset indent +sysctl dev.ice.<interface #>.fc +.Ed +.Pp +The available values for flow control are: +.Bd -literal -offset indent +0 = Disable flow control +1 = Enable Rx pause +2 = Enable Tx pause +3 = Enable Rx and Tx pause +.Ed +.Pp +Verify that link flow control was negotiated on the link by checking the +interface entry in +.Xr ifconfig 8 +and looking for the flags +.Dq txpause +and/or +.Dq rxpause +in the +.Dq media +status. +.Pp +The +.Nm +driver requires flow control on both the port and link partner. +If flow control is disabled on one of the sides, the port may appear to +hang on heavy traffic. +.Pp +For more information on priority flow control, refer to the +.Sx Data Center Bridging +section. +.Pp +The VF driver does not have access to flow control. +It must be managed from the host side. +.Ss Forward Error Correction +Forward Error Correction (FEC) improves link stability but increases latency. +Many high quality optics, direct attach cables, and backplane channels can +provide a stable link without FEC. +.Pp +For devices to benefit from this feature, link partners must have FEC enabled. +.Pp +If the +.Va allow_no_fec_modules_in_auto +sysctl is enabled Auto FEC negotiation will include +.Dq No FEC +in case the link partner does not have FEC enabled or is not FEC capable: +.Bd -literal -offset indent +sysctl dev.ice.<interface #>.allow_no_fec_modules_in_auto=1 +.Ed +.Pp +NOTE: This flag is currently not supported on the Intel Ethernet 830 +Series. +.Pp +To show the current FEC settings that are negotiated on the link: +.Bd -literal -offset indent +sysctl dev.ice.<interface #>.negotiated_fec +.Ed +.Pp +To view or set the FEC setting that was requested on the link: +.Bd -literal -offset indent +sysctl dev.ice.<interface #>.requested_fec +.Ed +.Pp +To see the valid FEC modes for the link: +.Bd -literal -offset indent +sysctl -d dev.ice.<interface #>.requested_fec +.Ed +.Ss Speed and Duplex Configuration +The speed and duplex settings cannot be hard set. +.Pp +To have the device change the speeds it will use in auto-negotiation or +force link with: +.Bd -literal -offset indent +sysctl dev.ice.<interface #>.advertise_speed=<mask> +.Ed +.Pp +Supported speeds will vary by device. +Depending on the speeds the device supports, valid bits used in a speed mask +could include: +.Bd -literal -offset indent +0x0 - Auto +0x2 - 100 Mbps +0x4 - 1 Gbps +0x8 - 2.5 Gbps +0x10 - 5 Gbps +0x20 - 10 Gbps +0x80 - 25 Gbps +0x100 - 40 Gbps +0x200 - 50 Gbps +0x400 - 100 Gbps +0x800 - 200 Gbps +.Ed +.Ss Disabling physical link when the interface is brought down +When the +.Va link_active_on_if_down +sysctl is set to +.Dq 0 , +the port's link will go down when the interface is brought down. +By default, link will stay up. +.Pp +To disable link when the interface is down: +.Bd -literal -offset indent +sysctl dev.ice.<interface #>.link_active_on_if_down=0 +.Ed +.Ss Firmware Logging +The +.Nm +driver allows for the generation of firmware logs for supported categories of +events, to help debug issues with Customer Support. +Refer to the +.Dq Intel Ethernet Adapters and Devices User Guide +for an overview of this feature and additional tips. +.Pp +At a high level, to capture a firmware log: +.Bl -enum -compact +.It +Set the configuration for the firmware log. +.It +Perform the necessary steps to reproduce the issue. +.It +Capture the firmware log. +.It +Stop capturing the firmware log. +.It +Reset the firmware log settings as needed. +.It +Work with Customer Support to debug the issue. +.El +.Pp +NOTE: Firmware logs are generated in a binary format and must be decoded by +Customer Support. +Information collected is related only to firmware and hardware for debug +purposes. +.Pp +Once the driver is loaded, it will create the +.Va fw_log +sysctl node under the debug section of the driver's sysctl list. +The driver groups these events into categories, called +.Dq modules . +Supported modules include: +.Pp +.Bl -tag -offset indent -compact -width "task_dispatch" +.It Va general +General (Bit 0) +.It Va ctrl +Control (Bit 1) +.It Va link +Link Management (Bit 2) +.It Va link_topo +Link Topology Detection (Bit 3) +.It Va dnl +Link Control Technology (Bit 4) +.It Va i2c +I2C (Bit 5) +.It Va sdp +SDP (Bit 6) +.It Va mdio +MDIO (Bit 7) +.It Va adminq +Admin Queue (Bit 8) +.It Va hdma +Host DMA (Bit 9) +.It Va lldp +LLDP (Bit 10) +.It Va dcbx +DCBx (Bit 11) +.It Va dcb +DCB (Bit 12) +.It Va xlr +XLR (function-level resets; Bit 13) +.It Va nvm +NVM (Bit 14) +.It Va auth +Authentication (Bit 15) +.It Va vpd +Vital Product Data (Bit 16) +.It Va iosf +Intel On-Chip System Fabric (Bit 17) +.It Va parser +Parser (Bit 18) +.It Va sw +Switch (Bit 19) +.It Va scheduler +Scheduler (Bit 20) +.It Va txq +TX Queue Management (Bit 21) +.It Va acl +ACL (Access Control List; Bit 22) +.It Va post +Post (Bit 23) +.It Va watchdog +Watchdog (Bit 24) +.It Va task_dispatch +Task Dispatcher (Bit 25) +.It Va mng +Manageability (Bit 26) +.It Va synce +SyncE (Bit 27) +.It Va health +Health (Bit 28) +.It Va tsdrv +Time Sync (Bit 29) +.It Va pfreg +PF Registration (Bit 30) +.It Va mdlver +Module Version (Bit 31) +.El +.Pp +The verbosity level of the firmware logs can be modified. +It is possible to set only one log level per module, and each level includes the +verbosity levels lower than it. +For instance, setting the level to +.Dq normal +will also log warning and error messages. +Available verbosity levels are: +.Pp +.Bl -item -offset indent -compact +.It +0 = none +.It +1 = error +.It +2 = warning +.It +3 = normal +.It +4 = verbose +.El +.Pp +To set the desired verbosity level for a module, use the following sysctl +command and then register it: +.Bd -literal -offset indent +sysctl dev.ice.<interface #>.debug.fw_log.severity.<module>=<level> +.Ed +.Pp +For example: +.Bd -literal -offset indent +sysctl dev.ice.0.debug.fw_log.severity.link=1 +sysctl dev.ice.0.debug.fw_log.severity.link_topo=2 +sysctl dev.ice.0.debug.fw_log.register=1 +.Ed +.Pp +To log firmware messages after booting, but before the driver initializes, use +.Xr kenv 1 +to set the tunable. +The +.Va on_load +setting tells the device to register the variable as soon as possible during +driver load. +For example: +.Bd -literal -offset indent +kenv dev.ice.0.debug.fw_log.severity.link=1 +kenv dev.ice.0.debug.fw_log.severity.link_topo=2 +kenv dev.ice.0.debug.fw_log.on_load=1 +.Ed +.Pp +To view the firmware logs and redirect them to a file, use the following +command: +.Bd -literal -offset indent +dmesg > log_output +.Ed +.Pp +NOTE: Logging a large number of modules or too high of a verbosity level will +add extraneous messages to dmesg and could hinder debug efforts. +.Ss Debug Dump +Intel Ethernet 800 Series devices support debug dump, +which allows gathering of runtime register values from the firmware for +.Dq clusters +of events and then write the results to a single dump file, for debugging +complicated issues in the field. +.Pp +This debug dump contains a snapshot of the device and its existing hardware +configuration, such as switch tables, transmit scheduler tables, and other +information. +Debug dump captures the current state of the specified cluster(s) and is a +stateless snapshot of the whole device. +.Pp +NOTE: Like with firmware logs, the contents of the debug dump are not +human-readable. +Work with Customer Support to decode the file. +.Pp +Debug dump is per device, not per PF. +.Pp +Debug dump writes all information to a single file. +.Pp +To generate a debug dump file in +.Fx +do the following: +.Pp +Specify the cluster(s) to include in the dump file, using a bitmask and the +following command: +.Bd -literal -offset indent +sysctl dev.ice.<interface #>.debug.dump.clusters=<bitmask> +.Ed +.Pp +To print the complete cluster bitmask and parameter list to the screen, +pass the +.Fl d +argument. +For example: +.Bd -literal -offset indent +sysctl -d dev.ice.0.debug.dump.clusters +.Ed +.Pp +Possible bitmask values for +.Va clusters +are: +.Bl -bullet -compact +.It +0 - Dump all clusters (only supported on Intel Ethernet E810 Series and +Intel Ethernet E830 Series) +.It +0x1 - Switch +.It +0x2 - ACL +.It +0x4 - Tx Scheduler +.It +0x8 - Profile Configuration +.It +0x20 - Link +.It +0x80 - DCB +.It +0x100 - L2P +.It +0x400000 - Manageability Transactions (only supported on Intel Ethernet +E810 Series) +.El +.Pp +For example, to dump the Switch, DCB, and L2P clusters, use the following: +.Bd -literal -offset indent +sysctl dev.ice.0.debug.dump.clusters=0x181 +.Ed +.Pp +To dump all clusters, use the following: +.Bd -literal -offset indent +sysctl dev.ice.0.debug.dump.clusters=0 +.Ed +.Pp +NOTE: Using 0 will skip Manageability Transactions data. +.Pp +If a single cluster is not specified, +the driver will dump all clusters to a single file. +Issue the debug dump command, using the following: +.Bd -literal -offset indent +sysctl -b dev.ice.<interface #>.debug.dump.dump=1 > dump.bin +.Ed +.Pp +NOTE: The driver will not receive the command if the sysctl is not set to +.Dq 1 . +.Pp +Replace +.Dq dump.bin +above with the preferred file name. +.Pp +To clear the +.Va clusters +mask before a subsequent debug dump and then do the dump: +.Bd -literal -offset indent +sysctl dev.ice.0.debug.dump.clusters=0 +sysctl dev.ice.0.debug.dump.dump=1 +.Ed +.Ss Debugging PHY Statistics +The ice driver supports the ability to obtain the values of the PHY registers +from Intel(R) Ethernet 810 Series devices in order to debug link and +connection issues during runtime. +.Pp +The driver provides information about: +.Bl -bullet +.It +Rx and Tx Equalization parameters +.It +RS FEC correctable and uncorrectable block counts +.El +.Pp +Use the following sysctl to read the PHY registers: +.Bd -literal -offset indent +sysctl dev.ice.<interface #>.debug.phy_statistics +.Ed +.Pp +NOTE: The contents of the registers are not human-readable. +Like with firmware logs and debug dump, work with Customer Support +to decode the file. +.Ss Transmit Balancing +Some Intel(R) Ethernet 800 Series devices allow for enabling a transmit +balancing feature to improve transmit performance under certain conditions. +When enabled, this feature should provide more consistent transmit +performance across queues and/or PFs and VFs. +.Pp +By default, transmit balancing is disabled in the NVM. +To enable this feature, use one of the following to persistently change the +setting for the device: +.Bl -bullet +.It +Use the Ethernet Port Configuration Tool (EPCT) to enable the +.Va tx_balancing +option. +Refer to the EPCT readme for more information. +.It +Enable the Transmit Balancing device setting in UEFI HII. +.El +.Pp +When the driver loads, it reads the transmit balancing setting from the NVM and +configures the device accordingly. +.Pp +NOTE: The user selection for transmit balancing in EPCT or HII is persistent +across reboots. +The system must be rebooted for the selected setting to take effect. +.Pp +This setting is device wide. +.Pp +The driver, NVM, and DDP package must all support this functionality to +enable the feature. +.Ss Thermal Monitoring +Intel(R) Ethernet 810 Series and Intel(R) Ethernet 830 Series devices can +display temperature data (in degrees Celsius) via: +.Bd -literal -offset indent +sysctl dev.ice.<interface #>.temp +.Ed +.Ss Network Memory Buffer Allocation +.Fx +may have a low number of network memory buffers (mbufs) by default. +If the number of mbufs available is too low, it may cause the driver to fail +to initialize and/or cause the system to become unresponsive. +Check to see if the system is mbuf-starved by running +.Ic netstat Fl m . +Increase the number of mbufs by editing the lines below in +.Pa /etc/sysctl.conf : +.Bd -literal -offset indent +kern.ipc.nmbclusters +kern.ipc.nmbjumbop +kern.ipc.nmbjumbo9 +kern.ipc.nmbjumbo16 +kern.ipc.nmbufs +.Ed +.Pp +The amount of memory that should be allocated is system specific, +and may require some trial and error. +Also, increasing the following in +.Pa /etc/sysctl.conf +could help increase network performance: +.Bd -literal -offset indent +kern.ipc.maxsockbuf +net.inet.tcp.sendspace +net.inet.tcp.recvspace +net.inet.udp.maxdgram +net.inet.udp.recvspace +.Ed +.Ss Additional Utilities +There are additional tools available from Intel to help configure and update +the adapters covered by this driver. +These tools can be downloaded directly from Intel at +.Lk https://downloadcenter.intel.com , +by searching for their names: +.Bl -bullet +.It +To change the behavior of the QSFP28 ports on E810-C adapters, use the Intel +.Sy Ethernet Port Configuration Tool - FreeBSD . +.It +To update the firmware on an adapter, use the Intel +.Sy Non-Volatile Memory (NVM) Update Utility for Intel Ethernet Network Adapters E810 series - FreeBSD +.El +.Ss Optics and auto-negotiation +Modules based on 100GBASE-SR4, +active optical cable (AOC), and active copper cable (ACC) +do not support auto-negotiation per the IEEE specification. +To obtain link with these modules, +auto-negotiation must be turned off on the link partner's switch ports. +.Pp +Note that adapters also support +all passive and active limiting direct attach cables +that comply with SFF-8431 v4.1 and SFF-8472 v10.4 specifications. +.Ss PCI-Express Slot Bandwidth +Some PCIe x8 slots are actually configured as x4 slots. +These slots have insufficient bandwidth +for full line rate with dual port and quad port devices. +In addition, +if a PCIe v4.0 or v3.0-capable adapter is placed into into a PCIe v2.x +slot, full bandwidth will not be possible. +.Pp +The driver detects this situation and +writes the following message in the system log: +.Bd -ragged -offset indent +PCI-Express bandwidth available for this device +may be insufficient for optimal performance. +Please move the device to a different PCI-e link +with more lanes and/or higher transfer rate. +.Ed +.Pp +If this error occurs, +moving the adapter to a true PCIe x8 or x16 slot will resolve the issue. +For best performance, install devices in the following PCI slots: +.Bl -bullet +.It +Any 100Gbps-capable Intel(R) Ethernet 800 Series device: Install in a +PCIe v4.0 x8 or v3.0 x16 slot +.It +A 200Gbps-capable Intel(R) Ethernet 830 Series device: Install in a +PCIe v5.0 x8 or v4.0 x16 slot +.El +.Pp +For questions related to hardware requirements, +refer to the documentation supplied with the adapter. +.Sh HARDWARE +The +.Nm +driver supports the following +Intel 800 series 1Gb to 200Gb Ethernet controllers: +.Pp +.Bl -bullet -compact +.It +Intel Ethernet Controller E810-C +.It +Intel Ethernet Controller E810-XXV +.It +Intel Ethernet Connection E822-C +.It +Intel Ethernet Connection E822-L +.It +Intel Ethernet Connection E823-C +.It +Intel Ethernet Connection E823-L +.It +Intel Ethernet Connection E825-C +.It +Intel Ethernet Connection E830-C +.It +Intel Ethernet Connection E830-CC +.It +Intel Ethernet Connection E830-L +.It +Intel Ethernet Connection E830-XXV +.It +Intel Ethernet Connection E835-C +.It +Intel Ethernet Connection E835-CC +.It +Intel Ethernet Connection E835-L +.It +Intel Ethernet Connection E835-XXV +.El +.Pp +The +.Nm +driver supports some adapters in this series with SFP28/QSFP28 cages +which have firmware that requires that Intel qualified modules are used; +these qualified modules are listed below. +This qualification check cannot be disabled by the driver. +.Pp +The +.Nm +driver supports 100Gb Ethernet adapters with these QSFP28 modules: +.Pp +.Bl -bullet -compact +.It +Intel 100G QSFP28 100GBASE-SR4 E100GQSFPSR28SRX +.It +Intel 100G QSFP28 100GBASE-SR4 SPTMBP1PMCDF +.It +Intel 100G QSFP28 100GBASE-CWDM4 SPTSBP3CLCCO +.It +Intel 100G QSFP28 100GBASE-DR SPTSLP2SLCDF +.El +.Pp +The +.Nm +driver supports 25Gb and 10Gb Ethernet adapters with these SFP28 modules: +.Pp +.Bl -bullet -compact +.It +Intel 10G/25G SFP28 25GBASE-SR E25GSFP28SR +.It +Intel 25G SFP28 25GBASE-SR E25GSFP28SRX (Extended Temp) +.It +Intel 25G SFP28 25GBASE-LR E25GSFP28LRX (Extended Temp) +.El +.Pp +The +.Nm +driver supports 10Gb and 1Gb Ethernet adapters with these SFP+ modules: +.Pp +.Bl -bullet -compact +.It +Intel 1G/10G SFP+ 10GBASE-SR E10GSFPSR +.It +Intel 1G/10G SFP+ 10GBASE-SR E10GSFPSRG1P5 +.It +Intel 1G/10G SFP+ 10GBASE-SR E10GSFPSRG2P5 +.It +Intel 10G SFP+ 10GBASE-SR E10GSFPSRX (Extended Temp) +.It +Intel 1G/10G SFP+ 10GBASE-LR E10GSFPLR +.El +.Sh LOADER TUNABLES +Tunables can be set at the +.Xr loader 8 +prompt before booting the kernel or stored in +.Xr loader.conf 5 . +See the +.Xr iflib 4 +man page for more information on using iflib sysctl variables as tunables. +.Bl -tag -width indent +.It Va hw.ice.enable_health_events +Set to 1 to enable firmware health event reporting across all devices. +Enabled by default. +.Pp +If enabled, when the driver receives a firmware health event message, it will +print out a description of the event to the kernel message buffer and if +applicable, possible actions to take to remedy it. +.It Va hw.ice.irdma +Set to 1 to enable the RDMA client interface, required by the +.Xr irdma 4 +driver. +Enabled by default. +.It Va hw.ice.rdma_max_msix +Set the maximum number of per-device MSI-X vectors that are allocated for use +by the +.Xr irdma 4 +driver. +Set to 64 by default. +.It Va hw.ice.debug.enable_tx_fc_filter +Set to 1 to enable the TX Flow Control filter across all devices. +Enabled by default. +.Pp +If enabled, the hardware will drop any transmitted Ethertype 0x8808 control +frames that do not originate from the hardware. +.It Va hw.ice.debug.enable_tx_lldp_filter +Set to 1 to enable the TX LLDP filter across all devices. +Enabled by default. +.Pp +If enabled, the hardware will drop any transmitted Ethertype 0x88cc LLDP frames +that do not originate from the hardware. +This must be disabled in order to use LLDP daemon software such as +.Xr lldpd 8 . +.It Va hw.ice.debug.ice_tx_balance_en +Set to 1 to allow the driver to use the 5-layer Tx Scheduler tree topology if +configured by the DDP package. +.Pp +Enabled by default. +.El +.Sh SYSCTL VARIABLES +.Bl -tag -width indent +.It Va dev.ice.#.current_speed +This is a display of the current link speed of the interface. +This is expected to match the speed of the media type in-use displayed by +.Xr ifconfig 8 . +.It Va dev.ice.#.fw_version +Displays the current firmware and NVM versions of the adapter. +This information should be submitted along with any support requests. +.It Va dev.ice.#.ddp_version +Displays the current DDP package version downloaded to the adapter. +This information should be submitted along with any support requests. +.It Va dev.ice.#.pba_number +Displays the Product Board Assembly Number. +May be used to help identify the type of adapter in use. +This sysctl may not exist depending on the adapter type. +.It Va dev.ice.#.hw.mac.* +This sysctl tree contains statistics collected by the hardware for the port. +.El +.Sh INTERRUPT STORMS +It is important to note that 100G operation can generate high +numbers of interrupts, often incorrectly being interpreted as +a storm condition in the kernel. +It is suggested that this be resolved by setting +.Va hw.intr_storm_threshold +to 0. +.Sh IOVCTL OPTIONS +The driver supports additional optional parameters for created VFs +(Virtual Functions) when using +.Xr iovctl 8 : +.Bl -tag -width indent +.It mac-addr Pq unicast-mac +Set the Ethernet MAC address that the VF will use. +If unspecified, the VF will use a randomly generated MAC address and +.Dq allow-set-mac +will be set to true. +.It mac-anti-spoof Pq bool +Prevent the VF from sending Ethernet frames with a source address +that does not match its own. +Enabled by default. +.It allow-set-mac Pq bool +Allow the VF to set its own Ethernet MAC address. +Disallowed by default. +.It allow-promisc Pq bool +Allow the VF to inspect all of the traffic sent to the port that it is created +on. +Disabled by default. +.It num-queues Pq uint16_t +Specify the number of queues the VF will have. +By default, this is set to the number of MSI-X vectors supported by the VF +minus one. +.It mirror-src-vsi Pq uint16_t +Specify which VSI the VF will mirror traffic from by setting this to a value +other than -1. +All traffic from that VSI will be mirrored to this VF. +Can be used as an alternative method to mirror RDMA traffic to another +interface than the method described in the +.Sx RDMA Monitoring +section. +Not affected by the +.Dq allow-promisc +parameter. +.It max-vlan-allowed Pq uint16_t +Specify maximum number of VLAN filters that the VF can use. +Receiving traffic on a VLAN requires a hardware filter which are a finite +resource; this is used to prevent a VF from starving other VFs or the PF of +filter resources. +By default, this is set to 16. +.It max-mac-filters Pq uint16_t +Specify maximum number of MAC address filters that the VF can use. +Each allowed MAC address requires a hardware filter which are a finite +resource; this is used to prevent a VF from starving other VFs or the PF of +filter resources. +The VF's default mac address does not count towards this limit. +By default, this is set to 64. +.El +.Pp +An up to date list of parameters and their defaults can be found by using +.Xr iovctl 8 +with the +.Fl S +option. +.Pp +For more information on standard and mandatory parameters, see +.Xr iovctl.conf 5 . +.Sh SUPPORT +For general information and support, go to the Intel support website at: +.Lk http://www.intel.com/support/ . +.Pp +If an issue is identified with this driver with a supported adapter, +email all the specific information related to the issue to +.Aq Mt freebsd@intel.com . +.Sh SEE ALSO +.Xr iflib 4 , +.Xr vlan 4 , +.Xr ifconfig 8 , +.Xr sysctl 8 +.Sh HISTORY +The +.Nm +device driver first appeared in +.Fx 12.2 . +.Sh AUTHORS +The +.Nm +driver was written by +.An Intel Corporation Aq Mt freebsd@intel.com . |