path: root/static/freebsd/man4/gve.4 3.html

<table class="head">
  <tr>
    <td class="head-ltitle">GVE(4)</td>
    <td class="head-vol">Device Drivers Manual</td>
    <td class="head-rtitle">GVE(4)</td>
  </tr>
</table>
<div class="manual-text">
<section class="Sh">
<h1 class="Sh" id="NAME"><a class="permalink" href="#NAME">NAME</a></h1>
<p class="Pp"><code class="Nm">gve</code> &#x2014; <span class="Nd">Ethernet
    driver for Google Virtual NIC (gVNIC)</span></p>
</section>
<section class="Sh">
<h1 class="Sh" id="SYNOPSIS"><a class="permalink" href="#SYNOPSIS">SYNOPSIS</a></h1>
<p class="Pp">To compile this driver into the kernel, place the following lines
    in your kernel configuration file:</p>
<div class="Bd Pp Bd-indent"><code class="Cd">device gve</code></div>
<p class="Pp">Alternatively, to load the driver as a module at boot time, place
    the following line in <a class="Xr">loader.conf(5)</a>:</p>
<div class="Bd Pp Bd-indent Li">
<pre>if_gve_load=&quot;YES&quot;</pre>
</div>
</section>
<section class="Sh">
<h1 class="Sh" id="DESCRIPTION"><a class="permalink" href="#DESCRIPTION">DESCRIPTION</a></h1>
<p class="Pp">gVNIC is a virtual network interface designed specifically for
    Google Compute Engine (GCE). It is required to support per-VM Tier-1
    networking performance, and for using certain VM shapes on GCE.</p>
<p class="Pp"><code class="Nm">gve</code> is the driver for gVNIC. It supports
    the following features:</p>
<p class="Pp"></p>
<ul class="Bl-bullet Bl-compact">
  <li>RX checksum offload</li>
  <li>TX chesksum offload</li>
  <li>TCP Segmentation Offload (TSO)</li>
  <li>Large Receive Offload (LRO) in software</li>
  <li>Jumbo frames</li>
  <li>Receive Side Scaling (RSS)</li>
</ul>
<p class="Pp">For more information on configuring this device, see
    <a class="Xr">ifconfig(8)</a>.</p>
</section>
<section class="Sh">
<h1 class="Sh" id="HARDWARE"><a class="permalink" href="#HARDWARE">HARDWARE</a></h1>
<p class="Pp"><code class="Nm">gve</code> binds to a single PCI device ID
    presented by gVNIC:</p>
<p class="Pp"></p>
<ul class="Bl-bullet Bl-compact">
  <li>0x1AE0:0x0042</li>
</ul>
</section>
<section class="Sh">
<h1 class="Sh" id="EXAMPLES"><a class="permalink" href="#EXAMPLES">EXAMPLES</a></h1>
<p class="Pp">Change the TX queue count to 4 for the gve0 interface:</p>
<div class="Bd Bd-indent">sysctl dev.gve.0.num_tx_queues=4</div>
<p class="Pp">Change the RX queue count to 4 for the gve0 interface:</p>
<div class="Bd Bd-indent">sysctl dev.gve.0.num_rx_queues=4</div>
<p class="Pp">Change the TX ring size to 512 for the gve0 interface:</p>
<div class="Bd Bd-indent">sysctl dev.gve.0.tx_ring_size=512</div>
<p class="Pp">Change the RX ring size to 512 for the gve0 interface:</p>
<div class="Bd Bd-indent">sysctl dev.gve.0.rx_ring_size=512</div>
</section>
<section class="Sh">
<h1 class="Sh" id="DIAGNOSTICS"><a class="permalink" href="#DIAGNOSTICS">DIAGNOSTICS</a></h1>
<p class="Pp">The following messages are recorded during driver
  initialization:</p>
<dl class="Bl-diag">
  <dt>Enabled MSIX with %d vectors</dt>
  <dd></dd>
  <dt>Configured device resources</dt>
  <dd></dd>
  <dt>Successfully attached %s</dt>
  <dd></dd>
  <dt>Deconfigured device resources</dt>
  <dd></dd>
</dl>
<p class="Pp">These messages are seen if driver initialization fails. Global
    (across-queues) allocation failures:</p>
<dl class="Bl-diag">
  <dt>Failed to configure device resources: err=%d</dt>
  <dd></dd>
  <dt>No compatible queue formats</dt>
  <dd></dd>
  <dt>Failed to allocate ifnet struct</dt>
  <dd></dd>
  <dt>Failed to allocate admin queue mem</dt>
  <dd></dd>
  <dt>Failed to alloc DMA mem for DescribeDevice</dt>
  <dd></dd>
  <dt>Failed to allocate QPL page</dt>
  <dd></dd>
</dl>
<p class="Pp">irq and BAR allocation failures:</p>
<dl class="Bl-diag">
  <dt>Failed to acquire any msix vectors</dt>
  <dd></dd>
  <dt>Tried to acquire %d msix vectors, got only %d</dt>
  <dd></dd>
  <dt>Failed to setup irq %d for Tx queue %d </dt>
  <dd></dd>
  <dt>Failed to setup irq %d for Rx queue %d </dt>
  <dd></dd>
  <dt>Failed to allocate irq %d for mgmnt queue</dt>
  <dd></dd>
  <dt>Failed to setup irq %d for mgmnt queue, err: %d</dt>
  <dd></dd>
  <dt>Failed to allocate BAR0</dt>
  <dd></dd>
  <dt>Failed to allocate BAR2</dt>
  <dd></dd>
  <dt>Failed to allocate msix table</dt>
  <dd></dd>
</dl>
<p class="Pp">Rx queue-specific allocation failures:</p>
<dl class="Bl-diag">
  <dt>No QPL left for rx ring %d</dt>
  <dd></dd>
  <dt>Failed to alloc queue resources for rx ring %d</dt>
  <dd></dd>
  <dt>Failed to alloc desc ring for rx ring %d</dt>
  <dd></dd>
  <dt>Failed to alloc data ring for rx ring %d</dt>
  <dd></dd>
</dl>
<p class="Pp">Tx queue-specific allocation failures:</p>
<dl class="Bl-diag">
  <dt>No QPL left for tx ring %d</dt>
  <dd></dd>
  <dt>Failed to alloc queue resources for tx ring %d</dt>
  <dd></dd>
  <dt>Failed to alloc desc ring for tx ring %d</dt>
  <dd></dd>
  <dt>Failed to vmap fifo, qpl_id = %d</dt>
  <dd></dd>
</dl>
<p class="Pp">The following messages are recorded when the interface detach
    fails:</p>
<dl class="Bl-diag">
  <dt>Failed to deconfigure device resources: err=%d</dt>
  <dd></dd>
</dl>
<p class="Pp">If bootverbose is on, the following messages are recorded when the
    interface is being brought up:</p>
<dl class="Bl-diag">
  <dt>Created %d rx queues</dt>
  <dd></dd>
  <dt>Created %d tx queues</dt>
  <dd></dd>
  <dt>MTU set to %d</dt>
  <dd></dd>
</dl>
<p class="Pp">The following messages are recorded when the interface is being
    brought down:</p>
<dl class="Bl-diag">
  <dt>Destroyed %d rx queues</dt>
  <dd></dd>
  <dt>Destroyed %d tx queues</dt>
  <dd></dd>
</dl>
<p class="Pp">These messages are seen if errors are encountered when bringing
    the interface up or down:</p>
<dl class="Bl-diag">
  <dt>Failed to destroy rxq %d, err: %d</dt>
  <dd></dd>
  <dt>Failed to destroy txq %d, err: %d</dt>
  <dd></dd>
  <dt>Failed to create rxq %d, err: %d</dt>
  <dd></dd>
  <dt>Failed to create txq %d, err: %d</dt>
  <dd></dd>
  <dt>Failed to set MTU to %d</dt>
  <dd></dd>
  <dt>Invalid new MTU setting. new mtu: %d max mtu: %d min mtu: %d</dt>
  <dd></dd>
  <dt>Cannot bring the iface up when detached</dt>
  <dd></dd>
  <dt>Reached max number of registered pages %lu &gt; %lu</dt>
  <dd></dd>
  <dt>Failed to init lro for rx ring %d</dt>
  <dd></dd>
</dl>
<p class="Pp">These messages are seen if any admin queue command fails:</p>
<dl class="Bl-diag">
  <dt>AQ command(%u): failed with status %d</dt>
  <dd></dd>
  <dt>AQ command(%u): unknown status code %d</dt>
  <dd></dd>
  <dt>AQ commands timed out, need to reset AQ</dt>
  <dd></dd>
  <dt>Unknown AQ command opcode %d</dt>
  <dd></dd>
</dl>
<p class="Pp">These messages appear if a TX timeout is detected:</p>
<dl class="Bl-diag">
  <dt>Found %d timed out packet(s) on txq%d, kicking it for completions</dt>
  <dd></dd>
  <dt>Found %d timed out packet(s) on txq%d with its last kick %ld sec ago which
    is less than the cooldown period %d. Resetting device</dt>
  <dd></dd>
</dl>
<p class="Pp">These messages are recorded when the device is being reset due to
    an error:</p>
<dl class="Bl-diag">
  <dt>Scheduling reset task!</dt>
  <dd></dd>
  <dt>Waiting until admin queue is released.</dt>
  <dd></dd>
  <dt>Admin queue released</dt>
  <dd></dd>
</dl>
<p class="Pp">If it was the NIC that requested the reset, this message is
    recorded:</p>
<dl class="Bl-diag">
  <dt>Device requested reset</dt>
  <dd></dd>
</dl>
<p class="Pp">If the reset fails during the reinitialization phase, this message
    is recorded:</p>
<dl class="Bl-diag">
  <dt>Restore failed!</dt>
  <dd></dd>
</dl>
<p class="Pp">These two messages correspond to the NIC alerting the driver to
    link state changes:</p>
<dl class="Bl-diag">
  <dt>Device link is up.</dt>
  <dd></dd>
  <dt>Device link is down.</dt>
  <dd></dd>
</dl>
<p class="Pp">Apart from these messages, the driver exposes per-queue packet and
    error counters as sysctl nodes. Global (across queues) counters can be read
    using <a class="Xr">netstat(1)</a>.</p>
</section>
<section class="Sh">
<h1 class="Sh" id="SYSCTL_VARIABLES"><a class="permalink" href="#SYSCTL_VARIABLES">SYSCTL
  VARIABLES</a></h1>
<p class="Pp"><code class="Nm">gve</code> exposes the following
    <a class="Xr">sysctl(8)</a> variables:</p>
<dl class="Bl-tag">
  <dt id="hw.gve.driver_version"><var class="Va">hw.gve.driver_version</var></dt>
  <dd>The driver version. This is read-only.</dd>
  <dt id="hw.gve.queue_format"><var class="Va">hw.gve.queue_format</var></dt>
  <dd>The queue format in use. This is read-only.</dd>
  <dt id="hw.gve.disable_hw_lro"><var class="Va">hw.gve.disable_hw_lro</var></dt>
  <dd>Setting this boot-time tunable to 1 disables Large Receive Offload (LRO)
      in the NIC. The default value is 0, which means hardware LRO is enabled by
      default. The software LRO stack in the kernel is always used. This sysctl
      variable needs to be set before loading the driver, using
      <a class="Xr">loader.conf(5)</a>.</dd>
  <dt id="hw.gve.allow_4k_rx_buffers"><var class="Va">hw.gve.allow_4k_rx_buffers</var></dt>
  <dd>Setting this boot-time tunable to 1 enables support for 4K RX Buffers. The
      default value is 0, which means 2K RX Buffers will be used. 4K RX Buffers
      are only supported on DQO_RDA and DQO_QPL queue formats. When enabled, 4K
      RX Buffers will be used either when HW LRO is enabled or mtu is greated
      than 2048. This sysctl variable needs to be set before loading the driver,
      using <a class="Xr">loader.conf(5)</a>.</dd>
  <dt id="dev.gve.X.num_rx_queues"><var class="Va">dev.gve.X.num_rx_queues and
    dev.gve.X.num_tx_queues</var></dt>
  <dd>Run-time tunables that represent the number of currently used RX/TX
      queues. The default value is the max number of RX/TX queues the device can
      support.
    <p class="Pp">This call turns down the interface while setting up the new
        queues, which may potentially cause any new packets to be dropped. This
        call can fail if the system is not able to provide the driver with
        enough resources. In that situation, the driver will revert to the
        previous number of RX/TX queues. If this also fails, a device reset will
        be triggered.</p>
    <p class="Pp">Note: sysctl nodes for queue stats remain available even if a
        queue is removed.</p>
  </dd>
  <dt id="dev.gve.X.rx_ring_size"><var class="Va">dev.gve.X.rx_ring_size and
    dev.gve.X.tx_ring_size</var></dt>
  <dd>Run-time tunables that represent the current ring size for RX/TX queues.
      The default value is set to device defaults for ring size.
    <p class="Pp">This call turns down the interface while setting up the queues
        with the new ring size, which may potentially cause any new packets to
        be dropped. This call can fail if the system is not able to provide the
        driver with enough resources. In that situation, the driver will try to
        revert to the previous ring size for RX/TX queues. If this also fails,
        the device will be in an unhealthy state and will need to be reloaded.
        This value must be a power of 2 and within the defined range.</p>
  </dd>
</dl>
</section>
<section class="Sh">
<h1 class="Sh" id="LIMITATIONS"><a class="permalink" href="#LIMITATIONS">LIMITATIONS</a></h1>
<p class="Pp"><code class="Nm">gve</code> does not support the transmission of
    VLAN-tagged packets. All VLAN-tagged traffic is dropped.</p>
</section>
<section class="Sh">
<h1 class="Sh" id="QUEUE_FORMATS"><a class="permalink" href="#QUEUE_FORMATS">QUEUE
  FORMATS</a></h1>
<p class="Pp"><code class="Nm">gve</code> features different datapath modes
    called queue formats:</p>
<p class="Pp"></p>
<ul class="Bl-bullet Bl-compact">
  <li>GQI_QPL: &quot;QPL&quot; stands for &quot;Queue Page List&quot; and refers
      to the fact that hardware expects a fixed bounce buffer and cannot access
      arbitrary memory. GQI is the older descriptor format. The G in
      &quot;GQI&quot; refers to an older generation of hardware, and the
      &quot;QI&quot; stands for &quot;Queue In-order&quot; referring to the fact
      that the NIC sends Tx and Rx completions in the same order as the one in
      which the corresponding descriptors were posted by the driver.</li>
  <li>DQO_RDA: DQO is the descriptor format required to take full advantage of
      next generation VM shapes. &quot;RDA&quot; stands for &quot;Raw DMA
      Addressing&quot; and refers to the fact that hardware can work with DMA-ed
      packets and does not expect them to be copied into or out of a fixed
      bounce buffer. The D in &quot;DQO&quot; refers to a newer generation of
      hardware, and the &quot;QO&quot; stands for &quot;Queue Out-of-order&quot;
      referring to the fact that the NIC might send Tx and Rx completions in an
      order different from the one in which the corresponding descriptors were
      posted by the driver.</li>
  <li>DQO_QPL: The next generation descriptor format in the &quot;QPL&quot;
      mode.</li>
</ul>
</section>
<section class="Sh">
<h1 class="Sh" id="SUPPORT"><a class="permalink" href="#SUPPORT">SUPPORT</a></h1>
<p class="Pp">Please email gvnic-drivers@google.com with the specifics of the
    issue encountered.</p>
</section>
<section class="Sh">
<h1 class="Sh" id="SEE_ALSO"><a class="permalink" href="#SEE_ALSO">SEE
  ALSO</a></h1>
<p class="Pp"><a class="Xr">netstat(1)</a>, <a class="Xr">loader.conf(5)</a>,
    <a class="Xr">ifconfig(8)</a>, <a class="Xr">sysctl(8)</a></p>
</section>
<section class="Sh">
<h1 class="Sh" id="HISTORY"><a class="permalink" href="#HISTORY">HISTORY</a></h1>
<p class="Pp">The <code class="Nm">gve</code> device driver first appeared in
    <span class="Ux">FreeBSD 13.3</span>.</p>
</section>
<section class="Sh">
<h1 class="Sh" id="AUTHORS"><a class="permalink" href="#AUTHORS">AUTHORS</a></h1>
<p class="Pp">The <code class="Nm">gve</code> driver was written by Google.</p>
</section>
</div>
<table class="foot">
  <tr>
    <td class="foot-date">May 20, 2025</td>
    <td class="foot-os">FreeBSD 15.0</td>
  </tr>
</table>