mirror of https://github.com/torvalds/linux.git
84bf1ac85a
The ice_put_rx_mbuf() function handles calling ice_put_rx_buf() for each
buffer in the current frame. This function was introduced as part of
handling multi-buffer XDP support in the ice driver.
It works by iterating over the buffers from first_desc up to 1 plus the
total number of fragments in the frame, cached from before the XDP program
was executed.
If the hardware posts a descriptor with a size of 0, the logic used in
ice_put_rx_mbuf() breaks. Such descriptors get skipped and don't get added
as fragments in ice_add_xdp_frag. Since the buffer isn't counted as a
fragment, we do not iterate over it in ice_put_rx_mbuf(), and thus we don't
call ice_put_rx_buf().
Because we don't call ice_put_rx_buf(), we don't attempt to re-use the
page or free it. This leaves a stale page in the ring, as we don't
increment next_to_alloc.
The ice_reuse_rx_page() assumes that the next_to_alloc has been incremented
properly, and that it always points to a buffer with a NULL page. Since
this function doesn't check, it will happily recycle a page over the top
of the next_to_alloc buffer, losing track of the old page.
Note that this leak only occurs for multi-buffer frames. The
ice_put_rx_mbuf() function always handles at least one buffer, so a
single-buffer frame will always get handled correctly. It is not clear
precisely why the hardware hands us descriptors with a size of 0 sometimes,
but it happens somewhat regularly with "jumbo frames" used by 9K MTU.
To fix ice_put_rx_mbuf(), we need to make sure to call ice_put_rx_buf() on
all buffers between first_desc and next_to_clean. Borrow the logic of a
similar function in i40e used for this same purpose. Use the same logic
also in ice_get_pgcnts().
Instead of iterating over just the number of fragments, use a loop which
iterates until the current index reaches to the next_to_clean element just
past the current frame. Unlike i40e, the ice_put_rx_mbuf() function does
call ice_put_rx_buf() on the last buffer of the frame indicating the end of
packet.
For non-linear (multi-buffer) frames, we need to take care when adjusting
the pagecnt_bias. An XDP program might release fragments from the tail of
the frame, in which case that fragment page is already released. Only
update the pagecnt_bias for the first descriptor and fragments still
remaining post-XDP program. Take care to only access the shared info for
fragmented buffers, as this avoids a significant cache miss.
The xdp_xmit value only needs to be updated if an XDP program is run, and
only once per packet. Drop the xdp_xmit pointer argument from
ice_put_rx_mbuf(). Instead, set xdp_xmit in the ice_clean_rx_irq() function
directly. This avoids needing to pass the argument and avoids an extra
bit-wise OR for each buffer in the frame.
Move the increment of the ntc local variable to ensure its updated *before*
all calls to ice_get_pgcnts() or ice_put_rx_mbuf(), as the loop logic
requires the index of the element just after the current frame.
Now that we use an index pointer in the ring to identify the packet, we no
longer need to track or cache the number of fragments in the rx_ring.
Cc: Christoph Petrausch <christoph.petrausch@deepl.com>
Cc: Jesper Dangaard Brouer <hawk@kernel.org>
Reported-by: Jaroslav Pulchart <jaroslav.pulchart@gooddata.com>
Closes: https://lore.kernel.org/netdev/CAK8fFZ4hY6GUJNENz3wY9jaYLZXGfpr7dnZxzGMYoE44caRbgw@mail.gmail.com/
Fixes:
|
||
|---|---|---|
| .. | ||
| devlink | ||
| Makefile | ||
| ice.h | ||
| ice_adapter.c | ||
| ice_adapter.h | ||
| ice_adminq_cmd.h | ||
| ice_arfs.c | ||
| ice_arfs.h | ||
| ice_base.c | ||
| ice_base.h | ||
| ice_common.c | ||
| ice_common.h | ||
| ice_controlq.c | ||
| ice_controlq.h | ||
| ice_dcb.c | ||
| ice_dcb.h | ||
| ice_dcb_lib.c | ||
| ice_dcb_lib.h | ||
| ice_dcb_nl.c | ||
| ice_dcb_nl.h | ||
| ice_ddp.c | ||
| ice_ddp.h | ||
| ice_debugfs.c | ||
| ice_devids.h | ||
| ice_dpll.c | ||
| ice_dpll.h | ||
| ice_eswitch.c | ||
| ice_eswitch.h | ||
| ice_eswitch_br.c | ||
| ice_eswitch_br.h | ||
| ice_ethtool.c | ||
| ice_ethtool.h | ||
| ice_ethtool_fdir.c | ||
| ice_fdir.c | ||
| ice_fdir.h | ||
| ice_flex_pipe.c | ||
| ice_flex_pipe.h | ||
| ice_flex_type.h | ||
| ice_flow.c | ||
| ice_flow.h | ||
| ice_fltr.c | ||
| ice_fltr.h | ||
| ice_fw_update.c | ||
| ice_fw_update.h | ||
| ice_fwlog.c | ||
| ice_fwlog.h | ||
| ice_gnss.c | ||
| ice_gnss.h | ||
| ice_hw_autogen.h | ||
| ice_hwmon.c | ||
| ice_hwmon.h | ||
| ice_idc.c | ||
| ice_idc_int.h | ||
| ice_irq.c | ||
| ice_irq.h | ||
| ice_lag.c | ||
| ice_lag.h | ||
| ice_lan_tx_rx.h | ||
| ice_lib.c | ||
| ice_lib.h | ||
| ice_main.c | ||
| ice_nvm.c | ||
| ice_nvm.h | ||
| ice_osdep.h | ||
| ice_parser.c | ||
| ice_parser.h | ||
| ice_parser_rt.c | ||
| ice_pf_vsi_vlan_ops.c | ||
| ice_pf_vsi_vlan_ops.h | ||
| ice_protocol_type.h | ||
| ice_ptp.c | ||
| ice_ptp.h | ||
| ice_ptp_consts.h | ||
| ice_ptp_hw.c | ||
| ice_ptp_hw.h | ||
| ice_repr.c | ||
| ice_repr.h | ||
| ice_sbq_cmd.h | ||
| ice_sched.c | ||
| ice_sched.h | ||
| ice_sf_eth.c | ||
| ice_sf_eth.h | ||
| ice_sf_vsi_vlan_ops.c | ||
| ice_sf_vsi_vlan_ops.h | ||
| ice_sriov.c | ||
| ice_sriov.h | ||
| ice_switch.c | ||
| ice_switch.h | ||
| ice_tc_lib.c | ||
| ice_tc_lib.h | ||
| ice_trace.h | ||
| ice_tspll.c | ||
| ice_tspll.h | ||
| ice_txrx.c | ||
| ice_txrx.h | ||
| ice_txrx_lib.c | ||
| ice_txrx_lib.h | ||
| ice_type.h | ||
| ice_vf_lib.c | ||
| ice_vf_lib.h | ||
| ice_vf_lib_private.h | ||
| ice_vf_mbx.c | ||
| ice_vf_mbx.h | ||
| ice_vf_vsi_vlan_ops.c | ||
| ice_vf_vsi_vlan_ops.h | ||
| ice_virtchnl.c | ||
| ice_virtchnl.h | ||
| ice_virtchnl_allowlist.c | ||
| ice_virtchnl_allowlist.h | ||
| ice_virtchnl_fdir.c | ||
| ice_virtchnl_fdir.h | ||
| ice_vlan.h | ||
| ice_vlan_mode.c | ||
| ice_vlan_mode.h | ||
| ice_vsi_vlan_lib.c | ||
| ice_vsi_vlan_lib.h | ||
| ice_vsi_vlan_ops.c | ||
| ice_vsi_vlan_ops.h | ||
| ice_xsk.c | ||
| ice_xsk.h | ||