linux

History

Alexander Lobakin 0315a075f1 net: fix premature exit from NAPI state polling in napi_disable() Commit `719c571970` ("net: make napi_disable() symmetric with enable") accidentally introduced a bug sometimes leading to a kernel BUG when bringing an iface up/down under heavy traffic load. Prior to this commit, napi_disable() was polling n->state until none of (NAPIF_STATE_SCHED \| NAPIF_STATE_NPSVC) is set and then always flip them. Now there's a possibility to get away with the NAPIF_STATE_SCHE unset as 'continue' drops us to the cmpxchg() call with an uninitialized variable, rather than straight to another round of the state check. Error path looks like: napi_disable(): unsigned long val, new; /* new is uninitialized / do { val = READ_ONCE(n->state); / NAPIF_STATE_NPSVC and/or NAPIF_STATE_SCHED is set / if (val & (NAPIF_STATE_SCHED \| NAPIF_STATE_NPSVC)) { / true / usleep_range(20, 200); continue; / go straight to the condition check / } new = val \| <...> } while (cmpxchg(&n->state, val, new) != val); / state == val, cmpxchg() writes garbage / napi_enable(): do { val = READ_ONCE(n->state); BUG_ON(!test_bit(NAPI_STATE_SCHED, &val)); / 50/50 boom */ <...> while the typical BUG splat is like: [ 172.652461] ------------[ cut here ]------------ [ 172.652462] kernel BUG at net/core/dev.c:6937! [ 172.656914] invalid opcode: 0000 [#1] PREEMPT SMP PTI [ 172.661966] CPU: 36 PID: 2829 Comm: xdp_redirect_cp Tainted: G I 5.15.0 #42 [ 172.670222] Hardware name: Intel Corporation S2600WFT/S2600WFT, BIOS SE5C620.86B.02.01.0014.082620210524 08/26/2021 [ 172.680646] RIP: 0010:napi_enable+0x5a/0xd0 [ 172.684832] Code: 07 49 81 cc 00 01 00 00 4c 89 e2 48 89 d8 80 e6 fb f0 48 0f b1 55 10 48 39 c3 74 10 48 8b 5d 10 f6 c7 04 75 3d f6 c3 01 75 b4 <0f> 0b 5b 5d 41 5c c3 65 ff 05 b8 e5 61 53 48 c7 c6 c0 f3 34 ad 48 [ 172.703578] RSP: 0018:ffffa3c9497477a8 EFLAGS: 00010246 [ 172.708803] RAX: ffffa3c96615a014 RBX: 0000000000000000 RCX: ffff8a4b575301a0 < snip > [ 172.782403] Call Trace: [ 172.784857] <TASK> [ 172.786963] ice_up_complete+0x6f/0x210 [ice] [ 172.791349] ice_xdp+0x136/0x320 [ice] [ 172.795108] ? ice_change_mtu+0x180/0x180 [ice] [ 172.799648] dev_xdp_install+0x61/0xe0 [ 172.803401] dev_xdp_attach+0x1e0/0x550 [ 172.807240] dev_change_xdp_fd+0x1e6/0x220 [ 172.811338] do_setlink+0xee8/0x1010 [ 172.814917] rtnl_setlink+0xe5/0x170 [ 172.818499] ? bpf_lsm_binder_set_context_mgr+0x10/0x10 [ 172.823732] ? security_capable+0x36/0x50 < snip > Fix this by replacing 'do { } while (cmpxchg())' with an "infinite" for-loop with an explicit break. From v1 [0]: - just use a for-loop to simplify both the fix and the existing code (Eric). [0] https://lore.kernel.org/netdev/20211110191126.1214-1-alexandr.lobakin@intel.com Fixes: `719c571970` ("net: make napi_disable() symmetric with enable") Suggested-by: Eric Dumazet <edumazet@google.com> # for-loop Signed-off-by: Alexander Lobakin <alexandr.lobakin@intel.com> Reviewed-by: Jesse Brandeburg <jesse.brandeburg@intel.com> Reviewed-by: Eric Dumazet <edumazet@google.com> Link: https://lore.kernel.org/r/20211110195605.1304-1-alexandr.lobakin@intel.com Signed-off-by: Jakub Kicinski <kuba@kernel.org>		2021-11-10 17:45:15 -08:00
..
Makefile	…
bpf_sk_storage.c	…
datagram.c	net: avoid double accounting for pure zerocopy skbs	2021-11-03 11:19:49 +00:00
datagram.h	…
dev.c	net: fix premature exit from NAPI state polling in napi_disable()	2021-11-10 17:45:15 -08:00
dev_addr_lists.c	…
dev_ioctl.c	ethtool: push the rtnl_lock into dev_ethtool()	2021-11-01 13:26:07 +00:00
devlink.c	devlink: fix flexible_array.cocci warning	2021-11-04 16:43:55 -07:00
drop_monitor.c	…
dst.c	…
dst_cache.c	…
failover.c	…
fib_notifier.c	…
fib_rules.c	…
filter.c	bpf, sockmap: sk_skb data_end access incorrect when src_reg = dst_reg	2021-11-09 01:05:34 +01:00
flow_dissector.c	Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net	2021-11-01 20:05:14 -07:00
flow_offload.c	…
gen_estimator.c	…
gen_stats.c	…
gro_cells.c	…
hwbm.c	…
link_watch.c	…
lwt_bpf.c	…
lwtunnel.c	…
neighbour.c	…
net-procfs.c	…
net-sysfs.c	Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net	2021-10-28 10:43:58 -07:00
net-sysfs.h	…
net-traces.c	…
net_namespace.c	…
netclassid_cgroup.c	…
netevent.c	…
netpoll.c	…
netprio_cgroup.c	…
of_net.c	…
page_pool.c	…
pktgen.c	…
ptp_classifier.c	…
request_sock.c	…
rtnetlink.c	net: rtnetlink: use __dev_addr_set()	2021-10-24 13:59:44 +01:00
scm.c	…
secure_seq.c	…
selftests.c	…
skbuff.c	net: avoid double accounting for pure zerocopy skbs	2021-11-03 11:19:49 +00:00
skmsg.c	Merge https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next	2021-11-01 19:59:46 -07:00
sock.c	net: fix possible NULL deref in sock_reserve_memory	2021-11-04 11:28:04 +00:00
sock_destructor.h	…
sock_diag.c	…
sock_map.c	bpf, sockmap: Use stricter sk state checks in sk_lookup_assign	2021-11-09 00:56:35 +01:00
sock_reuseport.c	…
stream.c	…
sysctl_net_core.c	…
timestamping.c	…
tso.c	…
utils.c	…
xdp.c	xdp: Remove redundant warning	2021-10-27 18:13:57 -07:00