mirror of https://github.com/torvalds/linux.git
90f9f5d64c
When creating a synthetic event based on an existing synthetic event that
had a stacktrace field and the new synthetic event used that field a
kernel crash occurred:
~# cd /sys/kernel/tracing
~# echo 's:stack unsigned long stack[];' > dynamic_events
~# echo 'hist:keys=prev_pid:s0=common_stacktrace if prev_state & 3' >> events/sched/sched_switch/trigger
~# echo 'hist:keys=next_pid:s1=$s0:onmatch(sched.sched_switch).trace(stack,$s1)' >> events/sched/sched_switch/trigger
The above creates a synthetic event that takes a stacktrace when a task
schedules out in a non-running state and passes that stacktrace to the
sched_switch event when that task schedules back in. It triggers the
"stack" synthetic event that has a stacktrace as its field (called "stack").
~# echo 's:syscall_stack s64 id; unsigned long stack[];' >> dynamic_events
~# echo 'hist:keys=common_pid:s2=stack' >> events/synthetic/stack/trigger
~# echo 'hist:keys=common_pid:s3=$s2,i0=id:onmatch(synthetic.stack).trace(syscall_stack,$i0,$s3)' >> events/raw_syscalls/sys_exit/trigger
The above makes another synthetic event called "syscall_stack" that
attaches the first synthetic event (stack) to the sys_exit trace event and
records the stacktrace from the stack event with the id of the system call
that is exiting.
When enabling this event (or using it in a historgram):
~# echo 1 > events/synthetic/syscall_stack/enable
Produces a kernel crash!
BUG: unable to handle page fault for address: 0000000000400010
#PF: supervisor read access in kernel mode
#PF: error_code(0x0000) - not-present page
PGD 0 P4D 0
Oops: Oops: 0000 [#1] SMP PTI
CPU: 6 UID: 0 PID: 1257 Comm: bash Not tainted 6.16.3+deb14-amd64 #1 PREEMPT(lazy) Debian 6.16.3-1
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.17.0-debian-1.17.0-1 04/01/2014
RIP: 0010:trace_event_raw_event_synth+0x90/0x380
Code: c5 00 00 00 00 85 d2 0f 84 e1 00 00 00 31 db eb 34 0f 1f 00 66 66 2e 0f 1f 84 00 00 00 00 00 66 66 2e 0f 1f 84 00 00 00 00 00 <49> 8b 04 24 48 83 c3 01 8d 0c c5 08 00 00 00 01 cd 41 3b 5d 40 0f
RSP: 0018:ffffd2670388f958 EFLAGS: 00010202
RAX: ffff8ba1065cc100 RBX: 0000000000000000 RCX: 0000000000000000
RDX: 0000000000000001 RSI: fffff266ffda7b90 RDI: ffffd2670388f9b0
RBP: 0000000000000010 R08: ffff8ba104e76000 R09: ffffd2670388fa50
R10: ffff8ba102dd42e0 R11: ffffffff9a908970 R12: 0000000000400010
R13: ffff8ba10a246400 R14: ffff8ba10a710220 R15: fffff266ffda7b90
FS: 00007fa3bc63f740(0000) GS:ffff8ba2e0f48000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000400010 CR3: 0000000107f9e003 CR4: 0000000000172ef0
Call Trace:
<TASK>
? __tracing_map_insert+0x208/0x3a0
action_trace+0x67/0x70
event_hist_trigger+0x633/0x6d0
event_triggers_call+0x82/0x130
trace_event_buffer_commit+0x19d/0x250
trace_event_raw_event_sys_exit+0x62/0xb0
syscall_exit_work+0x9d/0x140
do_syscall_64+0x20a/0x2f0
? trace_event_raw_event_sched_switch+0x12b/0x170
? save_fpregs_to_fpstate+0x3e/0x90
? _raw_spin_unlock+0xe/0x30
? finish_task_switch.isra.0+0x97/0x2c0
? __rseq_handle_notify_resume+0xad/0x4c0
? __schedule+0x4b8/0xd00
? restore_fpregs_from_fpstate+0x3c/0x90
? switch_fpu_return+0x5b/0xe0
? do_syscall_64+0x1ef/0x2f0
? do_fault+0x2e9/0x540
? __handle_mm_fault+0x7d1/0xf70
? count_memcg_events+0x167/0x1d0
? handle_mm_fault+0x1d7/0x2e0
? do_user_addr_fault+0x2c3/0x7f0
entry_SYSCALL_64_after_hwframe+0x76/0x7e
The reason is that the stacktrace field is not labeled as such, and is
treated as a normal field and not as a dynamic event that it is.
In trace_event_raw_event_synth() the event is field is still treated as a
dynamic array, but the retrieval of the data is considered a normal field,
and the reference is just the meta data:
// Meta data is retrieved instead of a dynamic array
str_val = (char *)(long)var_ref_vals[val_idx];
// Then when it tries to process it:
len = *((unsigned long *)str_val) + 1;
It triggers a kernel page fault.
To fix this, first when defining the fields of the first synthetic event,
set the filter type to FILTER_STACKTRACE. This is used later by the second
synthetic event to know that this field is a stacktrace. When creating
the field of the new synthetic event, have it use this FILTER_STACKTRACE
to know to create a stacktrace field to copy the stacktrace into.
Cc: stable@vger.kernel.org
Cc: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Tom Zanussi <zanussi@kernel.org>
Link: https://patch.msgid.link/20260122194824.6905a38e@gandalf.local.home
Fixes:
|
||
|---|---|---|
| .. | ||
| bpf | ||
| cgroup | ||
| configs | ||
| debug | ||
| dma | ||
| entry | ||
| events | ||
| futex | ||
| gcov | ||
| irq | ||
| kcsan | ||
| livepatch | ||
| liveupdate | ||
| locking | ||
| module | ||
| power | ||
| printk | ||
| rcu | ||
| sched | ||
| time | ||
| trace | ||
| unwind | ||
| .gitignore | ||
| Kconfig.freezer | ||
| Kconfig.hz | ||
| Kconfig.kexec | ||
| Kconfig.locks | ||
| Kconfig.preempt | ||
| Makefile | ||
| acct.c | ||
| async.c | ||
| audit.c | ||
| audit.h | ||
| audit_fsnotify.c | ||
| audit_tree.c | ||
| audit_watch.c | ||
| auditfilter.c | ||
| auditsc.c | ||
| backtracetest.c | ||
| bounds.c | ||
| capability.c | ||
| cfi.c | ||
| compat.c | ||
| configs.c | ||
| context_tracking.c | ||
| cpu.c | ||
| cpu_pm.c | ||
| crash_core.c | ||
| crash_core_test.c | ||
| crash_dump_dm_crypt.c | ||
| crash_reserve.c | ||
| cred.c | ||
| delayacct.c | ||
| dma.c | ||
| elfcorehdr.c | ||
| exec_domain.c | ||
| exit.c | ||
| exit.h | ||
| extable.c | ||
| fail_function.c | ||
| fork.c | ||
| freezer.c | ||
| gen_kheaders.sh | ||
| groups.c | ||
| hung_task.c | ||
| iomem.c | ||
| irq_work.c | ||
| jump_label.c | ||
| kallsyms.c | ||
| kallsyms_internal.h | ||
| kallsyms_selftest.c | ||
| kallsyms_selftest.h | ||
| kcmp.c | ||
| kcov.c | ||
| kexec.c | ||
| kexec_core.c | ||
| kexec_elf.c | ||
| kexec_file.c | ||
| kexec_internal.h | ||
| kheaders.c | ||
| kprobes.c | ||
| kstack_erase.c | ||
| ksyms_common.c | ||
| ksysfs.c | ||
| kthread.c | ||
| latencytop.c | ||
| module_signature.c | ||
| notifier.c | ||
| nscommon.c | ||
| nsproxy.c | ||
| nstree.c | ||
| padata.c | ||
| panic.c | ||
| params.c | ||
| pid.c | ||
| pid_namespace.c | ||
| pid_sysctl.h | ||
| profile.c | ||
| ptrace.c | ||
| range.c | ||
| reboot.c | ||
| regset.c | ||
| relay.c | ||
| resource.c | ||
| resource_kunit.c | ||
| rseq.c | ||
| scftorture.c | ||
| scs.c | ||
| seccomp.c | ||
| signal.c | ||
| smp.c | ||
| smpboot.c | ||
| smpboot.h | ||
| softirq.c | ||
| stacktrace.c | ||
| static_call.c | ||
| static_call_inline.c | ||
| stop_machine.c | ||
| sys.c | ||
| sys_ni.c | ||
| sysctl-test.c | ||
| sysctl.c | ||
| task_work.c | ||
| taskstats.c | ||
| torture.c | ||
| tracepoint.c | ||
| tsacct.c | ||
| ucount.c | ||
| uid16.c | ||
| uid16.h | ||
| umh.c | ||
| up.c | ||
| user-return-notifier.c | ||
| user.c | ||
| user_namespace.c | ||
| utsname.c | ||
| utsname_sysctl.c | ||
| vhost_task.c | ||
| vmcore_info.c | ||
| watch_queue.c | ||
| watchdog.c | ||
| watchdog_buddy.c | ||
| watchdog_perf.c | ||
| workqueue.c | ||
| workqueue_internal.h | ||