| commit | c86df29d11dfba27c0a1f5039cd6fe387fbf4239 | [log] [tgz] |
|---|---|---|
| author | Peter Zijlstra <[email protected]> | Thu Nov 03 13:00:14 2022 +0100 |
| committer | Daniel Borkmann <[email protected]> | Fri Nov 04 23:13:57 2022 +0100 |
| tree | 3e3bde30f24960b1459df5114c9214336af0f71a | |
| parent | 18acb7fac22ff7b36c7ea5a76b12996e7b7dbaba [diff] |
bpf: Convert BPF_DISPATCHER to use static_call() (not ftrace) The dispatcher function is currently abusing the ftrace __fentry__ call location for its own purposes -- this obviously gives trouble when the dispatcher and ftrace are both in use. A previous solution tried using __attribute__((patchable_function_entry())) which works, except it is GCC-8+ only, breaking the build on the earlier still supported compilers. Instead use static_call() -- which has its own annotations and does not conflict with ftrace -- to rewrite the dispatch function. By using: return static_call()(ctx, insni, bpf_func) you get a perfect forwarding tail call as function body (iow a single jmp instruction). By having the default static_call() target be bpf_dispatcher_nop_func() it retains the default behaviour (an indirect call to the argument function). Only once a dispatcher program is attached is the target rewritten to directly call the JIT'ed image. Signed-off-by: Peter Zijlstra (Intel) <[email protected]> Signed-off-by: Daniel Borkmann <[email protected]> Tested-by: Björn Töpel <[email protected]> Tested-by: Jiri Olsa <[email protected]> Acked-by: Björn Töpel <[email protected]> Acked-by: Jiri Olsa <[email protected]> Link: https://lkml.kernel.org/r/Y1/oBlK0yFk5c/[email protected] Link: https://lore.kernel.org/bpf/[email protected]