libbpf: auto-resize relocatable LOAD/STORE instructions #153

kernel-patches-bot · 2020-10-02T01:15:03Z

Pull request for series with
subject: libbpf: auto-resize relocatable LOAD/STORE instructions
version: 1
url: https://patchwork.kernel.org/project/bpf/list/?series=358607

kernel-patches-bot · 2020-10-02T01:15:04Z

Master branch: 6208689
series: https://patchwork.kernel.org/project/bpf/list/?series=358607
version: 1

kernel-patches-bot · 2020-10-02T02:36:14Z

Master branch: 360f898
series: https://patchwork.kernel.org/project/bpf/list/?series=358607
version: 1

kernel-patches-bot · 2020-10-02T14:54:17Z

Master branch: 48ca624
series: https://patchwork.kernel.org/project/bpf/list/?series=358607
version: 1

kernel-patches-bot · 2020-10-02T18:46:36Z

Master branch: 440c575
series: https://patchwork.kernel.org/project/bpf/list/?series=358607
version: 1

kernel-patches-bot · 2020-10-02T22:22:18Z

Master branch: 60a128b
series: https://patchwork.kernel.org/project/bpf/list/?series=358607
version: 1

kernel-patches-bot · 2020-10-02T22:30:03Z

Master branch: fb91db0
series: https://patchwork.kernel.org/project/bpf/list/?series=358607
version: 1

kernel-patches-bot · 2020-10-03T02:25:11Z

Master branch: 1028ae4
series: https://patchwork.kernel.org/project/bpf/list/?series=358607
version: 1

kernel-patches-bot · 2020-10-05T13:55:03Z

Master branch: b75597d
series: https://patchwork.kernel.org/project/bpf/list/?series=358607
version: 1

kernel-patches-bot · 2020-10-05T14:03:45Z

Master branch: 8731745
series: https://patchwork.kernel.org/project/bpf/list/?series=358607
version: 1

kernel-patches-bot · 2020-10-05T22:11:11Z

Master branch: 39d8f0d
series: https://patchwork.kernel.org/project/bpf/list/?series=358607
version: 1

kernel-patches-bot · 2020-10-05T22:50:27Z

Master branch: dca4121
series: https://patchwork.kernel.org/project/bpf/list/?series=358607
version: 1

…O-RE Add support for patching instructions of the following form: - rX = *(T *)(rY + <off>); - *(T *)(rX + <off>) = rY; - *(T *)(rX + <off>) = <imm>, where T is one of {u8, u16, u32, u64}. For such instructions, if the actual kernel field recorded in CO-RE relocation has a different size than the one recorded locally (e.g., from vmlinux.h), then libbpf will adjust T to an appropriate 1-, 2-, 4-, or 8-byte loads. In general, such transformation is not always correct and could lead to invalid final value being loaded or stored. But two classes of cases are always safe: - if both local and target (kernel) types are unsigned integers, but of different sizes, then it's OK to adjust load/store instruction according to the necessary memory size. Zero-extending nature of such instructions and unsignedness make sure that the final value is always correct; - pointer size mismatch between BPF target architecture (which is always 64-bit) and 32-bit host kernel architecture can be similarly resolved automatically, because pointer is essentially an unsigned integer. Loading 32-bit pointer into 64-bit BPF register with zero extension will leave correct pointer in the register. Both cases are necessary to support CO-RE on 32-bit kernels, as `unsigned long` in vmlinux.h generated from 32-bit kernel is 32-bit, but when compiled with BPF program for BPF target it will be treated by compiler as 64-bit integer. Similarly, pointers in vmlinux.h are 32-bit for kernel, but treated as 64-bit values by compiler for BPF target. Both problems are now resolved by libbpf for direct memory reads. But similar transformations are useful in general when kernel fields are "resized" from, e.g., unsigned int to unsigned long (or vice versa). Now, similar transformations for signed integers are not safe to perform as they will result in incorrect sign extension of the value. If such situation is detected, libbpf will emit helpful message and will poison the instruction. Not failing immediately means that it's possible to guard the instruction based on kernel version (or other conditions) and make sure it's not reachable. If there is a need to read signed integers that change sizes between different kernels, it's possible to use BPF_CORE_READ_BITFIELD() macro, which works both with bitfields and non-bitfield integers of any signedness and handles sign-extension properly. Also, bpf_core_read() with proper size and/or use of bpf_core_field_size() relocation could allow to deal with such complicated situations explicitly, if not so conventiently as direct memory reads. Selftests added in a separate patch in progs/test_core_autosize.c demonstrate both direct memory and probed use cases. BPF_CORE_READ() is not changed and it won't deal with such situations as automatically as direct memory reads due to the signedness integer limitations, which are much harder to detect and control with compiler macro magic. So it's encouraged to utilize direct memory reads as much as possible. Signed-off-by: Andrii Nakryiko <[email protected]>

Use generalized BTF parsing logic, making it possible to parse BTF both from ELF file, as well as a raw BTF dump. This makes it easier to write custom tests with manually generated BTFs. Signed-off-by: Andrii Nakryiko <[email protected]>

Add selftests validating libbpf's auto-resizing of load/store instructions when used with CO-RE relocations. An explicit and manual approach with using bpf_core_read() is also demonstrated and tested. Separate BPF program is supposed to fail due to using signed integers of sizes that differ from kernel's sizes. To reliably simulate 32-bit BTF (i.e., the one with sizeof(long) == sizeof(void *) == 4), selftest generates its own custom BTF and passes it as a replacement for real kernel BTF. This allows to test 32/64-bitness mix on all architectures. Signed-off-by: Andrii Nakryiko <[email protected]>

kernel-patches-bot · 2020-10-05T22:58:11Z

Master branch: dca4121
series: https://patchwork.kernel.org/project/bpf/list/?series=358607
version: 1

kernel-patches-bot · 2020-10-06T18:15:31Z

At least one diff in series https://patchwork.kernel.org/project/bpf/list/?series=358607 irrelevant now. Closing PR.

Add selftests for prog_kfunc feature probing. ./test_progs -t libbpf_probe_kfuncs #153 libbpf_probe_kfuncs:OK Summary: 1/0 PASSED, 0 SKIPPED, 0 FAILED Signed-off-by: Tao Chen <[email protected]>

Add selftests for prog_kfunc feature probing. ./test_progs -t libbpf_probe_kfuncs #153 libbpf_probe_kfuncs:OK Summary: 1/0 PASSED, 0 SKIPPED, 0 FAILED Cc: Tao Chen <[email protected]> Reviewed-by: Jiri Olsa <[email protected]> Signed-off-by: Tao Chen <[email protected]>

Add selftests for prog_kfunc feature probing. Thanks for Eduard providing the libbpf_probe_func_many test case. ./test_progs -t libbpf_probe_kfuncs #153 libbpf_probe_kfuncs:OK Summary: 1/0 PASSED, 0 SKIPPED, 0 FAILED ./test_progs -t libbpf_probe_kfuncs_many #154 libbpf_probe_kfuncs_many:OK Summary: 1/0 PASSED, 0 SKIPPED, 0 FAILED Cc: Tao Chen <[email protected]> Reviewed-by: Jiri Olsa <[email protected]> Co-developed-by: Eduard Zingerman <[email protected]> Signed-off-by: Eduard Zingerman <[email protected]> Signed-off-by: Tao Chen <[email protected]>

Add selftests for prog_kfunc feature probing. ./test_progs -t libbpf_probe_kfuncs #153 libbpf_probe_kfuncs:OK Summary: 1/0 PASSED, 0 SKIPPED, 0 FAILED Cc: Tao Chen <[email protected]> Reviewed-by: Jiri Olsa <[email protected]> Reviewed-by: Eduard Zingerman <[email protected]> Signed-off-by: Tao Chen <[email protected]>

kernel-patches-bot added bpf-next new V1 labels Oct 2, 2020

kernel-patches-bot force-pushed the series/358607=>bpf-next branch from 2853060 to 68cf33d Compare October 2, 2020 02:36

kernel-patches-bot force-pushed the series/358607=>bpf-next branch from 68cf33d to eef5844 Compare October 2, 2020 14:54

kernel-patches-bot force-pushed the series/358607=>bpf-next branch from eef5844 to 80b523f Compare October 2, 2020 18:46

kernel-patches-bot force-pushed the series/358607=>bpf-next branch from 80b523f to c0beecd Compare October 2, 2020 22:22

kernel-patches-bot force-pushed the series/358607=>bpf-next branch from c0beecd to 9e275f7 Compare October 2, 2020 22:30

kernel-patches-bot force-pushed the series/358607=>bpf-next branch from 9e275f7 to c234e66 Compare October 3, 2020 02:25

kernel-patches-bot force-pushed the series/358607=>bpf-next branch from c234e66 to 56f8bf7 Compare October 5, 2020 13:55

kernel-patches-bot force-pushed the series/358607=>bpf-next branch from 56f8bf7 to 6863120 Compare October 5, 2020 14:03

kernel-patches-bot force-pushed the series/358607=>bpf-next branch from 6863120 to 09ad3af Compare October 5, 2020 22:11

kernel-patches-bot force-pushed the series/358607=>bpf-next branch from 09ad3af to 1f016ee Compare October 5, 2020 22:50

kernel-patches-bot and others added 4 commits October 5, 2020 15:58

adding ci files

85dc717

kernel-patches-bot force-pushed the series/358607=>bpf-next branch from 1f016ee to 964cde9 Compare October 5, 2020 22:58

kernel-patches-bot added changes-requested and removed new labels Oct 6, 2020

kernel-patches-bot closed this Oct 6, 2020

kernel-patches-bot deleted the series/358607=>bpf-next branch October 7, 2020 01:45

libbpf: auto-resize relocatable LOAD/STORE instructions #153

libbpf: auto-resize relocatable LOAD/STORE instructions #153

Uh oh!

Conversation

kernel-patches-bot commented Oct 2, 2020

Uh oh!

kernel-patches-bot commented Oct 2, 2020

Uh oh!

kernel-patches-bot commented Oct 2, 2020

Uh oh!

kernel-patches-bot commented Oct 2, 2020

Uh oh!

kernel-patches-bot commented Oct 2, 2020

Uh oh!

kernel-patches-bot commented Oct 2, 2020

Uh oh!

kernel-patches-bot commented Oct 2, 2020

Uh oh!

kernel-patches-bot commented Oct 3, 2020

Uh oh!

kernel-patches-bot commented Oct 5, 2020

Uh oh!

kernel-patches-bot commented Oct 5, 2020

Uh oh!

kernel-patches-bot commented Oct 5, 2020

Uh oh!

kernel-patches-bot commented Oct 5, 2020

Uh oh!

kernel-patches-bot commented Oct 5, 2020

Uh oh!

kernel-patches-bot commented Oct 6, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants