mirror of
https://git.proxmox.com/git/mirror_ubuntu-kernels.git
synced 2025-12-30 16:15:56 +00:00
This patch adds support to BPF verifier to allow bpf program calling kernel function directly. The use case included in this set is to allow bpf-tcp-cc to directly call some tcp-cc helper functions (e.g. "tcp_cong_avoid_ai()"). Those functions have already been used by some kernel tcp-cc implementations. This set will also allow the bpf-tcp-cc program to directly call the kernel tcp-cc implementation, For example, a bpf_dctcp may only want to implement its own dctcp_cwnd_event() and reuse other dctcp_*() directly from the kernel tcp_dctcp.c instead of reimplementing (or copy-and-pasting) them. The tcp-cc kernel functions mentioned above will be white listed for the struct_ops bpf-tcp-cc programs to use in a later patch. The white listed functions are not bounded to a fixed ABI contract. Those functions have already been used by the existing kernel tcp-cc. If any of them has changed, both in-tree and out-of-tree kernel tcp-cc implementations have to be changed. The same goes for the struct_ops bpf-tcp-cc programs which have to be adjusted accordingly. This patch is to make the required changes in the bpf verifier. First change is in btf.c, it adds a case in "btf_check_func_arg_match()". When the passed in "btf->kernel_btf == true", it means matching the verifier regs' states with a kernel function. This will handle the PTR_TO_BTF_ID reg. It also maps PTR_TO_SOCK_COMMON, PTR_TO_SOCKET, and PTR_TO_TCP_SOCK to its kernel's btf_id. In the later libbpf patch, the insn calling a kernel function will look like: insn->code == (BPF_JMP | BPF_CALL) insn->src_reg == BPF_PSEUDO_KFUNC_CALL /* <- new in this patch */ insn->imm == func_btf_id /* btf_id of the running kernel */ [ For the future calling function-in-kernel-module support, an array of module btf_fds can be passed at the load time and insn->off can be used to index into this array. ] At the early stage of verifier, the verifier will collect all kernel function calls into "struct bpf_kfunc_desc". Those descriptors are stored in "prog->aux->kfunc_tab" and will be available to the JIT. Since this "add" operation is similar to the current "add_subprog()" and looking for the same insn->code, they are done together in the new "add_subprog_and_kfunc()". In the "do_check()" stage, the new "check_kfunc_call()" is added to verify the kernel function call instruction: 1. Ensure the kernel function can be used by a particular BPF_PROG_TYPE. A new bpf_verifier_ops "check_kfunc_call" is added to do that. The bpf-tcp-cc struct_ops program will implement this function in a later patch. 2. Call "btf_check_kfunc_args_match()" to ensure the regs can be used as the args of a kernel function. 3. Mark the regs' type, subreg_def, and zext_dst. At the later do_misc_fixups() stage, the new fixup_kfunc_call() will replace the insn->imm with the function address (relative to __bpf_call_base). If needed, the jit can find the btf_func_model by calling the new bpf_jit_find_kfunc_model(prog, insn). With the imm set to the function address, "bpftool prog dump xlated" will be able to display the kernel function calls the same way as it displays other bpf helper calls. gpl_compatible program is required to call kernel function. This feature currently requires JIT. The verifier selftests are adjusted because of the changes in the verbose log in add_subprog_and_kfunc(). Signed-off-by: Martin KaFai Lau <kafai@fb.com> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Link: https://lore.kernel.org/bpf/20210325015142.1544736-1-kafai@fb.com |
||
|---|---|---|
| .. | ||
| benchs | ||
| bpf_testmod | ||
| gnu | ||
| map_tests | ||
| prog_tests | ||
| progs | ||
| verifier | ||
| .gitignore | ||
| bench.c | ||
| bench.h | ||
| bpf_legacy.h | ||
| bpf_rand.h | ||
| bpf_rlimit.h | ||
| bpf_sockopt_helpers.h | ||
| bpf_tcp_helpers.h | ||
| bpf_util.h | ||
| btf_helpers.c | ||
| btf_helpers.h | ||
| cgroup_helpers.c | ||
| cgroup_helpers.h | ||
| config | ||
| flow_dissector_load.c | ||
| flow_dissector_load.h | ||
| get_cgroup_id_user.c | ||
| ima_setup.sh | ||
| Makefile | ||
| Makefile.docs | ||
| netcnt_common.h | ||
| network_helpers.c | ||
| network_helpers.h | ||
| README.rst | ||
| settings | ||
| test_bpftool_build.sh | ||
| test_bpftool_metadata.sh | ||
| test_bpftool.py | ||
| test_bpftool.sh | ||
| test_btf.h | ||
| test_cgroup_storage.c | ||
| test_cpp.cpp | ||
| test_dev_cgroup.c | ||
| test_doc_build.sh | ||
| test_flow_dissector.c | ||
| test_flow_dissector.sh | ||
| test_ftrace.sh | ||
| test_iptunnel_common.h | ||
| test_kmod.sh | ||
| test_lirc_mode2_user.c | ||
| test_lirc_mode2.sh | ||
| test_lpm_map.c | ||
| test_lru_map.c | ||
| test_lwt_ip_encap.sh | ||
| test_lwt_seg6local.sh | ||
| test_maps.c | ||
| test_maps.h | ||
| test_netcnt.c | ||
| test_offload.py | ||
| test_progs.c | ||
| test_progs.h | ||
| test_select_reuseport_common.h | ||
| test_skb_cgroup_id_user.c | ||
| test_skb_cgroup_id.sh | ||
| test_sock_addr.c | ||
| test_sock_addr.sh | ||
| test_sock.c | ||
| test_sockmap.c | ||
| test_stub.c | ||
| test_sysctl.c | ||
| test_tag.c | ||
| test_tc_edt.sh | ||
| test_tc_redirect.sh | ||
| test_tc_tunnel.sh | ||
| test_tcp_check_syncookie_user.c | ||
| test_tcp_check_syncookie.sh | ||
| test_tcp_hdr_options.h | ||
| test_tcpbpf.h | ||
| test_tcpnotify_user.c | ||
| test_tcpnotify.h | ||
| test_tunnel.sh | ||
| test_verifier_log.c | ||
| test_verifier.c | ||
| test_xdp_meta.sh | ||
| test_xdp_redirect.sh | ||
| test_xdp_veth.sh | ||
| test_xdp_vlan_mode_generic.sh | ||
| test_xdp_vlan_mode_native.sh | ||
| test_xdp_vlan.sh | ||
| test_xdping.sh | ||
| test_xsk.sh | ||
| testing_helpers.c | ||
| testing_helpers.h | ||
| trace_helpers.c | ||
| trace_helpers.h | ||
| urandom_read.c | ||
| vmtest.sh | ||
| with_addr.sh | ||
| with_tunnels.sh | ||
| xdping.c | ||
| xdping.h | ||
| xdpxceiver.c | ||
| xdpxceiver.h | ||
| xsk_prereqs.sh | ||
==================
BPF Selftest Notes
==================
General instructions on running selftests can be found in
`Documentation/bpf/bpf_devel_QA.rst`__.
__ /Documentation/bpf/bpf_devel_QA.rst#q-how-to-run-bpf-selftests
=========================
Running Selftests in a VM
=========================
It's now possible to run the selftests using ``tools/testing/selftests/bpf/vmtest.sh``.
The script tries to ensure that the tests are run with the same environment as they
would be run post-submit in the CI used by the Maintainers.
This script downloads a suitable Kconfig and VM userspace image from the system used by
the CI. It builds the kernel (without overwriting your existing Kconfig), recompiles the
bpf selftests, runs them (by default ``tools/testing/selftests/bpf/test_progs``) and
saves the resulting output (by default in ``~/.bpf_selftests``).
For more information on about using the script, run:
.. code-block:: console
$ tools/testing/selftests/bpf/vmtest.sh -h
.. note:: The script uses pahole and clang based on host environment setting.
If you want to change pahole and llvm, you can change `PATH` environment
variable in the beginning of script.
.. note:: The script currently only supports x86_64.
Additional information about selftest failures are
documented here.
profiler[23] test failures with clang/llvm <12.0.0
==================================================
With clang/llvm <12.0.0, the profiler[23] test may fail.
The symptom looks like
.. code-block:: c
// r9 is a pointer to map_value
// r7 is a scalar
17: bf 96 00 00 00 00 00 00 r6 = r9
18: 0f 76 00 00 00 00 00 00 r6 += r7
math between map_value pointer and register with unbounded min value is not allowed
// the instructions below will not be seen in the verifier log
19: a5 07 01 00 01 01 00 00 if r7 < 257 goto +1
20: bf 96 00 00 00 00 00 00 r6 = r9
// r6 is used here
The verifier will reject such code with above error.
At insn 18 the r7 is indeed unbounded. The later insn 19 checks the bounds and
the insn 20 undoes map_value addition. It is currently impossible for the
verifier to understand such speculative pointer arithmetic.
Hence `this patch`__ addresses it on the compiler side. It was committed on llvm 12.
__ https://reviews.llvm.org/D85570
The corresponding C code
.. code-block:: c
for (int i = 0; i < MAX_CGROUPS_PATH_DEPTH; i++) {
filepart_length = bpf_probe_read_str(payload, ...);
if (filepart_length <= MAX_PATH) {
barrier_var(filepart_length); // workaround
payload += filepart_length;
}
}
bpf_iter test failures with clang/llvm 10.0.0
=============================================
With clang/llvm 10.0.0, the following two bpf_iter tests failed:
* ``bpf_iter/ipv6_route``
* ``bpf_iter/netlink``
The symptom for ``bpf_iter/ipv6_route`` looks like
.. code-block:: c
2: (79) r8 = *(u64 *)(r1 +8)
...
14: (bf) r2 = r8
15: (0f) r2 += r1
; BPF_SEQ_PRINTF(seq, "%pi6 %02x ", &rt->fib6_dst.addr, rt->fib6_dst.plen);
16: (7b) *(u64 *)(r8 +64) = r2
only read is supported
The symptom for ``bpf_iter/netlink`` looks like
.. code-block:: c
; struct netlink_sock *nlk = ctx->sk;
2: (79) r7 = *(u64 *)(r1 +8)
...
15: (bf) r2 = r7
16: (0f) r2 += r1
; BPF_SEQ_PRINTF(seq, "%pK %-3d ", s, s->sk_protocol);
17: (7b) *(u64 *)(r7 +0) = r2
only read is supported
This is due to a llvm BPF backend bug. `The fix`__
has been pushed to llvm 10.x release branch and will be
available in 10.0.1. The patch is available in llvm 11.0.0 trunk.
__ https://reviews.llvm.org/D78466
bpf_verif_scale/loop6.o test failure with Clang 12
==================================================
With Clang 12, the following bpf_verif_scale test failed:
* ``bpf_verif_scale/loop6.o``
The verifier output looks like
.. code-block:: c
R1 type=ctx expected=fp
The sequence of 8193 jumps is too complex.
The reason is compiler generating the following code
.. code-block:: c
; for (i = 0; (i < VIRTIO_MAX_SGS) && (i < num); i++) {
14: 16 05 40 00 00 00 00 00 if w5 == 0 goto +64 <LBB0_6>
15: bc 51 00 00 00 00 00 00 w1 = w5
16: 04 01 00 00 ff ff ff ff w1 += -1
17: 67 05 00 00 20 00 00 00 r5 <<= 32
18: 77 05 00 00 20 00 00 00 r5 >>= 32
19: a6 01 01 00 05 00 00 00 if w1 < 5 goto +1 <LBB0_4>
20: b7 05 00 00 06 00 00 00 r5 = 6
00000000000000a8 <LBB0_4>:
21: b7 02 00 00 00 00 00 00 r2 = 0
22: b7 01 00 00 00 00 00 00 r1 = 0
; for (i = 0; (i < VIRTIO_MAX_SGS) && (i < num); i++) {
23: 7b 1a e0 ff 00 00 00 00 *(u64 *)(r10 - 32) = r1
24: 7b 5a c0 ff 00 00 00 00 *(u64 *)(r10 - 64) = r5
Note that insn #15 has w1 = w5 and w1 is refined later but
r5(w5) is eventually saved on stack at insn #24 for later use.
This cause later verifier failure. The bug has been `fixed`__ in
Clang 13.
__ https://reviews.llvm.org/D97479
BPF CO-RE-based tests and Clang version
=======================================
A set of selftests use BPF target-specific built-ins, which might require
bleeding-edge Clang versions (Clang 12 nightly at this time).
Few sub-tests of core_reloc test suit (part of test_progs test runner) require
the following built-ins, listed with corresponding Clang diffs introducing
them to Clang/LLVM. These sub-tests are going to be skipped if Clang is too
old to support them, they shouldn't cause build failures or runtime test
failures:
- __builtin_btf_type_id() [0_, 1_, 2_];
- __builtin_preserve_type_info(), __builtin_preserve_enum_value() [3_, 4_].
.. _0: https://reviews.llvm.org/D74572
.. _1: https://reviews.llvm.org/D74668
.. _2: https://reviews.llvm.org/D85174
.. _3: https://reviews.llvm.org/D83878
.. _4: https://reviews.llvm.org/D83242
Floating-point tests and Clang version
======================================
Certain selftests, e.g. core_reloc, require support for the floating-point
types, which was introduced in `Clang 13`__. The older Clang versions will
either crash when compiling these tests, or generate an incorrect BTF.
__ https://reviews.llvm.org/D83289