What is BTF (BPF Type Format)?

Internals of eBPF Programs - BTF

While BTF (BPF Type Format) might seem like just another acronym, it's one of the most crucial concepts to understand when developing production-grade eBPF applications. Among other things, BTF is what makes the eBPF CO-RE (Compile-Once, Run-Everywhere) possible.

But how exactly does BTF work, and in what ways does it facilitate CO-RE?

What can you do if your kernel doesn’t include BTF information?

And how do BTF-enabled tracepoints come into play?

Let’s explore these questions further…

---

The Problem

Most eBPF programs lack portability across different kernel versions. For example, a program compiled on kernel version 5.3 might fail to run on version 5.4.

This is because each kernel version comes with its own set of kernel headers, which define structs and memory layouts. Changes to these headers, even minor ones, can break eBPF programs.

Take structs, for instance. Imagine we have a structure representing a TCP header in kernel version 5.3:

In the next kernel release, 5.4, kernel developers might decide to place these fields into a new struct or rename the seq field to seque or perhaps move these fields up or down (changing their offset):

See the problem?

Your code may rely on specific fields or offsets, which are likely to change across kernel versions. Since the eBPF program itself has no control over these changes, there’s an inherent need for a solution to ensure the portability of eBPF programs.

The Solution

BPF Compile Once - Run Everywhere (CO-RE) concept allow us to alleviate this by creating portable, relocatable eBPF bytecode, eliminating the need to write or maintain multiple versions of the same program. You also no longer need to recompile for each specific distribution or kernel version. So, a single program can execute on multiple kernel versions.

The key components of CO-RE that make this possible:

• BPF Type Format (BTF): BTF is the format to describe the type information (data types, function information, ..) of kernel and eBPF programs

• Compiler (Clang): Generates CO-RE relocation information of the eBPF program based on BTF information. For example, if we’d access the seq field in the tcphdr struct, Clang records its field name, its type, and the struct it belongs to. Even if the target kernel has a modified version of the tcphdr struct, the field can still be located by BPF Loader using the CO-RE relocation information. This is also referred to as field offset relocation.

• BPF Loader (e.g. libbpf): Uses the eBPF program BTF and CO-RE information and matches them to a BTF information provided by the target kernel. It resolves types, updates offsets, and adjusts relocatable data to ensure the program functions correctly on the target kernel.

As you can see, BTF is one of the key components in making CO-RE possible. Let's take a closer look at its role.

BPF Type Format Use Cases

In simple terms, BTF provides information on how data structures and code are organized in memory. Or if you prefer, a more technical definition:

BTF is a minimalistic, compact format, inspired by Sun’s CTF (Compact C Type Format), which is used for representing kernel debug information since Solaris 9. BTF was created for similar purposes, with a focus on simplicity and compactness to allow its usage in the Linux kernel.

~ Enhancing the Linux Kernel with BTF Type information BY Andrii Nakryiko

Here's an example of what it looks like:

It’s possible, your kernel does NOT ship with BTF information. You can verify this using:

In case it doesn’t, then you have two options:

• Re-compile the kernel with the CONFIG_DEBUG_INFO_BTF=y option or upgrade the kernel, which is time-consuming, inconvenient — especially if the machine is in production.

• Provide the BTF information of that specific kernel alongside our program as well, for example by using projects like BTFhub.

There are several ways to make the eBPF programs portable so I’ll discuss this in a different post—Instead, the key question I want to address in the rest of this newsletter is:

Beside CO-RE, how could understanding BTF be relevant to you?

The first most relatable example I can think of is the bpftool.

bpftool

Nearly every tutorial mentions this command-line tool, which is primarily used for inspecting and managing eBPF programs, maps, and related kernel objects. It provides features like loading, attaching, and debugging eBPF code.

Here's an example:

This command lists the eBPF maps on the host and dumps the contents of the hello.bss map.

But how was bpftool able to pretty-print the variable name counter?

Under the hood, bpftool utilizes BTF information, which includes line and function details, allowing it to interleave source code with the output from translated or JIT-compiled program dumps.

In other words, one of BTF's use cases is mapping raw kernel data to human-readable strings by providing the necessary information to interpret these bytes, making the output much more human-friendly.

Without BTF, you would see something more like this:

BTF-Enabled Programs

Another interesting use case is developing BTF-enabled eBPF programs.

Consider a simple tracepoint example:

When developing eBPF programs, we usually define a struct that replicates the output of the /sys/kernel/tracing/events/sched/sched_process_exec/format file and use it as the context struct in our program.

However, BTF allows us to write tracepoints like this instead:

In the first example, the struct definition is tied to our kernel and may not work with other versions. In contrast, the second example uses the struct from the kernel it is running on, even if it was compiled on a different kernel version.

How is this possible?

In order to write an eBPF CO-RE program, we need a way to tell our eBPF programs what kind of kernel objects it operates on. bpftool allow us to generate an appropriate header file from the BTF information included with the kernel.

The vmlinux.h header file now includes internal kernel types and data structure information of the running kernel that an eBPF program may require, including the trace_event_raw_sched_process_exec required in the example above.

And it’s the following line, which you’ll find near the top of the vmlinux.h file, that causes Clang to embed CO-RE relocation information into the eBPF program ELF (Executable and Linkable Format) file:

And thanks to the BPF loader's ability to match the eBPF program BTF and CO-RE relocation data with the BTF provided by the target kernel, the program can run on a different kernel version than the one it was originally compiled for.

IMPORTANT: You need to pass the -g flag to the compiler to generate BTF information that you’ll need for CO-RE eBPF programs:

clang -O2 -g -target bpf -c my_ebpf_program.c -o my_ebpf_program.o

---

I hope you find this resource as interesting as I did. Stay tuned for more exciting developments and updates in the world of eBPF in next week's newsletter.

Until then, keep 🐝-ing!

Warm regards, Teodor