最近在研究ebpf的應(yīng)用，網(wǎng)上對較低版本的內(nèi)核和centos操作系統(tǒng)的相關(guān)資料較少，這里記錄一個自己環(huán)境配置&編譯運行一個ebpf的helloworld程序的過程。

環(huán)境是centos7.9，虛擬機(jī)安裝內(nèi)存需要分配高一些，后續(xù)編譯llvm很吃性能

基礎(chǔ)依賴安裝

升級內(nèi)核版本

ebpf需要至少內(nèi)核是4.9+以上的版本，這里選擇了4.18版本的內(nèi)核

下載4.18版本內(nèi)核的安裝包，下載鏈接

# 安裝4.18版本的內(nèi)核
yum install -y kernel-ml-4.18.0-1.el7.elrepo.x86_64.rpm
# 修改啟動內(nèi)核順序
yum install -y grub2-pc
grub-set-default 'CentOS Linux (4.18.0-1.el7.elrepo.x86_64) 7 (Core)'

重啟后可以確認(rèn)下是否切換到4.18版本的內(nèi)核

uname -sr
# Linux 4.18.0-1.el7.elrepo.x86_64

安裝gcc

編譯llvm 10+需要高版本gcc，這里使用scl軟件集的方式升級gcc到7.3.1

yum install centos-release-scl
yum install devtoolset-7  -y
scl enable devtoolset-7 bash
echo "source /opt/rh/devtoolset-7/enable" >> ~/.bash_profile
source /opt/rh/devtoolset-7/enable

# gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/opt/rh/devtoolset-7/root/usr/libexec/gcc/x86_64-redhat-linux/7/lto-wrapper
Target: x86_64-redhat-linux
Configured with: ../configure --enable-bootstrap --enable-languages=c,c++,fortran,lto --prefix=/opt/rh/devtoolset-7/root/usr --mandir=/opt/rh/devtoolset-7/root/usr/share/man --infodir=/opt/rh/devtoolset-7/root/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-shared --enable-threads=posix --enable-checking=release --enable-multilib --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --with-gcc-major-version-only --enable-plugin --with-linker-hash-style=gnu --enable-initfini-array --with-default-libstdcxx-abi=gcc4-compatible --with-isl=/builddir/build/BUILD/gcc-7.3.1-20180303/obj-x86_64-redhat-linux/isl-install --enable-libmpx --enable-gnu-indirect-function --with-tune=generic --with-arch_32=i686 --build=x86_64-redhat-linux
Thread model: posix
gcc version 7.3.1 20180303 (Red Hat 7.3.1-5) (GCC)

升級cmake

centos7默認(rèn)安裝的cmake版本較低，不滿足llvm10+的需求，需要升級

這里選擇了cmake 3.26.4版本

cd cmake-3.26.4
./configure
make
make install
ln -s /usr/local/bin/cmake /usr/bin/cmake

安裝llvm10+

libbpf框架的最新版本需要llvm10+的編譯支持，這里下載了11版本，解壓后編譯安裝 鏈接

這里make的需要的時間很長，建議掛著編譯去睡覺

unzip llvm-project-release-11.x.zip
cd llvm-project-release-11.x
mkdir build
cd build/
cmake -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_RTTI=ON -DLLVM_ENABLE_PROJECTS="clang;libcxx;libcxxabi" -G "Unix Makefiles" ../llvm
make
make install

借助libbpf-bootstrap腳手架跑一個helloworld程序

搭建腳手架

libbpf-bootstrap項目提供了一個快速構(gòu)建ebpf程序的框架，包括libbpf和bpftool兩大工具；項目包含一系列示例程序在examples/c文件夾中，并提供了一個相對通用的Makefile可以供我們了解一個ebpf程序是如何編譯起來的
項目地址

克隆libbpf-bootstrap項目，并更新子項目

git clone https://github.com/libbpf/libbpf-bootstrap.git
git submodule update --init --recursive

編寫一個helloworld程序

現(xiàn)在可以在examples/c文件夾下新建兩個文件，分別命名為hello.bpf.c和hello.c

// hello.bpf.c
#define BPF_NO_GLOBAL_DATA
#include <linux/bpf.h>
#include <bpf/bpf_helpers.h>

SEC("tracepoint/syscalls/sys_enter_execve")

int bpf_prog(void *ctx) {
  char msg[] = "Hello, World!";
  __bpf_printk("invoke bpf_prog: %s\n", msg);
  return 0;
}

char LICENSE[] SEC("license") = "Dual BSD/GPL";

// hello.c
#include <stdio.h>
#include <unistd.h>
#include <sys/resource.h>
#include <bpf/libbpf.h>
#include "hello.skel.h"

static int libbpf_print_fn(enum libbpf_print_level level, const char *format, va_list args)
{
    return vfprintf(stderr, format, args);
}

int main(int argc, char **argv)
{
    struct hello_bpf *skel;
    int err;

    libbpf_set_strict_mode(LIBBPF_STRICT_ALL);
    /* Set up libbpf errors and debug info callback */
    libbpf_set_print(libbpf_print_fn);

    /* Open BPF application */
    skel = hello_bpf__open();
    if (!skel) {
        fprintf(stderr, "Failed to open BPF skeleton\n");
        return 1;
    }   

    /* Load & verify BPF programs */
    err = hello_bpf__load(skel);
    if (err) {
        fprintf(stderr, "Failed to load and verify BPF skeleton\n");
        goto cleanup;
    }

    /* Attach tracepoint handler */
    err = hello_bpf__attach(skel);
    if (err) {
        fprintf(stderr, "Failed to attach BPF skeleton\n");
        goto cleanup;
    }

    printf("Successfully started! Please run `sudo cat /sys/kernel/debug/tracing/trace_pipe` "
           "to see output of the BPF programs.\n");

    for (;;) {
        /* trigger our BPF program */
        fprintf(stderr, ".");
        sleep(1);
    }

cleanup:
    hello_bpf__destroy(skel);
    return -err;
}

helloworld的程序hook了execve系統(tǒng)調(diào)用，在進(jìn)程創(chuàng)建時會調(diào)用execve系統(tǒng)調(diào)用進(jìn)入內(nèi)核空間，此時被注入內(nèi)核空間的ebpf字節(jié)碼打印出一個字符串。

需要注意的是：

• 這個打印行為并不是在用戶空間進(jìn)行的，需要通過/sys/kernel/debug/tracing/trace_pipe文件觀察打印輸出
• 在hello.bpf.c文件中在#include <bpf/bpf_helpers.h>之前定義了一個#define BPF_NO_GLOBAL_DATA宏，這里是關(guān)閉ebpf的全局變量功能，因為低版本的內(nèi)核不支持這個特性，項目的示例minimal和minimal_legacy很明確的標(biāo)注了這個問題

This version of minimal is modified to allow running on even older kernels that do not allow global variables. bpf_printk uses global variables unless BPF_NO_GLOBAL_DATA is defined before including bpf_helpers.h. Additionally, the global variable my_pid has been replaced with an array of one element to hold the process pid.

編譯運行helloworld程序

將自己的構(gòu)建目標(biāo)加入Makefile的APPS變量中，運行make $obj即可編譯自己的ebpf程序

# Makefile
APPS = minimal minimal_legacy bootstrap uprobe kprobe fentry usdt sockfilter tc ksyscall hello

運行make hello后，打印出整個項目的編譯流程，主要分為三個步驟如下

• 編譯libbpf
• 編譯bpftool
• 編譯hello

當(dāng)然編譯一個ebpf程序的過程不是這么簡單的過程，helloworld程序的具體編譯的過程會在后文介紹

# make hello
which: no cargo in (/opt/rh/devtoolset-7/root/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin)
  MKDIR    .output
  MKDIR    .output/libbpf
  LIB      libbpf.a
  MKDIR    /root/libbpf-bootstrap/examples/c/.output//libbpf/staticobjs
  CC       /root/libbpf-bootstrap/examples/c/.output//libbpf/staticobjs/bpf.o
  CC       /root/libbpf-bootstrap/examples/c/.output//libbpf/staticobjs/btf.o
  CC       /root/libbpf-bootstrap/examples/c/.output//libbpf/staticobjs/libbpf.o
  CC       /root/libbpf-bootstrap/examples/c/.output//libbpf/staticobjs/libbpf_errno.o
  CC       /root/libbpf-bootstrap/examples/c/.output//libbpf/staticobjs/netlink.o
  CC       /root/libbpf-bootstrap/examples/c/.output//libbpf/staticobjs/nlattr.o
  CC       /root/libbpf-bootstrap/examples/c/.output//libbpf/staticobjs/str_error.o
  CC       /root/libbpf-bootstrap/examples/c/.output//libbpf/staticobjs/libbpf_probes.o
  CC       /root/libbpf-bootstrap/examples/c/.output//libbpf/staticobjs/bpf_prog_linfo.o
  CC       /root/libbpf-bootstrap/examples/c/.output//libbpf/staticobjs/btf_dump.o
  CC       /root/libbpf-bootstrap/examples/c/.output//libbpf/staticobjs/hashmap.o
  CC       /root/libbpf-bootstrap/examples/c/.output//libbpf/staticobjs/ringbuf.o
  CC       /root/libbpf-bootstrap/examples/c/.output//libbpf/staticobjs/strset.o
  CC       /root/libbpf-bootstrap/examples/c/.output//libbpf/staticobjs/linker.o
  CC       /root/libbpf-bootstrap/examples/c/.output//libbpf/staticobjs/gen_loader.o
  CC       /root/libbpf-bootstrap/examples/c/.output//libbpf/staticobjs/relo_core.o
  CC       /root/libbpf-bootstrap/examples/c/.output//libbpf/staticobjs/usdt.o
  CC       /root/libbpf-bootstrap/examples/c/.output//libbpf/staticobjs/zip.o
  AR       /root/libbpf-bootstrap/examples/c/.output//libbpf/libbpf.a
  INSTALL  bpf.h libbpf.h btf.h libbpf_common.h libbpf_legacy.h bpf_helpers.h bpf_helper_defs.h bpf_tracing.h bpf_endian.h bpf_core_read.h skel_internal.h libbpf_version.h usdt.bpf.h
  INSTALL  /root/libbpf-bootstrap/examples/c/.output//libbpf/libbpf.pc
  INSTALL  /root/libbpf-bootstrap/examples/c/.output//libbpf/libbpf.a
  MKDIR    bpftool
  BPFTOOL  bpftool/bootstrap/bpftool
...                        libbfd: [ OFF ]
...               clang-bpf-co-re: [ on  ]
...                          llvm: [ on  ]
...                        libcap: [ OFF ]
  MKDIR    /root/libbpf-bootstrap/examples/c/.output/bpftool/bootstrap/libbpf/include/bpf
  INSTALL  /root/libbpf-bootstrap/examples/c/.output/bpftool/bootstrap/libbpf/include/bpf/hashmap.h
  INSTALL  /root/libbpf-bootstrap/examples/c/.output/bpftool/bootstrap/libbpf/include/bpf/relo_core.h
  INSTALL  /root/libbpf-bootstrap/examples/c/.output/bpftool/bootstrap/libbpf/include/bpf/libbpf_internal.h
  MKDIR    /root/libbpf-bootstrap/examples/c/.output/bpftool/bootstrap/
  MKDIR    /root/libbpf-bootstrap/examples/c/.output/bpftool/bootstrap/libbpf/
  MKDIR    /root/libbpf-bootstrap/examples/c/.output/bpftool/bootstrap/libbpf/staticobjs
  CC       /root/libbpf-bootstrap/examples/c/.output/bpftool/bootstrap/libbpf/staticobjs/bpf.o
  CC       /root/libbpf-bootstrap/examples/c/.output/bpftool/bootstrap/libbpf/staticobjs/btf.o
  CC       /root/libbpf-bootstrap/examples/c/.output/bpftool/bootstrap/libbpf/staticobjs/libbpf.o
  CC       /root/libbpf-bootstrap/examples/c/.output/bpftool/bootstrap/libbpf/staticobjs/libbpf_errno.o
  CC       /root/libbpf-bootstrap/examples/c/.output/bpftool/bootstrap/libbpf/staticobjs/netlink.o
  CC       /root/libbpf-bootstrap/examples/c/.output/bpftool/bootstrap/libbpf/staticobjs/nlattr.o
  CC       /root/libbpf-bootstrap/examples/c/.output/bpftool/bootstrap/libbpf/staticobjs/str_error.o
  CC       /root/libbpf-bootstrap/examples/c/.output/bpftool/bootstrap/libbpf/staticobjs/libbpf_probes.o
  CC       /root/libbpf-bootstrap/examples/c/.output/bpftool/bootstrap/libbpf/staticobjs/bpf_prog_linfo.o
  CC       /root/libbpf-bootstrap/examples/c/.output/bpftool/bootstrap/libbpf/staticobjs/btf_dump.o
  CC       /root/libbpf-bootstrap/examples/c/.output/bpftool/bootstrap/libbpf/staticobjs/hashmap.o
  CC       /root/libbpf-bootstrap/examples/c/.output/bpftool/bootstrap/libbpf/staticobjs/ringbuf.o
  CC       /root/libbpf-bootstrap/examples/c/.output/bpftool/bootstrap/libbpf/staticobjs/strset.o
  CC       /root/libbpf-bootstrap/examples/c/.output/bpftool/bootstrap/libbpf/staticobjs/linker.o
  CC       /root/libbpf-bootstrap/examples/c/.output/bpftool/bootstrap/libbpf/staticobjs/gen_loader.o
  CC       /root/libbpf-bootstrap/examples/c/.output/bpftool/bootstrap/libbpf/staticobjs/relo_core.o
  CC       /root/libbpf-bootstrap/examples/c/.output/bpftool/bootstrap/libbpf/staticobjs/usdt.o
  AR       /root/libbpf-bootstrap/examples/c/.output/bpftool/bootstrap/libbpf/libbpf.a
  INSTALL  bpf.h libbpf.h btf.h libbpf_common.h libbpf_legacy.h bpf_helpers.h bpf_helper_defs.h bpf_tracing.h bpf_endian.h bpf_core_read.h skel_internal.h libbpf_version.h usdt.bpf.h
  CC       /root/libbpf-bootstrap/examples/c/.output/bpftool/bootstrap/main.o
  CC       /root/libbpf-bootstrap/examples/c/.output/bpftool/bootstrap/common.o
  CC       /root/libbpf-bootstrap/examples/c/.output/bpftool/bootstrap/json_writer.o
  CC       /root/libbpf-bootstrap/examples/c/.output/bpftool/bootstrap/gen.o
  CC       /root/libbpf-bootstrap/examples/c/.output/bpftool/bootstrap/btf.o
  CC       /root/libbpf-bootstrap/examples/c/.output/bpftool/bootstrap/xlated_dumper.o
  CC       /root/libbpf-bootstrap/examples/c/.output/bpftool/bootstrap/btf_dumper.o
  CC       /root/libbpf-bootstrap/examples/c/.output/bpftool/bootstrap/disasm.o
  LINK     /root/libbpf-bootstrap/examples/c/.output/bpftool/bootstrap/bpftool
  BPF      .output/hello.bpf.o
  GEN-SKEL .output/hello.skel.h
  CC       .output/hello.o
  BINARY   hello

這里我們的helloworld程序就編譯好了，嘗試運行一下，需要使用root用戶：

# ./hello
libbpf: loading object 'hello_bpf' from buffer
libbpf: elf: section(2) .symtab, size 120, link 1, flags 0, type=2
libbpf: elf: section(3) tracepoint/syscalls/sys_enter_execve, size 200, link 0, flags 6, type=1
libbpf: sec 'tracepoint/syscalls/sys_enter_execve': found program 'bpf_prog' at insn offset 0 (0 bytes), code size 25 insns (200 bytes)
libbpf: elf: section(4) .rodata.str1.1, size 35, link 0, flags 32, type=1
libbpf: elf: section(5) license, size 13, link 0, flags 3, type=1
libbpf: license of hello_bpf is Dual BSD/GPL
libbpf: elf: section(6) .BTF, size 438, link 0, flags 0, type=1
libbpf: elf: section(7) .BTF.ext, size 160, link 0, flags 0, type=1
libbpf: looking for externs among 5 symbols...
libbpf: collected 0 externs total
libbpf: map '.rodata.str1.1' (global data): at sec_idx 4, offset 0, flags 80.
libbpf: map 0 is ".rodata.str1.1"
libbpf: map '.rodata.str1.1': skipped auto-creating...
Successfully started! Please run `sudo cat /sys/kernel/debug/tracing/trace_pipe` to see output of the BPF programs.

此時打開另一個終端查看/sys/kernel/debug/tracing/trace_pipe文件可以看到程序打?。?/p>

# cat /sys/kernel/debug/tracing/trace_pipe
           <...>-79115 [000] .... 67971.749552: 0: invoke bpf_prog: Hello, World!
           <...>-79117 [000] .... 67971.753866: 0: invoke bpf_prog: Hello, World!
           <...>-79119 [000] .... 67971.757380: 0: invoke bpf_prog: Hello, World!
           <...>-79120 [000] .... 67971.761711: 0: invoke bpf_prog: Hello, World!
           <...>-79121 [000] .... 67971.764028: 0: invoke bpf_prog: Hello, World!
           <...>-79123 [000] .... 67971.769068: 0: invoke bpf_prog: Hello, World!
           <...>-79124 [000] .... 67971.771786: 0: invoke bpf_prog: Hello, World!
           <...>-79126 [000] .... 67971.776243: 0: invoke bpf_prog: Hello, World!
           <...>-79127 [000] .... 67971.779373: 0: invoke bpf_prog: Hello, World!
           <...>-79129 [000] .... 67971.784824: 0: invoke bpf_prog: Hello, World!
           <...>-79131 [000] .... 67971.791846: 0: invoke bpf_prog: Hello, World!
           <...>-79132 [000] .... 67979.733858: 0: invoke bpf_prog: Hello, World!

這里我們的helloworld程序就運行成功了

to be continued文章來源地址http://www.zghlxwxcb.cn/news/detail-469945.html

到了這里，關(guān)于【從零開始的eBPF】跑一個helloworld程序的文章就介紹完了。如果您還想了解更多內(nèi)容，請在右上角搜索TOY模板網(wǎng)以前的文章或繼續(xù)瀏覽下面的相關(guān)文章，希望大家以后多多支持TOY模板網(wǎng)！