[lld][DebugInfo/BTF] Add BTF section merging and deduplication#183915
[lld][DebugInfo/BTF] Add BTF section merging and deduplication#183915caniko wants to merge 1 commit into
Conversation
|
Thank you for submitting a Pull Request (PR) to the LLVM Project! This PR will be automatically labeled and the relevant teams will be notified. If you wish to, you can add reviewers by using the "Reviewers" section on this page. If this is not working for you, it is probably because you do not have write permissions for the repository. In which case you can instead tag reviewers by name in a comment by using If you have received no comments on your PR for a week, you can request a review by "ping"ing the PR by adding a comment “Ping”. The common courtesy "ping" rate is once a week. Please remember that you are asking for valuable time from other developers. If you have further questions, they may be answered by the LLVM GitHub User Guide. You can also ask questions in a comment on this PR, on the LLVM Discord or on the forums. |
|
@llvm/pr-subscribers-debuginfo @llvm/pr-subscribers-lld Author: Can H. Tartanoglu (caniko) ChangesSummaryThis adds The approach: each compiler emits What's hereNew LLVM library code (
lld integration (
Tests:
MotivationThe kernel's Kconfig currently has: With compiler-generated BTF + linker-side merging, this restriction goes away. GNU ld already has this via libctf; this brings lld to parity, which matters because lld is what the kernel uses for Clang LTO builds. Algorithm notesThe dedup algorithm follows @anakryiko's design from libbpf (writeup). The composite type pass uses a hypothesis map for DFS graph equivalence, which correctly handles recursive types (linked lists, trees, etc.). I chose to put the library code in I'd appreciate review from @MaskRay on the lld integration side — I followed the cc @4ast Patch is 83.50 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/183915.diff 18 Files Affected:
diff --git a/lld/ELF/CMakeLists.txt b/lld/ELF/CMakeLists.txt
index e22897c2789d8..c2a9265f5281b 100644
--- a/lld/ELF/CMakeLists.txt
+++ b/lld/ELF/CMakeLists.txt
@@ -66,6 +66,7 @@ add_lld_library(lldELF
BinaryFormat
BitWriter
Core
+ DebugInfoBTF
DebugInfoDWARF
Demangle
DTLTO
diff --git a/lld/ELF/Config.h b/lld/ELF/Config.h
index 237df52194210..d9e279a5dbe03 100644
--- a/lld/ELF/Config.h
+++ b/lld/ELF/Config.h
@@ -56,6 +56,7 @@ struct Partition;
struct PhdrEntry;
class BssSection;
+class BtfSection;
class GdbIndexSection;
class GotPltSection;
class GotSection;
@@ -332,6 +333,7 @@ struct Config {
bool fixCortexA8;
bool formatBinary = false;
bool fortranCommon;
+ bool btfMerge;
bool gcSections;
bool gdbIndex;
bool gnuHash = false;
@@ -569,6 +571,7 @@ struct InStruct {
std::unique_ptr<SyntheticSection> riscvAttributes;
std::unique_ptr<BssSection> bss;
std::unique_ptr<BssSection> bssRelRo;
+ std::unique_ptr<BtfSection> btfSection;
std::unique_ptr<SyntheticSection> gnuProperty;
std::unique_ptr<SyntheticSection> gnuStack;
std::unique_ptr<GotSection> got;
diff --git a/lld/ELF/Driver.cpp b/lld/ELF/Driver.cpp
index d7bfa7357d4ed..6fa4cae5156e0 100644
--- a/lld/ELF/Driver.cpp
+++ b/lld/ELF/Driver.cpp
@@ -1436,6 +1436,7 @@ static void readConfigs(Ctx &ctx, opt::InputArgList &args) {
args.hasArg(OPT_fix_cortex_a8) && !args.hasArg(OPT_relocatable);
ctx.arg.fortranCommon =
args.hasFlag(OPT_fortran_common, OPT_no_fortran_common, false);
+ ctx.arg.btfMerge = args.hasFlag(OPT_btf_merge, OPT_no_btf_merge, false);
ctx.arg.gcSections = args.hasFlag(OPT_gc_sections, OPT_no_gc_sections, false);
ctx.arg.gnuUnique = args.hasFlag(OPT_gnu_unique, OPT_no_gnu_unique, true);
ctx.arg.gdbIndex = args.hasFlag(OPT_gdb_index, OPT_no_gdb_index, false);
diff --git a/lld/ELF/Options.td b/lld/ELF/Options.td
index c2111e58c12b9..55a5a00e3d8c9 100644
--- a/lld/ELF/Options.td
+++ b/lld/ELF/Options.td
@@ -281,6 +281,10 @@ defm gc_sections: B<"gc-sections",
"Enable garbage collection of unused sections",
"Disable garbage collection of unused sections (default)">;
+defm btf_merge: BB<"btf-merge",
+ "Merge and deduplicate .BTF sections",
+ "Do not merge .BTF sections (default)">;
+
defm gdb_index: BB<"gdb-index",
"Generate .gdb_index section",
"Do not generate .gdb_index section (default)">;
diff --git a/lld/ELF/SyntheticSections.cpp b/lld/ELF/SyntheticSections.cpp
index 2cfc88d8389b0..863aefc229cc0 100644
--- a/lld/ELF/SyntheticSections.cpp
+++ b/lld/ELF/SyntheticSections.cpp
@@ -32,6 +32,8 @@
#include "llvm/ADT/StringExtras.h"
#include "llvm/BinaryFormat/Dwarf.h"
#include "llvm/BinaryFormat/ELF.h"
+#include "llvm/DebugInfo/BTF/BTFBuilder.h"
+#include "llvm/DebugInfo/BTF/BTFDedup.h"
#include "llvm/DebugInfo/DWARF/DWARFAcceleratorTable.h"
#include "llvm/DebugInfo/DWARF/DWARFDebugPubTable.h"
#include "llvm/Support/DJB.h"
@@ -3676,6 +3678,53 @@ void GdbIndexSection::writeTo(uint8_t *buf) {
bool GdbIndexSection::isNeeded() const { return !chunks.empty(); }
+BtfSection::BtfSection(Ctx &ctx)
+ : SyntheticSection(ctx, ".BTF", SHT_PROGBITS, 0, 1) {}
+
+template <class ELFT>
+std::unique_ptr<BtfSection> BtfSection::create(Ctx &ctx) {
+ llvm::TimeTraceScope timeScope("Merge BTF");
+
+ const bool isLE = ELFT::Endianness == llvm::endianness::little;
+ llvm::BTFBuilder builder;
+
+ // Collect all .BTF input sections and merge them.
+ // Mark originals dead so they don't appear in the output.
+ for (InputSectionBase *s : ctx.inputSections) {
+ if (s->name != ".BTF" || !s->isLive())
+ continue;
+ s->markDead();
+ ArrayRef<uint8_t> data = s->content();
+ StringRef raw(reinterpret_cast<const char *>(data.data()), data.size());
+ Expected<uint32_t> idBase = builder.merge(raw, isLE);
+ if (!idBase) {
+ Warn(ctx) << s << ": failed to parse .BTF section: "
+ << toString(idBase.takeError());
+ continue;
+ }
+ }
+
+ auto ret = std::make_unique<BtfSection>(ctx);
+
+ if (builder.typesCount() == 0)
+ return ret;
+
+ // Deduplicate merged types.
+ if (Error e = llvm::btfDedup(builder)) {
+ Warn(ctx) << "BTF deduplication failed: " << toString(std::move(e));
+ // Fall through and emit un-deduped BTF.
+ builder.write(ret->outputData, isLE);
+ return ret;
+ }
+
+ builder.write(ret->outputData, isLE);
+ return ret;
+}
+
+void BtfSection::writeTo(uint8_t *buf) {
+ memcpy(buf, outputData.data(), outputData.size());
+}
+
VersionDefinitionSection::VersionDefinitionSection(Ctx &ctx)
: SyntheticSection(ctx, ".gnu.version_d", SHT_GNU_verdef, SHF_ALLOC,
sizeof(uint32_t)) {}
@@ -4907,6 +4956,11 @@ template <class ELFT> void elf::createSyntheticSections(Ctx &ctx) {
add(*ctx.in.debugNames);
}
+ if (ctx.arg.btfMerge) {
+ ctx.in.btfSection = BtfSection::create<ELFT>(ctx);
+ add(*ctx.in.btfSection);
+ }
+
if (ctx.arg.gdbIndex) {
ctx.in.gdbIndex = GdbIndexSection::create<ELFT>(ctx);
add(*ctx.in.gdbIndex);
diff --git a/lld/ELF/SyntheticSections.h b/lld/ELF/SyntheticSections.h
index 1ae03dc24a2f2..61e438155fb82 100644
--- a/lld/ELF/SyntheticSections.h
+++ b/lld/ELF/SyntheticSections.h
@@ -1013,6 +1013,22 @@ class GdbIndexSection final : public SyntheticSection {
size_t size;
};
+// Merges and deduplicates .BTF (BPF Type Format) sections from multiple input
+// object files into a single output .BTF section. Uses the BTFBuilder/BTFDedup
+// library to parse, merge, and deduplicate BTF type information.
+class BtfSection final : public SyntheticSection {
+public:
+ BtfSection(Ctx &);
+ template <typename ELFT>
+ static std::unique_ptr<BtfSection> create(Ctx &);
+ void writeTo(uint8_t *buf) override;
+ size_t getSize() const override { return outputData.size(); }
+ bool isNeeded() const override { return !outputData.empty(); }
+
+private:
+ SmallVector<uint8_t, 0> outputData;
+};
+
// For more information about .gnu.version and .gnu.version_r see:
// https://www.akkadia.org/drepper/symbol-versioning
diff --git a/lld/test/ELF/Inputs/btf-merge-int.s b/lld/test/ELF/Inputs/btf-merge-int.s
new file mode 100644
index 0000000000000..10b45a1de01cb
--- /dev/null
+++ b/lld/test/ELF/Inputs/btf-merge-int.s
@@ -0,0 +1,26 @@
+# Input file with a BTF section containing a single INT type "int" (4 bytes).
+.text
+.globl foo
+.type foo, @function
+foo:
+ ret
+
+.section .BTF,"",@progbits
+# BTF Header (24 bytes)
+.short 0xeb9f # magic
+.byte 1 # version
+.byte 0 # flags
+.long 24 # hdr_len
+.long 0 # type_off
+.long 16 # type_len (1 INT type = 12 + 4 bytes)
+.long 16 # str_off
+.long 5 # str_len ("\0int\0")
+# Type 1: INT "int" size=4
+.long 1 # name_off = 1 ("int")
+.long 0x01000000 # info: kind=INT(1), vlen=0
+.long 4 # size = 4
+.long 0x00000020 # INT encoding: bits=32, offset=0, encoding=0
+# String table
+.byte 0 # ""
+.ascii "int"
+.byte 0
diff --git a/lld/test/ELF/Inputs/btf-merge-long.s b/lld/test/ELF/Inputs/btf-merge-long.s
new file mode 100644
index 0000000000000..5cc930408c543
--- /dev/null
+++ b/lld/test/ELF/Inputs/btf-merge-long.s
@@ -0,0 +1,26 @@
+# Input file with a BTF section containing a single INT type "long" (8 bytes).
+.text
+.globl bar
+.type bar, @function
+bar:
+ ret
+
+.section .BTF,"",@progbits
+# BTF Header (24 bytes)
+.short 0xeb9f # magic
+.byte 1 # version
+.byte 0 # flags
+.long 24 # hdr_len
+.long 0 # type_off
+.long 16 # type_len (1 INT type = 12 + 4 bytes)
+.long 16 # str_off
+.long 6 # str_len ("\0long\0")
+# Type 1: INT "long" size=8
+.long 1 # name_off = 1 ("long")
+.long 0x01000000 # info: kind=INT(1), vlen=0
+.long 8 # size = 8
+.long 0x00000040 # INT encoding: bits=64, offset=0, encoding=0
+# String table
+.byte 0 # ""
+.ascii "long"
+.byte 0
diff --git a/lld/test/ELF/btf-merge-basic.s b/lld/test/ELF/btf-merge-basic.s
new file mode 100644
index 0000000000000..fac348ad8aae2
--- /dev/null
+++ b/lld/test/ELF/btf-merge-basic.s
@@ -0,0 +1,55 @@
+# REQUIRES: x86
+## Test basic --btf-merge functionality: merging .BTF sections from two input
+## files into a single output .BTF section.
+
+# RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %s -o %t1.o
+# RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %p/Inputs/btf-merge-long.s -o %t2.o
+
+## Verify that without --btf-merge, input .BTF sections pass through as-is.
+# RUN: ld.lld %t1.o %t2.o -o %t.no-merge
+# RUN: llvm-readelf -S %t.no-merge | FileCheck %s --check-prefix=NO-MERGE
+
+## Verify that --btf-merge produces a single merged .BTF section.
+# RUN: ld.lld --btf-merge %t1.o %t2.o -o %t.merged
+# RUN: llvm-readelf -S %t.merged | FileCheck %s --check-prefix=MERGED
+# RUN: llvm-readelf -x .BTF %t.merged | FileCheck %s --check-prefix=BTF-HEX
+
+## Verify --no-btf-merge disables merging.
+# RUN: ld.lld --btf-merge --no-btf-merge %t1.o %t2.o -o %t.disabled
+# RUN: llvm-readelf -S %t.disabled | FileCheck %s --check-prefix=NO-MERGE
+
+# NO-MERGE: .BTF
+
+## When merged, there should be exactly one .BTF section.
+# MERGED: .BTF PROGBITS
+# MERGED-NOT: .BTF PROGBITS
+
+## Check BTF header magic and that it's valid.
+## The hex dump should start with the BTF magic 9feb (LE for 0xeb9f).
+# BTF-HEX: Hex dump of section '.BTF':
+# BTF-HEX: 0x{{[0-9a-f]+}} 9feb0100
+
+.text
+.globl _start
+_start:
+ ret
+
+.section .BTF,"",@progbits
+# BTF Header (24 bytes)
+.short 0xeb9f # magic
+.byte 1 # version
+.byte 0 # flags
+.long 24 # hdr_len
+.long 0 # type_off
+.long 16 # type_len (1 INT type = 12 + 4 bytes)
+.long 16 # str_off
+.long 5 # str_len ("\0int\0")
+# Type 1: INT "int" size=4
+.long 1 # name_off = 1 ("int")
+.long 0x01000000 # info: kind=INT(1), vlen=0
+.long 4 # size = 4
+.long 0x00000020 # INT encoding: bits=32, offset=0, encoding=0
+# String table
+.byte 0 # ""
+.ascii "int"
+.byte 0
diff --git a/lld/test/ELF/btf-merge-dedup.s b/lld/test/ELF/btf-merge-dedup.s
new file mode 100644
index 0000000000000..d8940b4226be4
--- /dev/null
+++ b/lld/test/ELF/btf-merge-dedup.s
@@ -0,0 +1,45 @@
+# REQUIRES: x86
+## Test --btf-merge deduplication: two input files with identical BTF types
+## should produce a single merged .BTF section with duplicates removed.
+
+# RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %s -o %t1.o
+# RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %p/Inputs/btf-merge-int.s -o %t2.o
+
+## Both files have identical INT "int" types. After merge+dedup, only one
+## type should remain, making the output smaller than the sum of inputs.
+# RUN: ld.lld --btf-merge %t1.o %t2.o -o %t.merged
+# RUN: llvm-readelf -S %t.merged | FileCheck %s --check-prefix=MERGED
+
+## Verify the merged section exists and has valid BTF.
+# RUN: llvm-readelf -x .BTF %t.merged | FileCheck %s --check-prefix=BTF-HEX
+
+# MERGED: .BTF PROGBITS
+
+## Verify BTF magic is present.
+# BTF-HEX: Hex dump of section '.BTF':
+# BTF-HEX: 0x{{[0-9a-f]+}} 9feb0100
+
+.text
+.globl _start
+_start:
+ ret
+
+.section .BTF,"",@progbits
+# BTF Header (24 bytes)
+.short 0xeb9f # magic
+.byte 1 # version
+.byte 0 # flags
+.long 24 # hdr_len
+.long 0 # type_off
+.long 16 # type_len (1 INT type = 12 + 4 bytes)
+.long 16 # str_off
+.long 5 # str_len ("\0int\0")
+# Type 1: INT "int" size=4 (identical to Inputs/btf-merge-int.s)
+.long 1 # name_off = 1 ("int")
+.long 0x01000000 # info: kind=INT(1), vlen=0
+.long 4 # size = 4
+.long 0x00000020 # INT encoding: bits=32, offset=0, encoding=0
+# String table
+.byte 0 # ""
+.ascii "int"
+.byte 0
diff --git a/llvm/include/llvm/DebugInfo/BTF/BTFBuilder.h b/llvm/include/llvm/DebugInfo/BTF/BTFBuilder.h
new file mode 100644
index 0000000000000..a401788ec88f0
--- /dev/null
+++ b/llvm/include/llvm/DebugInfo/BTF/BTFBuilder.h
@@ -0,0 +1,96 @@
+//===- BTFBuilder.h - BTF builder/writer -----------------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+///
+/// \file
+/// Mutable in-memory representation of BTF type information.
+///
+/// BTFBuilder provides an interface for constructing and merging .BTF
+/// sections. Types and strings can be added individually or merged from
+/// raw .BTF section data parsed from ELF object files. The result can
+/// be serialized back to binary .BTF format.
+///
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_DEBUGINFO_BTF_BTFBUILDER_H
+#define LLVM_DEBUGINFO_BTF_BTFBUILDER_H
+
+#include "llvm/ADT/SmallVector.h"
+#include "llvm/ADT/StringRef.h"
+#include "llvm/DebugInfo/BTF/BTF.h"
+#include "llvm/Support/Compiler.h"
+#include "llvm/Support/Error.h"
+
+namespace llvm {
+
+/// A mutable container for BTF type information that supports construction,
+/// merging, and serialization.
+///
+/// Types are stored as contiguous raw bytes in native byte order.
+/// Type IDs are 1-based (ID 0 = void, never stored).
+class BTFBuilder {
+ // String table: concatenated NUL-terminated strings.
+ // Offset 0 is always the empty string (single NUL byte).
+ SmallVector<char, 0> Strings;
+
+ // Raw type data in native byte order. Types are stored sequentially,
+ // each as CommonType header followed by kind-specific tail data.
+ SmallVector<uint8_t, 0> TypeData;
+
+ // TypeOffsets[i] is the byte offset in TypeData for type ID (i+1).
+ SmallVector<uint32_t, 0> TypeOffsets;
+
+public:
+ LLVM_ABI BTFBuilder();
+
+ /// Add a string, returning its offset in the string table.
+ LLVM_ABI uint32_t addString(StringRef S);
+
+ /// Add a type header, returning the new 1-based type ID.
+ /// Append kind-specific tail data with addTail() immediately after.
+ LLVM_ABI uint32_t addType(const BTF::CommonType &Header);
+
+ /// Append kind-specific tail data for the most recently added type.
+ template <typename T> void addTail(const T &Data) {
+ const auto *Ptr = reinterpret_cast<const uint8_t *>(&Data);
+ TypeData.append(Ptr, Ptr + sizeof(Data));
+ }
+
+ /// Merge all types and strings from a raw .BTF section, remapping
+ /// type IDs and string offsets. Returns the first new type ID.
+ LLVM_ABI Expected<uint32_t> merge(StringRef RawBTFSection,
+ bool IsLittleEndian);
+
+ /// Look up a type by 1-based ID. Returns nullptr for invalid IDs.
+ LLVM_ABI const BTF::CommonType *findType(uint32_t Id) const;
+
+ /// Get raw bytes for a type entry (CommonType + tail data).
+ LLVM_ABI ArrayRef<uint8_t> getTypeBytes(uint32_t Id) const;
+
+ /// Get mutable raw bytes for a type entry.
+ LLVM_ABI MutableArrayRef<uint8_t> getMutableTypeBytes(uint32_t Id);
+
+ /// Look up a string by offset in the string table.
+ LLVM_ABI StringRef findString(uint32_t Offset) const;
+
+ /// Number of types, excluding void (type 0).
+ uint32_t typesCount() const { return TypeOffsets.size(); }
+
+ /// Compute the byte size of a type entry from its CommonType header.
+ LLVM_ABI static size_t typeByteSize(const BTF::CommonType *T);
+
+ /// Returns true if CommonType.Type is a type reference for this kind.
+ LLVM_ABI static bool hasTypeRef(uint32_t Kind);
+
+ /// Serialize to binary .BTF format, appending to Out.
+ LLVM_ABI void write(SmallVectorImpl<uint8_t> &Out,
+ bool IsLittleEndian) const;
+};
+
+} // namespace llvm
+
+#endif // LLVM_DEBUGINFO_BTF_BTFBUILDER_H
diff --git a/llvm/include/llvm/DebugInfo/BTF/BTFDedup.h b/llvm/include/llvm/DebugInfo/BTF/BTFDedup.h
new file mode 100644
index 0000000000000..302c1b13c0716
--- /dev/null
+++ b/llvm/include/llvm/DebugInfo/BTF/BTFDedup.h
@@ -0,0 +1,46 @@
+//===- BTFDedup.h - BTF type deduplication ---------------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+///
+/// \file
+/// BTF type deduplication algorithm.
+///
+/// Implements the 5-pass deduplication algorithm from libbpf:
+/// 1. String deduplication
+/// 2. Primitive/struct/enum type dedup with DFS equivalence checking
+/// 3. Reference type dedup (PTR, TYPEDEF, etc.)
+/// 4. Type compaction (remove duplicates, assign new IDs)
+/// 5. Type ID remapping
+///
+/// The algorithm achieves ~137x type reduction on a full Linux kernel
+/// (3.6M types -> 26K types).
+///
+/// Reference: https://nakryiko.com/posts/btf-dedup/
+///
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_DEBUGINFO_BTF_BTFDEDUP_H
+#define LLVM_DEBUGINFO_BTF_BTFDEDUP_H
+
+#include "llvm/Support/Compiler.h"
+#include "llvm/Support/Error.h"
+
+namespace llvm {
+
+class BTFBuilder;
+
+/// Deduplicate types in a BTFBuilder in-place.
+///
+/// After deduplication, structurally equivalent types are merged and
+/// all type IDs are remapped to point to canonical representatives.
+/// The resulting BTFBuilder can be serialized to produce a compact
+/// .BTF section.
+LLVM_ABI Error btfDedup(BTFBuilder &Builder);
+
+} // namespace llvm
+
+#endif // LLVM_DEBUGINFO_BTF_BTFDEDUP_H
diff --git a/llvm/lib/DebugInfo/BTF/BTFBuilder.cpp b/llvm/lib/DebugInfo/BTF/BTFBuilder.cpp
new file mode 100644
index 0000000000000..d9875133b4dfb
--- /dev/null
+++ b/llvm/lib/DebugInfo/BTF/BTFBuilder.cpp
@@ -0,0 +1,416 @@
+//===- BTFBuilder.cpp - BTF builder/writer implementation -----------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm/DebugInfo/BTF/BTFBuilder.h"
+#include "llvm/Support/Endian.h"
+#include "llvm/Support/SwapByteOrder.h"
+
+using namespace llvm;
+
+BTFBuilder::BTFBuilder() {
+ // String table starts with the empty string at offset 0.
+ Strings.push_back('\0');
+}
+
+uint32_t BTFBuilder::addString(StringRef S) {
+ uint32_t Offset = Strings.size();
+ Strings.append(S.begin(), S.end());
+ Strings.push_back('\0');
+ return Offset;
+}
+
+uint32_t BTFBuilder::addType(const BTF::CommonType &Header) {
+ TypeOffsets.push_back(TypeData.size());
+ const auto *Ptr = reinterpret_cast<const uint8_t *>(&Header);
+ TypeData.append(Ptr, Ptr + sizeof(Header));
+ return TypeOffsets.size(); // 1-based ID
+}
+
+const BTF::CommonType *BTFBuilder::findType(uint32_t Id) const {
+ if (Id == 0 || Id > TypeOffsets.size())
+ return nullptr;
+ return reinterpret_cast<const BTF::CommonType *>(
+ &TypeData[TypeOffsets[Id - 1]]);
+}
+
+// Returns {Start, Size} for a type's byte range, or {0, 0} for invalid IDs.
+static std::pair<uint32_t, uint32_t>
+typeBounds(uint32_t Id, const SmallVectorImpl<uint32_t> &TypeOffsets,
+ size_t TypeDataSize) {
+ if (Id == 0 || Id > TypeOffsets.size())
+ return {0, 0};
+ uint32_t Start = TypeOffsets[Id - 1];
+ uint32_t End =
+ (Id < TypeOffsets.size()) ? TypeOffsets[Id] : TypeDataSize;
+ return {Start, End - Start};
+}
+
+ArrayRef<uint8_t> BTFBuilder::getTypeBytes(uint32_t Id) const {
+ auto [Start, Size] = typeBounds(Id, TypeOffsets, TypeData.size());
+ if (Size == 0)
+ return {};
+ return ArrayRef<uint8_t>(&TypeData[Start], Size);
+}
+
+MutableArrayRef<uint8_t> BTFBuilder::getMutableTypeBytes(uint32_t Id) {
+ auto [Start, Size] = typeBounds(Id, TypeOffsets, TypeData.size());
+ if (Size == 0)
+ return {};
+ return MutableArrayRef<ui...
[truncated]
|
|
@llvm/pr-subscribers-lld-elf Author: Can H. Tartanoglu (caniko) ChangesSummaryThis adds The approach: each compiler emits What's hereNew LLVM library code (
lld integration (
Tests:
MotivationThe kernel's Kconfig currently has: With compiler-generated BTF + linker-side merging, this restriction goes away. GNU ld already has this via libctf; this brings lld to parity, which matters because lld is what the kernel uses for Clang LTO builds. Algorithm notesThe dedup algorithm follows @anakryiko's design from libbpf (writeup). The composite type pass uses a hypothesis map for DFS graph equivalence, which correctly handles recursive types (linked lists, trees, etc.). I chose to put the library code in I'd appreciate review from @MaskRay on the lld integration side — I followed the cc @4ast Patch is 83.50 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/183915.diff 18 Files Affected:
diff --git a/lld/ELF/CMakeLists.txt b/lld/ELF/CMakeLists.txt
index e22897c2789d8..c2a9265f5281b 100644
--- a/lld/ELF/CMakeLists.txt
+++ b/lld/ELF/CMakeLists.txt
@@ -66,6 +66,7 @@ add_lld_library(lldELF
BinaryFormat
BitWriter
Core
+ DebugInfoBTF
DebugInfoDWARF
Demangle
DTLTO
diff --git a/lld/ELF/Config.h b/lld/ELF/Config.h
index 237df52194210..d9e279a5dbe03 100644
--- a/lld/ELF/Config.h
+++ b/lld/ELF/Config.h
@@ -56,6 +56,7 @@ struct Partition;
struct PhdrEntry;
class BssSection;
+class BtfSection;
class GdbIndexSection;
class GotPltSection;
class GotSection;
@@ -332,6 +333,7 @@ struct Config {
bool fixCortexA8;
bool formatBinary = false;
bool fortranCommon;
+ bool btfMerge;
bool gcSections;
bool gdbIndex;
bool gnuHash = false;
@@ -569,6 +571,7 @@ struct InStruct {
std::unique_ptr<SyntheticSection> riscvAttributes;
std::unique_ptr<BssSection> bss;
std::unique_ptr<BssSection> bssRelRo;
+ std::unique_ptr<BtfSection> btfSection;
std::unique_ptr<SyntheticSection> gnuProperty;
std::unique_ptr<SyntheticSection> gnuStack;
std::unique_ptr<GotSection> got;
diff --git a/lld/ELF/Driver.cpp b/lld/ELF/Driver.cpp
index d7bfa7357d4ed..6fa4cae5156e0 100644
--- a/lld/ELF/Driver.cpp
+++ b/lld/ELF/Driver.cpp
@@ -1436,6 +1436,7 @@ static void readConfigs(Ctx &ctx, opt::InputArgList &args) {
args.hasArg(OPT_fix_cortex_a8) && !args.hasArg(OPT_relocatable);
ctx.arg.fortranCommon =
args.hasFlag(OPT_fortran_common, OPT_no_fortran_common, false);
+ ctx.arg.btfMerge = args.hasFlag(OPT_btf_merge, OPT_no_btf_merge, false);
ctx.arg.gcSections = args.hasFlag(OPT_gc_sections, OPT_no_gc_sections, false);
ctx.arg.gnuUnique = args.hasFlag(OPT_gnu_unique, OPT_no_gnu_unique, true);
ctx.arg.gdbIndex = args.hasFlag(OPT_gdb_index, OPT_no_gdb_index, false);
diff --git a/lld/ELF/Options.td b/lld/ELF/Options.td
index c2111e58c12b9..55a5a00e3d8c9 100644
--- a/lld/ELF/Options.td
+++ b/lld/ELF/Options.td
@@ -281,6 +281,10 @@ defm gc_sections: B<"gc-sections",
"Enable garbage collection of unused sections",
"Disable garbage collection of unused sections (default)">;
+defm btf_merge: BB<"btf-merge",
+ "Merge and deduplicate .BTF sections",
+ "Do not merge .BTF sections (default)">;
+
defm gdb_index: BB<"gdb-index",
"Generate .gdb_index section",
"Do not generate .gdb_index section (default)">;
diff --git a/lld/ELF/SyntheticSections.cpp b/lld/ELF/SyntheticSections.cpp
index 2cfc88d8389b0..863aefc229cc0 100644
--- a/lld/ELF/SyntheticSections.cpp
+++ b/lld/ELF/SyntheticSections.cpp
@@ -32,6 +32,8 @@
#include "llvm/ADT/StringExtras.h"
#include "llvm/BinaryFormat/Dwarf.h"
#include "llvm/BinaryFormat/ELF.h"
+#include "llvm/DebugInfo/BTF/BTFBuilder.h"
+#include "llvm/DebugInfo/BTF/BTFDedup.h"
#include "llvm/DebugInfo/DWARF/DWARFAcceleratorTable.h"
#include "llvm/DebugInfo/DWARF/DWARFDebugPubTable.h"
#include "llvm/Support/DJB.h"
@@ -3676,6 +3678,53 @@ void GdbIndexSection::writeTo(uint8_t *buf) {
bool GdbIndexSection::isNeeded() const { return !chunks.empty(); }
+BtfSection::BtfSection(Ctx &ctx)
+ : SyntheticSection(ctx, ".BTF", SHT_PROGBITS, 0, 1) {}
+
+template <class ELFT>
+std::unique_ptr<BtfSection> BtfSection::create(Ctx &ctx) {
+ llvm::TimeTraceScope timeScope("Merge BTF");
+
+ const bool isLE = ELFT::Endianness == llvm::endianness::little;
+ llvm::BTFBuilder builder;
+
+ // Collect all .BTF input sections and merge them.
+ // Mark originals dead so they don't appear in the output.
+ for (InputSectionBase *s : ctx.inputSections) {
+ if (s->name != ".BTF" || !s->isLive())
+ continue;
+ s->markDead();
+ ArrayRef<uint8_t> data = s->content();
+ StringRef raw(reinterpret_cast<const char *>(data.data()), data.size());
+ Expected<uint32_t> idBase = builder.merge(raw, isLE);
+ if (!idBase) {
+ Warn(ctx) << s << ": failed to parse .BTF section: "
+ << toString(idBase.takeError());
+ continue;
+ }
+ }
+
+ auto ret = std::make_unique<BtfSection>(ctx);
+
+ if (builder.typesCount() == 0)
+ return ret;
+
+ // Deduplicate merged types.
+ if (Error e = llvm::btfDedup(builder)) {
+ Warn(ctx) << "BTF deduplication failed: " << toString(std::move(e));
+ // Fall through and emit un-deduped BTF.
+ builder.write(ret->outputData, isLE);
+ return ret;
+ }
+
+ builder.write(ret->outputData, isLE);
+ return ret;
+}
+
+void BtfSection::writeTo(uint8_t *buf) {
+ memcpy(buf, outputData.data(), outputData.size());
+}
+
VersionDefinitionSection::VersionDefinitionSection(Ctx &ctx)
: SyntheticSection(ctx, ".gnu.version_d", SHT_GNU_verdef, SHF_ALLOC,
sizeof(uint32_t)) {}
@@ -4907,6 +4956,11 @@ template <class ELFT> void elf::createSyntheticSections(Ctx &ctx) {
add(*ctx.in.debugNames);
}
+ if (ctx.arg.btfMerge) {
+ ctx.in.btfSection = BtfSection::create<ELFT>(ctx);
+ add(*ctx.in.btfSection);
+ }
+
if (ctx.arg.gdbIndex) {
ctx.in.gdbIndex = GdbIndexSection::create<ELFT>(ctx);
add(*ctx.in.gdbIndex);
diff --git a/lld/ELF/SyntheticSections.h b/lld/ELF/SyntheticSections.h
index 1ae03dc24a2f2..61e438155fb82 100644
--- a/lld/ELF/SyntheticSections.h
+++ b/lld/ELF/SyntheticSections.h
@@ -1013,6 +1013,22 @@ class GdbIndexSection final : public SyntheticSection {
size_t size;
};
+// Merges and deduplicates .BTF (BPF Type Format) sections from multiple input
+// object files into a single output .BTF section. Uses the BTFBuilder/BTFDedup
+// library to parse, merge, and deduplicate BTF type information.
+class BtfSection final : public SyntheticSection {
+public:
+ BtfSection(Ctx &);
+ template <typename ELFT>
+ static std::unique_ptr<BtfSection> create(Ctx &);
+ void writeTo(uint8_t *buf) override;
+ size_t getSize() const override { return outputData.size(); }
+ bool isNeeded() const override { return !outputData.empty(); }
+
+private:
+ SmallVector<uint8_t, 0> outputData;
+};
+
// For more information about .gnu.version and .gnu.version_r see:
// https://www.akkadia.org/drepper/symbol-versioning
diff --git a/lld/test/ELF/Inputs/btf-merge-int.s b/lld/test/ELF/Inputs/btf-merge-int.s
new file mode 100644
index 0000000000000..10b45a1de01cb
--- /dev/null
+++ b/lld/test/ELF/Inputs/btf-merge-int.s
@@ -0,0 +1,26 @@
+# Input file with a BTF section containing a single INT type "int" (4 bytes).
+.text
+.globl foo
+.type foo, @function
+foo:
+ ret
+
+.section .BTF,"",@progbits
+# BTF Header (24 bytes)
+.short 0xeb9f # magic
+.byte 1 # version
+.byte 0 # flags
+.long 24 # hdr_len
+.long 0 # type_off
+.long 16 # type_len (1 INT type = 12 + 4 bytes)
+.long 16 # str_off
+.long 5 # str_len ("\0int\0")
+# Type 1: INT "int" size=4
+.long 1 # name_off = 1 ("int")
+.long 0x01000000 # info: kind=INT(1), vlen=0
+.long 4 # size = 4
+.long 0x00000020 # INT encoding: bits=32, offset=0, encoding=0
+# String table
+.byte 0 # ""
+.ascii "int"
+.byte 0
diff --git a/lld/test/ELF/Inputs/btf-merge-long.s b/lld/test/ELF/Inputs/btf-merge-long.s
new file mode 100644
index 0000000000000..5cc930408c543
--- /dev/null
+++ b/lld/test/ELF/Inputs/btf-merge-long.s
@@ -0,0 +1,26 @@
+# Input file with a BTF section containing a single INT type "long" (8 bytes).
+.text
+.globl bar
+.type bar, @function
+bar:
+ ret
+
+.section .BTF,"",@progbits
+# BTF Header (24 bytes)
+.short 0xeb9f # magic
+.byte 1 # version
+.byte 0 # flags
+.long 24 # hdr_len
+.long 0 # type_off
+.long 16 # type_len (1 INT type = 12 + 4 bytes)
+.long 16 # str_off
+.long 6 # str_len ("\0long\0")
+# Type 1: INT "long" size=8
+.long 1 # name_off = 1 ("long")
+.long 0x01000000 # info: kind=INT(1), vlen=0
+.long 8 # size = 8
+.long 0x00000040 # INT encoding: bits=64, offset=0, encoding=0
+# String table
+.byte 0 # ""
+.ascii "long"
+.byte 0
diff --git a/lld/test/ELF/btf-merge-basic.s b/lld/test/ELF/btf-merge-basic.s
new file mode 100644
index 0000000000000..fac348ad8aae2
--- /dev/null
+++ b/lld/test/ELF/btf-merge-basic.s
@@ -0,0 +1,55 @@
+# REQUIRES: x86
+## Test basic --btf-merge functionality: merging .BTF sections from two input
+## files into a single output .BTF section.
+
+# RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %s -o %t1.o
+# RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %p/Inputs/btf-merge-long.s -o %t2.o
+
+## Verify that without --btf-merge, input .BTF sections pass through as-is.
+# RUN: ld.lld %t1.o %t2.o -o %t.no-merge
+# RUN: llvm-readelf -S %t.no-merge | FileCheck %s --check-prefix=NO-MERGE
+
+## Verify that --btf-merge produces a single merged .BTF section.
+# RUN: ld.lld --btf-merge %t1.o %t2.o -o %t.merged
+# RUN: llvm-readelf -S %t.merged | FileCheck %s --check-prefix=MERGED
+# RUN: llvm-readelf -x .BTF %t.merged | FileCheck %s --check-prefix=BTF-HEX
+
+## Verify --no-btf-merge disables merging.
+# RUN: ld.lld --btf-merge --no-btf-merge %t1.o %t2.o -o %t.disabled
+# RUN: llvm-readelf -S %t.disabled | FileCheck %s --check-prefix=NO-MERGE
+
+# NO-MERGE: .BTF
+
+## When merged, there should be exactly one .BTF section.
+# MERGED: .BTF PROGBITS
+# MERGED-NOT: .BTF PROGBITS
+
+## Check BTF header magic and that it's valid.
+## The hex dump should start with the BTF magic 9feb (LE for 0xeb9f).
+# BTF-HEX: Hex dump of section '.BTF':
+# BTF-HEX: 0x{{[0-9a-f]+}} 9feb0100
+
+.text
+.globl _start
+_start:
+ ret
+
+.section .BTF,"",@progbits
+# BTF Header (24 bytes)
+.short 0xeb9f # magic
+.byte 1 # version
+.byte 0 # flags
+.long 24 # hdr_len
+.long 0 # type_off
+.long 16 # type_len (1 INT type = 12 + 4 bytes)
+.long 16 # str_off
+.long 5 # str_len ("\0int\0")
+# Type 1: INT "int" size=4
+.long 1 # name_off = 1 ("int")
+.long 0x01000000 # info: kind=INT(1), vlen=0
+.long 4 # size = 4
+.long 0x00000020 # INT encoding: bits=32, offset=0, encoding=0
+# String table
+.byte 0 # ""
+.ascii "int"
+.byte 0
diff --git a/lld/test/ELF/btf-merge-dedup.s b/lld/test/ELF/btf-merge-dedup.s
new file mode 100644
index 0000000000000..d8940b4226be4
--- /dev/null
+++ b/lld/test/ELF/btf-merge-dedup.s
@@ -0,0 +1,45 @@
+# REQUIRES: x86
+## Test --btf-merge deduplication: two input files with identical BTF types
+## should produce a single merged .BTF section with duplicates removed.
+
+# RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %s -o %t1.o
+# RUN: llvm-mc -filetype=obj -triple=x86_64-unknown-linux %p/Inputs/btf-merge-int.s -o %t2.o
+
+## Both files have identical INT "int" types. After merge+dedup, only one
+## type should remain, making the output smaller than the sum of inputs.
+# RUN: ld.lld --btf-merge %t1.o %t2.o -o %t.merged
+# RUN: llvm-readelf -S %t.merged | FileCheck %s --check-prefix=MERGED
+
+## Verify the merged section exists and has valid BTF.
+# RUN: llvm-readelf -x .BTF %t.merged | FileCheck %s --check-prefix=BTF-HEX
+
+# MERGED: .BTF PROGBITS
+
+## Verify BTF magic is present.
+# BTF-HEX: Hex dump of section '.BTF':
+# BTF-HEX: 0x{{[0-9a-f]+}} 9feb0100
+
+.text
+.globl _start
+_start:
+ ret
+
+.section .BTF,"",@progbits
+# BTF Header (24 bytes)
+.short 0xeb9f # magic
+.byte 1 # version
+.byte 0 # flags
+.long 24 # hdr_len
+.long 0 # type_off
+.long 16 # type_len (1 INT type = 12 + 4 bytes)
+.long 16 # str_off
+.long 5 # str_len ("\0int\0")
+# Type 1: INT "int" size=4 (identical to Inputs/btf-merge-int.s)
+.long 1 # name_off = 1 ("int")
+.long 0x01000000 # info: kind=INT(1), vlen=0
+.long 4 # size = 4
+.long 0x00000020 # INT encoding: bits=32, offset=0, encoding=0
+# String table
+.byte 0 # ""
+.ascii "int"
+.byte 0
diff --git a/llvm/include/llvm/DebugInfo/BTF/BTFBuilder.h b/llvm/include/llvm/DebugInfo/BTF/BTFBuilder.h
new file mode 100644
index 0000000000000..a401788ec88f0
--- /dev/null
+++ b/llvm/include/llvm/DebugInfo/BTF/BTFBuilder.h
@@ -0,0 +1,96 @@
+//===- BTFBuilder.h - BTF builder/writer -----------------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+///
+/// \file
+/// Mutable in-memory representation of BTF type information.
+///
+/// BTFBuilder provides an interface for constructing and merging .BTF
+/// sections. Types and strings can be added individually or merged from
+/// raw .BTF section data parsed from ELF object files. The result can
+/// be serialized back to binary .BTF format.
+///
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_DEBUGINFO_BTF_BTFBUILDER_H
+#define LLVM_DEBUGINFO_BTF_BTFBUILDER_H
+
+#include "llvm/ADT/SmallVector.h"
+#include "llvm/ADT/StringRef.h"
+#include "llvm/DebugInfo/BTF/BTF.h"
+#include "llvm/Support/Compiler.h"
+#include "llvm/Support/Error.h"
+
+namespace llvm {
+
+/// A mutable container for BTF type information that supports construction,
+/// merging, and serialization.
+///
+/// Types are stored as contiguous raw bytes in native byte order.
+/// Type IDs are 1-based (ID 0 = void, never stored).
+class BTFBuilder {
+ // String table: concatenated NUL-terminated strings.
+ // Offset 0 is always the empty string (single NUL byte).
+ SmallVector<char, 0> Strings;
+
+ // Raw type data in native byte order. Types are stored sequentially,
+ // each as CommonType header followed by kind-specific tail data.
+ SmallVector<uint8_t, 0> TypeData;
+
+ // TypeOffsets[i] is the byte offset in TypeData for type ID (i+1).
+ SmallVector<uint32_t, 0> TypeOffsets;
+
+public:
+ LLVM_ABI BTFBuilder();
+
+ /// Add a string, returning its offset in the string table.
+ LLVM_ABI uint32_t addString(StringRef S);
+
+ /// Add a type header, returning the new 1-based type ID.
+ /// Append kind-specific tail data with addTail() immediately after.
+ LLVM_ABI uint32_t addType(const BTF::CommonType &Header);
+
+ /// Append kind-specific tail data for the most recently added type.
+ template <typename T> void addTail(const T &Data) {
+ const auto *Ptr = reinterpret_cast<const uint8_t *>(&Data);
+ TypeData.append(Ptr, Ptr + sizeof(Data));
+ }
+
+ /// Merge all types and strings from a raw .BTF section, remapping
+ /// type IDs and string offsets. Returns the first new type ID.
+ LLVM_ABI Expected<uint32_t> merge(StringRef RawBTFSection,
+ bool IsLittleEndian);
+
+ /// Look up a type by 1-based ID. Returns nullptr for invalid IDs.
+ LLVM_ABI const BTF::CommonType *findType(uint32_t Id) const;
+
+ /// Get raw bytes for a type entry (CommonType + tail data).
+ LLVM_ABI ArrayRef<uint8_t> getTypeBytes(uint32_t Id) const;
+
+ /// Get mutable raw bytes for a type entry.
+ LLVM_ABI MutableArrayRef<uint8_t> getMutableTypeBytes(uint32_t Id);
+
+ /// Look up a string by offset in the string table.
+ LLVM_ABI StringRef findString(uint32_t Offset) const;
+
+ /// Number of types, excluding void (type 0).
+ uint32_t typesCount() const { return TypeOffsets.size(); }
+
+ /// Compute the byte size of a type entry from its CommonType header.
+ LLVM_ABI static size_t typeByteSize(const BTF::CommonType *T);
+
+ /// Returns true if CommonType.Type is a type reference for this kind.
+ LLVM_ABI static bool hasTypeRef(uint32_t Kind);
+
+ /// Serialize to binary .BTF format, appending to Out.
+ LLVM_ABI void write(SmallVectorImpl<uint8_t> &Out,
+ bool IsLittleEndian) const;
+};
+
+} // namespace llvm
+
+#endif // LLVM_DEBUGINFO_BTF_BTFBUILDER_H
diff --git a/llvm/include/llvm/DebugInfo/BTF/BTFDedup.h b/llvm/include/llvm/DebugInfo/BTF/BTFDedup.h
new file mode 100644
index 0000000000000..302c1b13c0716
--- /dev/null
+++ b/llvm/include/llvm/DebugInfo/BTF/BTFDedup.h
@@ -0,0 +1,46 @@
+//===- BTFDedup.h - BTF type deduplication ---------------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+///
+/// \file
+/// BTF type deduplication algorithm.
+///
+/// Implements the 5-pass deduplication algorithm from libbpf:
+/// 1. String deduplication
+/// 2. Primitive/struct/enum type dedup with DFS equivalence checking
+/// 3. Reference type dedup (PTR, TYPEDEF, etc.)
+/// 4. Type compaction (remove duplicates, assign new IDs)
+/// 5. Type ID remapping
+///
+/// The algorithm achieves ~137x type reduction on a full Linux kernel
+/// (3.6M types -> 26K types).
+///
+/// Reference: https://nakryiko.com/posts/btf-dedup/
+///
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_DEBUGINFO_BTF_BTFDEDUP_H
+#define LLVM_DEBUGINFO_BTF_BTFDEDUP_H
+
+#include "llvm/Support/Compiler.h"
+#include "llvm/Support/Error.h"
+
+namespace llvm {
+
+class BTFBuilder;
+
+/// Deduplicate types in a BTFBuilder in-place.
+///
+/// After deduplication, structurally equivalent types are merged and
+/// all type IDs are remapped to point to canonical representatives.
+/// The resulting BTFBuilder can be serialized to produce a compact
+/// .BTF section.
+LLVM_ABI Error btfDedup(BTFBuilder &Builder);
+
+} // namespace llvm
+
+#endif // LLVM_DEBUGINFO_BTF_BTFDEDUP_H
diff --git a/llvm/lib/DebugInfo/BTF/BTFBuilder.cpp b/llvm/lib/DebugInfo/BTF/BTFBuilder.cpp
new file mode 100644
index 0000000000000..d9875133b4dfb
--- /dev/null
+++ b/llvm/lib/DebugInfo/BTF/BTFBuilder.cpp
@@ -0,0 +1,416 @@
+//===- BTFBuilder.cpp - BTF builder/writer implementation -----------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm/DebugInfo/BTF/BTFBuilder.h"
+#include "llvm/Support/Endian.h"
+#include "llvm/Support/SwapByteOrder.h"
+
+using namespace llvm;
+
+BTFBuilder::BTFBuilder() {
+ // String table starts with the empty string at offset 0.
+ Strings.push_back('\0');
+}
+
+uint32_t BTFBuilder::addString(StringRef S) {
+ uint32_t Offset = Strings.size();
+ Strings.append(S.begin(), S.end());
+ Strings.push_back('\0');
+ return Offset;
+}
+
+uint32_t BTFBuilder::addType(const BTF::CommonType &Header) {
+ TypeOffsets.push_back(TypeData.size());
+ const auto *Ptr = reinterpret_cast<const uint8_t *>(&Header);
+ TypeData.append(Ptr, Ptr + sizeof(Header));
+ return TypeOffsets.size(); // 1-based ID
+}
+
+const BTF::CommonType *BTFBuilder::findType(uint32_t Id) const {
+ if (Id == 0 || Id > TypeOffsets.size())
+ return nullptr;
+ return reinterpret_cast<const BTF::CommonType *>(
+ &TypeData[TypeOffsets[Id - 1]]);
+}
+
+// Returns {Start, Size} for a type's byte range, or {0, 0} for invalid IDs.
+static std::pair<uint32_t, uint32_t>
+typeBounds(uint32_t Id, const SmallVectorImpl<uint32_t> &TypeOffsets,
+ size_t TypeDataSize) {
+ if (Id == 0 || Id > TypeOffsets.size())
+ return {0, 0};
+ uint32_t Start = TypeOffsets[Id - 1];
+ uint32_t End =
+ (Id < TypeOffsets.size()) ? TypeOffsets[Id] : TypeDataSize;
+ return {Start, End - Start};
+}
+
+ArrayRef<uint8_t> BTFBuilder::getTypeBytes(uint32_t Id) const {
+ auto [Start, Size] = typeBounds(Id, TypeOffsets, TypeData.size());
+ if (Size == 0)
+ return {};
+ return ArrayRef<uint8_t>(&TypeData[Start], Size);
+}
+
+MutableArrayRef<uint8_t> BTFBuilder::getMutableTypeBytes(uint32_t Id) {
+ auto [Start, Size] = typeBounds(Id, TypeOffsets, TypeData.size());
+ if (Size == 0)
+ return {};
+ return MutableArrayRef<ui...
[truncated]
|
There was a problem hiding this comment.
the motivation is thin: "enabling the LTO + Rust + BTF pipeline in the Linux kernel" needs more context — what specifically was broken or missing?
If the lld implementation is a thin wrapper of DebugInfoBTF, it doesn't need so many tests.
The five test .s files (btf-merge-basic.s, btf-merge-dedup.s, btf-merge-single.s, btf-merge-struct.s) embed near-identical BTF bytes inline. Can they be restructured and grouped to a small set of tests? If you read test history in this directory, I have many refactoring patches that group related tests with split-file. Newer tests tend to not use Inputs/ at all. The lack of a dump tool is also a problem. Dumping the section content as hex in lld/test/ELF is not ideal. Most tests should be on the LLVM BTF side.
In addition we should have -r and --gc-sections tests. - should not merge the section contents.
| @@ -281,6 +281,10 @@ defm gc_sections: B<"gc-sections", | |||
| "Enable garbage collection of unused sections", | |||
| "Disable garbage collection of unused sections (default)">; | |||
|
|
|||
| defm btf_merge: BB<"btf-merge", | |||
There was a problem hiding this comment.
should be alphabetically placed near other b-options
There was a problem hiding this comment.
Done. Moved after --branch-to-branch.
| // Collect all .BTF input sections and merge them. | ||
| // Mark originals dead so they don't appear in the output. | ||
| for (InputSectionBase *s : ctx.inputSections) { | ||
| if (s->name != ".BTF" || !s->isLive()) |
There was a problem hiding this comment.
Need a test to cover the s->isLive condition.
| } | ||
|
|
||
| auto ret = std::make_unique<BtfSection>(ctx); | ||
|
|
| @@ -4907,6 +4956,11 @@ template <class ELFT> void elf::createSyntheticSections(Ctx &ctx) { | |||
| add(*ctx.in.debugNames); | |||
| } | |||
|
|
|||
| if (ctx.arg.btfMerge) { | |||
There was a problem hiding this comment.
We can use this order: btf, debugNames, gdbIndex. Should use std::make_unique<BtfSection<ELFT>>(ctx);
MaskRay
left a comment
MaskRay
left a comment
There was a problem hiding this comment.
The header validation and byte-swapping logic in BTFBuilder::merge() (lines ~963–1000) duplicates BTFParser::parseBTF(). The cleanest fix would probably to refactor BTFParser::parseBTF to take a StringRef. I'll defer that to BTF reviewers.
| return ret; | ||
|
|
||
| // Deduplicate merged types. | ||
| if (Error e = llvm::btfDedup(builder)) { |
There was a problem hiding this comment.
btfDedup can be placed inside llvm::btf
| std::unique_ptr<BtfSection> BtfSection::create(Ctx &ctx) { | ||
| llvm::TimeTraceScope timeScope("Merge BTF"); | ||
|
|
||
| const bool isLE = ELFT::Endianness == llvm::endianness::little; |
There was a problem hiding this comment.
delete unneeded blank line
We need a big-endian test. Try aarch64be or ppc64
The kernel generates BTF via pahole's DWARF→BTF conversion. With LTO, pahole breaks on the merged DWARF because Rust compilation units produce DWARF constructs pahole doesn't understand. Even with
Agreed, the detailed type-level verification belongs in the LLVM unit tests (86 tests across BTFBuilder and BTFDedup already cover all 19 type kinds, recursive types, error paths, etc.). I've consolidated the lld tests to a single
Agreed, I've reduced the hex checks to just verifying BTF magic (confirming a valid merged section was produced). A proper
The duplication exists because |
smithp35
left a comment
smithp35
left a comment
There was a problem hiding this comment.
the motivation is thin: "enabling the LTO + Rust + BTF pipeline in the Linux kernel" needs more context — what specifically was broken or missing?
Closest external link I can find about this is https://lwn.net/Articles/991719/ "BTF, Rust, and the kernel toolchain"
Apologies in advance for naive questions about btf.
While I can get that an external tool to translate DWARF isn't the ideal architecture, has there been any consideration been given to updating pahole to filter out the Rust DWARF information that cannot be represented?
Are there any plans to implement this option in ld.bfd? Sure it won't support Rust/LTO (at least not yet), but parity between the toolchains to avoid pahole seems like a good idea.
If the lld implementation is a thin wrapper of DebugInfoBTF, it doesn't need so many tests.
The five test .s files (btf-merge-basic.s, btf-merge-dedup.s, btf-merge-single.s, btf-merge-struct.s) embed near-identical BTF bytes inline. Can they be restructured and grouped to a small set of tests? If you read test history in this directory, I have many refactoring patches that group related tests with
split-file. Newer tests tend to not useInputs/at all. The lack of a dump tool is also a problem. Dumping the section content as hex in lld/test/ELF is not ideal. Most tests should be on the LLVM BTF side.In addition we should have -r and --gc-sections tests.
-should not merge the section contents.
I agree with the sentiment that human readable dump output would help a lot. That may even enable bits of this patch to be split up into smaller parts that can be reviewed independently.
I note that GNU objdump https://sourceware.org/binutils/docs/binutils/objdump.html#index-CTF has some support for ctf output . Could that be used/extended to btf and implemented?
| @@ -63,6 +63,10 @@ defm branch_to_branch: BB<"branch-to-branch", | |||
| "Enable branch-to-branch optimization (default at -O2)", | |||
| "Disable branch-to-branch optimization (default at -O0 and -O1)">; | |||
|
|
|||
| defm btf_merge: BB<"btf-merge", | |||
There was a problem hiding this comment.
There's an easy to miss man page in lld/docs/ld.lld.1 that could do with an entry for btf-merge.
| @@ -1436,6 +1436,9 @@ static void readConfigs(Ctx &ctx, opt::InputArgList &args) { | |||
| args.hasArg(OPT_fix_cortex_a8) && !args.hasArg(OPT_relocatable); | |||
| ctx.arg.fortranCommon = | |||
| args.hasFlag(OPT_fortran_common, OPT_no_fortran_common, false); | |||
| ctx.arg.btfMerge = | |||
There was a problem hiding this comment.
Is there any reason not to enable by default? If the option is not implemented on all linkers then you'll need a lld specific command line option in a build system that needs to support both GNU and llvm toolchains.
There was a problem hiding this comment.
Good point. For the initial landing I'd prefer to keep it off by default — it's a large new feature and having an explicit opt-in lets us shake out any issues before it's on everywhere. Once it's proven in production I plan to flip the default in a follow-up. I'll also add the man page entry you noted.
| @@ -0,0 +1,96 @@ | |||
| //===- BTFBuilder.h - BTF builder/writer -----------------------*- C++ -*-===// | |||
There was a problem hiding this comment.
Will be worth adding reviewers from llvm/DebugInfo. While putting the code here is likely the best place, it may be out of the scope of a purely lld review for such a large amount of code.
There was a problem hiding this comment.
Sorry @caniko, I'm not suitable for reviewing new debug information formats, as I don't have the time to look into them, given the rest of my work and review load. I'm commenting here though because I briefly noticed that you're using the legacy style for header comments with the C++ annotation. Please take a look at the LLVM coding standards, which show the correct style to use.
There was a problem hiding this comment.
Understood on your review bandwidth. Thanks for catching the header comment style despite your load, I will update those to current LLVM standards.
|
Per @smithp35's suggestion, adding reviewers from llvm/DebugInfo for the new library code in @dwblaikie @jh7370 — would appreciate your review on the BTFBuilder / BTF dedup library portions of this PR. The lld integration is relatively thin, but the bulk of the new code lives under Also added a man page entry for Addressing @smithp35's review questions: Why not fix pahole instead? Pahole already has Having the compiler emit BTF directly and the linker merge it sidesteps the problem entirely — no DWARF→BTF conversion, no pahole dependency, and LTO + Rust + BTF all work together. Plans for ld.bfd? I'm not aware of plans on the GNU side, but GCC already emits BTF via Human-readable BTF dump / splitting the patch Agreed this would help. CTF support in GNU objdump → BTF? CTF (Compact C Type Format) and BTF (BPF Type Format) are structurally different formats despite some conceptual overlap — CTF comes from Solaris/DTrace, BTF from the Linux BPF subsystem. GNU objdump's |
|
✅ With the latest revision this PR passed the C/C++ code formatter. |
🐧 Linux x64 Test Results
✅ The build succeeded and all tests passed. |
cf1e0bf to
15e435d
Compare
There was a problem hiding this comment.
please only format the modified lines, not the whole file.
|
@caniko I have a few questions below:
|
The BTF generation from DWARF happens in a companion PR, #183929 (
With LTO, the linker performs cross-module inlining and merging before emitting DWARF. When you have a mixed C + Rust kernel build with LTO enabled, the linker can inline a Rust function into a C function (or vice versa), and the resulting DWARF ends up in a single CU that contains types from both languages. Pahole's
Yes, pahole handles things beyond type conversion: |
|
Thanks for There are some thing wrong for objtool. and also this failure Note that we have not reached pahole yet. |
One suggestion. Can we add an option in FunctionImport.cpp to avoid cross file inlining between C and Rust language (or broadly between different languages)? Does dwarf for rust have stable dwarf generation or the dwarf generation quite different from versions to versions. If it is stable, I guess pahole can try to learn those dwarf. If not stable, then probably we can use the above option for a while until rust dwarf generation becomes stable. |
These are all caused by For the kernel side,
Agreed that integration testing matters. I'll put together a working kernel build (bpf-next, non-LTO) with both PRs and post the results here before this is ready to land. The kernel Makefile/Kconfig changes needed to wire up
It wouldn't be sufficient. The problem isn't just cross-language inlining. With full LTO, the linker merges IR modules into one before codegen, so you end up with a single CU containing types from both languages regardless of inlining decisions. Blocking cross-language inlining would also give up a key optimization benefit of LTO in the first place. More fundamentally, even without LTO, pahole still can't convert Rust DWARF to BTF. Blocking inlining would only help with the mixed-CU filtering issue, not the underlying "pahole doesn't understand Rust types" issue.
It is not stable. |
Add support for merging and deduplicating .BTF (BPF Type Format) sections in lld's ELF linker, enabling the LTO + Rust + BTF pipeline in the Linux kernel. The implementation consists of: 1. BTFBuilder (llvm/lib/DebugInfo/BTF/BTFBuilder.cpp): A mutable BTF container that can parse, merge, and serialize BTF sections. Supports merging multiple .BTF sections with automatic type ID and string offset remapping. 2. BTFDedup (llvm/lib/DebugInfo/BTF/BTFDedup.cpp): A 5-pass dedup algorithm ported from libbpf's btf_dedup: - String deduplication via hash-based interning - Primitive type dedup (INT, FLOAT, ENUM, ENUM64, FWD) - Composite type dedup (STRUCT, UNION) with DFS cycle handling - Reference type dedup (PTR, TYPEDEF, CONST, VOLATILE, etc.) - Type compaction and ID remapping 3. BtfSection (lld/ELF/SyntheticSections.cpp): A SyntheticSection that collects .BTF input sections, merges and deduplicates them, and emits a single merged .BTF output section. Enabled via --btf-merge flag.
what is the status of it?
What is the story of adding support for all of them to BTF ? |
|
Status as of April 20, 2026: I now have a working bpf-next x86_64 non-LTO build at What I do not have green yet is the full reviewer-complete receipt set: full selftests/bpf is still blocked in the native path, I do not have ThinLTO/FullLTO receipts yet, and the final native vmlinux BTF is not yet equivalent to the pahole baseline. The remaining work has moved past the earlier final-link plumbing issue and is now in native-BTF fidelity. On the dedup copy concern: the intent is still to make linker-side merge/dedup the consolidation point rather than carry a permanent third implementation. I do have small receipts for clang-only, gcc-only, and mixed clang+gcc On Rust types: agreed that BTF should grow to express Rust-specific constructs in the long run. I see that as complementary to this work rather than competing with it. Even with future BTF extensions for Rust, we still need linker-level merge/dedup for native My next receipts are: full selftests/bpf on the native non-LTO build, ThinLTO/FullLTO probes, and final native-vs-pahole BTF comparison. I’ll post those before calling this ready. |
All sounds great. Thanks for the update |
|
Status update as of April 29, 2026, following the April 20 update. I have a fresh native-BTF validation rollup for bpf-next
The build and runtime plumbing is now in much better shape:
Broad I am not claiming native-vs-pahole BTF equivalence yet. The current classified C-dump comparison is:
The dominant residual buckets are anonymous enum text differences and pahole-only struct tags. The validation harness now emits |
|
Greetings @4ast, I would really value your feedback; guidance on next steps is appreciated! Thank you |
I don't see it in vmlinux.h and I don't think anyone reported such issue. Please post the analysis on bpf @ vger.kernel.org so that broad community can participate. Without pahole some difference in vmlinux.h is expected. Like kfuncs are probably not there, and also this patch and the thread: |
7 participants
Summary
This adds
--btf-mergesupport to lld's ELF linker, allowing it to merge and deduplicate.BTFsections from multiple input object files into a single output section.Companion to #183929 which adds
-gbtffor target-independent BTF emission. Together they enable native BTF for Linux kernel builds with Clang, including LTO + Rust configurations.Motivation
The Linux kernel requires BTF (
DEBUG_INFO_BTF=y) for the BPF subsystem. Today this is generated by pahole, which converts DWARF to BTF as a post-link step. This breaks in two specific ways:LTO + Rust: With LTO, the linker merges compilation units. Pahole then encounters Rust DWARF constructs it doesn't understand (e.g. Rust enums, trait objects). The
PAHOLE_HAS_LANG_EXCLUDEflag filters out Rust CUs, but with LTO the linker can interleave C and Rust code within the same CU, so filtering by language no longer works.Performance: Pahole's DWARF→BTF conversion is a sequential post-link pass over the entire binary's debug info. For large kernels this adds significant build time.
This produces the kernel's current Kconfig constraint:
The fix is to skip DWARF→BTF entirely: each compiler emits
.BTFdirectly (GCC already does via-gbtf, Clang support in #183929), and the linker merges/deduplicates them — the same way it already handles.gdb_index. This lifts the Kconfig restriction.What's here
New LLVM library code (
llvm/lib/DebugInfo/BTF/):BTFBuilder— mutable in-memory BTF container that can parse, merge, and serialize.BTFsectionsBTF::dedup()— 5-pass deduplication algorithm ported from libbpf'sbtf_dedup():lld integration (
lld/ELF/):BtfSection<ELFT>synthetic section that collects.BTFinputs, merges via BTFBuilder, deduplicates, and emits the result--btf-merge/--no-btf-mergeflags (off by default)-r(relocatable linking)Tests (86 unit tests + lld LIT tests):
-r,--gc-sections, COMDATisLive(), big-endianAlgorithm notes
The dedup algorithm follows @anakryiko's design from libbpf (writeup). The composite type pass uses a hypothesis map for DFS graph equivalence, which correctly handles recursive types (linked lists, trees, etc.). I chose to put the library code in
llvm/lib/DebugInfo/BTF/rather than inline in lld so other tools (llvm-objcopy, llvm-readobj) can reuse it.I'd appreciate review from @MaskRay on the lld integration side — I followed the
DebugNamesSectionpattern. And @anakryiko / @yonghong-song for the dedup algorithm correctness, since you wrote the original.cc @4ast