DIY Sanitizer: How to Add Your Own Pass in LLVM

Zheng Yu lucky
  2023-02-01 11:25:50 2023-02-01 11:25 Created   2024-02-26 08:11:09 2024-02-26 08:11 Updated      

In this passage, I will demonstrate how to integrate your pass into LLVM. The new pass is called “OverflowDefense”, and it is a simple pass that can be used to defend against buffer overflow attacks. The code of this pass can be found in my github repo . I will not go into the details of the pass implementation, but we will focus on how to integrate the pass into LLVM.

Firstly, we need to download the source code of LLVM. The version of LLVM used in this example is 15.0.6, which can be downloaded from here .

Build Command

As stated in the official document , LLVM will no longer support typed pointers in the future, and will instead use opaque pointers. However, the type information of the pointer is crucial for security researchers.

Thankfully, LLVM offers a way to disable the opaque pointer feature in LLVM 15. Hence, to preserve the type information of the pointer, we need to disable the opaque pointer feature in LLVM.

1
2
3
4
cd llvm-project
mkdir build && cd build
cmake -DLLVM_ENABLE_PROJECTS="clang;compiler-rt" -DCMAKE_BUILD_TYPE=Release -DCLANG_ENABLE_OPAQUE_POINTERS=OFF -G "Unix Makefiles" ../llvm
make -j`nproc`

When using LLVM 15, a warning message will appear indicating that the opaque pointer feature is deprecated. If you wish to disable this warning, you can add the following patch to the source code of LLVM.

1
2
3
4
5
6
7
8
9
10
11
12
--- a/llvm-project/llvm/include/llvm/IR/Type.h
+++ b/llvm-project/llvm/include/llvm/IR/Type.h
@@ -374,9 +374,9 @@ public:
 
   /// This method is deprecated without replacement. Pointer element types are
   /// not available with opaque pointers.
-  [[deprecated("Deprecated without replacement, see "
-               "https://llvm.org/docs/OpaquePointers.html for context and "
-               "migration instructions")]]
   Type *getPointerElementType() const {
     return getNonOpaquePointerElementType();
   }

Pass Implementation

To integrate the OverflowDefense Pass into LLVM, the following steps need to be taken:

  • Add the OverflowDefense.cpp file to the llvm-project/llvm/lib/Transforms/Instrumentation/ directory and the OverflowDefense.h header file to the llvm-project/llvm/include/llvm/Transforms/Instrumentation/ directory.
  • Implement the new pass manager since LLVM 14+ has added it. The new pass code is simpler than the old pass code.
  • Add your pass source code to the CMakeLists.txt directory.
  • Apply some patches to the source code of LLVM to enable the fsanitize=overflow-defense option and provide arguments to the pass.

Pass Code Framework

Because my pass need to instrument the code, I need to add my pass code to the llvm-project/llvm/lib/Transforms/Instrumentation/OverflowDefense.cpp file and add header file llvm-project/llvm/include/llvm/Transforms/Instrumentation/OverflowDefense.h. Here is the code of my header file.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
//===- Transform/Instrumentation/OverflowDefense.h - Overflow Defense -----===//

#ifndef LLVM_TRANSFORMS_INSTRUMENTATION_OVERFLOWDEFENSE_H
#define LLVM_TRANSFORMS_INSTRUMENTATION_OVERFLOWDEFENSE_H

#include "llvm/IR/PassManager.h"

namespace llvm {

struct OverflowDefenseOptions {
  OverflowDefenseOptions() : OverflowDefenseOptions(false, false){};
  OverflowDefenseOptions(bool Kernel, bool Recover);
  bool Kernel;
  bool Recover;
};

struct OverflowDefensePass : public PassInfoMixin<OverflowDefensePass> {
  OverflowDefensePass(OverflowDefenseOptions Options) : Options(Options) {}
  PreservedAnalyses run(Function &F, FunctionAnalysisManager &FAM);
  static bool isRequired() { return true; }

private:
  OverflowDefenseOptions Options;
};

struct ModuleOverflowDefensePass
    : public PassInfoMixin<ModuleOverflowDefensePass> {
  ModuleOverflowDefensePass(OverflowDefenseOptions Options)
      : Options(Options) {}
  PreservedAnalyses run(Module &M, ModuleAnalysisManager &AM);
  static bool isRequired() { return true; }

private:
  OverflowDefenseOptions Options;
};
} // namespace llvm

#endif // LLVM_TRANSFORMS_INSTRUMENTATION_OVERFLOWDEFENSE_H

In LLVM 14+, the llvm add the new pass manager, so we need to implement the new pass manager. The new pass code is much simpler than the old pass code. Here is the code of my pass. If you want to implement a function pass, you can refer to the code of run(Function &F, FunctionAnalysisManager &FAM) function as below:

1
2
3
4
5
6
7
PreservedAnalyses OverflowDefensePass::run(Function &F,
                                           FunctionAnalysisManager &FAM) {
  OverflowDefense Odef(*F.getParent(), Options);
  if (Odef.sanitizeFunction(F, FAM))
    return PreservedAnalyses::none();
  return PreservedAnalyses::all();
}

Compile Options

  • Add your pass source code to the CMakeLists.txt directory.
1
2
3
4
5
6
7
8
9
10
--- a/llvm-project/llvm/lib/Transforms/Instrumentation/CMakeLists.txt
+++ b/llvm-project/llvm/lib/Transforms/Instrumentation/CMakeLists.txt
@@ -7,6 +7,7 @@ add_llvm_component_library(LLVMInstrumentation
   GCOVProfiling.cpp
   MemProfiler.cpp
   MemorySanitizer.cpp
+  OverflowDefense.cpp
   IndirectCallPromotion.cpp
   Instrumentation.cpp
   InstrOrderFile.cpp
  • I hope that I can use fsanitize=overflow-defense to enable my pass, so I need to apply the following patchs to the source code of LLVM.
    • Add overflow-defense feature to Sanitizers.def and Features.def.
      1
      2
      3
      4
      5
      6
      7
      8
      9
      10
      11
      12
      13
      14
      --- a/llvm-project/clang/include/clang/Basic/Sanitizers.def
      +++ b/llvm-project/clang/include/clang/Basic/Sanitizers.def
      @@ -67,6 +67,12 @@ SANITIZER("memory", Memory)
      // Kernel MemorySanitizer (KMSAN)
      SANITIZER("kernel-memory", KernelMemory)

      +// OverflowDefense
      +SANITIZER("overflow-defense", OverflowDefense)
      +
      +// Kernel OverflowDefense (KOD)
      +SANITIZER("kernel-overflow-defense", KernelOverflowDefense)
      +
      // libFuzzer
      SANITIZER("fuzzer", Fuzzer)
      1
      2
      3
      4
      5
      6
      7
      8
      9
      10
      11
      12
      --- a/llvm-project/clang/include/clang/Basic/Features.def
      +++ b/llvm-project/clang/include/clang/Basic/Features.def
      @@ -96,6 +96,9 @@ FEATURE(nullability_nullable_result, true)
      FEATURE(memory_sanitizer,
              LangOpts.Sanitize.hasOneOf(SanitizerKind::Memory |
                                          SanitizerKind::KernelMemory))
      +FEATURE(overflow_defense,
      +        LangOpts.Sanitize.hasOneOf(SanitizerKind::OverflowDefense |
      +                                   SanitizerKind::KernelOverflowDefense))
      FEATURE(thread_sanitizer, LangOpts.Sanitize.has(SanitizerKind::Thread))
      FEATURE(dataflow_sanitizer, LangOpts.Sanitize.has(SanitizerKind::DataFlow))
      FEATURE(scudo, LangOpts.Sanitize.hasOneOf(SanitizerKind::Scudo))

How LLVM Call My Pass

These patch is used to tell the compiler how to use my pass and what arguments my pass need. My patch modifies the PassBuilder.cpp file in the LLVM source code to add support for the OverflowDefense pass. It defines a function named parseOdefPassOptions that parses the options for the OverflowDefensepass. These options includerecoverandkernel`, which are used to enable specific functionalities in the pass.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
--- a/llvm-project/llvm/lib/Passes/PassBuilder.cpp
+++ b/llvm-project/llvm/lib/Passes/PassBuilder.cpp
@@ -136,6 +136,7 @@
 #include "llvm/Transforms/Instrumentation/InstrProfiling.h"
 #include "llvm/Transforms/Instrumentation/MemProfiler.h"
 #include "llvm/Transforms/Instrumentation/MemorySanitizer.h"
+#include "llvm/Transforms/Instrumentation/OverflowDefense.h"
 #include "llvm/Transforms/Instrumentation/PGOInstrumentation.h"
 #include "llvm/Transforms/Instrumentation/PoisonChecking.h"
 #include "llvm/Transforms/Instrumentation/SanitizerCoverage.h"
@@ -688,6 +689,26 @@ Expected<MemorySanitizerOptions> parseMSanPassOptions(StringRef Params) {
   return Result;
 }
 
+Expected<OverflowDefenseOptions>
+parseOdefPassOptions(StringRef Params) {
+  OverflowDefenseOptions Result;
+  while (!Params.empty()) {
+    StringRef ParamName;
+    if (ParamName == "recover") {
+      Result.Recover = true;
+    } else if (ParamName == "kernel") {
+      Result.Kernel = true;
+    } else {
+      return make_error<StringError>(
+          formatv("invalid OverflowDefense pass parameter '{0}' ", ParamName)
+              .str(),
+          inconvertibleErrorCode());
+    }
+  }
+
+  return Result;
+}
+
 /// Parser of parameters for SimplifyCFG pass.
 Expected<SimplifyCFGOptions> parseSimplifyCFGOptions(StringRef Params) {
   SimplifyCFGOptions Result;

Runtime Library

Some passes may require runtime support to function properly. For example, AddressSanitizer requires a runtime library to intercept memory accesses and report errors. The runtime library is implemented in the llvm-project/compiler-rt directory. The runtime library is compiled into a static library. The compiler will link this library into the final executable. Now I need to implement the runtime library for my pass.

  • Add your runtime library code to the llvm-project/compiler-rt/lib/ directory. You can refer to the llvm-project/compiler-rt/lib/msan or other directory for the implementation of the runtime library.

  • Add code in CMakeLists.txt to tell the compiler which runtime library to build.

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    --- a/llvm-project/compiler-rt/test/CMakeLists.txt
    +++ b/llvm-project/compiler-rt/test/CMakeLists.txt
    @@ -79,7 +79,7 @@ if(COMPILER_RT_CAN_EXECUTE_TESTS)
     
         foreach(sanitizer ${COMPILER_RT_SANITIZERS_TO_BUILD})
           # cfi testing is gated on ubsan
    -      if(NOT ${sanitizer} STREQUAL cfi)
    +      if(NOT ${sanitizer} STREQUAL cfi AND NOT ${sanitizer} STREQUAL odef)
             compiler_rt_test_runtime(${sanitizer})
           endif()
         endforeach()

  • Add code in SanitizerArgs.cpp to enable the runtime library when compiling with fsanitize=overflow-defense.

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    --- a/llvm-project/clang/include/clang/Driver/SanitizerArgs.h
    +++ b/llvm-project/clang/include/clang/Driver/SanitizerArgs.h
    @@ -83,6 +83,7 @@ public:
       }
       bool needsTsanRt() const { return Sanitizers.has(SanitizerKind::Thread); }
       bool needsMsanRt() const { return Sanitizers.has(SanitizerKind::Memory); }
    +  bool needsOdefRt() const { return Sanitizers.has(SanitizerKind::OverflowDefense); }
       bool needsFuzzer() const { return Sanitizers.has(SanitizerKind::Fuzzer); }
       bool needsLsanRt() const {
         return Sanitizers.has(SanitizerKind::Leak) &&

  • Add code in CommonArgs.cpp to tell the compiler which runtime library to link.

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    --- a/llvm-project/clang/lib/Driver/ToolChains/CommonArgs.cpp
    +++ b/llvm-project/clang/lib/Driver/ToolChains/CommonArgs.cpp
    @@ -946,6 +946,11 @@ collectSanitizerRuntimes(const ToolChain &TC, const ArgList &Args,
         if (SanArgs.linkCXXRuntimes())
           StaticRuntimes.push_back("msan_cxx");
       }
    +  if (SanArgs.needsOdefRt() && SanArgs.linkRuntimes()) {
    +    StaticRuntimes.push_back("odef");
    +    if (SanArgs.linkCXXRuntimes())
    +      StaticRuntimes.push_back("odef_cxx");
    +  }
       if (!SanArgs.needsSharedRt() && SanArgs.needsTsanRt() &&
           SanArgs.linkRuntimes()) {
         StaticRuntimes.push_back("tsan");

Other Details

Pass calling sequence

Some passes need to be called before the optimization passes, and some passes need to be called after the optimization passes. You can refer to the llvm-project/clang/lib/CodeGen/BackendUtil.cpp file to see how to call the pass.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
--- a/llvm-project/clang/lib/CodeGen/BackendUtil.cpp
+++ b/llvm-project/clang/lib/CodeGen/BackendUtil.cpp
@@ -72,6 +72,7 @@
 #include "llvm/Transforms/Instrumentation/InstrProfiling.h"
 #include "llvm/Transforms/Instrumentation/MemProfiler.h"
 #include "llvm/Transforms/Instrumentation/MemorySanitizer.h"
+#include "llvm/Transforms/Instrumentation/OverflowDefense.h"
 #include "llvm/Transforms/Instrumentation/SanitizerCoverage.h"
 #include "llvm/Transforms/Instrumentation/ThreadSanitizer.h"
 #include "llvm/Transforms/ObjCARC.h"
@@ -660,6 +660,23 @@ static void addSanitizers(const Triple &TargetTriple,
     MSanPass(SanitizerKind::Memory, false);
     MSanPass(SanitizerKind::KernelMemory, true);
 
+    auto ODefPass = [&](SanitizerMask Mask, bool CompileKernel) {
+      if (LangOpts.Sanitize.has(Mask)) {
+        bool Recover = CodeGenOpts.SanitizeRecover.has(Mask);
+        OverflowDefenseOptions Opts(Recover, CompileKernel);
+
+        MPM.addPass(ModuleOverflowDefensePass(Opts));
+        FunctionPassManager FPM;
+        FPM.addPass(OverflowDefensePass(Opts));
+        if (Level != OptimizationLevel::O0) {
+          FPM.addPass(EarlyCSEPass());
+        }
+        MPM.addPass(createModuleToFunctionPassAdaptor(std::move(FPM)));
+      }
+    };
+    ODefPass(SanitizerKind::OverflowDefense, false);
+    ODefPass(SanitizerKind::KernelOverflowDefense, true);
+
     if (LangOpts.Sanitize.has(SanitizerKind::Thread)) {
       MPM.addPass(ModuleThreadSanitizerPass());
       MPM.addPass(createModuleToFunctionPassAdaptor(ThreadSanitizerPass()));

Architecture support

The following patch adds support for the x86_64 architecture. You can refer to the llvm-project/clang/lib/Driver/ToolChains/Linux.cpp file to see how to add support for other architectures.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
--- a/llvm-project/clang/lib/Driver/ToolChains/Linux.cpp
+++ b/llvm-project/clang/lib/Driver/ToolChains/Linux.cpp
@@ -758,8 +758,10 @@ SanitizerMask Linux::getSupportedSanitizers() const {
     Res |= SanitizerKind::Leak;
   if (IsX86_64 || IsMIPS64 || IsAArch64 || IsPowerPC64 || IsSystemZ)
     Res |= SanitizerKind::Thread;
-  if (IsX86_64)
+  if (IsX86_64) {
     Res |= SanitizerKind::KernelMemory;
+    Res |= SanitizerKind::OverflowDefense;
+  }
   if (IsX86 || IsX86_64)
     Res |= SanitizerKind::Function;
   if (IsX86_64 || IsMIPS64 || IsAArch64 || IsX86 || IsMIPS || IsArmArch ||
1
2
3
4
5
6
7
8
9
10
--- a/llvm-project/compiler-rt/cmake/Modules/AllSupportedArchDefs.cmake
+++ b/llvm-project/compiler-rt/cmake/Modules/AllSupportedArchDefs.cmake
@@ -55,6 +55,7 @@ else()
       ${PPC64} ${S390X} ${RISCV64} ${HEXAGON})
 endif()
 set(ALL_MSAN_SUPPORTED_ARCH ${X86_64} ${MIPS64} ${ARM64} ${PPC64} ${S390X})
+set(ALL_ODEF_SUPPORTED_ARCH ${X86_64})
 set(ALL_HWASAN_SUPPORTED_ARCH ${X86_64} ${ARM64})
 set(ALL_MEMPROF_SUPPORTED_ARCH ${X86_64})
 set(ALL_PROFILE_SUPPORTED_ARCH ${X86} ${X86_64} ${ARM32} ${ARM64} ${PPC32} ${PPC64}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
--- a/llvm-project/compiler-rt/cmake/config-ix.cmake
+++ b/llvm-project/compiler-rt/cmake/config-ix.cmake
@@ -583,6 +583,9 @@ if(APPLE)
   list_intersect(MSAN_SUPPORTED_ARCH
     ALL_MSAN_SUPPORTED_ARCH
     SANITIZER_COMMON_SUPPORTED_ARCH)
+  list_intersect(ODEF_SUPPORTED_ARCH
+    ALL_ODEF_SUPPORTED_ARCH
+    SANITIZER_COMMON_SUPPORTED_ARCH)
   list_intersect(HWASAN_SUPPORTED_ARCH
     ALL_HWASAN_SUPPORTED_ARCH
     SANITIZER_COMMON_SUPPORTED_ARCH)
@@ -638,6 +641,7 @@ else()
   filter_available_targets(DFSAN_SUPPORTED_ARCH ${ALL_DFSAN_SUPPORTED_ARCH})
   filter_available_targets(LSAN_SUPPORTED_ARCH ${ALL_LSAN_SUPPORTED_ARCH})
   filter_available_targets(MSAN_SUPPORTED_ARCH ${ALL_MSAN_SUPPORTED_ARCH})
+  filter_available_targets(ODEF_SUPPORTED_ARCH ${ALL_ODEF_SUPPORTED_ARCH})
   filter_available_targets(HWASAN_SUPPORTED_ARCH ${ALL_HWASAN_SUPPORTED_ARCH})
   filter_available_targets(MEMPROF_SUPPORTED_ARCH ${ALL_MEMPROF_SUPPORTED_ARCH})
   filter_available_targets(PROFILE_SUPPORTED_ARCH ${ALL_PROFILE_SUPPORTED_ARCH})
@@ -686,7 +690,7 @@ if(COMPILER_RT_SUPPORTED_ARCH)
 endif()
 message(STATUS "Compiler-RT supported architectures: ${COMPILER_RT_SUPPORTED_ARCH}")
 
-set(ALL_SANITIZERS asan;dfsan;msan;hwasan;tsan;safestack;cfi;scudo;ubsan_minimal;gwp_asan)
+set(ALL_SANITIZERS asan;dfsan;msan;hwasan;tsan;safestack;cfi;scudo;ubsan_minimal;gwp_asan;odef)
 set(COMPILER_RT_SANITIZERS_TO_BUILD all CACHE STRING
     "sanitizers to build if supported on the target (all;${ALL_SANITIZERS})")
 list_replace(COMPILER_RT_SANITIZERS_TO_BUILD all "${ALL_SANITIZERS}")
@@ -741,6 +745,13 @@ else()
   set(COMPILER_RT_HAS_MSAN FALSE)
 endif()
 
+if (COMPILER_RT_HAS_SANITIZER_COMMON AND ODEF_SUPPORTED_ARCH AND
+    OS_NAME MATCHES "Linux")
+  set(COMPILER_RT_HAS_ODEF TRUE)
+else()
+  set(COMPILER_RT_HAS_ODEF FALSE)
+endif()
+
 if (COMPILER_RT_HAS_SANITIZER_COMMON AND HWASAN_SUPPORTED_ARCH AND
     OS_NAME MATCHES "Linux|Android|Fuchsia")
   set(COMPILER_RT_HAS_HWASAN TRUE)

Build the compiler

After you have applied the patches, you can build the compiler as usual and then you should be able to use the overflow-defense sanitizer.

1
clang -g -O2 -fsanitize=overflow-defense my_program.c -o my_program
  • Post title：DIY Sanitizer: How to Add Your Own Pass in LLVM
  • Post author：Zheng Yu
  • Create time：2023-02-01 11:25:50
  • Post link：https://dataisland.org/2023/02/01/add-llvm-sanitizer/
  • Copyright Notice：All articles in this blog are licensed under BY-NC-SA unless stating additionally.
On this page
DIY Sanitizer: How to Add Your Own Pass in LLVM
  1. Build Command
  2. Pass Implementation
    1. Pass Code Framework
    2. Compile Options
    3. How LLVM Call My Pass
  3. Runtime Library
  4. Other Details
    1. Pass calling sequence
    2. Architecture support
  5. Build the compiler