dextool ~cpptooling-fix-assert-bug

dextool

Dextool is a framework for writing plugins using libclang. The main focus is tools for testing and static analysis.

The plugins in a standard installation of Dextool are:

• Analyze. Analyze C/C++ code to generate complexity numbers such as McCabe.
• C TestDouble. Analyze C code to generate a test double implementation.
• C++ TestDouble. Analyze C++ code to generate a test double implementation.
• Mutate. Mutation testing tool for C/C++.
• UML. Analyze C/C++ code to generate PlantUML diagrams.

Plugin Status

• Analyze: production ready.
• C TestDouble: production ready. The API of the generated code and how it behaves is stable.
• C++ TestDouble: production ready. The API of the generated code and how it behaves is stable.
• Mutate: production ready.
• UML: beta.

Installation

Dependencies

To build and run dextool, you will need the following packages:

• llvm >= 4.0 Tested versions are 4.0, 5.0, 6.0, 7.0, 8.0, 10.0, 11.0, 12.0, 13.0, 14.0. Note that these are the versions that are tested or have been tested. Dextool usually am able to work with never versions so if you version isn't in this list then please just try it. Otherwise leave an issue on github.
• llvm-xyz-dev >= 4.0
• libclang-xyz-dev >= 4.0
• cmake >= 3.5
• sqlite3 >= 3.24.0
• D compiler. See the following files for the tested versions. They are guaranteed to work.
• dmd minimal version
• dmd max version
• ldc minimal version
• ldc max version

NOTE only ldc is able to build a release build of dextool. dmd is for debug build. Which mean that for a normal user it is ldc that you should use.

Most of them can be installed using your package manager.

Installation instructions for Ubuntu is available here. Dockerfile are another alternative (they MAY be outdated until the CI is fixed to check them so if they fail for you ping me on github):

• Ubuntu
• Fedora

Build and Install

The easiest way to build and run dextool is to do it via dub:

dub run dextool -- -h

But if you want to, these are the steps to build it using CMake:

git clone https://github.com/joakim-brannstrom/dextool.git
cd dextool
mkdir build
cd build
cmake -DCMAKE_INSTALL_PREFIX=/path/to/where/to/install/dextool/binaries ..
make install

Done! Have fun. Don't be shy to report any issue that you find.

Common Build Errors

ldc is killed

To build dextool comfortably you need ~16Gbyte of RAM. Compiling dextool-mutate.o consume ~9Gbyte.

You probably ran out of memory when compiling dextool-mutate.o. Run cmake with -DLOW_MEM=ON and single threaded.

dub run dextool fail

This method of running dextool assume that the llvm-config commands found in $PATH is named and behave as the ones on Ubuntu. Such that llvm-config-12 returns the flags for llvm-12 and clang-12. There are also some assumptions of what the libraries that are installed are named. On Fedora for example the clang-libraries are clang-cpp and clang. So if this method of running dextool fail and you are on ubuntu then you need to install the necessary dependencies otherwise use the other installation method together with the more complex flags for specifying the llvm/clang versions.

component_tests Fail

The most common reason for why component_tests fail is that clang++ try to use the latest GCC that is installed, but the c++ standard library is not installed for that compiler.

Try to compile the following code with clang++:

#include <string>

int main(int argc, char **argv) {
    return 0;
}

clang++ -v test.cpp

If it fails with something like this:

test.cpp:1:10: fatal error: 'string' file not found

it means that you need to install the c++ standard library for your compiler.

In the output look for this line:

 /usr/bin/../lib/gcc/x86_64-linux-gnu/XYZ/../../../../include/c++

From that line we can deduce that the package to install in Ubuntu is:

sudo apt install libstdc++-XYZ-dev

Mismatch Clang and LLVM

To build dextool, the dev packages are required. Dextool is optimistic and assume that the latest and greatest version of llvm+libclang should be used. But this also requires that the dev packages are installed.

If you get this error:

libclang_interop.hpp:13:10: fatal error: clang/Analysis/CFG.h: No such file or directory
 #include <clang/Analysis/CFG.h>

It means that you need to install llvm-x.y-dev and libclang-x.y-dev for the version that Dextool detected.

Bypass llvm-config

cmake/introspect_llvm_.d try to derive the version of llvm/clang from llvm-config. If this fail or you want to force a specific version of llvm/clang you do it with these flags:

The following variables are defined:

• LIBCLANG_LDFLAGS - flags to use when linking with libclang
• LIBLLVMVERSION - libLLVM full version (e.g. 80_1)
• LIBLLVMMAJORVERSION - libLLVM major version (e.g. 8)
• LIBLLVM_LDFLAGS - flags to use when linking with libllvm
• LIBLLVMCXXFLAGS - the required flags to build C++ code using LLVM
• LIBLLVMCXXEXTRA_FLAGS - extra flags to use when build C++ code using LLVM
• LIBLLVM_FLAGS - the required flags by llvm-d such as version
• LIBLLVM_LIBS - the required libraries for linking LLVM

Lets say you have version 8.0.1 of LLVM installed but llvm-config returns 0.0.0. introspect_llvm.d will in this case fail to detect the version of LLVM thus what version of the bindings is unknown, which will default to the latest known by introspect_llvm.d. This is probably not the version you have. To tell cmake what version it is you can do the following:

# llvm-config --version
cmake -DLIBLLVM_VERSION="LLVM_8_0_1" \
-DLIBLLVM_MAJOR_VERSION="8"

If you also need to provide the includes and libs you would need to add the rest of the flags, otherwise they are derived from whatever llvm-config that is in $PATH.

# uses llvm-config --libdir to find where libclang.so is installed. Some
# additional flags are added but these are optional.
-DLIBLCANG_LDFLAGS="-Wl,--enable-new-dtags -Wl,--no-as-needed -L/foo/bar/libs -Wl,-rpath,/foo/bar/libs -l:libclang.so.8"
# llvm-config --libdir is searched for these libraries.
-DLIBCLANG_LIBS="-lclangFrontendTool -lclangRewriteFrontend -lclangDynamicASTMatchers -lclangFrontend -lclangASTMatchers -lclangParse -lclangSerialization -lclangRewrite -lclangSema -lclangEdit -lclangAnalysis -lclangAST -lclangLex -lclangBasic -l:libclang.so"
# llvm-config --cxxflags
-DLIBLLVM_CXX_FLAGS="-I/foo/bar/llvm-include -std=c++11 -fno-exceptions -fno-rtti"
# llvm-config --ldflags
-DLIBLLVM_LDFLAGS="-L/foo/smurf/libs -Wl,-rpath,/foo/smurf/libs"
# use llvm-config --libs and llvm-config --system-libs to find all libraries to link with.
# all those that are prefixed with libLLVM.
-DLIBLLVM_LIBS="-lLLVMXRay -lLLVMTextApi /*and maaaany more or just one depending on how you have installed LLVM*/"

SQLite link or missing

The sqlite3 library source code with a CMake build file in the vendor's directory. It is intended for those old OSs that have too old versions of SQLite.

To use it do something like this.

mkdir sqlite3
cd sqlite3 && cmake ../vendor/sqlite && make && cd ..
# setup dextool build to use it
mkdir build
cd build && cmake .. -DSQLITE3_LIB="-L/opt/sqlite -lsqlite3"

Cmake is unable to find the D compiler

If you have a D compiler installed in such a way that it isn't available in your $PATH you can specify it manually.

cmake .. -DD_COMPILER=/foo/bar/dmd/2.088/linux64/bin/dmd

My version of LLVM is not offically supported / do not work

First of all just test and see if dextool do build with your version of llvm. The tool introspect_llvm.d reuse the latest bindings if a new version is detected but no bindings exists. The C API rarely change so it normally just works.

If it doesn't then follow these steps:

• Follow the instructions in libs/libclang/README.md to generate the new bindings.
• Open the file cmake/introspect_llvm.d.
• Update the array of search paths in the function llvmSearchPaths with the new location.

Usage

See the usage examples in respective plugin directory.

Credit

Jacob Carlborg for his excellent DStep. It was used as a huge inspiration for this code base. Without DStep, Dextool wouldn't exist.