Using the DragonEgg plugin

As explained earlier, LLVM started as a project that was dependent on GCC when it still lacked its own C/C++ frontend. In those instances, to use LLVM, you needed to download a hacked GCC source tree called llvm-gcc and compile it in its entirety. Since the compilation involved the full GCC package, it was a very time-consuming and tricky task, requiring knowledge of all the necessary GNU lore to rebuild GCC by yourself. The DragonEgg project appeared as a clever solution to leverage the GCC plugin system, separating the LLVM logic in its own and much smaller code tree. In this way, the users no longer needed to rebuild the entire GCC package, but just a plugin, and then load it into GCC. DragonEgg is also the sole project under the LLVM project umbrella that is licensed under GPL.

Even with the rise of Clang, DragonEgg persists today because Clang only handles the C and C++ languages, while GCC is able to parse a wider variety of languages. By using the DragonEgg plugin, you can use GCC as a frontend to the LLVM compiler, being able to compile most of the languages supported by GCC: Ada, C, C++, and FORTRAN, with partial support for Go, Java, Obj-C, and Obj-C++.

The plugin acts by substituting the middle- and backend of GCC with the LLVM ones and performs all the compilation steps automatically, as you would expect from a first-class compiler driver. The compilation pipeline for this new scenario is represented in the following illustration:

If you wish, you can use the -fplugin-arg-dragonegg-emit-ir -S set of flags to stop the compilation pipeline at the LLVM IR generation phase and use LLVM tools to analyze and investigate the result of the frontend, or use the LLVM tools to finish the compilation yourself. We will see an example shortly.

As it is an LLVM side project, maintainers do not update DragonEgg at the same pace as the LLVM main project. The most recent stable version of DragonEgg at the time of this writing was version 3.3, which is bound to the toolset of LLVM 3.3. Therefore, if you generate LLVM bitcodes, that is, programs written on disk by using the LLVM IR, you cannot use LLVM tools of a version other than 3.3 to analyze this file, optimize, or proceed with the compilation. You can find the official DragonEgg website at http://dragonegg.llvm.org.

Building DragonEgg

To compile and install DragonEgg, first get the source from commands:

$ wget http://llvm.org/releases/3.3/dragonegg-3.3.src.tar.gz. 
$ tar xzvf dragonegg-3.3.src.tar.gz
$ cd dragonegg-3.3.src

If you wish to explore the current but unstable sources from SVN, use the following command:

$ svn checkout http://llvm.org/svn/llvm-project/dragonegg/trunk dragonegg

For the GIT mirror, use the following:

$ git clone http://llvm.org/git/dragonegg.git

To compile and install, you need to provide the LLVM installation path. The LLVM version must match the version of DragonEgg being installed. Assuming the same install prefix, /usr/local/llvm, from Chapter 1, Build and Install LLVM, and assuming GCC 4.6 is installed and present in your shell PATH variable, you should use the following commands:

$ GCC=gcc-4.6 LLVM_CONFIG=/usr/local/llvm/bin/llvm-config make
$ cp –a dragonegg.so /usr/local/llvm/lib

Note that the project lacks autotools or CMake project files. You should build directly by using the make command. If your gcc command already supplies the correct GCC version that you want to use, you can omit the GCC=gcc-4.6 prefix when running make. The plugin is the resulting shared library named dragonegg.so, and you can invoke it using the following GCC command line. Consider that you are compiling a classic "Hello, World!" C code.

$ gcc-4.6 –fplugin=/usr/local/llvm/lib/dragonegg.so hello.c –o hello

Tip

Although DragonEgg theoretically supports GCC version 4.5 and higher, GCC 4.6 is highly recommended. DragonEgg is not extensively tested and maintained in other GCC versions.

Understanding the compilation pipeline with DragonEgg and LLVM tools

If you wish to see the frontend in action, use the -S -fplugin-arg-dragonegg-emit-ir flag, which will emit a human-readable file with the LLVM IR code:

$ gcc-4.6 -fplugin=/usr/local/llvm/lib/dragonegg.so -S -fplugin-arg-dragonegg-emit-ir hello.c -o hello.ll
$ cat hello.ll

The ability to stop the compilation once the compiler translates the program to IR and serializing the in-memory representation to disk is a particular characteristic of LLVM. Most other compilers are unable to do this. After appreciating how LLVM IR represents your program, you can manually proceed with the compilation process by using several LLVM tools. The following command invokes a special assembler that converts LLVM in textual form to binary form, still stored on disk:

$ llvm-as hello.ll -o hello.bc
$ file hello.bc
hello.bc: LLVM bitcode

If you want, you can translate it back to human-readable form by using a special IR disassembler (llvm-dis). The following tool will apply target-independent optimizations while displaying to you statistics about successful code transformations:

$ opt -stats hello.bc -o hello.bc

The -stats flag is optional. Afterwards, you can use the LLVM backend tool to translate it to target-machine assembly language:

$ llc -stats hello.bc -o hello.S

Again, the -stats flag is optional. Since it is an assembly file, you can use either your GNU binutils assembler or the LLVM assembler. In the following command, we will use the LLVM assembler:

$ llvm-mc -filetype=obj hello.S -o hello.o

LLVM defaults to use your system linker because the LLVM linker project, lld, is currently in development and is not integrated into the core LLVM project. Therefore, if you do not have lld, you can finish the compilation by using your regular compiler driver, which will activate your system linker:

$ gcc hello.o -o hello

Keep in mind that, for performance reasons, the real LLVM compiler driver never serializes the program representation to disk in any stage, except for the object file, since it still lacks an integrated linker. It uses the in-memory representation and coordinates several LLVM components to carry on compilation.

Understanding the LLVM test suite

The LLVM test suite consists of an official set of programs and benchmarks used to test the LLVM compiler. The test suite is very useful to LLVM developers, which validates optimizations and compiler improvements by compiling and running such programs. If you are using an unstable release of LLVM, or if you hacked into LLVM sources and suspect that something is not working as it should, it is very useful to run the test suite by yourself. However, keep in mind that simpler LLVM regression and unit tests live in the LLVM main tree, and you can easily run them with make check-all. The test suite differs from the classic regression and unit tests because it contains entire benchmarks.

You must place the LLVM test suite in the LLVM source tree to allow the LLVM build system to recognize it. You can find the sources for version 3.4 at http://llvm.org/releases/3.4/test-suite-3.4.src.tar.gz.

To fetch the sources, use the following commands:

$ wget http://llvm.org/releases/3.4/test-suite-3.4.src.tar.gz
$ tar xzf test-suite-3.4.src.tar.gz
$ mv test-suite-3.4 llvm/projects/test-suite

If you otherwise prefer downloading it via SVN to get the most recent and possibly unstable version, use the following:

$ cd llvm/projects
$ svn checkout http://llvm.org/svn/llvm-project/test-suite/trunk test-suite

If you prefer GIT instead, use the following commands:

$ cd llvm/projects
$ git clone http://llvm.org/git/llvm-project/test-suite.git

You need to regenerate the build files of LLVM to use the test suite. In this special case, you cannot use CMake. You must stick with the classic configure script to work with the test suite. Repeat the configuration steps described in Chapter 1, Build and Install LLVM.

The test suite has a set of Makefiles that test and check benchmarks. You can also provide a custom Makefile that evaluates custom programs. Place the custom Makefile in the test suite's source directory using the naming template llvm/projects/test-suite/TEST.<custom>.Makefile, where the <custom> tag must be replaced by any name you want. Check llvm/projects/test-suite/TEST.example.Makefile for an example.

Tip

You need to regenerate LLVM build files to allow for a custom or changed Makefile to work.

During configuration, a directory for the test suite is created in the LLVM object directory where programs and benchmarks will run. To run and test the example Makefile, enter the object directory path from Chapter 1, Build and Install LLVM, and execute the following command lines:

$ cd your-llvm-build-folder/projects/test-suite
$ make TEST="example" report

Using LLDB

The LLDB (Low Level Debugger) project is a debugger built with the LLVM infrastructure, being actively developed and shipped as the debugger of Xcode 5 on Mac OS X. Since its development began in 2011, outside the scope of Xcode, LLDB had not yet released a stable version until the time of this writing. You can obtain LLDB sources at http://llvm.org/releases/3.4/lldb-3.4.src.tar.gz. Like many projects that depend on LLVM, you can easily build it by integrating it in the LLVM build system. To accomplish this, just put its source code in the LLVM tools folder, as in the following example:

$ wget http://llvm.org/releases/3.4/lldb-3.4.src.tar.gz
$ tar xvf lldb-3.4.src.tar.gz
$ mv lldb-3.4 llvm/tools/lldb

You can alternatively use its SVN repository to get the latest revision:

$ cd llvm/tools
$ svn checkout http://llvm.org/svn/llvm-project/lldb/trunk lldb

If you prefer, you can use its GIT mirror instead:

$ cd llvm/tools
$ git clone http://llvm.org/git/llvm-project/lldb.git

Note

LLDB is still experimental for GNU/Linux systems.

Before building it, note that LLDB has some software prerequisites: Swig, libedit (only for Linux), and Python. On Ubuntu systems, for example, you can solve these dependencies with the following command:

$ sudo apt-get install swig libedit-dev python

Remember that, as with other projects presented in this chapter, you need to regenerate LLVM build files to allow for LLDB compilation. Follow the same steps for building LLVM from source that we saw in Chapter 1, Build and Install LLVM.

To perform a simple test on your recent lldb installation, just run it with the -v flag to print its version:

$ lldb -v
lldb version 3.4 ( revision )

Exercising a debug session with LLDB

To see how it looks to use LLDB, we will start a debug session to analyze the Clang binary. The Clang binary contains many C++ symbols you can inspect. If you compiled the LLVM/Clang project with the default options, you have a Clang binary with debug symbols. This happens when you omit the --enable-optimized flag when running the configure script to generate LLVM Makefiles, or use -DCMAKE_BUILD_TYPE="Debug" when running CMake, which is the default build type.

If you are familiar with GDB, you may be interested in referring to the table at http://lldb.llvm.org/lldb-gdb.html, which maps common GDB commands to the LLDB counterpart.

In the same way as GDB, we start LLDB by passing as a command-line argument the path to the executable we want to debug:

$ lldb where-your-llvm-is-installed/bin/clang
Current executable set to 'where-your-llvm-is-installed/bin/clang' (x86_64).
(lldb) break main
Breakpoint 1: where = clang`main + 48 at driver.cpp:293, address = 0x000000001000109e0

To start debugging, we provide the command-line arguments to the Clang binary. We will use the -v argument, which should print the Clang version:

(lldb) run -v

After LLDB hits our breakpoint, feel free to step through each C++ line of code with the next command. As with GDB, LLDB accepts any command abbreviation, such as n instead of next, as long as it stays unambiguous:

(lldb) n

To see how LLDB prints C++ objects, step until you reach the line after declaring the argv or the ArgAllocator object and print it:

(lldb) n
(lldb) p ArgAllocator
(llvm::SpecificBumpPtrAllocator<char>) $0 = {
 Allocator = {
 SlabSize = 4096
 SizeThreshld = 4096
 DefaultSlabAllocator = (Allocator = llvm::MallocAllocator @ 0x00007f85f1497f68)
 Allocator = 0x0000007fffbff200
 CurSlab = 0x0000000000000000
 CurPtr = 0x0000000000000000
 End = 0x0000000000000000
 BytesAllocated = 0
 }
}

After you are satisfied, quit the debugger with the q command:

(lldb) q
Quitting LLDB will kill one or more processes. Do you really want to proceed: [Y/n] y

Introducing the libc++ standard library

The libc++ library is a C++ standard library rewrite for the LLVM project umbrella that supports the most recent C++ standards, including C++11 and C++1y, and that is dual-licensed under the MIT license and the UIUC license. The libc++ library is an important companion of Compiler-RT, being part of the runtime libraries used by Clang++ to build your final C++ executable, along with libclc (the OpenCL runtime library) when necessary. It differs from Compiler-RT because it is not crucial for you to build libc++. Clang is not limited to it and may link your program with the GNU libstdc++ in the absence of libc++. If you have both, you can choose which library Clang++ should use with the -stdlib switch. The libc++ library supports x86 and x86_64 processors and it was designed as a replacement to the GNU libstdc++ for Mac OS X and GNU/Linux systems.

Tip

libc++ support on GNU/Linux is still under way, and is not as stable as the Mac OS X one.

One of the major impediments to continue working in the GNU libstdc++, according to libc++ developers, is that it would require a major code rewrite to support the newer C++ standards, and that the mainline libstdc++ development switched to a GPLv3 license that some companies that back the LLVM project are unable to use. Notice that LLVM projects are routinely used in commercial products in a way that is incompatible with the GPL philosophy. In the face of these challenges, the LLVM community decided to work on a new C++ standard library chiefly for Mac OS X, with support for Linux.

The easiest way to get libc++ in your Apple computer is to install Xcode 4.2 or later.

If you intend to build the library yourself for your GNU/Linux machine, bear in mind that the C++ standard library is composed of the library itself and a lower-level layer that implements functionalities dealing with exception handling and Run-Time Type Information (RTTI). This separation of concerns allow the C++ standard library to be more easily ported to other systems. It also gives you different options when building your C++ standard library. You can build libc++ linked with either libsupc++, the GNU implementation of this lower-level layer, or with libc++abi, the implementation of the LLVM team. However, libc++abi currently only supports Mac OS X systems.

To build libc++ with libsupc++ in a GNU/Linux machine, start by downloading the source packages:

$ wget http://llvm.org/releases/3.4/libcxx-3.4.src.tar.gz
$ tar xvf libcxx-3.4.src.tar.gz
$ mv libcxx-3.4 libcxx

You still could not rely, until the time of this writing, on the LLVM build system to build the library for you as we did with other projects. Therefore, notice that we did not put libc++ sources into the LLVM source tree this time.

Alternatively, the SVN repository with the experimental top-of-trunk version is also available:

$ svn co http://llvm.org/svn/llvm-project/libcxx/trunk libcxx

You can also use the GIT mirror:

$ git clone http://llvm.org/git/llvm-project/libcxx.git

As soon as you have a working LLVM-based compiler, you need to generate the libc++ build files that specifically use your new LLVM-based compiler. In this example, we will assume that we have a working LLVM 3.4 compiler in our path.

To use libsupc++, we first need to find where you have its headers installed in your system. Since it is part of the regular GCC compiler for GNU/Linux, you can discover this by using the following commands:

$ echo | g++ -Wp,-v -x c++ - -fsyntax-only
#include "..." search starts here:
#include <...> search starts here:
/usr/include/c++/4.7.0
/usr/include/c++/4.7.0/x86_64-pc-linux-gnu
(Subsequent entries omitted)

In general, the first two paths indicate where the libsupc++ headers are. To confirm this, look for the presence of a libsupc++ header file such as bits/exception_ptr.h:

$ find /usr/include/c++/4.7.0 | grep bits/exception_ptr.h

Afterwards, generate libc++ build files to compile it with your LLVM-based compiler. To perform this, override the shell CC and CXX environment variables, which define the system C and C++ compilers, respectively, to use the LLVM compiler you want to embed with libc++. To use CMake to build libc++ with libsupc++, you will need to define the CMake parameters LIBCXX_CXX_ABI, which define the lower-level library to use, and LIBCXX_LIBSUPCXX_INCLUDE_PATHS, which is a semicolon-separated list of paths pointing to the folders with the libsupc++ include files that you just discovered:

$ mkdir where-you-want-to-build
$ cd where-you-want-to-build
$ CC=clang CXX=clang++ cmake -DLIBCXX_CXX_ABI=libstdc++ -DLIBCXX_LIBSUPCXX_INCLUDE_PATHS="/usr/include/c++/4.7.0;/usr/include/c++/4.7.0/x86_64-pc-linux-gnu" -DCMAKE_INSTALL_PREFIX="/usr" ../libcxx

At this stage, make sure that ../libcxx is the correct path to reach your libc++ source folder. Run the make command to build the project. Use sudo for the installation command, since we will install the library in /usr to allow clang++ to find the library later:

$ make && sudo make install

You can experiment with the new library and the newest C++ standards by using the -stdlib=libc++ flag when calling clang++ to compile your C++ project.

To see your new library in action, compile a simple C++ application with the following command:

$ clang++ -stdlib=libc++ hello.cpp -o hello

It is possible to perform a simple experiment with the readelf command to analyze the hello binary and confirm that it is indeed linked with your new libc++ library:

$ readelf d hello
Dynamic section at offset 0x2f00 contains 25 entries:
 Tag Type Name/Value
0x00000001 (NEEDED) Shared library: [libc++.so.1]

Subsequent entries are omitted in the preceding code. We see right at the first ELF dynamic section entry a specific request to load the libc++.so.1 shared library that we just compiled, confirming that our C++ binaries now use the new C++ standard library of LLVM. You can find additional information at the official project site, http://libcxx.llvm.org/.