This document is a manual for developers who wish to contribute to the HLVM project. It covers topics such as coding style, the build system and project rules. All the policies of the LLVM project apply here as well. This document just contains things specific to HLVM.
This section describes the HLVM build system.
The HLVM project uses the scons software construction tool to orchestrate its builds. This gives us superior dependency checking and a much more flexible tool for developing the build system. We started with "make" but were convinced of SCons superiority after trying it for a week. If you're not familiar with it, please read up on it and get a general understanding. You don't need detailed understanding because all you will most likely need to do is follow the instructions in this section.
Unlike some other build systems (e.g. autoconf/automake), the configuration and construction phases of building are not separated with scons. The configuration parameters are checked every time you build. While you might think that will slow things down, it doesn't. The configuration information is cached and proceeds quite quickly once your configuration is stable. If something changes in your environment, only those pieces affected will get re-configured. This saves a lot of time and hassle by telling you of any configuration errors on the very next build.
Configuration parameters are specified on the scons command line. The values of these parameters are stored in the options cache which is simply a file named .options_cache in the root source directory. If the file doesn't exist, it will be created the first time you run scons. On subsequent runs, the values are loaded from this cache and used unless new values are given on the command line. In this way, you only need to specify your options once and then they "stick" from that point forward.
HLVM has a variety of options to assist with configuring HLVM for your environment and build choices. Each option has a default that will work in many environments but might need adjusting for your environment. The default assumes various software packages are in /usr/local and that a fully debuggable version of HLVM should be built. The table below provides the full list of options.
| Option | Description | Default |
|---|---|---|
| assertions | Include assertions in the code. This can be enabled or disabled separate from the setting for debug. | 1 |
| debug | Build with debug options turned on. This includes #ifdef/#endif code as well as instructing the compiler to include debug information (-g) | 1 |
| inline | Cause inline code to be inline. This causes inline functions to really be inline. Its off by default for better debuggability. | 0 |
| optimize | Build object files with optimization. This turns on compiler optimization (-O3) to make the code efficient. | 0 |
| profile | Generate profiling aware code. This causes the compiler to construct the code so that it can be profiled (-pg on Unix). | 0 |
| small | Generate smaller code rather than faster. Instructs the code to favor smaller object files and executables than faster ones. Might be useful for memory constrained platforms. | 0 |
| strip | Strip executables of their symbols. This can significantly reduce the size of the executable. Useful for release preparation when trying to minimize the size of a tarball or RPM package. | 0 |
| prefix | Specify where to install HLVM. This is the root directory of where HLVM will be installed when you run scons install | /usr/local |
| confpath | Specify additional configuration dirs to search. This is a unix style path (colon separate) of directories to include in the search for software packages. These directories will be searched first. If you put your software in a non-standard place like /proj (Reid) or /more (Saem) then you'll want to use this option. | {empty} |
| with_llvm | Specify where LLVM is installed. | /usr/local |
| with_apr | Specify where apr is installed. | /usr/local/apr |
| with_apru | Specify where apr-utils is installed. | /usr/local/apr |
| with_libxml2 | Specify where LibXml2 is installed. | /usr/local |
| with_gperf | Specify where the gperf program is installed. | /usr/local/bin/gperf |
| with_llc | Specify where the LLVM compiler is installed. | /usr/local/bin/llc |
| with_llvmdis | Specify where the LLVM disassembler is installed. | /usr/local/bin/llvm-dis |
| with_llvmas | Specify where the LLVM assembler is installed. | /usr/local/bin/llvm-as |
| with_llvmgcc | Specify where the LLVM C compiler is installed. | /usr/local/bin/llvm-gcc |
| with_llvmgxx | Specify where the LLVM C++ compiler is installed. | /usr/local/bin/llvm-g++ |
| with_llvmar | Specify where the LLVM bytecode archiver is installed. | /usr/local/bin/llvm-g++ |
| with_llvm2cpp | Specify where the LLVM llvm2cpp program is installed. | /usr/local/bin/llvm2cpp |
| with_runtest | Specify where DejaGnu runtest program is installed. | /usr/local/bin/runtest |
| with_doxygen | Specify where the doxygen program is installed. | /usr/local/bin/doxygen |
| with_xsltproc | Specify where the XSLT processor is installed. | /usr/local/bin/xsltproc |
When you first run scons against HLVM, if you did not specify the confpath option or the configuration code cannot otherwise find a package it needs, you will be prompted to enter the applicable path names manually. You only need to do this once as the values you enter will be remembered in the .options_cache file. These prompts will repeat until you enter a path in which the package is properly recognized. If you wish to abort this, just enter exit or quit at any prompt.
This subsection describes the various build rules and extensions to SCons that HLVM uses.
For convenience, and to unclutter the SConscript files, the top level build directory contains a python module that provides the build facilities for HLVM. This module manipulates an SCons environment to set the variables and define the builder rules for HLVM. The public interface to this library is in the hlvm.py module. No other module should be imported into the SConscript files.
HLVM requires some specialized build rules beyond what SCons provides. While general construction of static libraries, shared libraries, and executables use the standard SCons builders, several more are defined in the build python module. The table below describes the HLVM specific builders.
| Builder | Source | Target | Description |
|---|---|---|---|
| RNGTokenizer | Relax/NG Schema (.rng) | *Tokenizer.h, *Tokenizer.cpp, *TokenHash.cpp | This builder scans a Relax/NG Schema for element, attribute, and enumerated value names. It then uses these tokens to build a perfect hash function (via gperf) for quickly recognizing the tokens and converting it to an enumeration value. This is provided to make recognizing the element and attributes names of an XML document faster. |
| RNGQuoteSource | Relax/NG Schema (.rng) | C++ Strings For Inclusion (.inc) | This builder converts a Relax/NG schema into a set of strings that can be #included into a C++ program as a literal string. It strips out comments, annotations, and converts special characters into the C++ equivalent. The purpose here is to allow a single source for the schema to be incorporated directly into the software so that no file I/O is necessaroy to access the schema. |
| Bytecode | C++ Source (.cpp) | LLVM Bytecode (.bc) | This builder compiles a C++ source file and produces an equivalent LLVM bytecode file. This allows us to incorporate C++ source into the Runtime as linkable bytecode. |
| BytecodeArchive | LLVM Bytecode (.bc) | LLVM Bytecode Archive (.bca) | This builder collects a group of LLVM Bytecode files together and creates a bytecode archive from them using llvm-ar |
| Cpp2LLVMCpp | C++ Source (.cxx) | C++ Source (.cpp) | This builder uses the llvm2cpp to turn source C++ into equivalent C++ calls against the LLVM Intermediate Representation (IR) to construct the same program or program fragment. The construction environment variable LLVM2CPPFLAGS can be set to control what options are passed to llvm2cpp. See the llvm2cpp manual page for details. Examples are in the hlvm/CodeGen/SConscript file. |
| Check | n/a | n/a | This builder is a little special. It is used to run the DejaGnu test suite. It is aliased on the command line as the target "check" so that the scons check command will invoke this build. |
| Doxygen | n/a | doxygen.tar.gz | This builder runs the doxygen documentation generator on the source code and builds a gzipped tar file containing doxygen's output. This allows the HLVM API documentation, generated from source code comments, to be packged into a single file. |
Building HLVM is no small feat. It depends on a lot of software that is quite version dependent. To bring a little sanity to the process, here is a step-by-step procedure we know to work.
In building HLVM, you'll be installing compilers and library that may already exist on your system. You don't want to overwrite your system versions of these things or it will wreak havoc on your system. So, we suggest that you start with a fresh directory. In the discussion that follows, we'll call it /proj (that's what Reid uses). But, it could be anything you want, as long as its new. ~/hlvm would work just as well.
Once you've found a suitable location for HLVM, create the following directory structure:
cd /proj mkdir gcc libxml2 apr apru gperf scons install
In the following sections you will build each of these packages and install them into /proj/install which will keep it separate from anything else in your system.
First, start with obtaining GCC 3.4.6. This will be the compiler that you use for all the remaining compilations. Use the following commands to obtain, build and install GCC 3.4.6:
cd /proj/gcc mkdir build svn co svn://gcc.gnu.org/svn/gcc/tags/gcc_3_4_6_release src cd ../build ../src/configure --prefix=/proj/install make make install
Now that you have gcc installed in a separate location, you will need to change your environment to ensure that that version of gcc is the one used in subsequent builds. Details vary from platform to platform, but on Linux, the following should work:
export PATH=/proj/install/bin:$PATH export LD_LIBRARY_PATH=/proj/install/lib:$LD_LIBRARY_PATH
The essential point is to change your environment so that programs and libraries installed into /proj/install will be found first. You should do this in any shell environment in which you'll be building HLVM related software.
This package is used for generating perfect hash functions. It is used by HLVM for fast recognition of XML element and attribute names. Its easy and fast to build:
cd /proj/gperf wget http://mirrors.kernel.org/gnu/gperf/gperf-2.7.2.tar.gz tar zxf gperf-2.7.2.tar.gz mkdir build cd build ../gperf-2.7.2/configure --prefix=/proj/install make make install
You can probably use any version after 2.7. We know it works with 2.7.2 and 3.0.1.
This package provides all XML services for HLVM. It is part of GNome and many other packages and quite stable. It should build quickly and easily for you. Use these commands:
cd /proj/libxml2 wget ftp://xmlsoft.org/libxml2/libxml2-2.6.24.tar.gz tar zxf libxml2-2.6.24.tar.gz mkdir build cd build ../libxml2-2.6.24/configure --prefix=/proj/install make make install
The Apache Portable Runtime is a portability layer used within the Apache HTTP Server. Although it is still undergoing active development, stable releases are available. HLVM uses APR for portability in the runtime. Build APR with the following commands:
cd /proj/apr
wget http://mirror.olnevhost.net/pub/apache/apr/apr-1.2.7.tar.gz
tar zxf apr-1.2.7.tar.gz
mkdir build
cd apr-1.2.7
./buildconf
cd ../build
../apr-1.2.7/configure --prefix=/proj/install --enable-debug \
--enable-threads --enable-other-child
make
make install
The apr-util package is some additional utilities that go with APR. Build apr-util with the following commands:
cd /proj/apru
wget http://mirror.olnevhost.net/pub/apache/apr/apr-util-1.2.7.tar.gz
tar zxf apr-1.2.7.tar.gz
mkdir build
cd apr-util-1.2.7
./buildconf
cd ../build
../apr-util-1.2.7/configure --prefix=/proj/install --enable-debug \
--enable-threads --enable-other-child
make
make install
For now, you must build LLVM from the CVS repository. Although LLVM is actively being developed, it is generally stable and this is safe. If you get tempted to use a release tarball, it will fail. HLVM depends on post-1.7 features of LLVM. When you build LLVM, use the "tools-only" target. This will avoid building the "runtime" portion of LLVM which was necessary for an older version of llvm-gcc (v3). Since we'll be using llvm-gcc4, this is unnecessary and will eliminate some chicken-and-egg type problems.
cd /proj/llvm cvs -d :pserver:anon@llvm-cvs.cs.uiuc.edu:/var/cvs/llvm login <return> cvs -z3 -d :pserver:anon@llvm-cvs.cs.uiuc.edu:/var/cvs/llvm co llvm mkdir build cd build ../llvm/configure --prefix=/proj/install make tools-only make install make ENABLE_OPTIMIZED=1 OPTIMIZE_OPTION=-O2 tools-only
Apple provides a Subversion repository for llvm-gcc4. You need this latest version because HLVM depends on some of the features. You might find this one a bit tricky. See the README.LLVM file in the top source directory for additional help and late breaking news.
cd /proj/llvm-gcc4
svn co svn://anonsvn.opensource.apple.com/svn/llvm cfe
mkdir build install
cd build
../cfe/configure --prefix=/proj/llvm-gcc4/cfe/install \
--enable-llvm=/proj/llvm/build --enable-languages=c,c++ --disable-threads \
--program-prefix=llvm-
make
make install
Note that the --disable-threads option is a temporary workaround until LLVM's PR822 (supporting weak-external linkage) is implemented. This should be fixed by September, 2006 at which time --disable-threads won't be needed any more.
IMPORTANT: You MUST install llvm-gcc4 into its own installation area. Do NOT be tempted to install it into /proj/install. If you do, it will overwrite your gcc 3.4.6 installation and you will have corrupted your build environment. You've been warned.
As noted earlier, HLVM uses the SCons tool for its builds. You need version 0.96.92. Install it like this:
cd /proj/scons wget http://internap.dl.sourceforge.net/sourceforge/scons/scons-0.96.92.tar.gz tar zxf scons-0.96.92.tar.gz python config.py install
Alternatively, you could use one of the SCons packages, such as:
cd /proj/scons wget http://internap.dl.sourceforge.net/sourceforge/scons/scons-0.96.92-1.noarch.rpm rpm --install scons-0.96.92-1.noarch.rpm
Note in the following that if you've actually used /proj as your build area then you don't need to provide any arguments to make as these paths are the default. Also note that the arguments are only needed the first time you build HLVM. Subsequently, these options will be remembered. See the description of SCons above.
cd /proj/hlvm
svn co svn://hlvm.org/hlvm hlvm
cd hlvm
make debug MYMODE=Debug MYPREFIX=/proj/install MYPATH=/proj/install \
MYLLVMGCC=/proj/llvm/cfe/install/bin
Contributions to HLVM must meet the following Coding Style requirements: