dpp ~fix-22

d++ - #include C and C++ headers in D files

| | |

Goal

To directly #include C and C++ headers in D files and have the same semantics and ease-of-use as if the file had been #included from C or C++ themselves. Warts and all, meaning that C enum declarations will pollute the global namespace, just as it does "back home".

This work was supported by Symmetry Investments.

Example

// c.h
#ifndef C_H
#define C_H

#define FOO_ID(x) (x*3)

int twice(int i);

#endif

// c.c
int twice(int i) { return i * 2; }

// foo.dpp
#include "c.h"
void main() {
    import std.stdio;
    writeln(twice(FOO_ID(5)));  // yes, it's using a C macro here!
}

At the shell:

$ gcc -c c.c
$ d++ foo.dpp c.o
$ ./foo
$ 30

Limitations

• It currently only supports C features, but C++ is planned.
• Using it on a C++ header will "work" if it's basically technically C, with extern(C++) instead of extern(C)
• Only known to work on Linux with libclang.so.6.0. It might work in different conditions.
• When used on multiple files, there might be problems with duplicate definitions depending on imports. This will be fixed.

This is alpha software. It has however produced programs that compile that #included several "real-life" C headers:

• nanomsg/nn.h, nanomsg/pubsub.h
• curl/curl.h
• stdio.h, stdlib.h
• pthread.h
• julia.h
• xlsxwriter.h
• libvirt/libvirt.h, libvirt/virterror.h
• libzfs
• openssl/ssl.h
• imapfilter.h
• libetpan/libetpan.h

Compilation however doesn't guarantee they work as expected and YMMV. Please consult the examples.

Command-line arguments

It is likely that the header or headers need -I flags to indicate paths to be searched, both by this executable and by libclang itself. The --include-path option can be used for that, once for each such path.

Use -h or --help to learn more.

Details

d++ is an executable that wraps a D compiler such as dmd (the default) so that D files with #include directives can be compiled.

It takes a .dpp file and outputs a valid D file that can be compiled. The original can't since D has no preprocessor, so the .dpp file is "quasi-D", or "D with #include directives". The only supported C preprocessor directive is #include.

The input .dpp file may also use C preprocessor macros defined in the file(s) it #includes, just as a C/C++ program would (see the example above). It may not, however, define macros of its own.

d++ goes through the input file line-by-line, and upon encountering an #include directive, parses the file to be included with libclang, loops over the definitions of data structures and functions therein and expands in-place the relevant D translations. e.g. if a header contains:

uint16_t foo(uin32_t a);

The output file will contain:

ushort foo(uint a);

d++ will also enclose each one of these original #include directives with either extern(C) {} or extern(C++) {} depending on the header file name and/or command-line options.

As part of expanding the #include, and as well as translating declarations, d++ will also insert text to define macros originally defined in the #included translation unit so that these macros can be used by the D program. The reason for this is that nearly every non-trivial C API requires the preprocessor to use properly. It is possible to mimic this usage in D with enums and CTFE, but the result is not guaranteed to be the same. The only way to use a C or C++ API as it was intended is by leveraging the preprocessor.

This means that only the #including .dpp file has access to constant macros, and any D module importing the .d file resulting from said .dpp file won't see those constants (e.g. #define THE_ANSWER 42). To mitigate this, dpp will introduce an enum for any macros that are string or integer constants but with the DPP_ENUM_ prefix. To see why, please consult github issue 103.

As a final pass before writing the output D file, d++ will run the C preprocessor (currently the cpp binary installed on the system) on the intermediary result of expanding all the #include directives so that any used macros are expanded, and the result is a D file that can be compiled.

In this fashion a user can write code that's not-quite-D-but-nearly that can "natively" call into a C/C++ API by #includeing the appropriate header(s).

Translation notes

enum

For convenience, this declaration:

enum Enum { foo, bar, baz }

Will generate this translation:

enum Enum { foo, bar, baz }
enum foo = Enum.foo;
enum bar = Enum.bar;
enum baz = Enum.baz;

This is to mimic C semantics with regards to the global namespace whilst also allowing one to, say, reflect on the enum type.

Renaming enums

There is the ability to rename C enums. With the following C definition:

enum FancyWidget { Widget_foo,  Widget_bar }

Then adding this to your .dpp file after the #include directive:

mixin dpp.EnumD!("Widget",      // the name of the new D enum
                 FancyWidget,   // the name of the original C enum
                 "Widget_");    // the prefix to cut out

will yield this translation:

enum Widget { foo, bar }

Names of structs, enums and unions

C has a different namespace for the aforementioned user-defined types. As such, this is legal C:

struct foo { int i; };
extern int foo;

The D translations just use the short name for these aggregates, and if there is a name collision with a variable or function, the latter two get renamed and have a pragma(mangle) added to avoid linker failures:

struct foo { int i; }
pragma(mangle, "foo") extern __gshared int foo_;

Functions or variables with a name that is a D keyword

Similary to name collisions with aggregates, they get an underscore appended and a pragma(mangle) added so they link:

void debug(const char* msg);

Becomes:

pragma(mangle, "debug")
void debug_(const(char)*);