djinn 0.1.1

Djinn

A templating language and code generator using D.

$ cat examples/hello.txt.dj
Hello [= retro("dlrow") ]!
[: enum one = 1; :]
1 + 1 = [= one + one ]
$ djinn --output-file - examples/hello.txt.dj
Hello world!
1 + 1 = 2

Features:

• Conveniently generate text using D ranges and other D code
• Supports compile-time embedding in D code
• Can be used with standalone tool
• @safe

Installation

You'll need a D compiler to both build and use Djinn. You might find one in your favourite package manager. Djinn templates get translated to D code and compiled with a DMD-compatible compiler when rendered.

You'll also need dub to use Djinn as a library, or build the command line tool. It might come with your D compiler.

You can add Djinn as a library dependency to an existing dub project using dub add djinn. You can also build the latest released version of the tool with dub build --build=release djinn.

Otherwise, dub build creates the command line tool by default.

$ dub build
$ ./djinn -h

The resulting binary is all you need on top of a working D compiler. You can install it somewhere in your $PATH.

Usage

You can use Djinn as a templating language inside D programs. For convenience, there's also a command line tool for generating text files. NB: Djinn templates can contain arbitrary D code, so the command line tool isn't intended for use with untrusted input.

As code generator

$ djinn -h
Usage: djinn [--help] [--output-file=<string>] [--output-dir=<string>]
               [--code] [--compiler=<string>] input

Positional arguments:
 input           input file path (- for standard input)

Optional arguments:
 --help, -h      display this help and exit
 --output-file, --of <string>
                 base output filename (- for standard output)
 --output-dir, --od <string>
                 output directory
 --code, -c      generate D code only
 --compiler <string>
                 dmd-compatible compiler to use

In the basic usage, the command line tool takes a file like foo.dj and creates a rendered text file called foo. You can also use Djinn as a Unix-style filter by passing - as the file name. Then it will read from standard input and write to standard output.

The djinn tool compiles templates as @safe D.

As embedded templating language

The translate() template returns a D source code string that you can splice into your code with a mixin statement. The translated code expects a string output range called output. Any variables in scope can be referenced inside your Djinn template.

string renderInvoice(InvoiceData data)
{
    import std.array : appender;
    auto output = appender!string;
    // "data" can be referenced inside invoice.dj
    mixin(translate!"invoice.dj");
    return output[];
}

DMD will only load files from places it's been configured to --- either using the -J switch, or the stringImportPaths parameter for dub.

If CTFE is too slow, you can try generating the whole D file as a Djinn template using the command line tool and the xlatinclude directive.

The greeter example demonstrates both the CTFE method and the xlatinclude method.

Djinn language

Djinn templates look a lot like whatever they're templating --- HTML, config files, whatever --- but with little snippets of code in them. The language was originally based on Jinja2, but the syntax was changed to make Djinn templates more readable when mixed with D code.

[= expressions ]

You can inject arbitrary D expressions by wrapping them in [= and ]:

There are [= 24 * 60 * 60 ] seconds in a day.

->

There are 86400 seconds in a day.

Everything std.stdio.write() supports, Djinn supports too. That includes ranges, structs, etc. You can also render multiple expressions by separating them with commas.

[= 'a', 'b', 'c' ]
[= iota('a', 'd') ]
[= [2, 7, 1, 8, 2, 8] ]

->

abc
abc
[2, 7, 1, 8, 2, 8]

If the first expression is a double-quoted string, it's treated like a format string for the other expressions.

[= "%0.2f", 1.0/3 ]
[= "%(%d, %)", iota(10) ]

->

0.33
0, 1, 2, 3, 4, 5, 6, 7, 8, 9

If you want a plain string literal as the first expression, use another string literal syntax, such as backticks (WYSIWYG strings). Alternatively, put an empty WYSIWYG string in front.

[= `%0.2f`, 1.0/3 ]
[= ``, "%0.2f", 1.0/3 ]

->

%0.2f0.333333
%0.2f0.333333

[: statements :]

You can inject D statements by wrapping them in [: and :]:

[:
import std.math;
:]
Pi is [= "%.6f", PI ].

->

Pi is 3.141593.

You can use any D code that's allowed in function local scope, including imports and function calls, and definitions for variables, functions and data types.

Everything between Djinn statements (or directives, explained later on) gets batched together and written using one write() statement. This reduces the number of curly braces you need to use (although you can still use them for more control if needed).

[: foreach (j; 0..4) :]
This is line [= j ].
[: foreach (j; 0..4) { :]
This is line [= j ], again.
[: } :]
The end.

->

This is line 0.
This is line 1.
This is line 2.
This is line 3.
This is line 0, again.
This is line 1, again.
This is line 2, again.
This is line 3, again.
The end.

[< directives >]

There are some extra directives special to Djinn (i.e., they're not D). You inject them into your template wrapped in [< and >].

include

This includes the given file in the template, interpreting it as if it were copy-pasted in:

[< include examples/normaldist.csv.dj >]

->

x,f(x)
-1.0,0.158655
-0.9,0.18406
-0.8,0.211855
-0.7,0.241964
-0.6,0.274253
-0.5,0.308538
-0.4,0.344578
-0.3,0.382089
-0.2,0.42074
-0.1,0.460172
0.0,0.5
0.1,0.539828
0.2,0.57926
0.3,0.617911
0.4,0.655422
0.5,0.691462
0.6,0.725747
0.7,0.758036
0.8,0.788145
0.9,0.81594

rawinclude

This copies the contents of the given file verbatim. No code is interpreted.

[< rawinclude examples/normaldist.csv.dj >]

->

x,f(x)
[: import std.mathspecial;
foreach (x; iota(-1.0, 1.0, 0.1)) :]
[= "%0.1f,%g", x, normalDistribution(x) ]

xlatinclude

This includes the given file, translating it into D code. There is no guarantee the D code will be stable across Djinn versions. xlatinclude is intended to help with using Djinn templates inside D code, as an alternative to using D CTFE.

[< xlatinclude examples/normaldist.csv.dj >]

->

# line 1 "examples/normaldist.csv.dj"
output.writeText("x,f(x)\n");
# line 2 "examples/normaldist.csv.dj"
 import std.mathspecial;
foreach (x; iota(-1.0, 1.0, 0.1)) 
output.writeText(
# line 4 "examples/normaldist.csv.dj"
format( "%0.1f,%g", x, normalDistribution(x) ),
# line 4 "examples/normaldist.csv.dj"
"\n");

raw and endraw

These directives can be used to wrap text to be rendered verbatim, without interpreting any Djinn code.

[< raw >]
This won't [< include >] anything.
[< endraw >]

->

This won't [< include >] anything.

You can optionally add a tag as an argument to the raw and endraw directives. An endraw directive will only match with a raw directive with the same tag, making it easier to wrap arbitrary Djinn code:

[< raw EXAMPLE >]
[< raw >]
This won't [< include >] anything.
[< endraw >]
[< endraw EXAMPLE >]

->

[< raw >]
This won't [< include >] anything.
[< endraw >]

Tags must use ASCII letters, numbers and underscores.

Whitespace control

If you're familiar with Jinja2, you might have already noticed a special rule of Djinn: any line that only contains whitespace and statements and directives has no direct effect on the output. In most cases, this rule is enough to let you format your Djinn templates in a readable way, without messing up the formatting of the output.

Djinn also supports using | to strip whitespace to the left or right of injected code. It acts like the '-' modifier in Jinja2:

This will all show
[=| ` on ` |]
one line.

This time only the
[=| ` first` ]
newline will get stripped.

This time only the
[= `second ` |]
newline will get stripped.

->

This will all show on one line.

This time only the first
newline will get stripped.

This time only the
second newline will get stripped.

Paths

Paths (for include, etc.) are relative to the source file and independent of the current directory. The exception is when using the djinn CLI and reading from standard input. In that case paths in the input file are interpreted relative to the current working directory, but paths inside included files are interpreted as normal.

If using Windows, it's recommended to use forward slashes (/) as directory separators for portability.

Contributing

Djinn mostly does what it needs to, so the best contributions are bug fixes, tests, tutorials, blog posts and performance improvements. Some Jinja2 features (like template inheritance) won't be copied in Djinn because D itself is already a powerful language.

Djinn templates have self-contained semantics --- i.e., there are no options that can change the output of a template. Any new features that change the output should be implemented as directives in the Djinn language.

Having said all that, there are some things that would be nice.

Better error messages

The current error messages are okay, but they can always be made more useful. In particular, some error locations are inaccurate because of known bugs.

Support for dub --single

It would be great if the djinn CLI supported alternative build systems, particularly dub single-file packages. That would help those who want to use third-party packages in Djinn scripts.

Auto-escaping

This is probably best implemented with a directive that sets a function that all expressions are passed through for rendering. Escape bypass can implemented using a SafeString type that all (templated) escape functions must leave untouched. Common escape functions (for XML, CSV, SQL, etc.) can be put in a separate library.

Possible sandbox mode

In theory, Djinn code generation templates can be sandboxed using syscall filtering (like OpenBSD's pledge). This will have to be thought through very, very carefully, given the huge attack surface.