mir-ion ~thunderbolt

Mir Ion

This library seeks to implement the Ion file format. We aim to support both versions of Ion (text & binary Ion), as well as providing an implementation of the format that is performant and easy-to-use.

Mir Ion serialization engine supports

• Text and binary Ion
• JSON
• MsgPack
• YAML
• Bloomberg

Mir Ion supports JSON de/serialization out of the box. On the other hand, you may want to use another our JSON library -- Asdf. It uses the same serialization User Defined Attributes.

Benchmarks

kostya/benchmarks

ARM64

Twitter data set

Test data from nativejson-benchmark

-------------------------------------------------
Binary Ion writing      0.564 GB/s
     is equivalent of   1.5 GB/s for JSON

JSON    -> binary Ion   0.75 GB/s

MsgPack -> binary Ion   0.749 GB/s
     is equivalent of   1.176 GB/s for JSON

Ion is 96% smaller then minimized JSON
   and 69% smaller then MsgPack
- - - - - - - - - - - - - - - - - - - - - - - - -
processed file: benchmark/twitter.json
number of iterations: 2000
os: 64bit osx
-------------------------------------------------

Kostya data et

Test data from kostya/benchmarks.

-------------------------------------------------
Binary Ion writing      0.228 GB/s
     is equivalent of   0.966 GB/s for JSON

JSON    -> binary Ion   0.41 GB/s

MsgPack -> binary Ion   0.285 GB/s
     is equivalent of   1.013 GB/s for JSON


Ion is 134% smaller then minimized JSON
   and 19% smaller then MsgPack
- - - - - - - - - - - - - - - - - - - - - - - - -
processed file: /tmp/1.json
number of iterations: 10
os: 64bit osx
-------------------------------------------------

Documentation

You can find the documentation for this library here. Additionally, for examples of the Ion format, you can check the Ion Cookbook

Exceptions

Mir Ion internals are @nogc and almost all are preallocated. If DIP1008 is enabled, then the parsing exception will contains additional information about error location in the file and last occured key.

Feature Status

• [x] Binary Ion Value parsing and skip-scan iteration.
• [x] Binary Ion conversions to D types.
• [x] Binary Ion conversions from D types.
• [x] Fast hash table for Ion Symbol Tables
• [x] Fast CTFE Symbol Table for deserialization
• [x] Ion Symbol Tables
• [x] JSON to Ion
• [x] Ion to JSON
• [x] Text Ion to Ion
• [x] Ion to Text Ion
• [x] Serialization API
• [x] Deserialization API
• [x] Precise decimal to floating conversion (except subnormals)
• [x] Precise floating to decimal conversion.
• [x] Local Symbol Tables
• [ ] Shared Symbol Tables
• [x] Chunked JSON reader
• [x] Chunked binary Ion Value Stream reader
• [x] MessagePack parsing
• [x] Ion to MessagePack