Dextrin

Dextrin is an Elixir implementation of DXN (Data eXchange Notation): a human-writable text format (.dxn), a compact binary format (.dxnb) built on CBOR, and a schema format (.dxns) that's just more DXN data — all three sharing one in-memory value representation and one extension mechanism.

This is the reference implementation. node-dextrin (Node.js, on npm) and php-dextrin (PHP, on Packagist) are sibling ports of the same format — see Other language implementations below.

{:ok, value} = Dextrin.decode(~s(%{x: 1, y: 2}))
value
#=> %{x: 1, y: 2}

Dextrin.encode(value)
#=> {:ok, "%{x:1,y:2}"}

{:ok, bytes} = Dextrin.encode_binary(value)
Dextrin.decode_binary(bytes)
#=> {:ok, %{x: 1, y: 2}}

A plain map's shorthand keys (x:) are themselves DXN keywords — %{x: 1} and %{:x => 1} are the exact same value, and keyword's own Elixir type, per DXN.md §1.3, is a real atom, which is what both decode/2 and decode_binary/2 produce by default (trusted: true). Encoding accepts a bare atom the same way, anywhere a keyword is expected — %{x: 1} or Dextrin.encode(:ok) both work with no conversion needed first.

That default assumes a source you control — your own config, your own application's data — not arbitrary untrusted/network input, where an unbounded String.to_atom/1 could exhaust the atom table. Pass trusted: false for that case; keyword then decodes as Dextrin.Keyword.t() instead:

Dextrin.decode(~s(%{x: 1, y: 2}), trusted: false)
#=> {:ok, %{%Dextrin.Keyword{name: "x"} => 1, %Dextrin.Keyword{name: "y"} => 2}}

A schema-backed struct's fields are the one place names do come back as plain strings regardless of trusted: — see the tutorial — since a schema always knows its field names up front.

.dxn's text grammar is compiled by Ichor — write the grammar once (priv/grammar/dxn.aether), get a lexer, parser, and (via Dextrin.Text.Actions) an evaluator with no hand-written parsing code. That compilation happens ahead of time (mix ichor.gen, checked in as generated source), not at dextrin's own build time, so only the small ichor_runtime support library the generated code actually calls ships as a real dependency — ichor proper (the Aether front-end, analysis, codegen) is dev-tooling only. .dxnb has no grammar to speak of — it's a direct, hand-rolled CBOR codec — so it's plain recursive Elixir working over the same shared value type.

Why

Most serialization formats pick one point on a spectrum: JSON is human-writable but loses precision (no distinct int/float boundary, no dates, no bytes) and has no extension story; Protobuf/Avro are compact and typed but need a separate schema-compiler step and aren't meant for a human to read or hand-edit; EDN is expressive and Elixir-friendly in spirit but has no first-party Elixir implementation and no binary counterpart. DXN's premise is that a .dxn text document and a .dxnb binary document should be the same value space — 30 scalar/collection/temporal/extended types (reference §1.3), precise enough for money (Decimal), exact ratios (Rational), and arbitrary precision integers, with struct and custom-tag as first-class, schema-describable extension points — encoded however density or readability happens to matter for a given use.

Components

• Dextrin — the four-function public API: decode/2, encode/2 (.dxn text) and decode_binary/2, encode_binary/2 (.dxnb binary). One error type, Dextrin.Error, for both.
• Value types (Dextrin.Symbol, Dextrin.Tuple, Dextrin.OrderedMap, Dextrin.SortedSet, Dextrin.Struct, Dextrin.Array, Dextrin.Duration, Dextrin.Rational, Dextrin.Uuid, Dextrin.Uri, Dextrin.Bytes, Dextrin.Char, Dextrin.CustomTag) — small wrapper structs for the DXN types Elixir has nothing native for without losing information. keyword is the one exception with two faces: a real atom by default (trusted: true), Dextrin.Keyword when decoded untrusted. Every other scalar/collection (integers, floats, strings, lists, plain maps, sets, dates, regexes, ...) decodes to the obvious native Elixir value.
• Dextrin.Text.Grammar/Actions/Printer/Formatter — the .dxn pipeline: an Ichor-compiled grammar (Grammar is a thin wrapper around the pregenerated Grammar.Native), an Ichor.Actions implementation that turns a parse into real values, a single-line printer (the reverse direction), and a multi-line pretty-formatter on top of it.
• Dextrin.Binary.Encoder/Decoder/Tags — the .dxnb pipeline: a direct recursive CBOR codec (not built on a generic CBOR library — see Dextrin.Binary.Encoder's own moduledoc for why) plus the private tag block and bit-layout constants it needs.
• Dextrin.Schema and Dextrin.Schema.* — compiles a .dxns document (itself just DXN data — no new grammar) into a Dextrin.Registry, enforced automatically, decode- and encode-side, wherever a registered struct name appears. Dextrin.Schema.Std ships a small standard library of common named types (PositiveInteger, NonEmptyString, ...); Dextrin.Schema.FileResolver resolves Namespace/Name references across separate .dxns files; Dextrin.Schema.Provider lets a struct's own library ship its DXN schema without that library ever depending on dextrin itself.
• Dextrin.Registry — the one extension point both struct and custom-tag share: register a tag decoder/encoder, a struct materializer, or a lazy schema resolver; also carries the trusted:/put_trusted/2 flag that decides how keyword decodes. Plain immutable data, threaded explicitly — never a process or ETS table.
• mix dextrin.* — validate, encode, decode, format, gen.schema (scaffold a .dxns file from an existing Elixir struct), and gen.unicode (regenerate the grammar's Unicode identifier ranges from the latest UCD data).

Installation

Add dextrin to your list of dependencies in mix.exs:

def deps do
  [
    {:dextrin, "~> 0.1.0"}
  ]
end

Other language implementations

DXN's format spec (guides/dxn/DXN.md) is implementation-independent — this library, node-dextrin, and php-dextrin all satisfy the same document, and stay close to this project's own in-memory shape-per-type choices wherever their host language allows, so recognizing one implementation gets you most of the way to recognizing the others.

Language	Package	Source
Elixir (this project)	dextrin on Hex.pm	joetjen/dextrin
Node.js	dextrin on npm	joetjen/node-dextrin
PHP	joetjen/dextrin on Packagist	joetjen/php-dextrin

Where to go next

• Tutorial — a step-by-step walkthrough of this library's features, building up to a small, working example that decodes, validates against a schema, and re-encodes real data.
• Examples — worked examples: config files, API payloads, event logs, and schema-validated records.
• Cheatsheet — quick reference for common Dextrin tasks.
• DXN tutorial and DXN reference — everything about the DXN format itself, independent of this Elixir implementation, plus DXN examples and a DXN cheatsheet.

Development

mix deps.get
mix precommit

mix precommit runs the full verification pass this project expects before a commit: mix format, mix compile --warnings-as-errors, mix credo --strict, mix sobelow, mix test, and mix dialyzer, in that order (fast/cheap checks first, dialyzer — the slowest, especially its first PLT build — last).

priv/grammar/dxn.aether's generated Unicode identifier ranges are regenerated with mix dextrin.gen.unicode — a deliberate, reviewed action on a Unicode version bump, never run automatically at build time (see that task's own docs). Either way, changing the grammar itself requires a mix ichor.gen step afterward, since lib/dextrin/text/grammar/native.ex is generated ahead of time, not produced at dextrin's own compile time (that's also what keeps ichor itself, and everything it depends on for parsing/codegen, only: :dev, runtime: false — only the small ichor_runtime package ships in a release) — see CONTRIBUTING.md.

See CONTRIBUTING.md for how to propose changes, and CHANGELOG.md for release history.

License

MIT — see LICENSE.