v0.4.0

This release contains quite a lot of changes, but only a few breaking changes.
First the highlight:

This release contains opt-in support for a version of the experimental number suffix feature currently under discussion in the KDL language repository.

import {parse} from "@bgotink/kdl";
import assert from "node:assert";

assert.throws(() => parse("node 10xp"));

assert.doesNotThrow(() =>
	parse("node 10px", {
		flags: {
			experimentalSuffixedNumbers: true,
		},
	}),
);

Changes:

• Start on KDL Query implementation
• Rewrite the entire tokenizer to use a lookup table, which provides a drastic speed boost
• The v1 parser is rebuilt from scratch and uses the same infrastructure as the existing v2 parser.
  This makes parsing a v1 document not only 2x faster but it stops us from having to bundle chevrotain with its heavy footprint.
• Fix issues with the format() function: improve its ability to format bare identifiers and fix unicode escapes.
• BREAKING CHANGE: in case multiple errors occur the parser will now throw an InvalidKdlError rather than an AggregateError.
  The InvalidKdlError can now have sub-errors in its errors property. You can use the flat() method to easily iterate over all errors contained within an InvalidKdlError.
• Add support for (a version of) the number suffix feature under discussion in the KDL language repository.
• BREAKING CHANGE: place support for number suffixes behind a parser flag.
  Note this is only a breaking change if you are updating from one of the 0.4.0-next releases.
• Expose value tags in the DeserializationContext API (#12).
• BREAKING CHANGE: The DeserializationContext#run function now leave the DeserializationContext unchanged if the deserializer throws an error instead of leaving the context in an inconsistent and unpredictable state.
  This means you can safely try multiple deserializers, which is further aided by the new context.run.try() function.

Full Changelog: v0.3.1...v0.4.0

v0.4.0-next.1

This pre-release contains quite a lot of changes, but only one breaking change:

• Start on KDL Query implementation
• Rewrite the entire tokenizer to use a lookup table, which provides a drastic speed boost
• The v1 parser is rebuilt from scratch and uses the same infrastructure as the existing v2 parser.
  This makes parsing a v1 document not only 2x faster but it stops us from having to bundle chevrotain with its heavy footprint.
• Fix issues with the format() function: improve its ability to format bare identifiers and fix unicode escapes.
• BREAKING CHANGE: in case multiple errors occur the parser will now throw an InvalidKdlError rather than an AggregateError.
  The InvalidKdlError can now have sub-errors in its errors property. You can use the flat() method to easily iterate over all errors contained within an InvalidKdlError.

This pre-release contains support for the experimental number suffix feature currently under discussion in the KDL language repository.

Full Changelog: v0.3.1...v0.4.0-next.1

v0.3.1

Fixes for two issues detected right after publishing v0.3.0:

• An off-by-one error in the position of some errors in invalid numbers
• An unescaped backslash in a regular expression makes multiline strings that end on x invalid

Full Changelog: v0.3.0...v0.3.1

v0.3.0

This release of @bgotink/kdl contains some major improvements and a few smaller breaking changes.

Breaking Changes

• The Location export is now called StoredLocation, TokenLocation is now called Location.
• The shape of StoredLocation has changed from location.startOffset to location.start.offset etc.
• The tag property has moved from the Entry to the Value. The getTag/setTag methods still exist on Entry.
• The Identifier and Value classes are now mutable, i.e. their value can be changed. Use the setName / setValue method instead of modifying the name / value property directly to prevent issues in parsed documents.

Error improvements

Parts of the tokenizer and parser have been rewritten to improve the errors thrown by the parse function. More errors are now considered "recoverable", which means the parser can continue to see if there are more errors so it can give the caller more information than only the first mistake in the source text. Some of the error messages have been extended with more useful information.

The InvalidKdlError class introduces two new properties start and end to point towards a specific location in the source text where the error occurs. These new properties allow for more precision and flexibility than the token property, which still exists. Many of the errors thrown by the parser are updated to point directly towards the character or characters that cause the error.

(De)Serialization Tools

This release introduces a new export: @bgotink/kdl/dessert which contains a bunch of utilities to deserialize KDL text into a JavaScript structure and serialize it back again.

The API doesn't enforce any programming paradigm, working with regular functions and with classes.

Here's an example with functions:

import {
	type DeserializationContext,
	type SerializationContext,
	deserialize,
	serialize,
} from "@bgotink/kdl/dessert";

type Tree = {value: number; left?: Tree; right?: Tree};

function treeDeserializer(ctx: DeserializationContext): Tree {
	return {
		value: ctx.argument.required("number"),
		left: ctx.child.single("left", treeDeserializer),
		right: ctx.child.single("right", treeDeserializer),
	};
}

export function readTree(node: Node): Tree {
	return deserialize(node, treeDeserializer);
}

function treeSerializer(ctx: SerializationContext, tree: Tree) {
	ctx.argument(tree.value);

	if (tree.left) {
		ctx.child("left", treeSerializer, tree.left);
	}
	if (tree.right) {
		ctx.child("right", treeSerializer, tree.right);
	}
}

export function writeTree(tree: Tree): Node {
	return serialize("root", treeDeserializer, tree);
}

and here's that same example using classes:

import {
	type DeserializationContext,
	type SerializationContext,
	deserialize,
	serialize,
} from "@bgotink/kdl/dessert";

class Tree {
	static deserialize(ctx: DeserializationContext): Tree {
		return new Tree(
			ctx.argument.required("number"),
			ctx.child.single("left", Tree),
			ctx.child.single("right", Tree),
		);
	}

	constructor(
		readonly value: number,
		readonly left?: Tree,
		readonly right?: Tree,
	) {}

	serialize(ctx: SerializationContext) {
		ctx.argument(this.value);

		if (this.left) {
			ctx.child("left", this.left);
		}
		if (this.right) {
			ctx.child("right", this.right);
		}
	}
}

export function readTree(node: Node): Tree {
	return deserialize(node, Tree);
}

export function writeTree(tree: Tree): Node {
	return serialize("root", tree);
}

Both of these examples turn the following KDL node into a Tree structure and back.

root 10 {
	left 5
	right 5 {
		left 2 { left 1; right 1 }
		right 3 { left 2; right 1 }
	}
}

The dessert API supports preserving comments and formatting when making modifications to a KDL file. This preservation can be enabled by linking the DeserializationContext used to deserialize the value from KDL to the SerializationContext used when serializing the value back to KDL. This is done via the SerializationContext's new source function.

This takes two steps:

1. Store the DeserializationContext while deserializing, e.g. by adding it to a hidden/private property on the returned object
2. Pass the stored DeserializationContext to the SerializationContext inside the serializer

It is important that step 2 is done at the start of the serializer. Doing so after making changes to the SerializationContext will throw an error.

Here we've taken the Tree class from above and added in the necessary code to enable preserving formatting and comments:

class Tree {
	static deserialize(ctx: DeserializationContext): Tree {
		const tree = new Tree(
			ctx.argument.required("number"),
			ctx.child.single("left", Tree),
			ctx.child.single("right", Tree),
		);
		tree.#deserializationCtx = ctx;
		return tree;
	}

	// We store the deserialization context
	#deserializationCtx?: DeserializationContext;

	constructor(
		readonly value: number,
		readonly left?: Tree,
		readonly right?: Tree,
	) {}

	serialize(ctx: SerializationContext) {
		// we pass the deserialization context used to create
		// this Tree instance to the serialization context, linking
		// them together
		ctx.source(this.#deserializationContext);
		ctx.argument(this.value);

		if (this.left) {
			ctx.child("left", this.left);
		}
		if (this.right) {
			ctx.child("right", this.right);
		}
	}
}

Full Changelog: v0.2.1...v0.3.0

v0.3.0-next.1

Preserving comments and formatting in dessert

This pre-release contains changes to the @bgotink/kdl/dessert API to support preserving comments and formatting when making modifications to a KDL file. This preservation can be enabled by linking the DeserializationContext used to deserialize the value from KDL to the SerializationContext used when serializing the value back to KDL. This is done via the SerializationContext's new source function.

This takes two steps:

1. Store the DeserializationContext while deserializing, e.g. by adding it to a hidden/private property on the returned object
2. Pass the stored DeserializationContext to the SerializationContext inside the serializer

It is important that step 2 is done at the start of the serializer. Doing so after making changes to the SerializationContext will throw an error.

Here we've taken the Tree class from the 0.3.0-next.0 release notes example and added in the necessary code to enable preserving formatting and comments:

class Tree {
	static deserialize(ctx: DeserializationContext): Tree {
		const tree = new Tree(
			ctx.argument.required("number"),
			ctx.child.single("left", Tree),
			ctx.child.single("right", Tree),
		);
		tree.#deserializationCtx = ctx;
		return tree;
	}

	// We store the deserialization context
	#deserializationCtx?: DeserializationContext;

	constructor(
		readonly value: number,
		readonly left?: Tree,
		readonly right?: Tree,
	) {}

	serialize(ctx: SerializationContext) {
		// we pass the deserialization context used to create
		// this Tree instance to the serialization context, linking
		// them together
		ctx.source(this.#deserializationContext);
		ctx.argument(this.value);

		if (this.left) {
			ctx.child("left", this.left);
		}
		if (this.right) {
			ctx.child("right", this.right);
		}
	}
}

Aligning the deserialization and serialization APIs

This pre-release improves the deserialization and serialization APIs:

• deserializers can now also accept parameters, just like serializers
• a new serialization function is now available: format, the counterpart to deserialization's parse

Changelog: v0.3.0-next.0...v0.3.0-next.1

v0.3.0-next.0

(De)Serialization Tools

This release introduces a new export: @bgotink/kdl/dessert which contains a bunch of utilities to deserialize KDL text into a JavaScript structure and serialize it back again.

The API doesn't enforce any programming paradigm, working with regular functions and with classes.

Here's an example with functions:

import {
	type DeserializationContext,
	type SerializationContext,
	deserialize,
	serialize,
} from "@bgotink/kdl/dessert";

type Tree = {value: number; left?: Tree; right?: Tree};

function treeDeserializer(ctx: DeserializationContext): Tree {
	return {
		value: ctx.argument.required("number"),
		left: ctx.child.single("left", treeDeserializer),
		right: ctx.child.single("right", treeDeserializer),
	};
}

export function readTree(node: Node): Tree {
	return deserialize(node, treeDeserializer);
}

function treeSerializer(ctx: SerializationContext, tree: Tree) {
	ctx.argument(tree.value);

	if (tree.left) {
		ctx.child("left", treeSerializer, tree.left);
	}
	if (tree.right) {
		ctx.child("right", treeSerializer, tree.right);
	}
}

export function writeTree(tree: Tree): Node {
	return serialize("root", treeDeserializer, tree);
}

and here's that same example using classes:

import {
	type DeserializationContext,
	type SerializationContext,
	deserialize,
	serialize,
} from "@bgotink/kdl/dessert";

class Tree {
	static deserialize(ctx: DeserializationContext): Tree {
		return new Tree(
			ctx.argument.required("number"),
			ctx.child.single("left", Tree),
			ctx.child.single("right", Tree),
		);
	}

	constructor(
		readonly value: number,
		readonly left?: Tree,
		readonly right?: Tree,
	) {}

	serialize(ctx: SerializationContext) {
		ctx.argument(this.value);

		if (this.left) {
			ctx.child("left", this.left);
		}
		if (this.right) {
			ctx.child("right", this.right);
		}
	}
}

export function readTree(node: Node): Tree {
	return deserialize(node, Tree);
}

export function writeTree(tree: Tree): Node {
	return serialize("root", tree);
}

Both of these examples turn the following KDL node into a Tree structure and back.

root 10 {
	left 5
	right 5 {
		left 2 { left 1; right 1 }
		right 3 { left 2; right 1 }
	}
}

BREAKING CHANGES

• The Location export is now called StoredLocation, TokenLocation is now called Location.
• The tag property has moved from the Entry to the Value. The getTag/setTag methods still exist on Entry.
• The Value class is now mutable, i.e. its value can be changed. Use the setValue method instead of modifying the value property directly to prevent issues in parsed documents.

Changelog: v0.2.1...v0.3.0-next.0

v0.2.1

Changelog: v0.2.0...v0.2.1

v0.2.0

What's Changed

The package now reads and writes KDL v2, a new version of the KDL language that's even better than v1.
Notable changes include

• Strings no longer have to be quoted if they don't contain certain disallowed characters and if there can't be any confusion with other data types like numbers or keywords

  print to=stderr Hello World
  // is equivalent to
  print to="stderr" "Hello" "World"

• Keywords are now prefixed with #: #true, #false, and #null
• New keywords: #inf, #-inf, and #nan
• Raw strings drop the r at the front, for example #"raw string"#
• Strings are now split into two kinds: single-line strings with single quotes, or multiline strings with three quotes
• Improved handling of whitespace throughout a KDL file

A new export @bgotink/kdl/v1-compat helps projects support both KDL v1 and v2 as well as automate the migration from v1 to v2.

Full Changelog: v0.1.6...v0.2.0

v0.1.7

Full Changelog: v0.1.6...v0.1.7

v0.2.0-next.6

Support KDL 2.0.0-draft.8

Full Changelog: v0.2.0-next.5...v0.2.0-next.6