v2.1.0 (Feb. 17, 2020)

This release focused on three items:

• CBOR tags (major type 6) for encoding and decoding
• Duplicate map key detection options for decoding
• Faster decoding with less memory use (to help offset overhead of new features)

Decoding got faster and memory use dropped more than expected, especially for 'keyasint' structs.

Here's how this library compares to another one using default options and test data from RFC 8392 A.1.

	fxamacker/cbor 2.1	ugorji/go 1.1.7
Encode CWT claims	457 ns/op, 176 B/op, 2 allocs/op	995 ns/op, 1424 B/op, 4 allocs/op
Decode CWT claims	796 ns/op, 176 B/op, 6 allocs/op	1105 ns/op, 568 B/op, 6 allocs/op

Changes include:

• #44 - Add support for CBOR tags (major type 6) with API for built-in and user-defined tags
• #122 - Add decoding options for duplicate map keys
• #147 - Decode "keyasint" structs 26% faster and with 57% fewer allocs (COSE, CWT, etc.)
• #125 - Add encoding option to tag or not tag time values
• #47 - Improve decoding speed (optimizations already identified during v1)
• Basic optimization for CBOR tags feature, more can be done in later releases
• #151 - Implement default encoding for uninitialized time.Time values when tag-required option is specified.
• Update README.md and benchmarks

There will be three ways to create EncMode (and similar API for DecMode):

Function (returns EncMode)	Encoding Mode
EncOptions{...}.EncMode()	immutable options, no tags
🆕 EncOptions{...}.EncModeWithTags(ts)	both options & tags are immutable
🆕 EncOptions{...}.EncModeWithSharedTags(ts)	immutable options & mutable shared tags

To minimize bloat, only tags 0 and 1 (datetime) are going to be part of the default build. The API provides a way for users to register and handle other CBOR tags for user-defined Go types.

In future releases, additional tags may be provided by the library in a modular way (possibly build flags or optional packages).

Special Thanks

@x448 for helping with v2.1 API, docs, and providing the general idea for DupMapKeyEnforcedAPF.
@kostko for providing feedback on the draft v2.1 API for CBOR tags.
@laurencelundblade for providing feedback to @x448 regarding CBOR Null & Undef interop with JSON.
@ZenGround0 for using this library in go-filecoin and requesting a useful feature that'll be in v2.2.

v2.1 fuzzing passed 380+ million execs before release, and passed 2.1+ billion execs on Feb 21, 2020.

v1.5.1 (Feb 9, 2020)

This release is outdated. Upgrading to v2.0.1 or newer is recommended.

Changes include:

• Backport v2.0 fix to v1.5 (sanitize decoded NaN and Infinity time values.) See issue #141.
• Add more unit tests for floating-point data.

Fuzzing passed 500+ million execs on Feb 9, 2020.

v2.0.1 (Feb 6, 2020)

Changes include:

• Fix: API bug introduced in v2.0.0 with (Un)Marshaler and (Un)MarshalerWithMode interface mixup
• Fix: Reject CBOR non-array data for structs decorated with "toarray"

More at #129 and #137.

Special thanks to ZenGround0 for reporting issues.

v2.0.1 passed 300+ million execs fuzzing at time of release.
v2.0.1 passed 2+ billion execs fuzzing on Feb 9, 2020.

v2.0.0 (Feb 2, 2020)

Changes in v2.0:

• Improved API.
• Faster performance.
• Reduced memory usage.
• Improved code quality using around 20 linters.
• Replaced EncOptions.TimeRFC3339 bool with TimeMode (5 settings).
• Decode NaN and Infinity time values to Go's "zero time" value.
• Removed deprecated v1 options and Valid() function.
• Many improvements to README.md.

Why 2.0?

v2 decoupled options from encoding/decoding functions so that:

• new features like CBOR tags can be added without breaking API changes.
• more encoding/decoding function signatures are identical to encoding/json.
• more function signatures can remain stable forever even as CBOR options & tags evolve.

Roadmap

• v2.1 (Feb. 9, 2020) support for CBOR tags (major type 6) and some decoder optimizations.
• v2.2 (Feb. 2020) options for handling duplicate map keys.

CBOR Tags (major type 6) was moved to milestone v2.1. and is over 50% done.

Status

v2.0 passed 1+ billion execs in coverage-guided fuzzing before release. This is the most well-tested and most reliable release of this library. Upgrading is highly recommended.

Improved API

More function signatures match encoding/json. These will never change their parameters or return types:
Marshal, Unmarshal, NewEncoder, NewDecoder, encoder.Encode, decoder.Decode.

"Mode" in this API means definite way of encoding or decoding.

EncMode and DecMode are interfaces created from EncOptions or DecOptions structs.

EncMode and DecMode use immutable options so their behavior won't accidentally change at runtime. Modes are intended to be reused and are safe for concurrent use.

// Create EncOptions, using either struct literal or a function.
opts := cbor.CanonicalEncOptions()

// If needed, modify options -- e.g. how time values should be encoded.
opts.Time = cbor.TimeUnix

// Create reusable EncMode interface with immutable options, safe for concurrent use.
em, err := opts.EncMode()   

// Use EncMode like encoding/json, with same function signatures.
b, err := em.Marshal(v)
// or
encoder := em.NewEncoder(w)
err := encoder.Encode(v)

Package Level Functions

These package level functions use default options without CBOR tags. Their API matches encoding/json. EncMode and DecMode also provide these functions with same signatures.

b, err := cbor.Marshal(v)
err := cbor.Unmarshal(b, &v)
encoder := cbor.NewEncoder(w)
decoder := cbor.NewDecoder(r)

EncOptions.Time

Replaced EncOptions.TimeRFC3339 bool with TimeMode:

• TimeUnix // secs, encodes to CBOR integer using fewest bytes
• TimeUnixMicro // μs, encodes to CBOR float with subsecs in fractional part
• TimeUnixDynamic // secs or μs, encodes to either int or float depending on empty subsecs
• TimeRFC3339 // secs, encodes to string
• TimeRFC3339Nano // ns, encodes to string with trailing zeros removed

v1.5.0 (Jan. 12, 2020)

This release adds encoding options while maintaining backward compatibility.

With EncOptions.ShortestFloat = ShortestFloat16, floating-point values (including subnormals) encode
float64 -> float32 -> float16 when values can round-trip. This is optimized by avoiding the round-trip test except for a subset of subnormal input values and using a faster test for other values.

Conversions for infinity and NaN use InfConvert and NaNConvert settings.

New functions like CanonicalEncOptions(), CTAP2EncOptions(), etc. return predefined EncOptions that can be modified to create custom configurations.

Changes include:

• Use x448/float16 (same team as this library) because all 4+ billion possible conversions are tested.
• Feature: Add funcs that return predefined EncOptions (commit b7b5733, e203b29)
  • CanonicalEncOptions() -- Canonical CBOR (RFC 7049)
  • CTAP2EncOptions() -- CTAP2 Canonical CBOR
  • CoreDetEncOptions() -- Core Deterministic Encoding (draft RFC, subject to change)
  • PreferredUnsortedEncOptions() - Preferred unsorted serialization (draft RFC, subject to change)
• Feature: Add ShortestFloat option for encoding (commit 29e78ee)
  • ShortestFloatNone (default)
  • ShortestFloat16
• Feature: Add NanConvertMode for encoding (commit 16c573c)
  • NaNConvertNone
  • NanConvert7e00
  • NanConvertQuiet
  • NaNConvertPreserveSignal
• Feature: Add InfConvertMode for encoding (commit 16c573c)
  • InfConvertNone
  • InfConvertFloat16
• Refactor and improve quality of code, docs, and tests

UPDATE: Fuzzing passed 4.75+ billion execs with Go 1.12 on Jan. 22, 2020.
Fuzzing passed 2.88+ billion execs with Go 1.13 on Jan. 31, 2020 (slower machine).

v1.4.0

Release v1.4.0 (Dec 25, 2019)

Changes include:

• Feature: Deprecate bool encoding options and add int SortMode (commit 3b78ee0)
• Reliability: Use float16 to float32 conversion func that had all 65536 results verified to be correct (commit 48850b2)
• Fix: Fix decoding of float16 subnormal numbers (commit 48850b2)

Fuzzing passed 532+ million execs with Go 1.12 at the time of release.
Fuzzing passed 4+ billion execs with Go 1.13 about 17-18 days after release.

v1.3.4

Release v1.3.4 (Dec 18, 2019)

v1.3.1 to v.1.3.4 benefited from extensive code review and refactoring.

Changes include:

• Fix: Limit nested levels to 32 for arrays, maps, tags to prevent stack exhaustion exploits (commit 3aa4328)
• Fix: Fix error when decoding to not-nil interface (commit d26d3cd)
• Misc: Refactor to improve readability and maintainability (commit d2d6a95)

Fuzzing reached 370+ million execs with Go 1.12 at the time of release.

Update: On Dec. 25, 2019, fuzzing passed 2+ billion execs with Go 1.13.

corpus: 1092 (46h8m ago), crashers: 0, restarts: 1/10000, execs: 2028736632 (3194/sec), cover: 2006, uptime: 176h27m

v1.3.3

Release v1.3.3 (Dec 9, 2019)

Changes include:

• Fix: Fix panic when encoding new type with float32 as its underlying type (commit bb1c06a)
• Fix: Change CBOR string validation error from SemanticError to SyntaxError (commit acaec05)
• Fix: Reject CBOR indefinite length byte/text string with tagged chunks (commit 28e2c0b)
• Fix: Reject CBOR indefinite length text string with invalid UTF-8 chunks (commit 9f1f677)
• Misc: Add unit tests based on latest 7049bis and remove tests made redundant by this (commit ac1c292)

Fuzzing reached 276+ million execs at the time of release.

UPDATE: It continued fuzzing and reached 2.79+ billion execs about 8 days after release.

workers: 2, corpus: 1074 (44h26m ago), crashers: 0, restarts: 1/10000, execs: 2796329664 (3386/sec), cover: 2011, uptime: 229h25m

v1.3.2

Release v1.3.2 (Nov 27, 2019)

This release checks for additional issues while decoding well-formed CBOR messages.

Changes include:

• Skip CBOR array/map elements on incompatible Go type
• Check if CBOR type can be used as Go map key when map key type is interface{}

A separate project, cbor-fuzz was updated to use new fxamacker/cbor features it had missed. Cover is noticably higher (better) during fuzzing.

UPDATE: Fuzzing reached 4.2+ billion execs on Dec 3, 2019. Corpus is temporarily low (as shown here) for v1.3.2, but it's already 1000+ after combining Go 1.12 + Go 1.13 fuzzing corpus folders after this run.

2019/12/03 15:37:19 workers: 2, corpus: 602 (12h17m ago), crashers: 0, restarts: 1/10000, execs: 4226276531 (7203/sec), cover: 2000, uptime: 162h59m

v1.3.1

Release v1.3.1 (Nov 24, 2019)

Issue #46 resulted in filing an errata to RFC 7049 (CBOR) after the same mistake was found in both 7049 and Wikipedia. RFC 7049 author (cabo) confirmed within an hour directly in #46 which was super nice of him.

I'll let fuzzing continue for 1-10 days, due in part to issue #46 and initial valid fuzzing corpus. Maybe it'll generate fewer corpus files this time to reach a good stopping point.

Most users of v1.3.0 won't notice any practical difference from these bugfixes. They involve data validation rules and an obscure difference in sorting rule for canonical encoding.

Changes include:

• Fix: Relax decoding restriction on CBOR int to Go float (commit 71ea0c5)
• Fix: Separate CTAP2 and RFC 7049 canonical encoding (commit 7164aa3)
• Fix: Reject indefinite-length byte/text string if chunks are indefinite-length (commit a4adae8)
• Fix: Reject CBOR primitive 2-byte simple value < 32 (commit aa44241)

This release passed 877k+ executions (19 hours, and still running) of coverage-guided fuzzing using fxamacker/cbor-fuzz.

workers: 2, corpus: 403 (48s ago), crashers: 0, restarts: 1/10000, execs: 877930249 (12405/sec), cover: 1501, uptime: 19h39m