Skip to content

Commit b2ab983

Browse files
authored
Update README.md for v2.1 (#158)
* Add CBOR tags example, "Encoding and Decoding CWT (CBOR Web Token) with CBOR Tags". * v2.1 passed 361+ million execs in coverage-guided fuzzing on Feb 17, 2020. * Package cbor is a fast & safe CBOR encoder & decoder (RFC 7049) with a standard API + toarray & keyasint struct tags, CBOR tags, float64->32->16, CTAP2 & Canonical CBOR, duplicate map key options, and is customizable via simple API.
1 parent 289df8c commit b2ab983

3 files changed

Lines changed: 97 additions & 26 deletions

File tree

README.md

Lines changed: 89 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
[![CBOR Library - Slideshow and Latest Docs.](https://github.com/fxamacker/images/raw/master/cbor/v2.1.0/cbor_slides.gif)](https://github.com/fxamacker/cbor/blob/master/README.md)
22

33
# CBOR library in Go
4-
[__`fxamacker/cbor`__](https://github.com/fxamacker/cbor) is a CBOR encoder and decoder in [Go](https://golang.org). v2.1 has a standard API, CBOR tags, 16/32/64-bit floats, options for duplicate map keys and more. Each release passes 375+ tests and 250+ million execs fuzzing.
4+
[__`fxamacker/cbor`__](https://github.com/fxamacker/cbor) is a CBOR encoder and decoder in [Go](https://golang.org). It has a standard API, CBOR tags, 16/32/64-bit floats, options for duplicate map keys and more. Each release passes 375+ tests and 250+ million execs fuzzing.
55

66
[![](https://github.com/fxamacker/cbor/workflows/ci/badge.svg)](https://github.com/fxamacker/cbor/actions?query=workflow%3Aci)
77
[![](https://github.com/fxamacker/cbor/workflows/cover%20%E2%89%A597%25/badge.svg)](https://github.com/fxamacker/cbor/actions?query=workflow%3A%22cover+%E2%89%A597%25%22)
@@ -12,7 +12,7 @@
1212

1313
__What is CBOR__? [CBOR](CBOR_GOLANG.md) ([RFC 7049](https://tools.ietf.org/html/rfc7049)) is a binary data format inspired by JSON and MessagePack. CBOR is used in [IETF](https://www.ietf.org) Internet Standards such as COSE ([RFC 8152](https://tools.ietf.org/html/rfc8152)) and CWT ([RFC 8392 CBOR Web Token](https://tools.ietf.org/html/rfc8392)). WebAuthn also uses CBOR.
1414

15-
This library is safe and fast. Here's how it compares to another library using test data from RFC 8392 A.1.
15+
This CBOR library is safe and fast. Here's how it compares to another library using test data from RFC 8392 A.1.
1616

1717
| | fxamacker/cbor 2.1 |ugorji/go 1.1.7 |
1818
|-------------------|-----------------------------|----------------------------------|
@@ -169,24 +169,26 @@ Latest version is v2.x, which has:
169169
* CoreDetEncOptions() is subject to change because it uses draft standard.
170170
* PreferredUnsortedEncOptions() is subject to change because it uses draft standard.
171171
* __Passed all tests__ – v2.x passed all 375+ tests on amd64, arm64, ppc64le and s390x with linux.
172-
* __Passed fuzzing__ – v2.1 passed 275+ million execs in coverage-guided fuzzing on Feb 17, 2020 and is still fuzzing.
172+
* __Passed fuzzing__ – v2.1 passed 361+ million execs in coverage-guided fuzzing on Feb 17, 2020.
173173

174174
__Why v2.x?__:
175175

176176
v1 required breaking API changes to support new features like CBOR tags, detection of duplicate map keys, and having more functions with identical signatures to `encoding/json`.
177177

178-
v2.1 is roughly 26% faster and uses 57% fewer allocs than v1 when decoding COSE and CWT using default options.
178+
v2.1 is roughly 26% faster and uses 57% fewer allocs than v1.x when decoding COSE and CWT using default options.
179179

180-
__Roadmap__:
180+
__Recent Activity__:
181181

182-
* v2.1 (Feb. 17, 2020)
182+
* Release v2.1 (Feb. 17, 2020)
183183
- [x] CBOR tags (major type 6) for encoding and decoding (pushed to master branch)
184184
- [x] Decoding options for duplicate map key detection: `DupMapKeyQuiet` (default) and `DupMapKeyEnforcedAPF`
185185
- [x] Decoding optimizations (pushed to master branch). Structs using keyasint tag (like COSE and CWT) is
186-
24-28% faster and 53-61% fewer allocs than v1.5 and v2.0.1.
186+
24-28% faster and 53-61% fewer allocs than both v1.5 and v2.0.1.
187+
188+
__Roadmap__:
187189

188-
* v2.2
189-
- [ ] CBOR BSTR <--> Go byte array. In the meantime, use CBOR BSTR with Go byte slice.
190+
* Milestone v2.2
191+
- [ ] CBOR BSTR <--> Go byte array. In the meantime, use CBOR BSTR with Go byte slice.
190192
- [ ] If time allows, add more CBOR tags as default or optional built-in tags.
191193

192194
<hr>
@@ -369,18 +371,18 @@ Many function signatures are identical to Go's encoding/json, such as:
369371
`Marshal`, `Unmarshal`, `NewEncoder`, `NewDecoder`, `encoder.Encode`, and `decoder.Decode`.
370372

371373
Interfaces identical or comparable to Go's encoding, encoding/json, or encoding/gob include:
372-
`Marshaler`, `UnMarshaler`, `BinaryMarshaler`, and `BinaryUnmarshaler`.
374+
`Marshaler`, `Unmarshaler`, `BinaryMarshaler`, and `BinaryUnmarshaler`.
373375

374376
Like `encoding/json`, `RawMessage` can be used to delay CBOR decoding or precompute CBOR encoding.
375377

376378
"Mode" in this API means defined way of encoding or decoding -- it links the standard API to CBOR options and CBOR tags.
377379

378380
EncMode and DecMode are interfaces created from EncOptions or DecOptions structs.
379-
For example, `em := cbor.EncOptions{...}.EncMode()` or `em := cbor.CanonicalEncOptions().EncMode()`.
381+
For example, `em, err := cbor.EncOptions{...}.EncMode()` or `em, err := cbor.CanonicalEncOptions().EncMode()`.
380382

381383
EncMode and DecMode use immutable options so their behavior won't accidentally change at runtime. Modes are intended to be reused and are safe for concurrent use.
382384

383-
__API for Default Mode of Encoding & Decoding__
385+
__API for Default Mode__
384386

385387
If default options are acceptable, then you don't need to create EncMode or DecMode.
386388
```
@@ -407,6 +409,12 @@ type EncOptions struct {
407409
408410
// EncMode returns an EncMode interface created from EncOptions.
409411
func (opts EncOptions) EncMode() (EncMode, error)
412+
413+
// EncModeWithTags returns EncMode with options and tags that are both immutable.
414+
func (opts EncOptions) EncModeWithTags(tags TagSet) (EncMode, error)
415+
416+
// EncModeWithSharedTags returns EncMode with immutable options and mutable shared tags.
417+
func (opts EncOptions) EncModeWithSharedTags(tags TagSet) (EncMode, error)
410418
```
411419

412420
__API for Predefined Encoding Options__
@@ -433,9 +441,42 @@ type DecOptions struct {
433441
434442
// DecMode returns a DecMode interface created from DecOptions.
435443
func (opts DecOptions) DecMode() (DecMode, error)
444+
445+
// DecModeWithTags returns DecMode with options and tags that are both immutable.
446+
func (opts DecOptions) DecModeWithTags(tags TagSet) (DecMode, error)
447+
448+
// DecModeWithSharedTags returns DecMode with immutable options and mutable shared tags.
449+
func (opts DecOptions) DecModeWithSharedTags(tags TagSet) (DecMode, error)
450+
```
451+
452+
__API for Using CBOR Tags__
453+
454+
`TagSet` can be used to associate user-defined Go type(s) to tag number(s). It's also used to create EncMode or DecMode. For example, `em := EncOptions{...}.EncModeWithTags(ts)` or `em := EncOptions{...}.EncModeWithSharedTags(ts)`. This allows every standard API exported by em (like `Marshal` and `NewEncoder`) to use the specified tags automatically.
455+
456+
`Tag` and `RawTag` can be used to encode/decode a tag number with a Go value, but `TagSet` is generally recommended.
457+
458+
```
459+
type TagSet interface {
460+
// Add adds given tag number(s), content type, and tag options to TagSet.
461+
Add(opts TagOptions, contentType reflect.Type, num uint64, nestedNum ...uint64) error
462+
463+
// Remove removes given tag content type from TagSet.
464+
Remove(contentType reflect.Type)
465+
}
466+
```
467+
468+
`Tag` and `RawTag` types can also be used to encode/decode tag number with Go value.
436469
```
470+
type Tag struct {
471+
Number uint64
472+
Content interface{}
473+
}
437474
438-
The `keyasint` and `toarray` struct tags are worth knowing. They can reduce programming effort, improve system performance, and reduce the size of serialized data.
475+
type RawTag struct {
476+
Number uint64
477+
Content RawMessage
478+
}
479+
```
439480

440481
See [API docs (godoc.org)](https://godoc.org/github.com/fxamacker/cbor) for more details and more functions. See [Usage section](#usage) for usage and code examples.
441482

@@ -451,8 +492,6 @@ Options for the decoding and encoding are listed here.
451492
|DecTagOptional |Tag numbers are only checked for validity if present for time values.|
452493
|DecTagRequired |Tag numbers must be provided for time values except for CBOR Null and CBOR Undefined.|
453494

454-
DecTagOptional and DecTagRequired will check if tag numbers are 0 and 1 (if present) and treat other tag numbers as an error for time values.
455-
456495
CBOR Null and CBOR Undefined are silently treated as Go's zero time instant. Go's `time` package provides `IsZero` function, which reports whether t represents the zero time instant, January 1, year 1, 00:00:00 UTC.
457496

458497
__Duplicate Map Key Options__
@@ -468,7 +507,7 @@ __Duplicate Map Key Options__
468507

469508
__Integers always encode to the shortest form that preserves value__. Encoding of other data types and map key sort order are determined by encoding options.
470509

471-
Encoder has options that can be set individually. These functions are provided to create and return a modifiable EncOptions struct with predefined settings.
510+
These functions are provided to create and return a modifiable EncOptions struct with predefined settings.
472511

473512
|Predefined EncOptions |Description |
474513
|-----------------------------|---------------------------------------------------------------------------------------|
@@ -633,11 +672,35 @@ if data, err := cbor.Marshal(v); err != nil {
633672
}
634673
```
635674

636-
__CBOR Tags__
675+
__Encoding and Decoding CWT (CBOR Web Token) with CBOR Tags__
676+
```
677+
// Use signedCWT struct defined in "Decoding CWT" example.
637678
638-
`TagSet` can be used to associate a user-defined Go type(s) to tag number(s). It's also used to create EncMode or DecMode. For example, `em := EncOptions{...}.EncModeWithTags(ts)` or `em := EncOptions{...}.EncModeWithSharedTags(ts)`. This allows every standard API exported by em (like `Marshal` and `NewEncoder`) to use the specified tags automatically.
679+
// Create TagSet (safe for concurrency).
680+
tags := cbor.NewTagSet()
681+
// Register tag COSE_Sign1 18 with signedCWT type.
682+
tags.Add(
683+
cbor.TagOptions{EncTag: cbor.EncTagRequired, DecTag: cbor.DecTagRequired},
684+
reflect.TypeOf(signedCWT{}),
685+
18)
639686
640-
`Tag` and `RawTag` can be used to encode/decode a tag number with a Go value, but `TagSet` is generally recommended.
687+
// Create DecMode with immutable tags.
688+
dm, _ := cbor.DecOptions{}.DecModeWithTags(tags)
689+
690+
// Unmarshal to signedCWT with tag support.
691+
var v signedCWT
692+
if err := dm.Unmarshal(data, &v); err != nil {
693+
return err
694+
}
695+
696+
// Create EncMode with immutable tags.
697+
em, _ := cbor.EncOptions{}.EncModeWithTags(tags)
698+
699+
// Marshal signedCWT with tag number.
700+
if data, err := cbor.Marshal(v); err != nil {
701+
return err
702+
}
703+
```
641704

642705
For more examples, see [examples_test.go](example_test.go).
643706

@@ -702,9 +765,9 @@ See [Benchmarks for fxamacker/cbor](CBOR_BENCHMARKS.md).
702765

703766
__Over 375 tests__ must pass on 4 architectures before tagging a release. They include all RFC 7049 examples, bugs found by fuzzing, 2 maliciously crafted CBOR data, and over 87 tests with malformed data.
704767

705-
__Code coverage__ must not fall below 95% when tagging a release. Code coverage is 97.9% (`go test -cover`) for cbor v2.0 which is among the highest for libraries (in Go) of this type.
768+
__Code coverage__ must not fall below 95% when tagging a release. Code coverage is 98.5% (`go test -cover`) for cbor v2.1 which is among the highest for libraries (in Go) of this type.
706769

707-
__Coverage-guided fuzzing__ must pass 250+ million execs before tagging a release. E.g. v1.4 passed 532+ million execs in coverage-guided fuzzing at the time of release and reached 4+ billion execs 18 days later. Fuzzing uses [fxamacker/cbor-fuzz](https://github.com/fxamacker/cbor-fuzz). Default corpus has:
770+
__Coverage-guided fuzzing__ must pass 250+ million execs before tagging a release. Fuzzing uses [fxamacker/cbor-fuzz](https://github.com/fxamacker/cbor-fuzz). Default corpus has:
708771

709772
* 2 files related to WebAuthn (FIDO U2F key).
710773
* 3 files with custom struct.
@@ -714,6 +777,8 @@ __Coverage-guided fuzzing__ must pass 250+ million execs before tagging a releas
714777

715778
Over 1,100 files (corpus) are used for fuzzing because it includes fuzz-generated corpus.
716779

780+
To prevent excessive delays, fuzzing is not restarted for a release if changes are limited to docs and comments.
781+
717782
## Versions and API Changes
718783
This project uses [Semantic Versioning](https://semver.org), so the API is always backwards compatible unless the major version number changes.
719784

@@ -743,11 +808,12 @@ Please read the license for additional disclaimers and terms.
743808

744809
__Making this library better__
745810

746-
* Montgomery Edwards⁴⁴⁸ for contributing float16 conversion code, updating the docs, creating comparison charts & slideshow, filing issues, nudging me to ask for feedback from users, and helping with design of v2.0 and v2.1 API.
811+
* Montgomery Edwards⁴⁴⁸ for [x448/float16](https://github.com/x448/float16), updating the docs, creating charts & slideshow, filing issues, nudging me to ask for feedback from users, helping with design of v2.0-v2.1 API, and general idea for DupMapKeyEnforcedAPF.
747812
* Stefan Tatschner for using this library in [sep](https://git.sr.ht/~rumpelsepp/sep), being the 1st to discover my CBOR library, requesting time.Time in issue #1, and submitting this library in a [PR to cbor.io](https://github.com/cbor/cbor.github.io/pull/56) on Aug 12, 2019.
748813
* Yawning Angel for using this library to [oasis-core](https://github.com/oasislabs/oasis-core), and requesting BinaryMarshaler in issue #5.
749814
* Jernej Kos for requesting RawMessage in issue #11 and offering feedback on v2.1 API for CBOR tags.
750-
* ZenGround0 for using this library in [go-filecoin](https://github.com/filecoin-project/go-filecoin), filing "toarray" bug in issue #129, and requesting CBOR BSTR <--> Go array in #133.
815+
* ZenGround0 for using this library in [go-filecoin](https://github.com/filecoin-project/go-filecoin), filing "toarray" bug in issue #129, and requesting
816+
CBOR BSTR <--> Go array in #133.
751817
* Keith Randall for [fixing Go bugs and providing workarounds](https://github.com/golang/go/issues/36400) so we don't have to wait for new versions of Go.
752818

753819
__Help clarifying CBOR RFC 7049 or 7049bis__

doc.go

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,8 +2,10 @@
22
// Licensed under the MIT License. See LICENSE in the project root for license information.
33

44
/*
5-
Package cbor provides a fuzz-tested CBOR encoder and decoder with full support
6-
for float16, Canonical CBOR, CTAP2 Canonical CBOR, and custom settings.
5+
Package cbor is a fast & safe CBOR encoder & decoder (RFC 7049) with a
6+
standard API + toarray & keyasint struct tags, CBOR tags, float64->32->16,
7+
CTAP2 & Canonical CBOR, duplicate map key options, and is customizable via
8+
simple API.
79
810
CBOR encoding options allow "preferred serialization" by encoding integers and floats
911
to their smallest forms (like float16) when values fit.

example_test.go

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -454,7 +454,10 @@ func Example_signedCWTWithTag() {
454454

455455
// Register tag COSE_Sign1 18 with signedCWT type.
456456
tags := cbor.NewTagSet()
457-
if err := tags.Add(cbor.TagOptions{EncTag: cbor.EncTagRequired, DecTag: cbor.DecTagRequired}, reflect.TypeOf(signedCWT{}), 18); err != nil {
457+
if err := tags.Add(
458+
cbor.TagOptions{EncTag: cbor.EncTagRequired, DecTag: cbor.DecTagRequired},
459+
reflect.TypeOf(signedCWT{}),
460+
18); err != nil {
458461
fmt.Println("error:", err)
459462
}
460463

0 commit comments

Comments
 (0)