You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
* Add CBOR tags example, "Encoding and Decoding CWT (CBOR Web Token) with CBOR Tags".
* v2.1 passed 361+ million execs in coverage-guided fuzzing on Feb 17, 2020.
* Package cbor is a fast & safe CBOR encoder & decoder (RFC 7049) with a
standard API + toarray & keyasint struct tags, CBOR tags, float64->32->16,
CTAP2 & Canonical CBOR, duplicate map key options, and is customizable via
simple API.
Copy file name to clipboardExpand all lines: README.md
+89-23Lines changed: 89 additions & 23 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,7 +1,7 @@
1
1
[](https://github.com/fxamacker/cbor/blob/master/README.md)
2
2
3
3
# CBOR library in Go
4
-
[__`fxamacker/cbor`__](https://github.com/fxamacker/cbor) is a CBOR encoder and decoder in [Go](https://golang.org). v2.1 has a standard API, CBOR tags, 16/32/64-bit floats, options for duplicate map keys and more. Each release passes 375+ tests and 250+ million execs fuzzing.
4
+
[__`fxamacker/cbor`__](https://github.com/fxamacker/cbor) is a CBOR encoder and decoder in [Go](https://golang.org). It has a standard API, CBOR tags, 16/32/64-bit floats, options for duplicate map keys and more. Each release passes 375+ tests and 250+ million execs fuzzing.
__What is CBOR__? [CBOR](CBOR_GOLANG.md) ([RFC 7049](https://tools.ietf.org/html/rfc7049)) is a binary data format inspired by JSON and MessagePack. CBOR is used in [IETF](https://www.ietf.org) Internet Standards such as COSE ([RFC 8152](https://tools.ietf.org/html/rfc8152)) and CWT ([RFC 8392 CBOR Web Token](https://tools.ietf.org/html/rfc8392)). WebAuthn also uses CBOR.
14
14
15
-
This library is safe and fast. Here's how it compares to another library using test data from RFC 8392 A.1.
15
+
This CBOR library is safe and fast. Here's how it compares to another library using test data from RFC 8392 A.1.
@@ -169,24 +169,26 @@ Latest version is v2.x, which has:
169
169
* CoreDetEncOptions() is subject to change because it uses draft standard.
170
170
* PreferredUnsortedEncOptions() is subject to change because it uses draft standard.
171
171
*__Passed all tests__ – v2.x passed all 375+ tests on amd64, arm64, ppc64le and s390x with linux.
172
-
*__Passed fuzzing__ – v2.1 passed 275+ million execs in coverage-guided fuzzing on Feb 17, 2020 and is still fuzzing.
172
+
*__Passed fuzzing__ – v2.1 passed 361+ million execs in coverage-guided fuzzing on Feb 17, 2020.
173
173
174
174
__Why v2.x?__:
175
175
176
176
v1 required breaking API changes to support new features like CBOR tags, detection of duplicate map keys, and having more functions with identical signatures to `encoding/json`.
177
177
178
-
v2.1 is roughly 26% faster and uses 57% fewer allocs than v1 when decoding COSE and CWT using default options.
178
+
v2.1 is roughly 26% faster and uses 57% fewer allocs than v1.x when decoding COSE and CWT using default options.
179
179
180
-
__Roadmap__:
180
+
__Recent Activity__:
181
181
182
-
* v2.1 (Feb. 17, 2020)
182
+
*Release v2.1 (Feb. 17, 2020)
183
183
-[x] CBOR tags (major type 6) for encoding and decoding (pushed to master branch)
184
184
-[x] Decoding options for duplicate map key detection: `DupMapKeyQuiet` (default) and `DupMapKeyEnforcedAPF`
185
185
-[x] Decoding optimizations (pushed to master branch). Structs using keyasint tag (like COSE and CWT) is
186
-
24-28% faster and 53-61% fewer allocs than v1.5 and v2.0.1.
186
+
24-28% faster and 53-61% fewer allocs than both v1.5 and v2.0.1.
187
+
188
+
__Roadmap__:
187
189
188
-
* v2.2
189
-
-[ ] CBOR BSTR <--> Go byte array. In the meantime, use CBOR BSTR with Go byte slice.
190
+
*Milestone v2.2
191
+
-[ ] CBOR BSTR <--> Go byte array. In the meantime, use CBOR BSTR with Go byte slice.
190
192
-[ ] If time allows, add more CBOR tags as default or optional built-in tags.
191
193
192
194
<hr>
@@ -369,18 +371,18 @@ Many function signatures are identical to Go's encoding/json, such as:
369
371
`Marshal`, `Unmarshal`, `NewEncoder`, `NewDecoder`, `encoder.Encode`, and `decoder.Decode`.
370
372
371
373
Interfaces identical or comparable to Go's encoding, encoding/json, or encoding/gob include:
372
-
`Marshaler`, `UnMarshaler`, `BinaryMarshaler`, and `BinaryUnmarshaler`.
374
+
`Marshaler`, `Unmarshaler`, `BinaryMarshaler`, and `BinaryUnmarshaler`.
373
375
374
376
Like `encoding/json`, `RawMessage` can be used to delay CBOR decoding or precompute CBOR encoding.
375
377
376
378
"Mode" in this API means defined way of encoding or decoding -- it links the standard API to CBOR options and CBOR tags.
377
379
378
380
EncMode and DecMode are interfaces created from EncOptions or DecOptions structs.
379
-
For example, `em:= cbor.EncOptions{...}.EncMode()` or `em := cbor.CanonicalEncOptions().EncMode()`.
381
+
For example, `em, err := cbor.EncOptions{...}.EncMode()` or `em, err := cbor.CanonicalEncOptions().EncMode()`.
380
382
381
383
EncMode and DecMode use immutable options so their behavior won't accidentally change at runtime. Modes are intended to be reused and are safe for concurrent use.
382
384
383
-
__API for Default Mode of Encoding & Decoding__
385
+
__API for Default Mode__
384
386
385
387
If default options are acceptable, then you don't need to create EncMode or DecMode.
386
388
```
@@ -407,6 +409,12 @@ type EncOptions struct {
407
409
408
410
// EncMode returns an EncMode interface created from EncOptions.
409
411
func (opts EncOptions) EncMode() (EncMode, error)
412
+
413
+
// EncModeWithTags returns EncMode with options and tags that are both immutable.
`TagSet` can be used to associate user-defined Go type(s) to tag number(s). It's also used to create EncMode or DecMode. For example, `em := EncOptions{...}.EncModeWithTags(ts)` or `em := EncOptions{...}.EncModeWithSharedTags(ts)`. This allows every standard API exported by em (like `Marshal` and `NewEncoder`) to use the specified tags automatically.
455
+
456
+
`Tag` and `RawTag` can be used to encode/decode a tag number with a Go value, but `TagSet` is generally recommended.
457
+
458
+
```
459
+
type TagSet interface {
460
+
// Add adds given tag number(s), content type, and tag options to TagSet.
461
+
Add(opts TagOptions, contentType reflect.Type, num uint64, nestedNum ...uint64) error
462
+
463
+
// Remove removes given tag content type from TagSet.
464
+
Remove(contentType reflect.Type)
465
+
}
466
+
```
467
+
468
+
`Tag` and `RawTag` types can also be used to encode/decode tag number with Go value.
436
469
```
470
+
type Tag struct {
471
+
Number uint64
472
+
Content interface{}
473
+
}
437
474
438
-
The `keyasint` and `toarray` struct tags are worth knowing. They can reduce programming effort, improve system performance, and reduce the size of serialized data.
475
+
type RawTag struct {
476
+
Number uint64
477
+
Content RawMessage
478
+
}
479
+
```
439
480
440
481
See [API docs (godoc.org)](https://godoc.org/github.com/fxamacker/cbor) for more details and more functions. See [Usage section](#usage) for usage and code examples.
441
482
@@ -451,8 +492,6 @@ Options for the decoding and encoding are listed here.
451
492
|DecTagOptional |Tag numbers are only checked for validity if present for time values.|
452
493
|DecTagRequired |Tag numbers must be provided for time values except for CBOR Null and CBOR Undefined.|
453
494
454
-
DecTagOptional and DecTagRequired will check if tag numbers are 0 and 1 (if present) and treat other tag numbers as an error for time values.
455
-
456
495
CBOR Null and CBOR Undefined are silently treated as Go's zero time instant. Go's `time` package provides `IsZero` function, which reports whether t represents the zero time instant, January 1, year 1, 00:00:00 UTC.
457
496
458
497
__Duplicate Map Key Options__
@@ -468,7 +507,7 @@ __Duplicate Map Key Options__
468
507
469
508
__Integers always encode to the shortest form that preserves value__. Encoding of other data types and map key sort order are determined by encoding options.
470
509
471
-
Encoder has options that can be set individually. These functions are provided to create and return a modifiable EncOptions struct with predefined settings.
510
+
These functions are provided to create and return a modifiable EncOptions struct with predefined settings.
__Encoding and Decoding CWT (CBOR Web Token) with CBOR Tags__
676
+
```
677
+
// Use signedCWT struct defined in "Decoding CWT" example.
637
678
638
-
`TagSet` can be used to associate a user-defined Go type(s) to tag number(s). It's also used to create EncMode or DecMode. For example, `em := EncOptions{...}.EncModeWithTags(ts)` or `em := EncOptions{...}.EncModeWithSharedTags(ts)`. This allows every standard API exported by em (like `Marshal` and `NewEncoder`) to use the specified tags automatically.
679
+
// Create TagSet (safe for concurrency).
680
+
tags := cbor.NewTagSet()
681
+
// Register tag COSE_Sign1 18 with signedCWT type.
`Tag` and `RawTag` can be used to encode/decode a tag number with a Go value, but `TagSet` is generally recommended.
687
+
// Create DecMode with immutable tags.
688
+
dm, _ := cbor.DecOptions{}.DecModeWithTags(tags)
689
+
690
+
// Unmarshal to signedCWT with tag support.
691
+
var v signedCWT
692
+
if err := dm.Unmarshal(data, &v); err != nil {
693
+
return err
694
+
}
695
+
696
+
// Create EncMode with immutable tags.
697
+
em, _ := cbor.EncOptions{}.EncModeWithTags(tags)
698
+
699
+
// Marshal signedCWT with tag number.
700
+
if data, err := cbor.Marshal(v); err != nil {
701
+
return err
702
+
}
703
+
```
641
704
642
705
For more examples, see [examples_test.go](example_test.go).
643
706
@@ -702,9 +765,9 @@ See [Benchmarks for fxamacker/cbor](CBOR_BENCHMARKS.md).
702
765
703
766
__Over 375 tests__ must pass on 4 architectures before tagging a release. They include all RFC 7049 examples, bugs found by fuzzing, 2 maliciously crafted CBOR data, and over 87 tests with malformed data.
704
767
705
-
__Code coverage__ must not fall below 95% when tagging a release. Code coverage is 97.9% (`go test -cover`) for cbor v2.0 which is among the highest for libraries (in Go) of this type.
768
+
__Code coverage__ must not fall below 95% when tagging a release. Code coverage is 98.5% (`go test -cover`) for cbor v2.1 which is among the highest for libraries (in Go) of this type.
706
769
707
-
__Coverage-guided fuzzing__ must pass 250+ million execs before tagging a release. E.g. v1.4 passed 532+ million execs in coverage-guided fuzzing at the time of release and reached 4+ billion execs 18 days later. Fuzzing uses [fxamacker/cbor-fuzz](https://github.com/fxamacker/cbor-fuzz). Default corpus has:
770
+
__Coverage-guided fuzzing__ must pass 250+ million execs before tagging a release. Fuzzing uses [fxamacker/cbor-fuzz](https://github.com/fxamacker/cbor-fuzz). Default corpus has:
708
771
709
772
* 2 files related to WebAuthn (FIDO U2F key).
710
773
* 3 files with custom struct.
@@ -714,6 +777,8 @@ __Coverage-guided fuzzing__ must pass 250+ million execs before tagging a releas
714
777
715
778
Over 1,100 files (corpus) are used for fuzzing because it includes fuzz-generated corpus.
716
779
780
+
To prevent excessive delays, fuzzing is not restarted for a release if changes are limited to docs and comments.
781
+
717
782
## Versions and API Changes
718
783
This project uses [Semantic Versioning](https://semver.org), so the API is always backwards compatible unless the major version number changes.
719
784
@@ -743,11 +808,12 @@ Please read the license for additional disclaimers and terms.
743
808
744
809
__Making this library better__
745
810
746
-
* Montgomery Edwards⁴⁴⁸ for contributing float16 conversion code, updating the docs, creating comparison charts & slideshow, filing issues, nudging me to ask for feedback from users, and helping with design of v2.0 and v2.1 API.
811
+
* Montgomery Edwards⁴⁴⁸ for [x448/float16](https://github.com/x448/float16), updating the docs, creating charts & slideshow, filing issues, nudging me to ask for feedback from users, helping with design of v2.0-v2.1 API, and general idea for DupMapKeyEnforcedAPF.
747
812
* Stefan Tatschner for using this library in [sep](https://git.sr.ht/~rumpelsepp/sep), being the 1st to discover my CBOR library, requesting time.Time in issue #1, and submitting this library in a [PR to cbor.io](https://github.com/cbor/cbor.github.io/pull/56) on Aug 12, 2019.
748
813
* Yawning Angel for using this library to [oasis-core](https://github.com/oasislabs/oasis-core), and requesting BinaryMarshaler in issue #5.
749
814
* Jernej Kos for requesting RawMessage in issue #11 and offering feedback on v2.1 API for CBOR tags.
750
-
* ZenGround0 for using this library in [go-filecoin](https://github.com/filecoin-project/go-filecoin), filing "toarray" bug in issue #129, and requesting CBOR BSTR <--> Go array in #133.
815
+
* ZenGround0 for using this library in [go-filecoin](https://github.com/filecoin-project/go-filecoin), filing "toarray" bug in issue #129, and requesting
816
+
CBOR BSTR <--> Go array in #133.
751
817
* Keith Randall for [fixing Go bugs and providing workarounds](https://github.com/golang/go/issues/36400) so we don't have to wait for new versions of Go.
0 commit comments