Module serde_byte_buf 

Serde helper for encoding a Vec<u8> field as a CBOR byte string while remaining tolerant of the legacy array-of-integers encoding produced by serde’s blanket impl Serialize for Vec<T>.

Why both paths exist

Vec<u8>’s derived Serialize takes the generic sequence path, so ciborium previously emitted one CBOR item per byte (major type 4, array) instead of a single byte string (major type 2). For bytes >= 24 this costs two bytes per byte of payload instead of one, inflating stored block payloads by up to ~2x. serialize here switches to serialize_bytes to fix that going forward.

Blocks written before this change are permanent in object storage (they are never rewritten), so deserialize must keep accepting the legacy array form indefinitely. This is not a temporary compatibility shim — do not delete the visit_seq path in a future cleanup; array-form blobs remain in the lake forever (analytics/src/replication.rs also copies payload blobs verbatim between lakes, so array-form objects can keep arriving after the cutover indefinitely).

Why deserialize_byte_buf, not deserialize_bytes

ciborium’s deserialize_byte_buf routes both CBOR hints to the visitor: a byte-string header goes to visit_byte_buf, and an array header goes to visit_seq. deserialize_bytes also routes array headers to visit_seq, but only accepts a byte-string header when its length fits in the decoder’s fixed scratch buffer (4096 bytes) and otherwise errors out — rejecting both definite-length byte strings longer than that and indefinite-length byte strings entirely. Real block payloads run into the megabytes, so this module calls deserialize_byte_buf, never deserialize_bytes.

This dual-hint routing works only because CBOR is self-describing (the byte-string vs. array distinction is carried in the header). Reusing this module with a non-self-describing format (bincode, postcard) would need a different approach.

Functions

deserialize

Deserializes from either a CBOR byte string (current form) or a CBOR array of integers (legacy form, permanent in object storage).

serialize

Serializes v as a CBOR byte string.