An unmaterialized sequence of bytes that may be pasted into a mutable byte array.
Modulebytebuild-0.3.17.0Haskell2010
Data.Bytes.Builder
Contents
- Bounded Primitives
- Evaluation
- Materialized Byte Sequences
- Byte Sequence Encodings
- Encode Integral Types
- Human-Readable
- Unsigned Words
- 64-bit
- 32-bit
- 16-bit
- 8-bit
- Machine-Readable
- One
- Big Endian
- Little Endian
- LEB128
- VLQ
- Many
- Big Endian
- Little Endian
- Prefixing with Length
- Encode Floating-Point Types
- Human-Readable
- Replication
- Control
- Rebuild
- IO
Synopsis
- newtype Builder
- value fromBounded
- value run
- value runOnto
- value runOntoLength
- value reversedOnto
- value putMany
- value putManyConsLength
- value bytes
- value chunks
- value copy
- value copyCons
- value copy2
- value insert
- value byteArray
- value shortByteString
- value textUtf8
- value textJsonString
- value shortTextUtf8
- value shortTextJsonString
- value cstring
- value cstring#
- value cstringLen
- value stringUtf8
- value sevenEightRight
- value sevenEightSmile
- value word64Dec
- value word32Dec
- value word16Dec
- value word8Dec
- value wordDec
- value naturalDec
- value int64Dec
- value int32Dec
- value int16Dec
- value int8Dec
- value intDec
- value integerDec
- value word64PaddedUpperHex
- value word32PaddedUpperHex
- value word16PaddedUpperHex
- value word16PaddedLowerHex
- value word16LowerHex
- value word16UpperHex
- value word8PaddedUpperHex
- value word8LowerHex
- value ascii
- value ascii2
- value ascii3
- value ascii4
- value ascii5
- value ascii6
- value ascii7
- value ascii8
- value char
- value word8
- value word256BE
- value word128BE
- value word64BE
- value word32BE
- value word16BE
- value int64BE
- value int32BE
- value int16BE
- value word256LE
- value word128LE
- value word64LE
- value word32LE
- value word16LE
- value int64LE
- value int32LE
- value int16LE
- value intLEB128
- value int32LEB128
- value int64LEB128
- value wordLEB128
- value word16LEB128
- value word32LEB128
- value word64LEB128
- value wordVlq
- value word32Vlq
- value word64Vlq
- value word8Array
- value word16ArrayBE
- value word32ArrayBE
- value word64ArrayBE
- value word128ArrayBE
- value word256ArrayBE
- value int64ArrayBE
- value int32ArrayBE
- value int16ArrayBE
- value word16ArrayLE
- value word32ArrayLE
- value word64ArrayLE
- value word128ArrayLE
- value word256ArrayLE
- value int64ArrayLE
- value int32ArrayLE
- value int16ArrayLE
- value consLength
- value consLength32LE
- value consLength32BE
- value consLength64BE
- value doubleDec
- value replicate
- value flush
- value rebuild
- value hPut
- Packagebytebuild-0.3.17.0
- Exports108
- LanguageHaskell2010
- LicenceBSD-3-Clause
- SourceUnsafe.hs
Bounded Primitives
Convert a bounded builder to an unbounded one. If the size
is a constant, use Arithmetic.Nat.constant as the first argument
to let GHC conjure up this value for you.
Evaluation
Run a builder.
value
runOnto Run a builder. The resulting chunks are consed onto the beginning of an existing sequence of chunks.
value
runOntoLength Variant of runOnto that additionally returns the number of bytes consed onto the suffix.
value
reversedOnto Variant of runOnto that conses the additional chunks in reverse order.
value
putMany Run a builder against lots of elements. This fills the same
underlying buffer over and over again. Do not let the argument to
the callback escape from the callback (i.e. do not write it to an
IORef). Also, do not unsafeFreezeByteArray any of the mutable
byte arrays in the callback. The intent is that the callback will
write the buffer out.
value
putManyConsLength :: (Foldable f, MonadIO m)=> Nat nNumber of bytes used by the serialization of the length
-> (Int -> Builder n)Length serialization function
-> IntSize of shared chunk (use 8176 if uncertain)
-> (a -> Builder)Value builder
-> f aCollection of values
-> (MutableBytes RealWorld -> m b)Consume chunks.
-> m ()
Variant of putMany that prefixes each pushed array of chunks with the number of bytes that the chunks in each batch required. (This excludes the bytes required to encode the length itself.) This is useful for chunked HTTP encoding.
Materialized Byte Sequences
Paste byte chunks into a builder.
Create a builder from a byte sequence. This always results in a
call to memcpy. This is beneficial when the byte sequence is
known to be small (less than 256 bytes).
Variant of copy that additionally pastes an extra byte in front of the bytes.
Create a builder from two byte sequences. This always results in two
calls to memcpy. This is beneficial when the byte sequences are
known to be small (less than 256 bytes).
Create a builder from a byte sequence. This never calls memcpy.
Instead, it pushes a chunk that references the argument byte sequence.
This wastes the remaining space in the active chunk, so it may adversely
affect performance if used carelessly. See flush for a way to mitigate
this problem. This functions is most beneficial when the byte sequence
is known to be large (more than 8192 bytes).
Create a builder from an unsliced byte sequence. Implemented with bytes.
Create a builder from a short bytestring. Implemented with bytes.
Create a builder from text. The text will be UTF-8 encoded.
Create a builder from text. The text will be UTF-8 encoded.
Create a builder from text. The text will be UTF-8 encoded, and JSON special characters will be escaped. Additionally, the result is surrounded by double quotes. For example:
foo ==> "foo"(no escape sequences)\_"_/ ==> "\\_\"_/"(escapes backslashes and quotes)hello<ESC>world ==> "hello\u001Bworld"(where<ESC>is code point 0x1B)
Create a builder from a NUL-terminated CString. This ignores any
textual encoding, copying bytes until NUL is reached.
Create a builder from a C string with explicit length. The builder must be executed before the C string is freed.
Create a builder from a cons-list of Char. These must be UTF-8 encoded.
Byte Sequence Encodings
Encode seven bytes into eight so that the encoded form is eight-bit clean. Specifically segment the input bytes inot 7-bit groups (lowest-to-highest index byte, most-to-least significant bit within a byte), pads the last group with trailing zeros, and forms octects by prepending a zero to each group.
The name was chosen because this pads the input bits with zeros on the right, and also because this was likely the originally-indended behavior of the SMILE standard (see sevenEightSmile). Right padding the input bits to a multiple of seven, as in this variant, is consistent with base64 encodings (which encodes 3 bytes in 4) and base85 (which encodes 4 bytes in 5).
Encode seven bytes into eight so that the encoded form is eight-bit clean. Specifically segment the input bytes inot 7-bit groups (lowest-to-highest index byte, most-to-least significant bit within a byte), then pad each group with zeros on the left until each group is an octet.
The name was chosen because this is the implementation that is used (probably unintentionally) in the reference SMILE implementation, and so is expected tp be accepted by existing SMILE consumers.
Encode Integral Types
Human-Readable
Encodes an unsigned 64-bit integer as decimal. This encoding never starts with a zero unless the argument was zero.
Encodes an unsigned 16-bit integer as decimal. This encoding never starts with a zero unless the argument was zero.
Encodes an unsigned 16-bit integer as decimal. This encoding never starts with a zero unless the argument was zero.
Encodes an unsigned 8-bit integer as decimal. This encoding never starts with a zero unless the argument was zero.
Encodes an unsigned machine-sized integer as decimal. This encoding never starts with a zero unless the argument was zero.
Encodes an unsigned arbitrary-precision integer as decimal. This encoding never starts with a zero unless the argument was zero.
Encodes a signed 64-bit integer as decimal. This encoding never starts with a zero unless the argument was zero. Negative numbers are preceded by a minus sign. Positive numbers are not preceded by anything.
Encodes a signed 32-bit integer as decimal. This encoding never starts with a zero unless the argument was zero. Negative numbers are preceded by a minus sign. Positive numbers are not preceded by anything.
Encodes a signed 16-bit integer as decimal. This encoding never starts with a zero unless the argument was zero. Negative numbers are preceded by a minus sign. Positive numbers are not preceded by anything.
Encodes a signed 8-bit integer as decimal. This encoding never starts with a zero unless the argument was zero. Negative numbers are preceded by a minus sign. Positive numbers are not preceded by anything.
Encodes a signed machine-sized integer as decimal. This encoding never starts with a zero unless the argument was zero. Negative numbers are preceded by a minus sign. Positive numbers are not preceded by anything.
Encode a signed arbitrary-precision integer as decimal. This encoding never starts with a zero unless the argument was zero. Negative numbers are preceded by a minus sign. Positive numbers are not preceded by anything.
Unsigned Words
64-bit
Encode a 64-bit unsigned integer as hexadecimal, zero-padding
the encoding to 16 digits. This uses uppercase for the alphabetical
digits. For example, this encodes the number 1022 as 00000000000003FE.
32-bit
Encode a 32-bit unsigned integer as hexadecimal, zero-padding
the encoding to 8 digits. This uses uppercase for the alphabetical
digits. For example, this encodes the number 1022 as 000003FE.
16-bit
Encode a 16-bit unsigned integer as hexadecimal, zero-padding
the encoding to 4 digits. This uses uppercase for the alphabetical
digits. For example, this encodes the number 1022 as 03FE.
Encode a 16-bit unsigned integer as hexadecimal, zero-padding
the encoding to 4 digits. This uses lowercase for the alphabetical
digits. For example, this encodes the number 1022 as 03fe.
Encode a 16-bit unsigned integer as hexadecimal without leading
zeroes. This uses lowercase for the alphabetical digits. For
example, this encodes the number 1022 as 3fe.
Encode a 16-bit unsigned integer as hexadecimal without leading
zeroes. This uses uppercase for the alphabetical digits. For
example, this encodes the number 1022 as 3FE.
8-bit
Encode a 8-bit unsigned integer as hexadecimal, zero-padding
the encoding to 2 digits. This uses uppercase for the alphabetical
digits. For example, this encodes the number 11 as 0B.
Encode a 16-bit unsigned integer as hexadecimal without leading
zeroes. This uses lowercase for the alphabetical digits. For
example, this encodes the number 1022 as 3FE.
Encode an ASCII char. Precondition: Input must be an ASCII character. This is not checked.
Encode two ASCII characters. Precondition: Must be an ASCII characters. This is not checked.
Encode three ASCII characters. Precondition: Must be an ASCII characters. This is not checked.
Encode four ASCII characters. Precondition: Must be an ASCII characters. This is not checked.
Encode five ASCII characters. Precondition: Must be an ASCII characters. This is not checked.
Encode six ASCII characters. Precondition: Must be an ASCII characters. This is not checked.
Encode seven ASCII characters. Precondition: Must be an ASCII characters. This is not checked.
Encode eight ASCII characters. Precondition: Must be an ASCII characters. This is not checked.
Encode a UTF-8 char. This only uses as much space as is required.
Machine-Readable
One
Requires exactly 1 byte.
Big Endian
Requires exactly 32 bytes. Dump the octets of a 256-bit word in a big-endian fashion.
Requires exactly 16 bytes. Dump the octets of a 128-bit word in a big-endian fashion.
Requires exactly 8 bytes. Dump the octets of a 64-bit word in a big-endian fashion.
Requires exactly 4 bytes. Dump the octets of a 32-bit word in a big-endian fashion.
Requires exactly 2 bytes. Dump the octets of a 16-bit word in a big-endian fashion.
Requires exactly 8 bytes. Dump the octets of a 64-bit signed integer in a big-endian fashion.
Requires exactly 4 bytes. Dump the octets of a 32-bit signed integer in a big-endian fashion.
Requires exactly 2 bytes. Dump the octets of a 16-bit signed integer in a big-endian fashion.
Little Endian
Requires exactly 32 bytes. Dump the octets of a 256-bit word in a little-endian fashion.
Requires exactly 16 bytes. Dump the octets of a 128-bit word in a little-endian fashion.
Requires exactly 8 bytes. Dump the octets of a 64-bit word in a little-endian fashion.
Requires exactly 4 bytes. Dump the octets of a 32-bit word in a little-endian fashion.
Requires exactly 2 bytes. Dump the octets of a 16-bit word in a little-endian fashion.
Requires exactly 8 bytes. Dump the octets of a 64-bit signed integer in a little-endian fashion.
Requires exactly 4 bytes. Dump the octets of a 32-bit signed integer in a little-endian fashion.
Requires exactly 2 bytes. Dump the octets of a 16-bit signed integer in a little-endian fashion.
LEB128
Encode a signed machine-sized integer with LEB-128. This uses zig-zag encoding.
Encode a 32-bit signed integer with LEB-128. This uses zig-zag encoding.
Encode a 64-bit signed integer with LEB-128. This uses zig-zag encoding.
Encode a machine-sized word with LEB-128.
Encode a 16-bit word with LEB-128.
Encode a 32-bit word with LEB-128.
Encode a 64-bit word with LEB-128.
VLQ
Encode a machine-sized word with VLQ.
Encode a 32-bit word with VLQ.
Encode a 64-bit word with VLQ.
Many
Big Endian
Little Endian
Prefixing with Length
value
consLength Prefix a builder with the number of bytes that it requires.
Variant of consLength32BE the encodes the length in a little-endian fashion.
Prefix a builder with its size in bytes. This size is presented as a big-endian 32-bit word. The need to prefix a builder with its length shows up a numbers of wire protocols including those of PostgreSQL and Apache Kafka. Note the equivalence:
forall (n :: Int) (x :: Builder).
let sz = sizeofByteArray (run n (consLength32BE x))
consLength32BE x === word32BE (fromIntegral sz) <> xHowever, using consLength32BE is much more efficient here since it only materializes the ByteArray once.
Prefix a builder with its size in bytes. This size is presented as a big-endian 64-bit word. See consLength32BE.
Encode Floating-Point Types
Human-Readable
Encode a double-floating-point number, using decimal notation or
scientific notation depending on the magnitude. This has undefined
behavior when representing +inf, -inf, and NaN. It will not
crash, but the generated numbers will be nonsense.
Replication
value
replicate Replicate a byte the given number of times.
Control
Push the buffer currently being filled onto the chunk list, allocating a new active buffer of the requested size. This is helpful when a small builder is sandwhiched between two large zero-copy builders:
insert bigA <> flush 1 <> word8 0x42 <> insert bigBWithout flush 1, word8 0x42 would see the zero-byte active
buffer that insert returned, decide that it needed more space,
and allocate a 4080-byte buffer to which only a single byte
would be written.
Rebuild
This function and the documentation for it are copied from Takano Akio's fast-builder library.
rebuild b is equivalent to b, but it allows GHC to assume
that b will be run at most once. This can enable various
optimizations that greately improve performance.
There are two types of typical situations where a use of rebuild is often a win:
When constructing a builder using a recursive function. e.g.
rebuild $ foldr ....When constructing a builder using a conditional expression. e.g.
rebuild $ case x of ...