HORIZON HASKELLDocslts/ghc-10.0.x77a86622026-10-31Search packagesCtrl K
Theme: |

GHC 10.0.0.20260917 · lts/ghc-10.0.x · 77a8662 · 2026-10-31

Moduleos-string-2.0.11Haskell2010

System.OsString.Data.ByteString.Short

A compact representation suitable for storing short byte strings in memory.

In typical use cases it can be imported alongside Data.ByteString, e.g.

import qualified Data.ByteString       as B
import qualified Data.ByteString.Short as B
         (ShortByteString, toShort, fromShort)

Other ShortByteString operations clash with Data.ByteString or Prelude functions however, so they should be imported qualified with a different alias e.g.

import qualified Data.ByteString.Short as B.Short
Contents
  1. The ShortByteString type
  2. Memory overhead
  3. Heap fragmentation
  4. Introducing and eliminating ShortByteStrings
  5. Basic interface
  6. Transforming ShortByteStrings
  7. Reducing ShortByteStrings (folds)
  8. Special folds
  9. Generating and unfolding ByteStrings
  10. Substrings
  11. Breaking strings
  12. Predicates
  13. Search for arbitrary substrings
  14. Searching ShortByteStrings
  15. Searching by equality
  16. Searching with a predicate
  17. Indexing ShortByteStrings
  18. Low level conversions
  19. Packing CStrings and pointers
  20. Using ShortByteStrings as CStrings
Synopsis
  • Packageos-string-2.0.11
  • Exports74
  • LanguageHaskell2010
  • LicenceBSD-3-Clause
  • SourceShort.hs

The ShortByteString type

newtypenewtype ShortByteString

A compact representation of a Word8 vector.

It has a lower memory overhead than a ByteString and does not contribute to heap fragmentation. It can be converted to or from a ByteString (at the cost of copying the string data). It supports very few other operations.

Instances14IsList, Eq, Data, Ord, Read, Show, …

Memory overhead

With GHC, the memory overheads are as follows, expressed in words and in bytes (words are 4 and 8 bytes on 32 or 64bit machines respectively).

  • B.ByteString unshared: 8 words; 32 or 64 bytes.

  • B.ByteString shared substring: 4 words; 16 or 32 bytes.

  • ShortByteString: 4 words; 16 or 32 bytes.

For the string data itself, both ShortByteString and B.ByteString use one byte per element, rounded up to the nearest word. For example, including the overheads, a length 10 ShortByteString would take 16 + 12 = 28 bytes on a 32bit platform and 32 + 16 = 48 bytes on a 64bit platform.

These overheads can all be reduced by 1 word (4 or 8 bytes) when the ShortByteString or B.ByteString is unpacked into another constructor.

For example:

data ThingId = ThingId {-# UNPACK #-} !Int
                       {-# UNPACK #-} !ShortByteString

This will take 1 + 1 + 3 words (the ThingId constructor + unpacked Int + unpacked ShortByteString), plus the words for the string data.

Heap fragmentation

With GHC, the B.ByteString representation uses pinned memory, meaning it cannot be moved by the GC. This is usually the right thing to do for larger strings, but for small strings using pinned memory can lead to heap fragmentation which wastes space. The ShortByteString type (and the Text type from the text package) use unpinned memory so they do not contribute to heap fragmentation. In addition, with GHC, small unpinned strings are allocated in the same way as normal heap allocations, rather than in a separate pinned area.

Introducing and eliminating ShortByteStrings

Basic interface

O(1) Extract the last element of a ShortByteString, which must be finite and non-empty. An exception will be thrown in the case of an empty ShortByteString.

This is a partial function, consider using unsnoc instead.

O(n) Extract the elements after the head of a ShortByteString, which must be non-empty. An exception will be thrown in the case of an empty ShortByteString.

This is a partial function, consider using uncons instead.

Note: copies the entire byte array

O(1) Extract the first element of a ShortByteString, which must be non-empty. An exception will be thrown in the case of an empty ShortByteString.

This is a partial function, consider using uncons instead.

Transforming ShortByteStrings

Reducing ShortByteStrings (folds)

valuefoldl :: (a -> Word8 -> a) -> a -> ShortByteString -> a

foldl, applied to a binary operator, a starting value (typically the left-identity of the operator), and a ShortByteString, reduces the ShortByteString using the binary operator, from left to right.

valuefoldr :: (Word8 -> a -> a) -> a -> ShortByteString -> a

foldr, applied to a binary operator, a starting value (typically the right-identity of the operator), and a ShortByteString, reduces the ShortByteString using the binary operator, from right to left.

Special folds

Generating and unfolding ByteStrings

valueunfoldr :: (a -> Maybe (Word8, a)) -> a -> ShortByteString

O(n), where n is the length of the result. The unfoldr function is analogous to the List 'unfoldr'. unfoldr builds a ShortByteString from a seed value. The function takes the element and returns Nothing if it is done producing the ShortByteString or returns Just (a,b), in which case, a is the next byte in the string, and b is the seed value for further production.

This function is not efficient/safe. It will build a list of [Word8] and run the generator until it returns Nothing, otherwise recurse infinitely, then finally create a ShortByteString.

If you know the maximum length, consider using unfoldrN.

Examples:

   unfoldr (\x -> if x <= 5 then Just (x, x + 1) else Nothing) 0
== pack [0, 1, 2, 3, 4, 5]

Substrings

Breaking strings

O(n) Break a ShortByteString into pieces separated by the byte argument, consuming the delimiter. I.e.

split 10  "a\nb\nd\ne" == ["a","b","d","e"]   -- fromEnum '\n' == 10
split 97  "aXaXaXa"    == ["","X","X","X",""] -- fromEnum 'a' == 97
split 120 "x"          == ["",""]             -- fromEnum 'x' == 120
split undefined ""     == []                  -- and not [""]

and

intercalate [c] . split c == id
split == splitWith . (==)

Note: copies the substrings

O(n) Splits a ShortByteString into components delimited by separators, where the predicate returns True for a separator element. The resulting components do not contain the separators. Two adjacent separators result in an empty component in the output. eg.

splitWith (==97) "aabbaca" == ["","","bb","c",""] -- fromEnum 'a' == 97
splitWith undefined ""     == []                  -- and not [""]

Predicates

Search for arbitrary substrings

valuebreakSubstring
  1. :: ShortByteString

    String to search for

  2. -> ShortByteString

    String to search in

  3. -> (ShortByteString, ShortByteString)

    Head and tail of string broken at substring

Break a string on a substring, returning a pair of the part of the string prior to the match, and the rest of the string.

The following relationships hold:

break (== c) l == breakSubstring (singleton c) l

For example, to tokenise a string, dropping delimiters:

tokenise x y = h : if null t then [] else tokenise x (drop (length x) t)
    where (h,t) = breakSubstring x y

To skip to the first occurrence of a string:

snd (breakSubstring x y)

To take the parts of a string before a delimiter:

fst (breakSubstring x y)

Note that calling `breakSubstring x` does some preprocessing work, so you should avoid unnecessarily duplicating breakSubstring calls with the same pattern.

Searching ShortByteStrings

Searching by equality

Searching with a predicate

O(n) The find function takes a predicate and a ShortByteString, and returns the first element in matching the predicate, or Nothing if there is no such element.

find f p = case findIndex f p of Just n -> Just (p ! n) ; _ -> Nothing

Indexing ShortByteStrings

Low level conversions

Packing CStrings and pointers

O(n). Construct a new ShortByteString from a CString. The resulting ShortByteString is an immutable copy of the original CString, and is managed on the Haskell heap. The original CString must be null terminated.

O(n). Construct a new ShortByteString from a CStringLen. The resulting ShortByteString is an immutable copy of the original CStringLen. The ShortByteString is a normal Haskell value and will be managed on the Haskell heap.

Using ShortByteStrings as CStrings

valueuseAsCString :: ShortByteString -> (CString -> IO a) -> IO a

O(n) construction. Use a ShortByteString with a function requiring a null-terminated CString. The CString is a copy and will be freed automatically; it must not be stored or used after the subcomputation finishes.

O(n) construction. Use a ShortByteString with a function requiring a CStringLen. As for useAsCString this function makes a copy of the original ShortByteString. It must not be stored or used after the subcomputation finishes.

Beware that this function does not add a terminating NUL byte at the end of CStringLen. If you need to construct a pointer to a null-terminated sequence, use useAsCString (and measure length independently if desired).