View Source arrow_buffer (arrow v0.1.0)
Buffer implementation for arrow. This module adds suppport for buffers, or Contiguous Memory Regions.
There are multiple things to know about buffers[1]:
- Each value it stores is called an element or a slot[2].
- Each slot's length (in bytes) is a positive integer. As a result when we say that a slot has a length of 1, we mean that each slot has a length of 1 byte.
- The buffer's length in the metadata refers to the unpadded binary's size in bytes.
- All buffers have a size that is a multiple of 64. If their data's length is not a multiple of 64, it must be padded (in this implementation, by zeros).
- Null values are represented in this implementation by zeros.
- In this implementation buffers can be initialized from raw bytes as data apart datatypes supported by Arrow. This is so that the Validity Bitmap Buffer can be initialized.
[1]: https://arrow.apache.org/docs/format/Glossary.html#term-buffer
[2]: https://arrow.apache.org/docs/format/Glossary.html#term-slotLink to this section Summary
Functions
Creates a new buffer from a list of Erlang values or binaries, given its type
Creates a new buffer from a list of Erlang values or binaries, given its type and length
Returns the size of the buffer inclusive of padding in bytes.
Returns an Arrow buffer binary given a buffer.
Returns a list of Erlang values or binaries from a buffer.
Link to this section Functions
-spec from_erlang(Value :: [arrow_type:native_type()] | binary(), Type :: arrow_type:arrow_longhand_type()) -> Buffer :: #buffer{}.
-spec from_erlang(Data :: [arrow_type:native_type()] | binary(), Type :: arrow_type:arrow_longhand_type(), DataLen :: pos_integer() | undefined) -> Buffer :: #buffer{}.
-spec size(Buffer :: #buffer{}) -> pos_integer().
-spec to_arrow(Buffer :: #buffer{}) -> binary().
-spec to_erlang(Buffer :: #buffer{}) -> [arrow_type:native_type()].