package flatbuf

Import Path
	github.com/apache/arrow-go/v18/arrow/internal/flatbuf (on go.dev)

Dependency Relation
	imports 2 packages, and imported by 3 packages


Package-Level Type Names (total 61)
/* sort by: | */
/ Opaque binary data (*Binary) Init(buf []byte, i flatbuffers.UOffsetT) (*Binary) Table() flatbuffers.Table *Binary : github.com/google/flatbuffers/go.FlatBuffer func GetRootAsBinary(buf []byte, offset flatbuffers.UOffsetT) *Binary
/ Logically the same as Binary, but the internal representation uses a view / struct that contains the string length and either the string's entire data / inline (for small strings) or an inlined prefix, an index of another buffer, / and an offset pointing to a slice in that buffer (for non-small strings). / / Since it uses a variable number of data buffers, each Field with this type / must have a corresponding entry in `variadicBufferCounts`. (*BinaryView) Init(buf []byte, i flatbuffers.UOffsetT) (*BinaryView) Table() flatbuffers.Table *BinaryView : github.com/google/flatbuffers/go.FlatBuffer func GetRootAsBinaryView(buf []byte, offset flatbuffers.UOffsetT) *BinaryView
/ Length of the data (this is aligned so there can be a gap between this and / the metadata). (*Block) Init(buf []byte, i flatbuffers.UOffsetT) / Length of the metadata / Length of the data (this is aligned so there can be a gap between this and / the metadata). / Length of the metadata / Index to the start of the RecordBlock (note this is past the Message header) / Index to the start of the RecordBlock (note this is past the Message header) (*Block) Table() flatbuffers.Table *Block : github.com/google/flatbuffers/go.FlatBuffer func (*Footer).Dictionaries(obj *Block, j int) bool func (*Footer).RecordBatches(obj *Block, j int) bool
/ Optional compression for the memory buffers constituting IPC message / bodies. Intended for use with RecordBatch but could be used for other / message types / Compressor library. / For LZ4_FRAME, each compressed buffer must consist of a single frame. (*BodyCompression) Init(buf []byte, i flatbuffers.UOffsetT) / Indicates the way the record batch body was compressed / Compressor library. / For LZ4_FRAME, each compressed buffer must consist of a single frame. / Indicates the way the record batch body was compressed (*BodyCompression) Table() flatbuffers.Table *BodyCompression : github.com/google/flatbuffers/go.FlatBuffer func GetRootAsBodyCompression(buf []byte, offset flatbuffers.UOffsetT) *BodyCompression func (*RecordBatch).Compression(obj *BodyCompression) *BodyCompression func (*RecordBatch).Compression(obj *BodyCompression) *BodyCompression
/ Provided for forward compatibility in case we need to support different / strategies for compressing the IPC message body (like whole-body / compression rather than buffer-level) in the future ( BodyCompressionMethod) String() string BodyCompressionMethod : expvar.Var BodyCompressionMethod : fmt.Stringer func (*BodyCompression).Method() BodyCompressionMethod func BodyCompressionAddMethod(builder *flatbuffers.Builder, method BodyCompressionMethod) func (*BodyCompression).MutateMethod(n BodyCompressionMethod) bool const BodyCompressionMethodBUFFER
(*Bool) Init(buf []byte, i flatbuffers.UOffsetT) (*Bool) Table() flatbuffers.Table *Bool : github.com/google/flatbuffers/go.FlatBuffer func GetRootAsBool(buf []byte, offset flatbuffers.UOffsetT) *Bool
/ ---------------------------------------------------------------------- / A Buffer represents a single contiguous memory segment (*Buffer) Init(buf []byte, i flatbuffers.UOffsetT) / The absolute length (in bytes) of the memory buffer. The memory is found / from offset (inclusive) to offset + length (non-inclusive). When building / messages using the encapsulated IPC message, padding bytes may be written / after a buffer, but such padding bytes do not need to be accounted for in / the size here. / The absolute length (in bytes) of the memory buffer. The memory is found / from offset (inclusive) to offset + length (non-inclusive). When building / messages using the encapsulated IPC message, padding bytes may be written / after a buffer, but such padding bytes do not need to be accounted for in / the size here. / The relative offset into the shared memory page where the bytes for this / buffer starts / The relative offset into the shared memory page where the bytes for this / buffer starts (*Buffer) Table() flatbuffers.Table *Buffer : github.com/google/flatbuffers/go.FlatBuffer func (*SparseMatrixIndexCSR).IndicesBuffer(obj *Buffer) *Buffer func (*SparseMatrixIndexCSR).IndptrBuffer(obj *Buffer) *Buffer func (*SparseMatrixIndexCSX).IndicesBuffer(obj *Buffer) *Buffer func (*SparseMatrixIndexCSX).IndptrBuffer(obj *Buffer) *Buffer func (*SparseTensor).Data(obj *Buffer) *Buffer func (*SparseTensorIndexCOO).IndicesBuffer(obj *Buffer) *Buffer func (*Tensor).Data(obj *Buffer) *Buffer func (*RecordBatch).Buffers(obj *Buffer, j int) bool func (*SparseMatrixIndexCSR).IndicesBuffer(obj *Buffer) *Buffer func (*SparseMatrixIndexCSR).IndptrBuffer(obj *Buffer) *Buffer func (*SparseMatrixIndexCSX).IndicesBuffer(obj *Buffer) *Buffer func (*SparseMatrixIndexCSX).IndptrBuffer(obj *Buffer) *Buffer func (*SparseTensor).Data(obj *Buffer) *Buffer func (*SparseTensorIndexCOO).IndicesBuffer(obj *Buffer) *Buffer func (*SparseTensorIndexCSF).IndicesBuffers(obj *Buffer, j int) bool func (*SparseTensorIndexCSF).IndptrBuffers(obj *Buffer, j int) bool func (*Tensor).Data(obj *Buffer) *Buffer
( CompressionType) String() string CompressionType : expvar.Var CompressionType : fmt.Stringer func (*BodyCompression).Codec() CompressionType func BodyCompressionAddCodec(builder *flatbuffers.Builder, codec CompressionType) func (*BodyCompression).MutateCodec(n CompressionType) bool const CompressionTypeLZ4_FRAME const CompressionTypeZSTD
/ Date is either a 32-bit or 64-bit signed integer type representing an / elapsed time since UNIX epoch (1970-01-01), stored in either of two units: / / * Milliseconds (64 bits) indicating UNIX time elapsed since the epoch (no / leap seconds), where the values are evenly divisible by 86400000 / * Days (32 bits) since the UNIX epoch (*Date) Init(buf []byte, i flatbuffers.UOffsetT) (*Date) MutateUnit(n DateUnit) bool (*Date) Table() flatbuffers.Table (*Date) Unit() DateUnit *Date : github.com/google/flatbuffers/go.FlatBuffer func GetRootAsDate(buf []byte, offset flatbuffers.UOffsetT) *Date
( DateUnit) String() string DateUnit : expvar.Var DateUnit : fmt.Stringer func (*Date).Unit() DateUnit func DateAddUnit(builder *flatbuffers.Builder, unit DateUnit) func (*Date).MutateUnit(n DateUnit) bool const DateUnitDAY const DateUnitMILLISECOND
/ Exact decimal value represented as an integer value in two's / complement. Currently 32-bit (4-byte), 64-bit (8-byte), / 128-bit (16-byte) and 256-bit (32-byte) integers are used. / The representation uses the endianness indicated in the Schema. / Number of bits per value. The accepted widths are 32, 64, 128 and 256. / We use bitWidth for consistency with Int::bitWidth. (*Decimal) Init(buf []byte, i flatbuffers.UOffsetT) / Number of bits per value. The accepted widths are 32, 64, 128 and 256. / We use bitWidth for consistency with Int::bitWidth. / Total number of decimal digits / Number of digits after the decimal point "." / Total number of decimal digits / Number of digits after the decimal point "." (*Decimal) Table() flatbuffers.Table *Decimal : github.com/google/flatbuffers/go.FlatBuffer func GetRootAsDecimal(buf []byte, offset flatbuffers.UOffsetT) *Decimal
/ For sending dictionary encoding information. Any Field can be / dictionary-encoded, but in this case none of its children may be / dictionary-encoded. / There is one vector / column per dictionary, but that vector / column / may be spread across multiple dictionary batches by using the isDelta / flag (*DictionaryBatch) Data(obj *RecordBatch) *RecordBatch (*DictionaryBatch) Id() int64 (*DictionaryBatch) Init(buf []byte, i flatbuffers.UOffsetT) / If isDelta is true the values in the dictionary are to be appended to a / dictionary with the indicated id. If isDelta is false this dictionary / should replace the existing dictionary. (*DictionaryBatch) MutateId(n int64) bool / If isDelta is true the values in the dictionary are to be appended to a / dictionary with the indicated id. If isDelta is false this dictionary / should replace the existing dictionary. (*DictionaryBatch) Table() flatbuffers.Table *DictionaryBatch : github.com/google/flatbuffers/go.FlatBuffer func GetRootAsDictionaryBatch(buf []byte, offset flatbuffers.UOffsetT) *DictionaryBatch
(*DictionaryEncoding) DictionaryKind() DictionaryKind / The known dictionary id in the application where this data is used. In / the file or streaming formats, the dictionary ids are found in the / DictionaryBatch messages / The dictionary indices are constrained to be non-negative integers. If / this field is null, the indices must be signed int32. To maximize / cross-language compatibility and performance, implementations are / recommended to prefer signed integer types over unsigned integer types / and to avoid uint64 indices unless they are required by an application. (*DictionaryEncoding) Init(buf []byte, i flatbuffers.UOffsetT) / The dictionary indices are constrained to be non-negative integers. If / this field is null, the indices must be signed int32. To maximize / cross-language compatibility and performance, implementations are / recommended to prefer signed integer types over unsigned integer types / and to avoid uint64 indices unless they are required by an application. / By default, dictionaries are not ordered, or the order does not have / semantic meaning. In some statistical, applications, dictionary-encoding / is used to represent ordered categorical data, and we provide a way to / preserve that metadata here (*DictionaryEncoding) MutateDictionaryKind(n DictionaryKind) bool / The known dictionary id in the application where this data is used. In / the file or streaming formats, the dictionary ids are found in the / DictionaryBatch messages / By default, dictionaries are not ordered, or the order does not have / semantic meaning. In some statistical, applications, dictionary-encoding / is used to represent ordered categorical data, and we provide a way to / preserve that metadata here (*DictionaryEncoding) Table() flatbuffers.Table *DictionaryEncoding : github.com/google/flatbuffers/go.FlatBuffer func GetRootAsDictionaryEncoding(buf []byte, offset flatbuffers.UOffsetT) *DictionaryEncoding func (*Field).Dictionary(obj *DictionaryEncoding) *DictionaryEncoding func (*Field).Dictionary(obj *DictionaryEncoding) *DictionaryEncoding
/ ---------------------------------------------------------------------- / Dictionary encoding metadata / Maintained for forwards compatibility, in the future / Dictionaries might be explicit maps between integers and values / allowing for non-contiguous index values ( DictionaryKind) String() string DictionaryKind : expvar.Var DictionaryKind : fmt.Stringer func (*DictionaryEncoding).DictionaryKind() DictionaryKind func DictionaryEncodingAddDictionaryKind(builder *flatbuffers.Builder, dictionaryKind DictionaryKind) func (*DictionaryEncoding).MutateDictionaryKind(n DictionaryKind) bool const DictionaryKindDenseArray
(*Duration) Init(buf []byte, i flatbuffers.UOffsetT) (*Duration) MutateUnit(n TimeUnit) bool (*Duration) Table() flatbuffers.Table (*Duration) Unit() TimeUnit *Duration : github.com/google/flatbuffers/go.FlatBuffer func GetRootAsDuration(buf []byte, offset flatbuffers.UOffsetT) *Duration
/ ---------------------------------------------------------------------- / Endianness of the platform producing the data ( Endianness) String() string Endianness : expvar.Var Endianness : fmt.Stringer func (*Schema).Endianness() Endianness func SchemaAddEndianness(builder *flatbuffers.Builder, endianness Endianness) func (*Schema).MutateEndianness(n Endianness) bool const EndiannessBig const EndiannessLittle
/ Represents Arrow Features that might not have full support / within implementations. This is intended to be used in / two scenarios: / 1. A mechanism for readers of Arrow Streams / and files to understand that the stream or file makes / use of a feature that isn't supported or unknown to / the implementation (and therefore can meet the Arrow / forward compatibility guarantees). / 2. A means of negotiating between a client and server / what features a stream is allowed to use. The enums / values here are intented to represent higher level / features, additional details maybe negotiated / with key-value pairs specific to the protocol. / / Enums added to this list should be assigned power-of-two values / to facilitate exchanging and comparing bitmaps for supported / features. ( Feature) String() string Feature : expvar.Var Feature : fmt.Stringer func (*Schema).Features(j int) Feature func (*Schema).MutateFeatures(j int, n Feature) bool const FeatureCOMPRESSED_BODY const FeatureDICTIONARY_REPLACEMENT const FeatureUNUSED
/ ---------------------------------------------------------------------- / A field represents a named column in a record / row batch or child of a / nested type. / Present only if the field is dictionary encoded. / children apply only to nested data types like Struct, List and Union. For / primitive types children will have length 0. (*Field) ChildrenLength() int / children apply only to nested data types like Struct, List and Union. For / primitive types children will have length 0. / User-defined metadata (*Field) CustomMetadataLength() int / This is the type of the decoded value if the field is dictionary encoded. / Present only if the field is dictionary encoded. (*Field) Init(buf []byte, i flatbuffers.UOffsetT) / Whether or not this field can contain nulls. Should be true in general. (*Field) MutateTypeType(n Type) bool / Name is not required, in i.e. a List / Name is not required, in i.e. a List / Whether or not this field can contain nulls. Should be true in general. (*Field) Table() flatbuffers.Table / This is the type of the decoded value if the field is dictionary encoded. (*Field) TypeType() Type *Field : github.com/google/flatbuffers/go.FlatBuffer func GetRootAsField(buf []byte, offset flatbuffers.UOffsetT) *Field func (*Field).Children(obj *Field, j int) bool func (*Schema).Fields(obj *Field, j int) bool
/ ---------------------------------------------------------------------- / Data structures for describing a table row batch (a collection of / equal-length Arrow arrays) / Metadata about a field at some level of a nested type tree (but not / its children). / / For example, a List<Int16> with values `[[1, 2, 3], null, [4], [5, 6], null]` / would have {length: 5, null_count: 2} for its List node, and {length: 6, / null_count: 0} for its Int16 node, as separate FieldNode structs (*FieldNode) Init(buf []byte, i flatbuffers.UOffsetT) / The number of value slots in the Arrow array at this level of a nested / tree / The number of value slots in the Arrow array at this level of a nested / tree / The number of observed nulls. Fields with null_count == 0 may choose not / to write their physical validity bitmap out as a materialized buffer, / instead setting the length of the bitmap buffer to 0. / The number of observed nulls. Fields with null_count == 0 may choose not / to write their physical validity bitmap out as a materialized buffer, / instead setting the length of the bitmap buffer to 0. (*FieldNode) Table() flatbuffers.Table *FieldNode : github.com/google/flatbuffers/go.FlatBuffer func (*RecordBatch).Nodes(obj *FieldNode, j int) bool
/ Number of bytes per value (*FixedSizeBinary) Init(buf []byte, i flatbuffers.UOffsetT) / Number of bytes per value (*FixedSizeBinary) Table() flatbuffers.Table *FixedSizeBinary : github.com/google/flatbuffers/go.FlatBuffer func GetRootAsFixedSizeBinary(buf []byte, offset flatbuffers.UOffsetT) *FixedSizeBinary
(*FixedSizeList) Init(buf []byte, i flatbuffers.UOffsetT) / Number of list items per value / Number of list items per value (*FixedSizeList) Table() flatbuffers.Table *FixedSizeList : github.com/google/flatbuffers/go.FlatBuffer func GetRootAsFixedSizeList(buf []byte, offset flatbuffers.UOffsetT) *FixedSizeList
(*FloatingPoint) Init(buf []byte, i flatbuffers.UOffsetT) (*FloatingPoint) MutatePrecision(n Precision) bool (*FloatingPoint) Precision() Precision (*FloatingPoint) Table() flatbuffers.Table *FloatingPoint : github.com/google/flatbuffers/go.FlatBuffer func GetRootAsFloatingPoint(buf []byte, offset flatbuffers.UOffsetT) *FloatingPoint
(*Int) BitWidth() int32 (*Int) Init(buf []byte, i flatbuffers.UOffsetT) (*Int) IsSigned() bool (*Int) MutateBitWidth(n int32) bool (*Int) MutateIsSigned(n bool) bool (*Int) Table() flatbuffers.Table *Int : github.com/google/flatbuffers/go.FlatBuffer func GetRootAsInt(buf []byte, offset flatbuffers.UOffsetT) *Int func (*DictionaryEncoding).IndexType(obj *Int) *Int func (*SparseMatrixIndexCSR).IndicesType(obj *Int) *Int func (*SparseMatrixIndexCSR).IndptrType(obj *Int) *Int func (*SparseMatrixIndexCSX).IndicesType(obj *Int) *Int func (*SparseMatrixIndexCSX).IndptrType(obj *Int) *Int func (*SparseTensorIndexCOO).IndicesType(obj *Int) *Int func (*SparseTensorIndexCSF).IndicesType(obj *Int) *Int func (*SparseTensorIndexCSF).IndptrType(obj *Int) *Int func (*DictionaryEncoding).IndexType(obj *Int) *Int func (*SparseMatrixIndexCSR).IndicesType(obj *Int) *Int func (*SparseMatrixIndexCSR).IndptrType(obj *Int) *Int func (*SparseMatrixIndexCSX).IndicesType(obj *Int) *Int func (*SparseMatrixIndexCSX).IndptrType(obj *Int) *Int func (*SparseTensorIndexCOO).IndicesType(obj *Int) *Int func (*SparseTensorIndexCSF).IndicesType(obj *Int) *Int func (*SparseTensorIndexCSF).IndptrType(obj *Int) *Int
(*Interval) Init(buf []byte, i flatbuffers.UOffsetT) (*Interval) MutateUnit(n IntervalUnit) bool (*Interval) Table() flatbuffers.Table (*Interval) Unit() IntervalUnit *Interval : github.com/google/flatbuffers/go.FlatBuffer func GetRootAsInterval(buf []byte, offset flatbuffers.UOffsetT) *Interval
( IntervalUnit) String() string IntervalUnit : expvar.Var IntervalUnit : fmt.Stringer func (*Interval).Unit() IntervalUnit func IntervalAddUnit(builder *flatbuffers.Builder, unit IntervalUnit) func (*Interval).MutateUnit(n IntervalUnit) bool const IntervalUnitDAY_TIME const IntervalUnitMONTH_DAY_NANO const IntervalUnitYEAR_MONTH
/ ---------------------------------------------------------------------- / user defined key value pairs to add custom metadata to arrow / key namespacing is the responsibility of the user (*KeyValue) Init(buf []byte, i flatbuffers.UOffsetT) (*KeyValue) Key() []byte (*KeyValue) Table() flatbuffers.Table (*KeyValue) Value() []byte *KeyValue : github.com/google/flatbuffers/go.FlatBuffer func GetRootAsKeyValue(buf []byte, offset flatbuffers.UOffsetT) *KeyValue func (*Field).CustomMetadata(obj *KeyValue, j int) bool func (*Footer).CustomMetadata(obj *KeyValue, j int) bool func (*Message).CustomMetadata(obj *KeyValue, j int) bool func (*Schema).CustomMetadata(obj *KeyValue, j int) bool
/ Same as Binary, but with 64-bit offsets, allowing to represent / extremely large data values. (*LargeBinary) Init(buf []byte, i flatbuffers.UOffsetT) (*LargeBinary) Table() flatbuffers.Table *LargeBinary : github.com/google/flatbuffers/go.FlatBuffer func GetRootAsLargeBinary(buf []byte, offset flatbuffers.UOffsetT) *LargeBinary
/ Same as List, but with 64-bit offsets, allowing to represent / extremely large data values. (*LargeList) Init(buf []byte, i flatbuffers.UOffsetT) (*LargeList) Table() flatbuffers.Table *LargeList : github.com/google/flatbuffers/go.FlatBuffer func GetRootAsLargeList(buf []byte, offset flatbuffers.UOffsetT) *LargeList
/ Same as ListView, but with 64-bit offsets and sizes, allowing to represent / extremely large data values. (*LargeListView) Init(buf []byte, i flatbuffers.UOffsetT) (*LargeListView) Table() flatbuffers.Table *LargeListView : github.com/google/flatbuffers/go.FlatBuffer func GetRootAsLargeListView(buf []byte, offset flatbuffers.UOffsetT) *LargeListView
/ Same as Utf8, but with 64-bit offsets, allowing to represent / extremely large data values. (*LargeUtf8) Init(buf []byte, i flatbuffers.UOffsetT) (*LargeUtf8) Table() flatbuffers.Table *LargeUtf8 : github.com/google/flatbuffers/go.FlatBuffer func GetRootAsLargeUtf8(buf []byte, offset flatbuffers.UOffsetT) *LargeUtf8
(*List) Init(buf []byte, i flatbuffers.UOffsetT) (*List) Table() flatbuffers.Table *List : github.com/google/flatbuffers/go.FlatBuffer func GetRootAsList(buf []byte, offset flatbuffers.UOffsetT) *List
/ Represents the same logical types that List can, but contains offsets and / sizes allowing for writes in any order and sharing of child values among / list values. (*ListView) Init(buf []byte, i flatbuffers.UOffsetT) (*ListView) Table() flatbuffers.Table *ListView : github.com/google/flatbuffers/go.FlatBuffer func GetRootAsListView(buf []byte, offset flatbuffers.UOffsetT) *ListView
/ A Map is a logical nested type that is represented as / / List<entries: Struct<key: K, value: V>> / / In this layout, the keys and values are each respectively contiguous. We do / not constrain the key and value types, so the application is responsible / for ensuring that the keys are hashable and unique. Whether the keys are sorted / may be set in the metadata for this field. / / In a field with Map type, the field has a child Struct field, which then / has two children: key type and the second the value type. The names of the / child fields may be respectively "entries", "key", and "value", but this is / not enforced. / / Map / ```text / - child[0] entries: Struct / - child[0] key: K / - child[1] value: V / ``` / Neither the "entries" field nor the "key" field may be nullable. / / The metadata is structured so that Arrow systems without special handling / for Map can make Map an alias for List. The "layout" attribute for the Map / field must have the same contents as a List. (*Map) Init(buf []byte, i flatbuffers.UOffsetT) / Set to true if the keys within each value are sorted / Set to true if the keys within each value are sorted (*Map) Table() flatbuffers.Table *Map : github.com/google/flatbuffers/go.FlatBuffer func GetRootAsMap(buf []byte, offset flatbuffers.UOffsetT) *Map
(*Message) BodyLength() int64 (*Message) CustomMetadata(obj *KeyValue, j int) bool (*Message) CustomMetadataLength() int (*Message) Header(obj *flatbuffers.Table) bool (*Message) HeaderType() MessageHeader (*Message) Init(buf []byte, i flatbuffers.UOffsetT) (*Message) MutateBodyLength(n int64) bool (*Message) MutateHeaderType(n MessageHeader) bool (*Message) MutateVersion(n MetadataVersion) bool (*Message) Table() flatbuffers.Table (*Message) Version() MetadataVersion *Message : github.com/google/flatbuffers/go.FlatBuffer func GetRootAsMessage(buf []byte, offset flatbuffers.UOffsetT) *Message
/ ---------------------------------------------------------------------- / The root Message type / This union enables us to easily send different message types without / redundant storage, and in the future we can easily add new message types. / / Arrow implementations do not need to implement all of the message types, / which may include experimental metadata types. For maximum compatibility, / it is best to send data using RecordBatch ( MessageHeader) String() string MessageHeader : expvar.Var MessageHeader : fmt.Stringer func (*Message).HeaderType() MessageHeader func MessageAddHeaderType(builder *flatbuffers.Builder, headerType MessageHeader) func (*Message).MutateHeaderType(n MessageHeader) bool const MessageHeaderDictionaryBatch const MessageHeaderNONE const MessageHeaderRecordBatch const MessageHeaderSchema const MessageHeaderSparseTensor const MessageHeaderTensor
( MetadataVersion) String() string MetadataVersion : expvar.Var MetadataVersion : fmt.Stringer func (*Footer).Version() MetadataVersion func (*Message).Version() MetadataVersion func FooterAddVersion(builder *flatbuffers.Builder, version MetadataVersion) func MessageAddVersion(builder *flatbuffers.Builder, version MetadataVersion) func (*Footer).MutateVersion(n MetadataVersion) bool func (*Message).MutateVersion(n MetadataVersion) bool func github.com/apache/arrow-go/v18/arrow/internal.HasValidityBitmap(id arrow.Type, version MetadataVersion) bool const MetadataVersionV1 const MetadataVersionV2 const MetadataVersionV3 const MetadataVersionV4 const MetadataVersionV5 const github.com/apache/arrow-go/v18/arrow/internal.CurMetadataVersion
/ These are stored in the flatbuffer in the Type union below (*Null) Init(buf []byte, i flatbuffers.UOffsetT) (*Null) Table() flatbuffers.Table *Null : github.com/google/flatbuffers/go.FlatBuffer func GetRootAsNull(buf []byte, offset flatbuffers.UOffsetT) *Null
( Precision) String() string Precision : expvar.Var Precision : fmt.Stringer func (*FloatingPoint).Precision() Precision func FloatingPointAddPrecision(builder *flatbuffers.Builder, precision Precision) func (*FloatingPoint).MutatePrecision(n Precision) bool const PrecisionDOUBLE const PrecisionHALF const PrecisionSINGLE
/ A data header describing the shared memory layout of a "record" or "row" / batch. Some systems call this a "row batch" internally and others a "record / batch". / Nodes correspond to the pre-ordered flattened logical schema / Buffers correspond to the pre-ordered flattened buffer tree / / The number of buffers appended to this list depends on the schema. For / example, most primitive arrays will have 2 buffers, 1 for the validity / bitmap and 1 for the values. For struct arrays, there will only be a / single buffer for the validity (nulls) bitmap (*RecordBatch) BuffersLength() int / Buffers correspond to the pre-ordered flattened buffer tree / / The number of buffers appended to this list depends on the schema. For / example, most primitive arrays will have 2 buffers, 1 for the validity / bitmap and 1 for the values. For struct arrays, there will only be a / single buffer for the validity (nulls) bitmap / Optional compression of the message body (*RecordBatch) Init(buf []byte, i flatbuffers.UOffsetT) / number of records / rows. The arrays in the batch should all have this / length / number of records / rows. The arrays in the batch should all have this / length / Some types such as Utf8View are represented using a variable number of buffers. / For each such Field in the pre-ordered flattened logical schema, there will be / an entry in variadicBufferCounts to indicate the number of number of variadic / buffers which belong to that Field in the current RecordBatch. / / For example, the schema / col1: Struct<alpha: Int32, beta: BinaryView, gamma: Float64> / col2: Utf8View / contains two Fields with variadic buffers so variadicBufferCounts will have / two entries, the first counting the variadic buffers of `col1.beta` and the / second counting `col2`'s. / / This field may be omitted if and only if the schema contains no Fields with / a variable number of buffers, such as BinaryView and Utf8View. / Nodes correspond to the pre-ordered flattened logical schema (*RecordBatch) NodesLength() int (*RecordBatch) Table() flatbuffers.Table / Optional compression of the message body / Some types such as Utf8View are represented using a variable number of buffers. / For each such Field in the pre-ordered flattened logical schema, there will be / an entry in variadicBufferCounts to indicate the number of number of variadic / buffers which belong to that Field in the current RecordBatch. / / For example, the schema / col1: Struct<alpha: Int32, beta: BinaryView, gamma: Float64> / col2: Utf8View / contains two Fields with variadic buffers so variadicBufferCounts will have / two entries, the first counting the variadic buffers of `col1.beta` and the / second counting `col2`'s. / / This field may be omitted if and only if the schema contains no Fields with / a variable number of buffers, such as BinaryView and Utf8View. (*RecordBatch) VariadicBufferCountsLength() int *RecordBatch : github.com/google/flatbuffers/go.FlatBuffer func GetRootAsRecordBatch(buf []byte, offset flatbuffers.UOffsetT) *RecordBatch func (*DictionaryBatch).Data(obj *RecordBatch) *RecordBatch func (*DictionaryBatch).Data(obj *RecordBatch) *RecordBatch
/ Contains two child arrays, run_ends and values. / The run_ends child array must be a 16/32/64-bit integer array / which encodes the indices at which the run with the value in / each corresponding index in the values child array ends. / Like list/struct types, the value array can be of any type. (*RunEndEncoded) Init(buf []byte, i flatbuffers.UOffsetT) (*RunEndEncoded) Table() flatbuffers.Table *RunEndEncoded : github.com/google/flatbuffers/go.FlatBuffer func GetRootAsRunEndEncoded(buf []byte, offset flatbuffers.UOffsetT) *RunEndEncoded
(*RunLengthEncoded) Init(buf []byte, i flatbuffers.UOffsetT) (*RunLengthEncoded) Table() flatbuffers.Table *RunLengthEncoded : github.com/google/flatbuffers/go.FlatBuffer func GetRootAsRunLengthEncoded(buf []byte, offset flatbuffers.UOffsetT) *RunLengthEncoded
/ ---------------------------------------------------------------------- / A Schema describes the columns in a row batch (*Schema) CustomMetadata(obj *KeyValue, j int) bool (*Schema) CustomMetadataLength() int / endianness of the buffer / it is Little Endian by default / if endianness doesn't match the underlying system then the vectors need to be converted / Features used in the stream/file. (*Schema) FeaturesLength() int (*Schema) Fields(obj *Field, j int) bool (*Schema) FieldsLength() int (*Schema) Init(buf []byte, i flatbuffers.UOffsetT) / endianness of the buffer / it is Little Endian by default / if endianness doesn't match the underlying system then the vectors need to be converted / Features used in the stream/file. (*Schema) Table() flatbuffers.Table *Schema : github.com/google/flatbuffers/go.FlatBuffer func GetRootAsSchema(buf []byte, offset flatbuffers.UOffsetT) *Schema func (*Footer).Schema(obj *Schema) *Schema func (*Footer).Schema(obj *Schema) *Schema
( SparseMatrixCompressedAxis) String() string SparseMatrixCompressedAxis : expvar.Var SparseMatrixCompressedAxis : fmt.Stringer func (*SparseMatrixIndexCSX).CompressedAxis() SparseMatrixCompressedAxis func SparseMatrixIndexCSXAddCompressedAxis(builder *flatbuffers.Builder, compressedAxis SparseMatrixCompressedAxis) func (*SparseMatrixIndexCSX).MutateCompressedAxis(n SparseMatrixCompressedAxis) bool const SparseMatrixCompressedAxisColumn const SparseMatrixCompressedAxisRow
/ Compressed Sparse Row format, that is matrix-specific. / The type of values in indicesBuffer / indicesBuffer stores the location and size of the array that / contains the column indices of the corresponding non-zero values. / The type of index value is long. / / For example, the indices of the above X is: / / indices(X) = [1, 2, 2, 1, 3, 0, 2, 3, 1]. / / Note that the indices are sorted in lexicographical order for each row. / indptrBuffer stores the location and size of indptr array that / represents the range of the rows. / The i-th row spans from indptr[i] to indptr[i+1] in the data. / The length of this array is 1 + (the number of rows), and the type / of index value is long. / / For example, let X be the following 6x4 matrix: / / X := [[0, 1, 2, 0], / [0, 0, 3, 0], / [0, 4, 0, 5], / [0, 0, 0, 0], / [6, 0, 7, 8], / [0, 9, 0, 0]]. / / The array of non-zero values in X is: / / values(X) = [1, 2, 3, 4, 5, 6, 7, 8, 9]. / / And the indptr of X is: / / indptr(X) = [0, 2, 3, 5, 5, 8, 10]. / The type of values in indicesBuffer / The type of values in indptrBuffer / indptrBuffer stores the location and size of indptr array that / represents the range of the rows. / The i-th row spans from indptr[i] to indptr[i+1] in the data. / The length of this array is 1 + (the number of rows), and the type / of index value is long. / / For example, let X be the following 6x4 matrix: / / X := [[0, 1, 2, 0], / [0, 0, 3, 0], / [0, 4, 0, 5], / [0, 0, 0, 0], / [6, 0, 7, 8], / [0, 9, 0, 0]]. / / The array of non-zero values in X is: / / values(X) = [1, 2, 3, 4, 5, 6, 7, 8, 9]. / / And the indptr of X is: / / indptr(X) = [0, 2, 3, 5, 5, 8, 10]. / The type of values in indptrBuffer (*SparseMatrixIndexCSR) Init(buf []byte, i flatbuffers.UOffsetT) (*SparseMatrixIndexCSR) Table() flatbuffers.Table *SparseMatrixIndexCSR : github.com/google/flatbuffers/go.FlatBuffer func GetRootAsSparseMatrixIndexCSR(buf []byte, offset flatbuffers.UOffsetT) *SparseMatrixIndexCSR
/ Compressed Sparse format, that is matrix-specific. / Which axis, row or column, is compressed / The type of values in indicesBuffer / indicesBuffer stores the location and size of the array that / contains the column indices of the corresponding non-zero values. / The type of index value is long. / / For example, the indices of the above X is: / ```text / indices(X) = [1, 2, 2, 1, 3, 0, 2, 3, 1]. / ``` / Note that the indices are sorted in lexicographical order for each row. / indptrBuffer stores the location and size of indptr array that / represents the range of the rows. / The i-th row spans from `indptr[i]` to `indptr[i+1]` in the data. / The length of this array is 1 + (the number of rows), and the type / of index value is long. / / For example, let X be the following 6x4 matrix: / ```text / X := [[0, 1, 2, 0], / [0, 0, 3, 0], / [0, 4, 0, 5], / [0, 0, 0, 0], / [6, 0, 7, 8], / [0, 9, 0, 0]]. / ``` / The array of non-zero values in X is: / ```text / values(X) = [1, 2, 3, 4, 5, 6, 7, 8, 9]. / ``` / And the indptr of X is: / ```text / indptr(X) = [0, 2, 3, 5, 5, 8, 10]. / ``` / The type of values in indicesBuffer / The type of values in indptrBuffer / indptrBuffer stores the location and size of indptr array that / represents the range of the rows. / The i-th row spans from `indptr[i]` to `indptr[i+1]` in the data. / The length of this array is 1 + (the number of rows), and the type / of index value is long. / / For example, let X be the following 6x4 matrix: / ```text / X := [[0, 1, 2, 0], / [0, 0, 3, 0], / [0, 4, 0, 5], / [0, 0, 0, 0], / [6, 0, 7, 8], / [0, 9, 0, 0]]. / ``` / The array of non-zero values in X is: / ```text / values(X) = [1, 2, 3, 4, 5, 6, 7, 8, 9]. / ``` / And the indptr of X is: / ```text / indptr(X) = [0, 2, 3, 5, 5, 8, 10]. / ``` / The type of values in indptrBuffer (*SparseMatrixIndexCSX) Init(buf []byte, i flatbuffers.UOffsetT) / Which axis, row or column, is compressed (*SparseMatrixIndexCSX) Table() flatbuffers.Table *SparseMatrixIndexCSX : github.com/google/flatbuffers/go.FlatBuffer func GetRootAsSparseMatrixIndexCSX(buf []byte, offset flatbuffers.UOffsetT) *SparseMatrixIndexCSX
/ Sparse tensor index / The location and size of the tensor's data (*SparseTensor) Init(buf []byte, i flatbuffers.UOffsetT) / The number of non-zero values in a sparse tensor. (*SparseTensor) MutateSparseIndexType(n SparseTensorIndex) bool (*SparseTensor) MutateTypeType(n Type) bool / The dimensions of the tensor, optionally named. / The number of non-zero values in a sparse tensor. / The type of data contained in a value cell. / Currently only fixed-width value types are supported, / no strings or nested types. / The dimensions of the tensor, optionally named. (*SparseTensor) ShapeLength() int / Sparse tensor index (*SparseTensor) SparseIndexType() SparseTensorIndex (*SparseTensor) Table() flatbuffers.Table / The type of data contained in a value cell. / Currently only fixed-width value types are supported, / no strings or nested types. (*SparseTensor) TypeType() Type *SparseTensor : github.com/google/flatbuffers/go.FlatBuffer func GetRootAsSparseTensor(buf []byte, offset flatbuffers.UOffsetT) *SparseTensor
( SparseTensorIndex) String() string SparseTensorIndex : expvar.Var SparseTensorIndex : fmt.Stringer func (*SparseTensor).SparseIndexType() SparseTensorIndex func SparseTensorAddSparseIndexType(builder *flatbuffers.Builder, sparseIndexType SparseTensorIndex) func (*SparseTensor).MutateSparseIndexType(n SparseTensorIndex) bool const SparseTensorIndexNONE const SparseTensorIndexSparseMatrixIndexCSX const SparseTensorIndexSparseTensorIndexCOO const SparseTensorIndexSparseTensorIndexCSF
/ ---------------------------------------------------------------------- / EXPERIMENTAL: Data structures for sparse tensors / Coordinate (COO) format of sparse tensor index. / / COO's index list are represented as a NxM matrix, / where N is the number of non-zero values, / and M is the number of dimensions of a sparse tensor. / / indicesBuffer stores the location and size of the data of this indices / matrix. The value type and the stride of the indices matrix is / specified in indicesType and indicesStrides fields. / / For example, let X be a 2x3x4x5 tensor, and it has the following / 6 non-zero values: / ```text / X[0, 1, 2, 0] := 1 / X[1, 1, 2, 3] := 2 / X[0, 2, 1, 0] := 3 / X[0, 1, 3, 0] := 4 / X[0, 1, 2, 1] := 5 / X[1, 2, 0, 4] := 6 / ``` / In COO format, the index matrix of X is the following 4x6 matrix: / ```text / [[0, 0, 0, 0, 1, 1], / [1, 1, 1, 2, 1, 2], / [2, 2, 3, 1, 2, 0], / [0, 1, 0, 0, 3, 4]] / ``` / When isCanonical is true, the indices is sorted in lexicographical order / (row-major order), and it does not have duplicated entries. Otherwise, / the indices may not be sorted, or may have duplicated entries. / The location and size of the indices matrix's data / The type of values in indicesBuffer / Non-negative byte offsets to advance one value cell along each dimension / If omitted, default to row-major order (C-like). (*SparseTensorIndexCOO) IndicesStridesLength() int / The type of values in indicesBuffer (*SparseTensorIndexCOO) Init(buf []byte, i flatbuffers.UOffsetT) / The location and size of the indices matrix's data / This flag is true if and only if the indices matrix is sorted in / row-major order, and does not have duplicated entries. / This sort order is the same as of Tensorflow's SparseTensor, / but it is inverse order of SciPy's canonical coo_matrix / (SciPy employs column-major order for its coo_matrix). / Non-negative byte offsets to advance one value cell along each dimension / If omitted, default to row-major order (C-like). / This flag is true if and only if the indices matrix is sorted in / row-major order, and does not have duplicated entries. / This sort order is the same as of Tensorflow's SparseTensor, / but it is inverse order of SciPy's canonical coo_matrix / (SciPy employs column-major order for its coo_matrix). (*SparseTensorIndexCOO) Table() flatbuffers.Table *SparseTensorIndexCOO : github.com/google/flatbuffers/go.FlatBuffer func GetRootAsSparseTensorIndexCOO(buf []byte, offset flatbuffers.UOffsetT) *SparseTensorIndexCOO
/ Compressed Sparse Fiber (CSF) sparse tensor index. / indicesBuffers stores values of nodes. / Each tensor dimension corresponds to a buffer in indicesBuffers. / For example, the indicesBuffers for the above X is: / ```text / indicesBuffer(X) = [ / [0, 1], / [0, 1, 1], / [0, 0, 1, 1], / [1, 2, 0, 2, 0, 0, 1, 2] / ]. / ``` / axisOrder stores the sequence in which dimensions were traversed to / produce the prefix tree. / For example, the axisOrder for the above X is: / ```text / axisOrder(X) = [0, 1, 2, 3]. / ``` (*SparseTensorIndexCSF) AxisOrderLength() int / The type of values in indicesBuffers / indicesBuffers stores values of nodes. / Each tensor dimension corresponds to a buffer in indicesBuffers. / For example, the indicesBuffers for the above X is: / ```text / indicesBuffer(X) = [ / [0, 1], / [0, 1, 1], / [0, 0, 1, 1], / [1, 2, 0, 2, 0, 0, 1, 2] / ]. / ``` (*SparseTensorIndexCSF) IndicesBuffersLength() int / indptrBuffers stores the sparsity structure. / Each two consecutive dimensions in a tensor correspond to a buffer in / indptrBuffers. A pair of consecutive values at `indptrBuffers[dim][i]` / and `indptrBuffers[dim][i + 1]` signify a range of nodes in / `indicesBuffers[dim + 1]` who are children of `indicesBuffers[dim][i]` node. / / For example, the indptrBuffers for the above X is: / ```text / indptrBuffer(X) = [ / [0, 2, 3], / [0, 1, 3, 4], / [0, 2, 4, 5, 8] / ]. / ``` / The type of values in indicesBuffers / CSF is a generalization of compressed sparse row (CSR) index. / See [smith2017knl](http://shaden.io/pub-files/smith2017knl.pdf) / / CSF index recursively compresses each dimension of a tensor into a set / of prefix trees. Each path from a root to leaf forms one tensor / non-zero index. CSF is implemented with two arrays of buffers and one / arrays of integers. / / For example, let X be a 2x3x4x5 tensor and let it have the following / 8 non-zero values: / ```text / X[0, 0, 0, 1] := 1 / X[0, 0, 0, 2] := 2 / X[0, 1, 0, 0] := 3 / X[0, 1, 0, 2] := 4 / X[0, 1, 1, 0] := 5 / X[1, 1, 1, 0] := 6 / X[1, 1, 1, 1] := 7 / X[1, 1, 1, 2] := 8 / ``` / As a prefix tree this would be represented as: / ```text / 0 1 / / \ | / 0 1 1 / / / \ | / 0 0 1 1 / /| /| | /| | / 1 2 0 2 0 0 1 2 / ``` / The type of values in indptrBuffers / indptrBuffers stores the sparsity structure. / Each two consecutive dimensions in a tensor correspond to a buffer in / indptrBuffers. A pair of consecutive values at `indptrBuffers[dim][i]` / and `indptrBuffers[dim][i + 1]` signify a range of nodes in / `indicesBuffers[dim + 1]` who are children of `indicesBuffers[dim][i]` node. / / For example, the indptrBuffers for the above X is: / ```text / indptrBuffer(X) = [ / [0, 2, 3], / [0, 1, 3, 4], / [0, 2, 4, 5, 8] / ]. / ``` (*SparseTensorIndexCSF) IndptrBuffersLength() int / CSF is a generalization of compressed sparse row (CSR) index. / See [smith2017knl](http://shaden.io/pub-files/smith2017knl.pdf) / / CSF index recursively compresses each dimension of a tensor into a set / of prefix trees. Each path from a root to leaf forms one tensor / non-zero index. CSF is implemented with two arrays of buffers and one / arrays of integers. / / For example, let X be a 2x3x4x5 tensor and let it have the following / 8 non-zero values: / ```text / X[0, 0, 0, 1] := 1 / X[0, 0, 0, 2] := 2 / X[0, 1, 0, 0] := 3 / X[0, 1, 0, 2] := 4 / X[0, 1, 1, 0] := 5 / X[1, 1, 1, 0] := 6 / X[1, 1, 1, 1] := 7 / X[1, 1, 1, 2] := 8 / ``` / As a prefix tree this would be represented as: / ```text / 0 1 / / \ | / 0 1 1 / / / \ | / 0 0 1 1 / /| /| | /| | / 1 2 0 2 0 0 1 2 / ``` / The type of values in indptrBuffers (*SparseTensorIndexCSF) Init(buf []byte, i flatbuffers.UOffsetT) / axisOrder stores the sequence in which dimensions were traversed to / produce the prefix tree. / For example, the axisOrder for the above X is: / ```text / axisOrder(X) = [0, 1, 2, 3]. / ``` (*SparseTensorIndexCSF) Table() flatbuffers.Table *SparseTensorIndexCSF : github.com/google/flatbuffers/go.FlatBuffer func GetRootAsSparseTensorIndexCSF(buf []byte, offset flatbuffers.UOffsetT) *SparseTensorIndexCSF
/ A Struct_ in the flatbuffer metadata is the same as an Arrow Struct / (according to the physical memory layout). We used Struct_ here as / Struct is a reserved word in Flatbuffers (*Struct_) Init(buf []byte, i flatbuffers.UOffsetT) (*Struct_) Table() flatbuffers.Table *Struct_ : github.com/google/flatbuffers/go.FlatBuffer func GetRootAsStruct_(buf []byte, offset flatbuffers.UOffsetT) *Struct_
/ The location and size of the tensor's data (*Tensor) Init(buf []byte, i flatbuffers.UOffsetT) / Non-negative byte offsets to advance one value cell along each dimension / If omitted, default to row-major order (C-like). (*Tensor) MutateTypeType(n Type) bool / The type of data contained in a value cell. Currently only fixed-width / value types are supported, no strings or nested types / The dimensions of the tensor, optionally named (*Tensor) ShapeLength() int / The dimensions of the tensor, optionally named / Non-negative byte offsets to advance one value cell along each dimension / If omitted, default to row-major order (C-like). (*Tensor) StridesLength() int (*Tensor) Table() flatbuffers.Table / The type of data contained in a value cell. Currently only fixed-width / value types are supported, no strings or nested types (*Tensor) TypeType() Type *Tensor : github.com/google/flatbuffers/go.FlatBuffer func GetRootAsTensor(buf []byte, offset flatbuffers.UOffsetT) *Tensor
/ ---------------------------------------------------------------------- / Data structures for dense tensors / Shape data for a single axis in a tensor (*TensorDim) Init(buf []byte, i flatbuffers.UOffsetT) / Length of dimension / Name of the dimension, optional / Length of dimension (*TensorDim) Table() flatbuffers.Table *TensorDim : github.com/google/flatbuffers/go.FlatBuffer func GetRootAsTensorDim(buf []byte, offset flatbuffers.UOffsetT) *TensorDim func (*SparseTensor).Shape(obj *TensorDim, j int) bool func (*Tensor).Shape(obj *TensorDim, j int) bool
/ Time is either a 32-bit or 64-bit signed integer type representing an / elapsed time since midnight, stored in either of four units: seconds, / milliseconds, microseconds or nanoseconds. / / The integer `bitWidth` depends on the `unit` and must be one of the following: / * SECOND and MILLISECOND: 32 bits / * MICROSECOND and NANOSECOND: 64 bits / / The allowed values are between 0 (inclusive) and 86400 (=24*60*60) seconds / (exclusive), adjusted for the time unit (for example, up to 86400000 / exclusive for the MILLISECOND unit). / This definition doesn't allow for leap seconds. Time values from / measurements with leap seconds will need to be corrected when ingesting / into Arrow (for example by replacing the value 86400 with 86399). (*Time) BitWidth() int32 (*Time) Init(buf []byte, i flatbuffers.UOffsetT) (*Time) MutateBitWidth(n int32) bool (*Time) MutateUnit(n TimeUnit) bool (*Time) Table() flatbuffers.Table (*Time) Unit() TimeUnit *Time : github.com/google/flatbuffers/go.FlatBuffer func GetRootAsTime(buf []byte, offset flatbuffers.UOffsetT) *Time
/ Timestamp is a 64-bit signed integer representing an elapsed time since a / fixed epoch, stored in either of four units: seconds, milliseconds, / microseconds or nanoseconds, and is optionally annotated with a timezone. / / Timestamp values do not include any leap seconds (in other words, all / days are considered 86400 seconds long). / / Timestamps with a non-empty timezone / ------------------------------------ / / If a Timestamp column has a non-empty timezone value, its epoch is / 1970-01-01 00:00:00 (January 1st 1970, midnight) in the *UTC* timezone / (the Unix epoch), regardless of the Timestamp's own timezone. / / Therefore, timestamp values with a non-empty timezone correspond to / physical points in time together with some additional information about / how the data was obtained and/or how to display it (the timezone). / / For example, the timestamp value 0 with the timezone string "Europe/Paris" / corresponds to "January 1st 1970, 00h00" in the UTC timezone, but the / application may prefer to display it as "January 1st 1970, 01h00" in / the Europe/Paris timezone (which is the same physical point in time). / / One consequence is that timestamp values with a non-empty timezone / can be compared and ordered directly, since they all share the same / well-known point of reference (the Unix epoch). / / Timestamps with an unset / empty timezone / ----------------------------------------- / / If a Timestamp column has no timezone value, its epoch is / 1970-01-01 00:00:00 (January 1st 1970, midnight) in an *unknown* timezone. / / Therefore, timestamp values without a timezone cannot be meaningfully / interpreted as physical points in time, but only as calendar / clock / indications ("wall clock time") in an unspecified timezone. / / For example, the timestamp value 0 with an empty timezone string / corresponds to "January 1st 1970, 00h00" in an unknown timezone: there / is not enough information to interpret it as a well-defined physical / point in time. / / One consequence is that timestamp values without a timezone cannot / be reliably compared or ordered, since they may have different points of / reference. In particular, it is *not* possible to interpret an unset / or empty timezone as the same as "UTC". / / Conversion between timezones / ---------------------------- / / If a Timestamp column has a non-empty timezone, changing the timezone / to a different non-empty value is a metadata-only operation: / the timestamp values need not change as their point of reference remains / the same (the Unix epoch). / / However, if a Timestamp column has no timezone value, changing it to a / non-empty value requires to think about the desired semantics. / One possibility is to assume that the original timestamp values are / relative to the epoch of the timezone being set; timestamp values should / then adjusted to the Unix epoch (for example, changing the timezone from / empty to "Europe/Paris" would require converting the timestamp values / from "Europe/Paris" to "UTC", which seems counter-intuitive but is / nevertheless correct). / / Guidelines for encoding data from external libraries / ---------------------------------------------------- / / Date & time libraries often have multiple different data types for temporal / data. In order to ease interoperability between different implementations the / Arrow project has some recommendations for encoding these types into a Timestamp / column. / / An "instant" represents a physical point in time that has no relevant timezone / (for example, astronomical data). To encode an instant, use a Timestamp with / the timezone string set to "UTC", and make sure the Timestamp values / are relative to the UTC epoch (January 1st 1970, midnight). / / A "zoned date-time" represents a physical point in time annotated with an / informative timezone (for example, the timezone in which the data was / recorded). To encode a zoned date-time, use a Timestamp with the timezone / string set to the name of the timezone, and make sure the Timestamp values / are relative to the UTC epoch (January 1st 1970, midnight). / / (There is some ambiguity between an instant and a zoned date-time with the / UTC timezone. Both of these are stored the same in Arrow. Typically, / this distinction does not matter. If it does, then an application should / use custom metadata or an extension type to distinguish between the two cases.) / / An "offset date-time" represents a physical point in time combined with an / explicit offset from UTC. To encode an offset date-time, use a Timestamp / with the timezone string set to the numeric timezone offset string / (e.g. "+03:00"), and make sure the Timestamp values are relative to / the UTC epoch (January 1st 1970, midnight). / / A "naive date-time" (also called "local date-time" in some libraries) / represents a wall clock time combined with a calendar date, but with / no indication of how to map this information to a physical point in time. / Naive date-times must be handled with care because of this missing / information, and also because daylight saving time (DST) may make / some values ambiguous or nonexistent. A naive date-time may be / stored as a struct with Date and Time fields. However, it may also be / encoded into a Timestamp column with an empty timezone. The timestamp / values should be computed "as if" the timezone of the date-time values / was UTC; for example, the naive date-time "January 1st 1970, 00h00" would / be encoded as timestamp value 0. (*Timestamp) Init(buf []byte, i flatbuffers.UOffsetT) (*Timestamp) MutateUnit(n TimeUnit) bool (*Timestamp) Table() flatbuffers.Table / The timezone is an optional string indicating the name of a timezone, / one of: / / * As used in the Olson timezone database (the "tz database" or / "tzdata"), such as "America/New_York". / * An absolute timezone offset of the form "+XX:XX" or "-XX:XX", / such as "+07:30". / / Whether a timezone string is present indicates different semantics about / the data (see above). (*Timestamp) Unit() TimeUnit *Timestamp : github.com/google/flatbuffers/go.FlatBuffer func GetRootAsTimestamp(buf []byte, offset flatbuffers.UOffsetT) *Timestamp
( TimeUnit) String() string TimeUnit : expvar.Var TimeUnit : fmt.Stringer func (*Duration).Unit() TimeUnit func (*Time).Unit() TimeUnit func (*Timestamp).Unit() TimeUnit func DurationAddUnit(builder *flatbuffers.Builder, unit TimeUnit) func TimeAddUnit(builder *flatbuffers.Builder, unit TimeUnit) func TimestampAddUnit(builder *flatbuffers.Builder, unit TimeUnit) func (*Duration).MutateUnit(n TimeUnit) bool func (*Time).MutateUnit(n TimeUnit) bool func (*Timestamp).MutateUnit(n TimeUnit) bool const TimeUnitMICROSECOND const TimeUnitMILLISECOND const TimeUnitNANOSECOND const TimeUnitSECOND
/ ---------------------------------------------------------------------- / Top-level Type value, enabling extensible type-specific metadata. We can / add new logical types to Type without breaking backwards compatibility ( Type) String() string Type : expvar.Var Type : fmt.Stringer func (*Field).TypeType() Type func (*SparseTensor).TypeType() Type func (*Tensor).TypeType() Type func FieldAddTypeType(builder *flatbuffers.Builder, typeType Type) func SparseTensorAddTypeType(builder *flatbuffers.Builder, typeType Type) func TensorAddTypeType(builder *flatbuffers.Builder, typeType Type) func (*Field).MutateTypeType(n Type) bool func (*SparseTensor).MutateTypeType(n Type) bool func (*Tensor).MutateTypeType(n Type) bool const TypeBinary const TypeBinaryView const TypeBool const TypeDate const TypeDecimal const TypeDuration const TypeFixedSizeBinary const TypeFixedSizeList const TypeFloatingPoint const TypeInt const TypeInterval const TypeLargeBinary const TypeLargeList const TypeLargeListView const TypeLargeUtf8 const TypeList const TypeListView const TypeMap const TypeNONE const TypeNull const TypeRunEndEncoded const TypeStruct_ const TypeTime const TypeTimestamp const TypeUnion const TypeUtf8 const TypeUtf8View
/ A union is a complex type with children in Field / By default ids in the type vector refer to the offsets in the children / optionally typeIds provides an indirection between the child offset and the type id / for each child `typeIds[offset]` is the id used in the type vector (*Union) Init(buf []byte, i flatbuffers.UOffsetT) (*Union) Mode() UnionMode (*Union) MutateMode(n UnionMode) bool (*Union) MutateTypeIds(j int, n int32) bool (*Union) Table() flatbuffers.Table (*Union) TypeIds(j int) int32 (*Union) TypeIdsLength() int *Union : github.com/google/flatbuffers/go.FlatBuffer func GetRootAsUnion(buf []byte, offset flatbuffers.UOffsetT) *Union
( UnionMode) String() string UnionMode : expvar.Var UnionMode : fmt.Stringer func (*Union).Mode() UnionMode func UnionAddMode(builder *flatbuffers.Builder, mode UnionMode) func (*Union).MutateMode(n UnionMode) bool const UnionModeDense const UnionModeSparse
/ Unicode with UTF-8 encoding (*Utf8) Init(buf []byte, i flatbuffers.UOffsetT) (*Utf8) Table() flatbuffers.Table *Utf8 : github.com/google/flatbuffers/go.FlatBuffer func GetRootAsUtf8(buf []byte, offset flatbuffers.UOffsetT) *Utf8
/ Logically the same as Utf8, but the internal representation uses a view / struct that contains the string length and either the string's entire data / inline (for small strings) or an inlined prefix, an index of another buffer, / and an offset pointing to a slice in that buffer (for non-small strings). / / Since it uses a variable number of data buffers, each Field with this type / must have a corresponding entry in `variadicBufferCounts`. (*Utf8View) Init(buf []byte, i flatbuffers.UOffsetT) (*Utf8View) Table() flatbuffers.Table *Utf8View : github.com/google/flatbuffers/go.FlatBuffer func GetRootAsUtf8View(buf []byte, offset flatbuffers.UOffsetT) *Utf8View
Package-Level Functions (total 239)
func BoolStart(builder *flatbuffers.Builder)
func CreateBlock(builder *flatbuffers.Builder, offset int64, metaDataLength int32, bodyLength int64) flatbuffers.UOffsetT
func DateAddUnit(builder *flatbuffers.Builder, unit DateUnit)
func DateStart(builder *flatbuffers.Builder)
func DecimalAddBitWidth(builder *flatbuffers.Builder, bitWidth int32)
func DecimalAddPrecision(builder *flatbuffers.Builder, precision int32)
func DecimalAddScale(builder *flatbuffers.Builder, scale int32)
func FieldAddNullable(builder *flatbuffers.Builder, nullable bool)
func FieldAddTypeType(builder *flatbuffers.Builder, typeType Type)
/ User-defined metadata
/ User-defined metadata
func GetRootAsInt(buf []byte, offset flatbuffers.UOffsetT) *Int
func GetRootAsMap(buf []byte, offset flatbuffers.UOffsetT) *Map
func IntAddBitWidth(builder *flatbuffers.Builder, bitWidth int32)
func IntAddIsSigned(builder *flatbuffers.Builder, isSigned bool)
func IntStart(builder *flatbuffers.Builder)
func ListStart(builder *flatbuffers.Builder)
func MapAddKeysSorted(builder *flatbuffers.Builder, keysSorted bool)
func MapStart(builder *flatbuffers.Builder)
func MessageAddBodyLength(builder *flatbuffers.Builder, bodyLength int64)
func NullStart(builder *flatbuffers.Builder)
/ indicesBuffer stores the location and size of the array that / contains the column indices of the corresponding non-zero values. / The type of index value is long. / / For example, the indices of the above X is: / / indices(X) = [1, 2, 2, 1, 3, 0, 2, 3, 1]. / / Note that the indices are sorted in lexicographical order for each row.
/ indicesBuffer stores the location and size of the array that / contains the column indices of the corresponding non-zero values. / The type of index value is long. / / For example, the indices of the above X is: / ```text / indices(X) = [1, 2, 2, 1, 3, 0, 2, 3, 1]. / ``` / Note that the indices are sorted in lexicographical order for each row.
func SparseTensorAddNonZeroLength(builder *flatbuffers.Builder, nonZeroLength int64)
/ The location and size of the tensor's data
func TensorAddTypeType(builder *flatbuffers.Builder, typeType Type)
/ Name of the dimension, optional
/ The location and size of the tensor's data
func TimeAddBitWidth(builder *flatbuffers.Builder, bitWidth int32)
func TimeAddUnit(builder *flatbuffers.Builder, unit TimeUnit)
/ The timezone is an optional string indicating the name of a timezone, / one of: / / * As used in the Olson timezone database (the "tz database" or / "tzdata"), such as "America/New_York". / * An absolute timezone offset of the form "+XX:XX" or "-XX:XX", / such as "+07:30". / / Whether a timezone string is present indicates different semantics about / the data (see above).
func TimeStart(builder *flatbuffers.Builder)
func Utf8Start(builder *flatbuffers.Builder)
Package-Level Variables (total 30)
Package-Level Constants (total 67)
/ Each constituent buffer is first compressed with the indicated / compressor, and then written with the uncompressed length in the first 8 / bytes as a 64-bit little-endian signed integer followed by the compressed / buffer bytes (and then padding as required by the protocol). The / uncompressed length may be set to -1 to indicate that the data that / follows is not compressed, which can be useful for cases where / compression does not yield appreciable savings.
/ The stream makes use of compressed bodies as described / in Message.fbs.
/ The stream makes use of multiple full dictionaries with the / same ID and assumes clients implement dictionary replacement / correctly.
/ Needed to make flatbuffers happy.
/ 0.1.0 (October 2016).
/ 0.2.0 (February 2017). Non-backwards compatible with V1.
/ 0.3.0 -> 0.7.1 (May - December 2017). Non-backwards compatible with V2.
/ >= 0.8.0 (December 2017). Non-backwards compatible with V3.
/ >= 1.0.0 (July 2020). Backwards compatible with V4 (V5 readers can read V4 / metadata and IPC messages). Implementations are recommended to provide a / V4 compatibility mode with V5 format changes disabled. / / Incompatible changes between V4 and V5: / - Union buffer layout has changed. In V5, Unions don't have a validity / bitmap buffer.
const TypeBinary Type = 4
const TypeBinaryView Type = 23
const TypeBool Type = 6
const TypeDate Type = 8
const TypeDecimal Type = 7
const TypeDuration Type = 18
const TypeInt Type = 2
const TypeInterval Type = 11
const TypeLargeBinary Type = 19
const TypeLargeList Type = 21
const TypeLargeUtf8 Type = 20
const TypeList Type = 12
const TypeListView Type = 25
const TypeMap Type = 17
const TypeNONE Type = 0
const TypeNull Type = 1
const TypeStruct_ Type = 13
const TypeTime Type = 9
const TypeTimestamp Type = 10
const TypeUnion Type = 14
const TypeUtf8 Type = 5
const TypeUtf8View Type = 24