package compute
Import Path
github.com/apache/arrow-go/v18/arrow/compute (on go.dev)
Dependency Relation
imports 36 packages, and imported by 2 packages
Involved Source Files
arithmetic.go
cast.go
datum.go
datumkind_string.go
Package compute is a native-go implementation of an Acero-like
arrow compute engine. It requires go1.18+
While consumers of Arrow that are able to use CGO could utilize the
C Data API (using the cdata package) and could link against the
acero library directly, there are consumers who cannot use CGO. This
is an attempt to provide for those users, and in general create a
native-go arrow compute engine.
The overwhelming majority of things in this package require go1.18 as
it utilizes generics. The files in this package and its sub-packages
are all excluded from being built by go versions lower than 1.18 so
that the larger Arrow module itself is still compatible with go1.17.
Everything in this package should be considered Experimental for now.
exec.go
executor.go
expression.go
fieldref.go
fieldref_hash.go
funckind_string.go
functions.go
registry.go
scalar_bool.go
scalar_compare.go
scalar_set_lookup.go
selection.go
utils.go
vector_hash.go
vector_run_ends.go
Code Examples
package main
import (
"context"
"fmt"
"log"
"github.com/apache/arrow-go/v18/arrow"
"github.com/apache/arrow-go/v18/arrow/array"
"github.com/apache/arrow-go/v18/arrow/compute"
"github.com/apache/arrow-go/v18/arrow/compute/exec"
"github.com/apache/arrow-go/v18/arrow/memory"
)
func main() {
pool := memory.NewGoAllocator()
ctx := context.Background()
execCtx := compute.DefaultExecCtx()
ctx = compute.SetExecCtx(ctx, execCtx)
add42 := compute.NewScalarFunction("add_42", compute.Arity{
NArgs: 1,
}, compute.FunctionDoc{
Summary: "Returns the input values plus 42",
ArgNames: []string{"input"},
})
if err := add42.AddNewKernel(
[]exec.InputType{
// We accept a single argument (array) of Int8 type.
{
Kind: exec.InputExact,
Type: arrow.PrimitiveTypes.Int8,
},
},
// We'll return a single Int8 array.
exec.NewOutputType(arrow.PrimitiveTypes.Int8),
func(ctx *exec.KernelCtx, span *exec.ExecSpan, result *exec.ExecResult) error {
// The second buffer contains the values. Both for the input and the output arrays.
for i, x := range span.Values[0].Array.Buffers[1].Buf {
result.Buffers[1].Buf[i] = x + 42
}
return nil
},
nil,
); err != nil {
log.Fatal(err)
}
execCtx.Registry.AddFunction(add42, true)
inputArrayBuilder := array.NewInt8Builder(pool)
for i := 0; i < 16; i++ {
inputArrayBuilder.Append(int8(i))
}
inputArray := inputArrayBuilder.NewArray()
outputArrayDatum, err := compute.CallFunction(
compute.SetExecCtx(context.Background(), execCtx),
"add_42",
nil,
&compute.ArrayDatum{Value: inputArray.Data()},
)
if err != nil {
log.Fatal(err)
}
fmt.Println(array.NewInt8Data(outputArrayDatum.(*compute.ArrayDatum).Value).Int8Values())
}
Package-Level Type Names (total 46)
NoCheckOverflow bool
( ArithmeticOptions) TypeName() string
ArithmeticOptions : FunctionOptions
func AbsoluteValue(ctx context.Context, opts ArithmeticOptions, input Datum) (Datum, error)
func Acos(ctx context.Context, opts ArithmeticOptions, arg Datum) (Datum, error)
func Add(ctx context.Context, opts ArithmeticOptions, left, right Datum) (Datum, error)
func Asin(ctx context.Context, opts ArithmeticOptions, arg Datum) (Datum, error)
func Cos(ctx context.Context, opts ArithmeticOptions, arg Datum) (Datum, error)
func Divide(ctx context.Context, opts ArithmeticOptions, left, right Datum) (Datum, error)
func Ln(ctx context.Context, opts ArithmeticOptions, arg Datum) (Datum, error)
func Log10(ctx context.Context, opts ArithmeticOptions, arg Datum) (Datum, error)
func Log1p(ctx context.Context, opts ArithmeticOptions, arg Datum) (Datum, error)
func Log2(ctx context.Context, opts ArithmeticOptions, arg Datum) (Datum, error)
func Logb(ctx context.Context, opts ArithmeticOptions, x, base Datum) (Datum, error)
func Multiply(ctx context.Context, opts ArithmeticOptions, left, right Datum) (Datum, error)
func Negate(ctx context.Context, opts ArithmeticOptions, input Datum) (Datum, error)
func Power(ctx context.Context, opts ArithmeticOptions, base, exp Datum) (Datum, error)
func ShiftLeft(ctx context.Context, opts ArithmeticOptions, lhs, rhs Datum) (Datum, error)
func ShiftRight(ctx context.Context, opts ArithmeticOptions, lhs, rhs Datum) (Datum, error)
func Sin(ctx context.Context, opts ArithmeticOptions, arg Datum) (Datum, error)
func Subtract(ctx context.Context, opts ArithmeticOptions, left, right Datum) (Datum, error)
func Tan(ctx context.Context, opts ArithmeticOptions, arg Datum) (Datum, error)
Arity defines the number of required arguments for a function.
Naming conventions are taken from https://en.wikipedia.org/wiki/Arity
IsVarArgs bool
NArgs int
func Binary() Arity
func Nullary() Arity
func Ternary() Arity
func Unary() Arity
func VarArgs(minArgs int) Arity
func Function.Arity() Arity
func NewMetaFunction(name string, arity Arity, doc FunctionDoc, impl MetaFunctionImpl) *MetaFunction
func NewScalarFunction(name string, arity Arity, doc FunctionDoc) *ScalarFunction
func NewVectorFunction(name string, arity Arity, doc FunctionDoc) *VectorFunction
ArrayDatum references an array.Data object which can be used to create
array instances from if needed.
Value arrow.ArrayData
(*ArrayDatum) Chunks() []arrow.Array
(*ArrayDatum) Equals(other Datum) bool
( ArrayDatum) Kind() DatumKind
(*ArrayDatum) Len() int64
(*ArrayDatum) MakeArray() arrow.Array
(*ArrayDatum) NullN() int64
(*ArrayDatum) Release()
(*ArrayDatum) String() string
(*ArrayDatum) ToScalar() (scalar.Scalar, error)
(*ArrayDatum) Type() arrow.DataType
*ArrayDatum : ArrayLikeDatum
*ArrayDatum : Datum
*ArrayDatum : github.com/apache/arrow-go/v18/arrow/scalar.TypeToScalar
*ArrayDatum : expvar.Var
*ArrayDatum : fmt.Stringer
ArrayLikeDatum is an interface for treating a Datum similarly to an Array,
so that it is easy to differentiate between Record/Table/Collection and Scalar,
Array/ChunkedArray for ease of use. Chunks will return an empty slice for Scalar,
a slice with 1 element for Array, and the slice of chunks for a chunked array.
( ArrayLikeDatum) Chunks() []arrow.Array
( ArrayLikeDatum) Equals(Datum) bool
( ArrayLikeDatum) Kind() DatumKind
( ArrayLikeDatum) Len() int64
( ArrayLikeDatum) NullN() int64
( ArrayLikeDatum) Release()
( ArrayLikeDatum) String() string
( ArrayLikeDatum) Type() arrow.DataType
*ArrayDatum
*ChunkedDatum
*ScalarDatum
ArrayLikeDatum : Datum
ArrayLikeDatum : expvar.Var
ArrayLikeDatum : fmt.Stringer
Call is a function call with specific arguments which are themselves other
expressions. A call can also have options that are specific to the function
in question. It must be bound to determine the shape and type.
Deprecated: use substrait-go expression functions instead.
(*Call) Equals(other Expression) bool
(*Call) FieldRef() *FieldRef
(*Call) Hash() uint64
(*Call) IsBound() bool
(*Call) IsNullLiteral() bool
(*Call) IsSatisfiable() bool
(*Call) IsScalarExpr() bool
(*Call) Release()
(*Call) String() string
(*Call) Type() arrow.DataType
*Call : Expression
*Call : expvar.Var
*Call : fmt.Stringer
type CastOptions = kernels.CastOptions (struct)
ChunkedDatum contains a chunked array for use with expressions and compute.
Value *arrow.Chunked
(*ChunkedDatum) Chunks() []arrow.Array
(*ChunkedDatum) Equals(other Datum) bool
( ChunkedDatum) Kind() DatumKind
(*ChunkedDatum) Len() int64
(*ChunkedDatum) NullN() int64
(*ChunkedDatum) Release()
(*ChunkedDatum) String() string
(*ChunkedDatum) Type() arrow.DataType
*ChunkedDatum : ArrayLikeDatum
*ChunkedDatum : Datum
*ChunkedDatum : expvar.Var
*ChunkedDatum : fmt.Stringer
Datum is a variant interface for wrapping the various Arrow data structures
for now the various Datum types just hold a Value which is the type they
are wrapping, but it might make sense in the future for those types
to actually be aliases or embed their types instead. Not sure yet.
( Datum) Equals(Datum) bool
( Datum) Kind() DatumKind
( Datum) Len() int64
( Datum) Release()
( Datum) String() string
*ArrayDatum
ArrayLikeDatum (interface)
*ChunkedDatum
EmptyDatum
*RecordDatum
*ScalarDatum
*TableDatum
TableLikeDatum (interface)
Datum : expvar.Var
Datum : fmt.Stringer
func AbsoluteValue(ctx context.Context, opts ArithmeticOptions, input Datum) (Datum, error)
func Acos(ctx context.Context, opts ArithmeticOptions, arg Datum) (Datum, error)
func Add(ctx context.Context, opts ArithmeticOptions, left, right Datum) (Datum, error)
func Asin(ctx context.Context, opts ArithmeticOptions, arg Datum) (Datum, error)
func Atan(ctx context.Context, arg Datum) (Datum, error)
func Atan2(ctx context.Context, x, y Datum) (Datum, error)
func CallFunction(ctx context.Context, funcName string, opts FunctionOptions, args ...Datum) (Datum, error)
func CastDatum(ctx context.Context, val Datum, opts *CastOptions) (Datum, error)
func Cos(ctx context.Context, opts ArithmeticOptions, arg Datum) (Datum, error)
func Divide(ctx context.Context, opts ArithmeticOptions, left, right Datum) (Datum, error)
func Filter(ctx context.Context, values, filter Datum, options FilterOptions) (Datum, error)
func IsIn(ctx context.Context, opts SetOptions, values Datum) (Datum, error)
func IsInSet(ctx context.Context, valueSet, values Datum) (Datum, error)
func Ln(ctx context.Context, opts ArithmeticOptions, arg Datum) (Datum, error)
func Log10(ctx context.Context, opts ArithmeticOptions, arg Datum) (Datum, error)
func Log1p(ctx context.Context, opts ArithmeticOptions, arg Datum) (Datum, error)
func Log2(ctx context.Context, opts ArithmeticOptions, arg Datum) (Datum, error)
func Logb(ctx context.Context, opts ArithmeticOptions, x, base Datum) (Datum, error)
func Multiply(ctx context.Context, opts ArithmeticOptions, left, right Datum) (Datum, error)
func Negate(ctx context.Context, opts ArithmeticOptions, input Datum) (Datum, error)
func NewDatum(value interface{}) Datum
func NewDatumWithoutOwning(value interface{}) Datum
func Power(ctx context.Context, opts ArithmeticOptions, base, exp Datum) (Datum, error)
func Round(ctx context.Context, opts RoundOptions, arg Datum) (Datum, error)
func RoundToMultiple(ctx context.Context, opts RoundToMultipleOptions, arg Datum) (Datum, error)
func RunEndDecode(ctx context.Context, arg Datum) (Datum, error)
func RunEndEncode(ctx context.Context, opts RunEndEncodeOptions, arg Datum) (Datum, error)
func ShiftLeft(ctx context.Context, opts ArithmeticOptions, lhs, rhs Datum) (Datum, error)
func ShiftRight(ctx context.Context, opts ArithmeticOptions, lhs, rhs Datum) (Datum, error)
func Sign(ctx context.Context, input Datum) (Datum, error)
func Sin(ctx context.Context, opts ArithmeticOptions, arg Datum) (Datum, error)
func Subtract(ctx context.Context, opts ArithmeticOptions, left, right Datum) (Datum, error)
func Take(ctx context.Context, opts TakeOptions, values, indices Datum) (Datum, error)
func Tan(ctx context.Context, opts ArithmeticOptions, arg Datum) (Datum, error)
func Unique(ctx context.Context, values Datum) (Datum, error)
func Function.Execute(context.Context, FunctionOptions, ...Datum) (Datum, error)
func KernelExecutor.WrapResults(ctx context.Context, out <-chan Datum, chunkedArgs bool) Datum
func (*MetaFunction).Execute(ctx context.Context, opts FunctionOptions, args ...Datum) (Datum, error)
func (*ScalarFunction).Execute(ctx context.Context, opts FunctionOptions, args ...Datum) (Datum, error)
func (*VectorFunction).Execute(ctx context.Context, opts FunctionOptions, args ...Datum) (Datum, error)
func AbsoluteValue(ctx context.Context, opts ArithmeticOptions, input Datum) (Datum, error)
func Acos(ctx context.Context, opts ArithmeticOptions, arg Datum) (Datum, error)
func Add(ctx context.Context, opts ArithmeticOptions, left, right Datum) (Datum, error)
func Asin(ctx context.Context, opts ArithmeticOptions, arg Datum) (Datum, error)
func Atan(ctx context.Context, arg Datum) (Datum, error)
func Atan2(ctx context.Context, x, y Datum) (Datum, error)
func CallFunction(ctx context.Context, funcName string, opts FunctionOptions, args ...Datum) (Datum, error)
func CastDatum(ctx context.Context, val Datum, opts *CastOptions) (Datum, error)
func Cos(ctx context.Context, opts ArithmeticOptions, arg Datum) (Datum, error)
func DatumIsValue(d Datum) bool
func Divide(ctx context.Context, opts ArithmeticOptions, left, right Datum) (Datum, error)
func Filter(ctx context.Context, values, filter Datum, options FilterOptions) (Datum, error)
func FilterTable(ctx context.Context, tbl arrow.Table, filter Datum, opts *FilterOptions) (arrow.Table, error)
func IsIn(ctx context.Context, opts SetOptions, values Datum) (Datum, error)
func IsInSet(ctx context.Context, valueSet, values Datum) (Datum, error)
func Ln(ctx context.Context, opts ArithmeticOptions, arg Datum) (Datum, error)
func Log10(ctx context.Context, opts ArithmeticOptions, arg Datum) (Datum, error)
func Log1p(ctx context.Context, opts ArithmeticOptions, arg Datum) (Datum, error)
func Log2(ctx context.Context, opts ArithmeticOptions, arg Datum) (Datum, error)
func Logb(ctx context.Context, opts ArithmeticOptions, x, base Datum) (Datum, error)
func Multiply(ctx context.Context, opts ArithmeticOptions, left, right Datum) (Datum, error)
func Negate(ctx context.Context, opts ArithmeticOptions, input Datum) (Datum, error)
func Power(ctx context.Context, opts ArithmeticOptions, base, exp Datum) (Datum, error)
func Round(ctx context.Context, opts RoundOptions, arg Datum) (Datum, error)
func RoundToMultiple(ctx context.Context, opts RoundToMultipleOptions, arg Datum) (Datum, error)
func RunEndDecode(ctx context.Context, arg Datum) (Datum, error)
func RunEndEncode(ctx context.Context, opts RunEndEncodeOptions, arg Datum) (Datum, error)
func ShiftLeft(ctx context.Context, opts ArithmeticOptions, lhs, rhs Datum) (Datum, error)
func ShiftRight(ctx context.Context, opts ArithmeticOptions, lhs, rhs Datum) (Datum, error)
func Sign(ctx context.Context, input Datum) (Datum, error)
func Sin(ctx context.Context, opts ArithmeticOptions, arg Datum) (Datum, error)
func Subtract(ctx context.Context, opts ArithmeticOptions, left, right Datum) (Datum, error)
func Take(ctx context.Context, opts TakeOptions, values, indices Datum) (Datum, error)
func Tan(ctx context.Context, opts ArithmeticOptions, arg Datum) (Datum, error)
func Unique(ctx context.Context, values Datum) (Datum, error)
func (*ArrayDatum).Equals(other Datum) bool
func ArrayLikeDatum.Equals(Datum) bool
func (*ChunkedDatum).Equals(other Datum) bool
func Datum.Equals(Datum) bool
func EmptyDatum.Equals(other Datum) bool
func Function.Execute(context.Context, FunctionOptions, ...Datum) (Datum, error)
func KernelExecutor.CheckResultType(out Datum) error
func KernelExecutor.Execute(context.Context, *ExecBatch, chan<- Datum) error
func KernelExecutor.WrapResults(ctx context.Context, out <-chan Datum, chunkedArgs bool) Datum
func (*MetaFunction).Execute(ctx context.Context, opts FunctionOptions, args ...Datum) (Datum, error)
func (*RecordDatum).Equals(other Datum) bool
func (*ScalarDatum).Equals(other Datum) bool
func (*ScalarFunction).Execute(ctx context.Context, opts FunctionOptions, args ...Datum) (Datum, error)
func (*TableDatum).Equals(other Datum) bool
func TableLikeDatum.Equals(Datum) bool
func (*VectorFunction).Execute(ctx context.Context, opts FunctionOptions, args ...Datum) (Datum, error)
DatumKind is an enum used for denoting which kind of type a datum is encapsulating
( DatumKind) String() string
DatumKind : expvar.Var
DatumKind : fmt.Stringer
func ArrayDatum.Kind() DatumKind
func ArrayLikeDatum.Kind() DatumKind
func ChunkedDatum.Kind() DatumKind
func Datum.Kind() DatumKind
func EmptyDatum.Kind() DatumKind
func RecordDatum.Kind() DatumKind
func ScalarDatum.Kind() DatumKind
func TableDatum.Kind() DatumKind
func TableLikeDatum.Kind() DatumKind
const KindArray
const KindChunked
const KindNone
const KindRecord
const KindScalar
const KindTable
EmptyDatum is the null case, a Datum with nothing in it.
( EmptyDatum) Equals(other Datum) bool
( EmptyDatum) Kind() DatumKind
( EmptyDatum) Len() int64
( EmptyDatum) Release()
( EmptyDatum) String() string
EmptyDatum : Datum
EmptyDatum : expvar.Var
EmptyDatum : fmt.Stringer
ExecBatch is a unit of work for kernel execution. It contains a collection
of Array and Scalar values.
ExecBatch is semantically similar to a RecordBatch but for a SQL-style
execution context. It represents a collection or records, but constant
"columns" are represented by Scalar values rather than having to be
converted into arrays with repeated values.
Guarantee is a predicate Expression guaranteed to evaluate to true for
all rows in this batch.
Guarantee Expression
Len is the semantic length of this ExecBatch. When the values are
all scalars, the length should be set to 1 for non-aggregate kernels.
Otherwise the length is taken from the array values. Aggregate kernels
can have an ExecBatch formed by projecting just the partition columns
from a batch in which case it would have scalar rows with length > 1
If the array values are of length 0, then the length is 0 regardless of
whether any values are Scalar.
Values []Datum
( ExecBatch) NumValues() int
func ExecSpanFromBatch(batch *ExecBatch) *exec.ExecSpan
func KernelExecutor.Execute(context.Context, *ExecBatch, chan<- Datum) error
ExecCtx holds simple contextual information for execution
such as the default ChunkSize for batch iteration, whether or not
to ensure contiguous preallocations for kernels that want preallocation,
and a reference to the desired function registry to use.
An ExecCtx should be placed into a context.Context by using
SetExecCtx and GetExecCtx to pass it along for execution.
ChunkSize is the size used when iterating batches for execution
ChunkSize elements will be operated on as a time unless an argument
is a chunkedarray with a chunk that is smaller
ExecChannelSize is the size of the channel used for passing
exec results to the WrapResults function.
NumParallel determines the number of parallel goroutines
allowed for parallel executions.
PreallocContiguous determines whether preallocating memory for
execution of compute attempts to preallocate a full contiguous
buffer for all of the chunks beforehand.
Registry allows specifying the Function Registry to utilize
when searching for kernel implementations.
func DefaultExecCtx() ExecCtx
func GetExecCtx(ctx context.Context) ExecCtx
func SetExecCtx(ctx context.Context, e ExecCtx) context.Context
Expression is an interface for mapping one datum to another. An expression
is one of:
A literal Datum
A reference to a single (potentially nested) field of an input Datum
A call to a compute function, with arguments specified by other Expressions
Deprecated: use substrait-go expressions instead.
( Expression) Equals(Expression) bool
FieldRef returns a pointer to the underlying field reference, or nil if
this expression is not a field reference.
( Expression) Hash() uint64
IsBound returns true if this expression has been bound to a particular
Datum and/or Schema.
IsNullLiteral returns true if this expression is a literal and entirely
null.
IsSatisfiable returns true if this expression could evaluate to true
IsScalarExpr returns true if this expression is composed only of scalar
literals, field references and calls to scalar functions.
Release releases the underlying bound C++ memory that is allocated when
a Bind is performed. Any bound expression should get released to ensure
no memory leaks.
( Expression) String() string
Type returns the datatype this expression will evaluate to.
*Call
*Literal
*Parameter
Expression : expvar.Var
Expression : fmt.Stringer
func And(lhs, rhs Expression, ops ...Expression) Expression
func Cast(ex Expression, dt arrow.DataType) Expression
func DeserializeExpr(mem memory.Allocator, buf *memory.Buffer) (Expression, error)
func Equal(lhs, rhs Expression) Expression
func Greater(lhs, rhs Expression) Expression
func GreaterEqual(lhs, rhs Expression) Expression
func IsNull(lhs Expression, nanIsNull bool) Expression
func IsValid(lhs Expression) Expression
func Less(lhs, rhs Expression) Expression
func LessEqual(lhs, rhs Expression) Expression
func NewCall(name string, args []Expression, opts FunctionOptions) Expression
func NewFieldRef(field string) Expression
func NewLiteral(arg interface{}) Expression
func NewRef(ref FieldRef) Expression
func Not(expr Expression) Expression
func NotEqual(lhs, rhs Expression) Expression
func NullLiteral(dt arrow.DataType) Expression
func Or(lhs, rhs Expression, ops ...Expression) Expression
func Project(values []Expression, names []string) Expression
func And(lhs, rhs Expression, ops ...Expression) Expression
func And(lhs, rhs Expression, ops ...Expression) Expression
func Cast(ex Expression, dt arrow.DataType) Expression
func Equal(lhs, rhs Expression) Expression
func Greater(lhs, rhs Expression) Expression
func GreaterEqual(lhs, rhs Expression) Expression
func IsNull(lhs Expression, nanIsNull bool) Expression
func IsValid(lhs Expression) Expression
func Less(lhs, rhs Expression) Expression
func LessEqual(lhs, rhs Expression) Expression
func NewCall(name string, args []Expression, opts FunctionOptions) Expression
func Not(expr Expression) Expression
func NotEqual(lhs, rhs Expression) Expression
func Or(lhs, rhs Expression, ops ...Expression) Expression
func Or(lhs, rhs Expression, ops ...Expression) Expression
func Project(values []Expression, names []string) Expression
func SerializeExpr(expr Expression, mem memory.Allocator) (*memory.Buffer, error)
func (*Call).Equals(other Expression) bool
func Expression.Equals(Expression) bool
func (*Literal).Equals(other Expression) bool
func (*Parameter).Equals(other Expression) bool
FieldPath represents a path to a nested field using indices of child fields.
For example, given the indices {5, 9, 3} the field could be retrieved with:
schema.Field(5).Type().(*arrow.StructType).Field(9).Type().(*arrow.StructType).Field(3)
Attempting to retrieve a child field using a FieldPath which is not valid for a given
schema will get an error such as an out of range index, or an empty path.
FieldPaths provide for drilling down to potentially nested children for convenience
of accepting a slice of fields, a schema or a datatype (which should contain child fields).
A fieldpath can also be used to retrieve a child arrow.Array or column from a record batch.
Get retrieves the corresponding nested child field by drilling through the schema's
fields as per the field path.
GetColumn will return the correct child array by traversing the fieldpath
going to the nested arrays of the columns in the record batch.
GetField is equivalent to GetFieldFromType(field.Type)
GetFieldFromSlice treats the slice as the top layer of fields, so the first value
in the field path will index into the slice, and then drill down from there.
GetFieldFromType returns the nested field from a datatype by drilling into it's
child fields.
( FieldPath) String() string
FieldPath : expvar.Var
FieldPath : fmt.Stringer
func (*FieldRef).FieldPath() FieldPath
func FieldRef.FindAll(fields []arrow.Field) []FieldPath
func FieldRef.FindAllField(field arrow.Field) []FieldPath
func FieldRef.FindOne(schema *arrow.Schema) (FieldPath, error)
func FieldRef.FindOneOrNone(schema *arrow.Schema) (FieldPath, error)
func FieldRef.FindOneOrNoneRecord(root arrow.RecordBatch) (FieldPath, error)
func FieldRefPath(p FieldPath) FieldRef
FieldRef is a descriptor of a (potentially nested) field within a schema.
Unlike FieldPath (which is exclusively indices of child fields), FieldRef
may reference a field by name. It can be constructed from either
a field index, field name, or field path.
Nested fields can be referenced as well, given the schema:
arrow.NewSchema([]arrow.Field{
{Name: "a", Type: arrow.StructOf(arrow.Field{Name: "n", Type: arrow.Null})},
{Name: "b", Type: arrow.PrimitiveTypes.Int32},
})
the following all indicate the nested field named "n":
FieldRefPath(FieldPath{0, 0})
FieldRefList("a", 0)
FieldRefList("a", "n")
FieldRefList(0, "n")
NewFieldRefFromDotPath(".a[0]")
FieldPaths matching a FieldRef are retrieved with the FindAll* functions
Multiple matches are possible because field names may be duplicated within
a schema. For example:
aIsAmbiguous := arrow.NewSchema([]arrow.Field{
{Name: "a", Type: arrow.PrimitiveTypes.Int32},
{Name: "a", Type: arrow.PrimitiveTypes.Float32},
})
matches := FieldRefName("a").FindAll(aIsAmbiguous)
assert.Len(matches, 2)
assert.True(matches[0].Get(aIsAmbiguous).Equals(aIsAmbiguous.Field(0))
assert.True(matches[1].Get(aIsAmbiguous).Equals(aIsAmbiguous.Field(1))
(*FieldRef) Equals(other FieldRef) bool
FieldPath returns the fieldpath that this FieldRef uses, otherwise
an empty FieldPath if it's not a FieldPath reference
FindAll returns all the fieldpaths which this FieldRef matches in the given
slice of fields.
FindAllField returns all the fieldpaths that this FieldRef matches against
the type of the given field.
FindOne returns an error if the field isn't matched or if there are multiple matches
otherwise it returns the path to the single valid match.
FindOneOrNone is a convenience helper that will either return 1 fieldpath,
or an empty fieldpath, and will return an error if there are multiple matches.
FindOneOrNoneRecord is like FindOneOrNone but for the schema of a record,
returning an error only if there are multiple matches.
GetAllColumns gets all the matching column arrays from the given record that
this FieldRef references.
GetOneColumnOrNone returns either a nil or the referenced array if it can be
found, erroring only if there is an ambiguous multiple matches.
GetOneField will return a pointer to a field or an error if it is not found
or if there are multiple matches.
GetOneOrNone will return a field or a nil if the field is found or not, and
only errors if there are multiple matches.
Hash produces a hash of this field reference and takes in a seed so that
it can maintain consistency across multiple places / processes /etc.
IsFieldPath returns true if this FieldRef uses a fieldpath
IsName returns true if this fieldref is a name reference
IsNested returns true if this FieldRef expects to represent
a nested field.
Name returns the name of the field this references if it is
a Name reference, otherwise the empty string
( FieldRef) String() string
*FieldRef : github.com/polarsignals/frostdb/query/logicalplan.Named
FieldRef : expvar.Var
FieldRef : fmt.Stringer
func FieldRefIndex(i int) FieldRef
func FieldRefList(elems ...interface{}) FieldRef
func FieldRefName(n string) FieldRef
func FieldRefPath(p FieldPath) FieldRef
func NewFieldRefFromDotPath(dotpath string) (out FieldRef, err error)
func (*Call).FieldRef() *FieldRef
func Expression.FieldRef() *FieldRef
func Literal.FieldRef() *FieldRef
func (*Parameter).FieldRef() *FieldRef
func NewRef(ref FieldRef) Expression
func (*FieldRef).Equals(other FieldRef) bool
type FilterOptions = kernels.FilterOptions (struct)
FuncKind is an enum representing the type of a function
( FuncKind) String() string
FuncKind : expvar.Var
FuncKind : fmt.Stringer
func Function.Kind() FuncKind
const FuncHashAgg
const FuncMeta
const FuncScalar
const FuncScalarAgg
const FuncVector
( Function) Arity() Arity
( Function) DefaultOptions() FunctionOptions
( Function) DispatchBest(...arrow.DataType) (exec.Kernel, error)
( Function) DispatchExact(...arrow.DataType) (exec.Kernel, error)
( Function) Doc() FunctionDoc
( Function) Execute(context.Context, FunctionOptions, ...Datum) (Datum, error)
( Function) Kind() FuncKind
( Function) Name() string
( Function) NumKernels() int
( Function) Validate() error
*MetaFunction
*ScalarFunction
*VectorFunction
Function : github.com/polarsignals/frostdb/query/logicalplan.Named
func FunctionRegistry.GetFunction(name string) (Function, bool)
func FunctionRegistry.AddFunction(fn Function, allowOverwrite bool) bool
func FunctionRegistry.CanAddFunction(fn Function, allowOverwrite bool) bool
Symbolic names (identifiers) for the function arguments.
Can be used to generate nicer function signatures.
A detailed description of the function, meant to follow the summary.
Whether or not options are required for function execution.
If false, then either there are no options for this function,
or there is a usable default options value.
Name of the options struct type, if any
A one-line summary of the function, using a verb.
For example, "Add two numeric arrays or scalars"
func Function.Doc() FunctionDoc
func NewMetaFunction(name string, arity Arity, doc FunctionDoc, impl MetaFunctionImpl) *MetaFunction
func NewScalarFunction(name string, arity Arity, doc FunctionDoc) *ScalarFunction
func NewVectorFunction(name string, arity Arity, doc FunctionDoc) *VectorFunction
var EmptyFuncDoc
FunctionOptions can be any type which has a TypeName function. The fields
of the type will be used (via reflection) to determine the information to
propagate when serializing to pass to the C++ for execution.
( FunctionOptions) TypeName() string
ArithmeticOptions
MakeStructOptions
NullOptions
SetLookupOptions
*SetOptions
StrptimeOptions
github.com/apache/arrow-go/v18/arrow/compute/internal/kernels.CastOptions
github.com/apache/arrow-go/v18/arrow/compute/internal/kernels.FilterOptions
github.com/apache/arrow-go/v18/arrow/compute/internal/kernels.RoundOptions
github.com/apache/arrow-go/v18/arrow/compute/internal/kernels.RoundState
github.com/apache/arrow-go/v18/arrow/compute/internal/kernels.RoundToMultipleOptions
github.com/apache/arrow-go/v18/arrow/compute/internal/kernels.RunEndEncodeState
github.com/apache/arrow-go/v18/arrow/compute/internal/kernels.TakeOptions
*github.com/apache/thrift/lib/go/thrift.ValidationError
github.com/orsinium-labs/enum.Enum[...]
google.golang.org/protobuf/internal/encoding/text.Token
func Function.DefaultOptions() FunctionOptions
func FunctionOptionsCloneable.Clone() FunctionOptions
func CallFunction(ctx context.Context, funcName string, opts FunctionOptions, args ...Datum) (Datum, error)
func NewCall(name string, args []Expression, opts FunctionOptions) Expression
func SerializeOptions(opts FunctionOptions, mem memory.Allocator) (*memory.Buffer, error)
func Function.Execute(context.Context, FunctionOptions, ...Datum) (Datum, error)
func FunctionOptionsEqual.Equals(FunctionOptions) bool
func (*MetaFunction).Execute(ctx context.Context, opts FunctionOptions, args ...Datum) (Datum, error)
func (*ScalarFunction).Execute(ctx context.Context, opts FunctionOptions, args ...Datum) (Datum, error)
func (*ScalarFunction).SetDefaultOptions(opts FunctionOptions)
func (*SetLookupOptions).Equals(other FunctionOptions) bool
func (*VectorFunction).Execute(ctx context.Context, opts FunctionOptions, args ...Datum) (Datum, error)
func (*VectorFunction).SetDefaultOptions(opts FunctionOptions)
( FunctionOptionsCloneable) Clone() FunctionOptions
( FunctionOptionsEqual) Equals(FunctionOptions) bool
*SetLookupOptions
( FunctionRegistry) AddAlias(target, source string) bool
( FunctionRegistry) AddFunction(fn Function, allowOverwrite bool) bool
( FunctionRegistry) CanAddAlias(target, source string) bool
( FunctionRegistry) CanAddFunction(fn Function, allowOverwrite bool) bool
( FunctionRegistry) GetFunction(name string) (Function, bool)
( FunctionRegistry) GetFunctionNames() []string
( FunctionRegistry) NumFunctions() int
func GetFunctionRegistry() FunctionRegistry
func NewChildRegistry(parent FunctionRegistry) FunctionRegistry
func NewRegistry() FunctionRegistry
func NewChildRegistry(parent FunctionRegistry) FunctionRegistry
func RegisterScalarArithmetic(reg FunctionRegistry)
func RegisterScalarBoolean(reg FunctionRegistry)
func RegisterScalarCast(reg FunctionRegistry)
func RegisterScalarComparisons(reg FunctionRegistry)
func RegisterScalarSetLookup(reg FunctionRegistry)
func RegisterVectorHash(reg FunctionRegistry)
func RegisterVectorRunEndFuncs(reg FunctionRegistry)
func RegisterVectorSelection(reg FunctionRegistry)
KernelExecutor is the interface for all executors to initialize and
call kernel execution functions on batches.
CheckResultType checks the actual result type against the resolved
output type. If the types don't match an error is returned
Clear resets the state in the executor so that it can be reused.
Execute the kernel for the provided batch and pass the resulting
Datum values to the provided channel.
Init must be called *after* the kernel's init method and any
KernelState must be set into the KernelCtx *before* calling
this Init method. This is to facilitate the case where
Init may be expensive and does not need to be called
again for each execution of the kernel. For example,
the same lookup table can be re-used for all scanned batches
in a dataset filter.
WrapResults exists for the case where an executor wants to post process
the batches of result datums. Such as creating a ChunkedArray from
multiple output batches or so on. Results from individual batch
executions should be read from the out channel, and WrapResults should
return the final Datum result.
func NewScalarExecutor() KernelExecutor
Literal is an expression denoting a literal Datum which could be any value
as a scalar, an array, or so on.
Deprecated: use substrait-go expressions Literal instead.
Literal Datum
(*Literal) Equals(other Expression) bool
( Literal) FieldRef() *FieldRef
(*Literal) Hash() uint64
(*Literal) IsBound() bool
(*Literal) IsNullLiteral() bool
(*Literal) IsSatisfiable() bool
(*Literal) IsScalarExpr() bool
(*Literal) Release()
(*Literal) String() string
(*Literal) Type() arrow.DataType
*Literal : Expression
*Literal : expvar.Var
*Literal : fmt.Stringer
FieldMetadata []*arrow.Metadata
FieldNames []string
FieldNullability []bool
( MakeStructOptions) TypeName() string
MakeStructOptions : FunctionOptions
MetaFunction is a function which dispatches to other functions, the impl
must not be nil.
For Array, ChunkedArray and Scalar datums, this may rely on the execution
of concrete function types, but this must handle other Datum kinds on its
own.
(*MetaFunction) Arity() Arity
(*MetaFunction) DefaultOptions() FunctionOptions
(*MetaFunction) DispatchBest(...arrow.DataType) (exec.Kernel, error)
(*MetaFunction) DispatchExact(...arrow.DataType) (exec.Kernel, error)
(*MetaFunction) Doc() FunctionDoc
(*MetaFunction) Execute(ctx context.Context, opts FunctionOptions, args ...Datum) (Datum, error)
(*MetaFunction) Kind() FuncKind
(*MetaFunction) Name() string
( MetaFunction) NumKernels() int
(*MetaFunction) Validate() error
*MetaFunction : Function
*MetaFunction : github.com/polarsignals/frostdb/query/logicalplan.Named
func NewMetaFunction(name string, arity Arity, doc FunctionDoc, impl MetaFunctionImpl) *MetaFunction
MetaFunctionImpl is the signature needed for implementing a MetaFunction
which is a function that dispatches to another function instead.
func NewMetaFunction(name string, arity Arity, doc FunctionDoc, impl MetaFunctionImpl) *MetaFunction
type NullMatchingBehavior = kernels.NullMatchingBehavior (basic type)
NanIsNull bool
( NullOptions) TypeName() string
NullOptions : FunctionOptions
type NullSelectionBehavior = kernels.NullSelectionBehavior (basic type)
Parameter represents a field reference and needs to be bound in order to determine
its type and shape.
Deprecated: use substrait-go field references instead.
(*Parameter) Equals(other Expression) bool
(*Parameter) FieldRef() *FieldRef
(*Parameter) Hash() uint64
(*Parameter) IsBound() bool
( Parameter) IsNullLiteral() bool
(*Parameter) IsSatisfiable() bool
(*Parameter) IsScalarExpr() bool
(*Parameter) Release()
(*Parameter) String() string
(*Parameter) Type() arrow.DataType
*Parameter : Expression
*Parameter : expvar.Var
*Parameter : fmt.Stringer
RecordDatum contains an array.Record for passing a full record to an expression
or to compute.
Value arrow.RecordBatch
(*RecordDatum) Equals(other Datum) bool
( RecordDatum) Kind() DatumKind
(*RecordDatum) Len() int64
(*RecordDatum) Release()
(*RecordDatum) Schema() *arrow.Schema
( RecordDatum) String() string
*RecordDatum : Datum
*RecordDatum : TableLikeDatum
RecordDatum : expvar.Var
RecordDatum : fmt.Stringer
type RoundOptions = kernels.RoundOptions (struct) type RoundToMultipleOptions = kernels.RoundToMultipleOptions (struct) type RunEndEncodeOptions = kernels.RunEndEncodeState (struct)
ScalarDatum contains a scalar value
Value scalar.Scalar
( ScalarDatum) Chunks() []arrow.Array
(*ScalarDatum) Equals(other Datum) bool
( ScalarDatum) Kind() DatumKind
( ScalarDatum) Len() int64
(*ScalarDatum) NullN() int64
(*ScalarDatum) Release()
(*ScalarDatum) String() string
(*ScalarDatum) ToScalar() (scalar.Scalar, error)
(*ScalarDatum) Type() arrow.DataType
*ScalarDatum : ArrayLikeDatum
*ScalarDatum : Datum
*ScalarDatum : github.com/apache/arrow-go/v18/arrow/scalar.TypeToScalar
*ScalarDatum : expvar.Var
*ScalarDatum : fmt.Stringer
A ScalarFunction is a function that executes element-wise operations
on arrays or scalars, and therefore whose results generally do not
depend on the order of the values in the arguments. Accepts and returns
arrays that are all of the same size. These functions roughly correspond
to the functions used in most SQL expressions.
AddKernel adds the provided kernel to the list of kernels
this function has. A copy of the kernel is added to the slice of kernels,
which means that a given kernel object can be created, added and then
reused to add other kernels.
AddNewKernel constructs a new kernel with the provided signature
and execution/init functions and then adds it to the function's list of
kernels. This assumes default null handling (intersection of validity bitmaps)
(*ScalarFunction) Arity() Arity
(*ScalarFunction) DefaultOptions() FunctionOptions
(*ScalarFunction) DispatchBest(vals ...arrow.DataType) (exec.Kernel, error)
(*ScalarFunction) DispatchExact(vals ...arrow.DataType) (exec.Kernel, error)
(*ScalarFunction) Doc() FunctionDoc
Execute uses the passed in context, function options and arguments to eagerly
execute the function using kernel dispatch, batch iteration and memory
allocation details as defined by the kernel.
If opts is nil, then the DefaultOptions() will be used.
(*ScalarFunction) Kernels() []*exec.ScalarKernel
(*ScalarFunction) Kind() FuncKind
(*ScalarFunction) Name() string
(*ScalarFunction) NumKernels() int
(*ScalarFunction) SetDefaultOptions(opts FunctionOptions)
(*ScalarFunction) Validate() error
*ScalarFunction : Function
*ScalarFunction : github.com/polarsignals/frostdb/query/logicalplan.Named
func NewScalarFunction(name string, arity Arity, doc FunctionDoc) *ScalarFunction
Deprecated: Use SetOptions instead
SkipNulls bool
ValueSet Datum
(*SetLookupOptions) Equals(other FunctionOptions) bool
(*SetLookupOptions) FromStructScalar(sc *scalar.Struct) error
(*SetLookupOptions) Release()
( SetLookupOptions) TypeName() string
SetLookupOptions : FunctionOptions
*SetLookupOptions : FunctionOptionsEqual
*SetLookupOptions : github.com/apache/arrow-go/v18/arrow/scalar.TypeFromScalar
NullBehavior NullMatchingBehavior
ValueSet Datum
(*SetOptions) TypeName() string
*SetOptions : FunctionOptions
func IsIn(ctx context.Context, opts SetOptions, values Datum) (Datum, error)
Format string
Unit arrow.TimeUnit
( StrptimeOptions) TypeName() string
StrptimeOptions : FunctionOptions
TableDatum contains a table so that multiple record batches can be worked with
together as a single table for being passed to compute and expression handling.
Value arrow.Table
(*TableDatum) Equals(other Datum) bool
( TableDatum) Kind() DatumKind
(*TableDatum) Len() int64
(*TableDatum) Release()
(*TableDatum) Schema() *arrow.Schema
( TableDatum) String() string
*TableDatum : Datum
*TableDatum : TableLikeDatum
TableDatum : expvar.Var
TableDatum : fmt.Stringer
TableLikeDatum is an interface type for specifying either a RecordBatch or a
Table as both contain a schema as opposed to just a single data type.
( TableLikeDatum) Equals(Datum) bool
( TableLikeDatum) Kind() DatumKind
( TableLikeDatum) Len() int64
( TableLikeDatum) Release()
( TableLikeDatum) Schema() *arrow.Schema
( TableLikeDatum) String() string
*RecordDatum
*TableDatum
TableLikeDatum : Datum
TableLikeDatum : expvar.Var
TableLikeDatum : fmt.Stringer
type TakeOptions = kernels.TakeOptions (struct)
(*VectorFunction) AddKernel(kernel exec.VectorKernel) error
(*VectorFunction) AddNewKernel(inTypes []exec.InputType, outType exec.OutputType, execFn exec.ArrayKernelExec, init exec.KernelInitFn) error
(*VectorFunction) Arity() Arity
(*VectorFunction) DefaultOptions() FunctionOptions
(*VectorFunction) DispatchBest(vals ...arrow.DataType) (exec.Kernel, error)
(*VectorFunction) DispatchExact(vals ...arrow.DataType) (exec.Kernel, error)
(*VectorFunction) Doc() FunctionDoc
(*VectorFunction) Execute(ctx context.Context, opts FunctionOptions, args ...Datum) (Datum, error)
(*VectorFunction) Kernels() []*exec.VectorKernel
(*VectorFunction) Kind() FuncKind
(*VectorFunction) Name() string
(*VectorFunction) NumKernels() int
(*VectorFunction) SetDefaultOptions(opts FunctionOptions)
(*VectorFunction) Validate() error
*VectorFunction : Function
*VectorFunction : github.com/polarsignals/frostdb/query/logicalplan.Named
func NewVectorFunction(name string, arity Arity, doc FunctionDoc) *VectorFunction
Package-Level Functions (total 106)
AbsoluteValue returns the AbsoluteValue for each element in the input
argument. It accepts either a scalar or an array.
ArithmeticOptions specifies whether or not to check for overflows,
performance is faster if not explicitly checking for overflows but
will error on an overflow if CheckOverflow is true.
Add performs an addition between the passed in arguments (scalar or array)
and returns the result. If one argument is a scalar and the other is an
array, the scalar value is added to each value of the array.
ArithmeticOptions specifies whether or not to check for overflows,
performance is faster if not explicitly checking for overflows but
will error on an overflow if NoCheckOverflow is false (default).
And constructs a tree of calls to and_kleene for boolean And logic taking
an arbitrary number of values.
CallFunction is a one-shot invoker for all types of functions.
It will perform kernel-dispatch, argument checking, iteration of
ChunkedArray inputs and wrapping of outputs.
To affect the execution options, you must call SetExecCtx and pass
the resulting context in here.
CanCast returns true if there is an implementation for casting an array
or scalar value from the specified DataType to the other data type.
func Cast(ex Expression, dt arrow.DataType) Expression
CastArray is a convenience function for casting an Array to another type.
It is equivalent to constructing a Datum for the array and using
CallFunction(ctx, "cast", ...).
CastDatum is a convenience function for casting a Datum to another type.
It is equivalent to calling CallFunction(ctx, "cast", opts, Datum) and
should work for Scalar, Array or ChunkedArray Datums.
func CastFromExtension(ctx *exec.KernelCtx, batch *exec.ExecSpan, out *exec.ExecResult) error func CastStruct(ctx *exec.KernelCtx, batch *exec.ExecSpan, out *exec.ExecResult) error
CastToType is a convenience function equivalent to calling
CastArray(ctx, val, compute.SafeCastOptions(toType))
DatumIsValue returns true if the datum passed is a Scalar, Array
or ChunkedArray type (e.g. it contains a specific value not a
group of values)
func DefaultCastOptions(safe bool) *CastOptions
DefaultExecCtx returns the default exec context which will be used
if there is no ExecCtx set into the context for execution.
This can be called to get a copy of the default values which can
then be modified to set into a context.
The default exec context uses the following values:
- ChunkSize = DefaultMaxChunkSize (MaxInt64)
- PreallocContiguous = true
- Registry = GetFunctionRegistry()
- ExecChannelSize = 10
- NumParallel = runtime.NumCPU()
func DefaultFilterOptions() *FilterOptions func DefaultTakeOptions() *TakeOptions func DeserializeExpr(mem memory.Allocator, buf *memory.Buffer) (Expression, error)
Divide performs a division between the passed in arguments (scalar or array)
and returns the result. If one argument is a scalar and the other is an
array, the scalar value is used with each value of the array.
ArithmeticOptions specifies whether or not to check for overflows,
performance is faster if not explicitly checking for overflows but
will error on an overflow if NoCheckOverflow is false (default).
Will error on divide by zero regardless of whether or not checking for
overflows.
Equal is a convenience function for the equal function
ExecSpanFromBatch constructs and returns a new ExecSpan from the values
inside of the ExecBatch which could be scalar or arrays.
This is mostly used for tests but is also a convenience method for other
cases.
FieldRefIndex is a convenience function to construct a FieldPath reference
of a single index
FieldRefList takes an arbitrary number of arguments which can be either
strings or ints. This will panic if anything other than a string or int
is passed in.
FieldRefName constructs a FieldRef by name
FieldRefPath constructs a FieldRef from a given FieldPath
Filter is a wrapper convenience that is equivalent to calling
CallFunction(ctx, "filter", &options, values, filter) for filtering
an input array (values) by a boolean array (filter). The two inputs
must be the same length.
FilterArray is a convenience method for calling Filter without having
to manually construct the intervening Datum objects (they will be
created for you internally here).
func FilterRecordBatch(ctx context.Context, batch arrow.RecordBatch, filter arrow.Array, opts *FilterOptions) (arrow.RecordBatch, error) func FilterTable(ctx context.Context, tbl arrow.Table, filter Datum, opts *FilterOptions) (arrow.Table, error)
GetExecCtx returns an embedded ExecCtx from the provided context.
If it does not contain an ExecCtx, then the default one is returned.
Greater is shorthand for NewCall("greater",....)
GreaterEqual is shorthand for NewCall("greater_equal",....)
IsNull creates an expression that returns true if the passed in expression is
null. Optionally treating NaN as null if desired.
IsValid is the inverse of IsNull
Less is shorthand for NewCall("less",....)
LessEqual is shorthand for NewCall("less_equal",....)
Multiply performs a multiplication between the passed in arguments (scalar or array)
and returns the result. If one argument is a scalar and the other is an
array, the scalar value is multiplied against each value of the array.
ArithmeticOptions specifies whether or not to check for overflows,
performance is faster if not explicitly checking for overflows but
will error on an overflow if NoCheckOverflow is false (default).
Negate returns a result containing the negation of each element in the
input argument. It accepts either a scalar or an array.
ArithmeticOptions specifies whether or not to check for overflows,
or to throw an error on unsigned types.
NewCall constructs an expression that represents a specific function call with
the given arguments and options.
func NewCastOptions(dt arrow.DataType, safe bool) *CastOptions func NewChildRegistry(parent FunctionRegistry) FunctionRegistry
NewDatum will construct the appropriate Datum type based on what is passed in
as the argument.
An arrow.Array gets an ArrayDatum
An array.Chunked gets a ChunkedDatum
An array.Record gets a RecordDatum
An array.Table gets a TableDatum
A scalar.Scalar gets a ScalarDatum
Anything else is passed to scalar.MakeScalar and receives a scalar
datum of that appropriate type.
func NewDatumWithoutOwning(value interface{}) Datum
NewFieldRef is shorthand for NewRef(FieldRefName(field))
NewFieldRefFromDotPath parses a dot path into a field ref.
dot_path = '.' name
| '[' digit+ ']'
| dot_path+
Examples
".alpha" => FieldRefName("alpha")
"[2]" => FieldRefIndex(2)
".beta[3]" => FieldRefList("beta", 3)
"[5].gamma.delta[7]" => FieldRefList(5, "gamma", "delta", 7)
".hello world" => FieldRefName("hello world")
`.\[y\]\\tho\.\` => FieldRef(`[y]\tho.\`)
Note: when parsing a name, a '\' preceding any other character will be
dropped from the resulting name. therefore if a name must contain the characters
'.', '\', '[' or ']' then they must be escaped with a preceding '\'.
NewLiteral constructs a new literal expression from any value. It is passed
to NewDatum which will construct the appropriate Datum and/or scalar
value for the type provided.
NewMetaFunction constructs a new MetaFunction which will call the provided
impl for dispatching with the expected arity.
Will panic if impl is nil.
NewRef constructs a parameter expression which refers to a specific field
func NewRegistry() FunctionRegistry func NewScalarExecutor() KernelExecutor
NewScalarFunction constructs a new ScalarFunction object with the passed in
name, arity and function doc.
func NewVectorFunction(name string, arity Arity, doc FunctionDoc) *VectorFunction
Not creates a call to "invert" for the value specified.
NotEqual creates a call to not_equal
func NullLiteral(dt arrow.DataType) Expression
Or constructs a tree of calls to or_kleene for boolean Or logic taking
an arbitrary number of values.
Power returns base**exp for each element in the input arrays. Should work
for both Arrays and Scalars
Project is shorthand for `make_struct` to produce a record batch output
from a group of expressions.
func RegisterScalarArithmetic(reg FunctionRegistry) func RegisterScalarBoolean(reg FunctionRegistry) func RegisterScalarCast(reg FunctionRegistry) func RegisterScalarComparisons(reg FunctionRegistry) func RegisterScalarSetLookup(reg FunctionRegistry) func RegisterVectorHash(reg FunctionRegistry) func RegisterVectorRunEndFuncs(reg FunctionRegistry)
RegisterVectorSelection registers functions that select specific
values from arrays such as Take and Filter
func RoundToMultiple(ctx context.Context, opts RoundToMultipleOptions, arg Datum) (Datum, error) func RunEndEncode(ctx context.Context, opts RunEndEncodeOptions, arg Datum) (Datum, error) func RunEndEncodeArray(ctx context.Context, opts RunEndEncodeOptions, input arrow.Array) (arrow.Array, error) func SafeCastOptions(dt arrow.DataType) *CastOptions
SerializeExpr serializes expressions by converting them to Metadata and
storing this in the schema of a Record. Embedded arrays and scalars are
stored in its columns. Finally the record is written as an IPC file
func SerializeOptions(opts FunctionOptions, mem memory.Allocator) (*memory.Buffer, error)
SetExecCtx returns a new child context containing the passed in ExecCtx
ShiftLeft only accepts integral types and shifts each element of the
first argument to the left by the value of the corresponding element
in the second argument.
The value to shift by should be >= 0 and < precision of the type.
ShiftRight only accepts integral types and shifts each element of the
first argument to the right by the value of the corresponding element
in the second argument.
The value to shift by should be >= 0 and < precision of the type.
Sign returns -1, 0, or 1 depending on the sign of each element in the
input. For x in the input:
if x > 0: 1
if x < 0: -1
if x == 0: 0
Sub performs a subtraction between the passed in arguments (scalar or array)
and returns the result. If one argument is a scalar and the other is an
array, the scalar value is subtracted from each value of the array.
ArithmeticOptions specifies whether or not to check for overflows,
performance is faster if not explicitly checking for overflows but
will error on an overflow if NoCheckOverflow is false (default).
func TakeArrayOpts(ctx context.Context, values, indices arrow.Array, opts TakeOptions) (arrow.Array, error) func UnsafeCastOptions(dt arrow.DataType) *CastOptions
Package-Level Variables (total 11)
EmptyFuncDoc is a reusable empty function doc definition for convenience.
var ErrIndexRange error var ErrInvalid error var ErrNoChildren error var ErrNoMatch error
GetAllocator retrieves the allocator from the context, or returns
memory.DefaultAllocator if there was no allocator in the provided
context.
WithAllocator returns a new context with the provided allocator
embedded into the context.
Package-Level Constants (total 29)
const DefaultMaxChunkSize = 9223372036854775807
A function that computes grouped summary statistics from array
input and an array of group identifiers.
A function that dispatches to other functions and does not contain
its own kernels.
A function that performs scalar data operations on whole arrays
of data. Can generally process Array or Scalar values. The size
of the output will be the same as the size (or broadcasted size,
in the case of mixing Array and Scalar inputs) of the input.
A function that computes a scalar summary statistic from array input.
A function with array input and output whose behavior depends on
the values of the entire arrays passed, rather than the value of
each scalar value.
const KindChunked DatumKind = 3 // chunked_array const KindRecord DatumKind = 4 // record_batch const KindScalar DatumKind = 1 // scalar const NullMatchingEmitNull kernels.NullMatchingBehavior = 2 const NullMatchingMatch kernels.NullMatchingBehavior = 0 const NullMatchingSkip kernels.NullMatchingBehavior = 1
Round to nearest integer less than or equal in magnitude (aka "floor")
Round ties with DOWN (aka "round half towards negative infinity")
Round ties to nearest even integer
Round ties to nearest odd integer
Round ties with AwayFromZero (aka "round half towards infinity")
Round ties with TowardsZero (aka "round half away from infinity")
Round ties with UP (aka "round half towards positive infinity")
Round negative values with DOWN and positive values with UP
Get integral part without fractional digits (aka "trunc")
Round to nearest integer greater than or equal in magnitude (aka "ceil")
const SelectionDropNulls kernels.NullSelectionBehavior = 0 const SelectionEmitNulls kernels.NullSelectionBehavior = 1 const UnknownLength int64 = -1![]() |
The pages are generated with Golds v0.8.2. (GOOS=linux GOARCH=amd64) Golds is a Go 101 project developed by Tapir Liu. PR and bug reports are welcome and can be submitted to the issue list. Please follow @zigo_101 (reachable from the left QR code) to get the latest news of Golds. |