package util

Import Path
	github.com/apache/arrow-go/v18/arrow/util (on go.dev)

Dependency Relation
	imports 9 packages, and imported by 3 packages


Involved Source Files

	    byte_size.go
	    protobuf_reflect.go


Package-Level Type Names (total 4)


	/* sort by:  |  */
	
		ProtobufFieldReflection represents the metadata and values of a protobuf field

		
			Field arrow.Field
			
				// The field's metadata, if any

			
				// Field name

			
				// Fields can be nullable

			
				// The field's data type

		
			( ProtobufFieldReflection) Equal(o arrow.Field) bool
			( ProtobufFieldReflection) Fingerprint() string
			(*ProtobufFieldReflection) GetDescriptor() protoreflect.FieldDescriptor
			( ProtobufFieldReflection) HasMetadata() bool
			( ProtobufFieldReflection) String() string
		
			 ProtobufFieldReflection : expvar.Var
			 ProtobufFieldReflection : fmt.Stringer


	
		ProtobufMessageFieldReflection links together the message and it's fields

		
			Field arrow.Field
			
				// The field's metadata, if any

			
				// Field name

			
				// Fields can be nullable

			
				// The field's data type

		
			
				AppendValueOrNull add the value of a protobuf field to an arrow array builder

			( ProtobufMessageFieldReflection) Equal(o arrow.Field) bool
			( ProtobufMessageFieldReflection) Fingerprint() string
			( ProtobufMessageFieldReflection) GetDescriptor() protoreflect.FieldDescriptor
			( ProtobufMessageFieldReflection) HasMetadata() bool
			( ProtobufMessageFieldReflection) String() string
		
			 ProtobufMessageFieldReflection : expvar.Var
			 ProtobufMessageFieldReflection : fmt.Stringer


	
		ProtobufMessageReflection represents the metadata and values of a protobuf message

		
			
				Record returns an arrow.RecordBatch for a protobuf message

			
				Schema returns an arrow.Schema representing a protobuf message

		
			func NewProtobufMessageReflection(msg proto.Message, options ...option) *ProtobufMessageReflection


	
		ProtobufTypeHandler provides options on how protobuf fields should be handled in the conversion to arrow

		
			func WithEnumHandler(enumHandler ProtobufTypeHandler) option
			func WithOneOfHandler(oneOfHandler ProtobufTypeHandler) option
		
			const EnumDictionary
			const EnumNumber
			const EnumValue
			const OneOfDenseUnion
			const OneOfNull




Package-Level Functions (total 7)


	
		NewProtobufMessageReflection initialises a ProtobufMessageReflection
		can be used to convert a protobuf message into an arrow Record


	
		TotalArraySize returns the sum of the number of bytes in each buffer referenced by the Array.


	
		TotalRecordSize return the sum of bytes in each buffer referenced by the RecordBatch.


	
		WithEnumHandler is an option for a ProtobufMessageReflection
		WithEnumHandler enables customisation of the protobuf Enum type in the arrow schema
		By default, the Enums are mapped to arrow.Dictionary


	
		WithExclusionPolicy is an option for a ProtobufMessageReflection
		WithExclusionPolicy acts as a deny filter on the fields of a protobuf message
		i.e. prevents them from being included in the schema.
		A use case for this is to exclude fields containing PII.


	
		WithFieldNameFormatter is an option for a ProtobufMessageReflection
		WithFieldNameFormatter enables customisation of the field names in the arrow schema
		By default, the field names are taken from the protobuf message (.proto file)


	
		WithOneOfHandler is an option for a ProtobufMessageReflection
		WithOneOfHandler enables customisation of the protobuf oneOf type in the arrow schema
		By default, the oneOfs are mapped to separate columns




Package-Level Constants (total 5)


	
		EnumDictionary uses both the numeric and string value and maps to an arrow.Dictionary


	
		EnumNumber uses the Enum numeric value


	
		EnumValue uses the Enum string value


	
		OneOfDenseUnion maps the protobuf OneOf to an arrow.DENSE_UNION


	
		OneOfNull means do not wrap oneOfs in a union, they are treated as separate fields