package storage

Import Path
	github.com/polarsignals/frostdb/storage (on go.dev)

Dependency Relation
	imports 20 packages, and imported by one package

Involved Source Files

	    bucket.go
	    iceberg.go
Package-Level Type Names (total 5)

	/* sort by: alphabet | popularity */
	 type Bucket (interface)
		Bucket is an objstore.Bucket that also supports reading files via a ReaderAt interface.

		Methods (total 12)
			( Bucket) Attributes(ctx context.Context, name string) (objstore.ObjectAttributes, error)
				Attributes returns information about the specified object.

			( Bucket) Close() error
			( Bucket) Delete(ctx context.Context, name string) error
				Delete removes the object with the given name.
				If object does not exist in the moment of deletion, Delete should throw error.

			( Bucket) Exists(ctx context.Context, name string) (bool, error)
				Exists checks if the given object exists in the bucket.

			( Bucket) Get(ctx context.Context, name string) (io.ReadCloser, error)
				Get returns a reader for the given object name.

			( Bucket) GetRange(ctx context.Context, name string, off, length int64) (io.ReadCloser, error)
				GetRange returns a new range reader for the given object name and range.

			( Bucket) GetReaderAt(ctx context.Context, name string) (io.ReaderAt, error)
			( Bucket) IsAccessDeniedErr(err error) bool
				IsAccessDeniedErr returns true if access to object is denied.

			( Bucket) IsObjNotFoundErr(err error) bool
				IsObjNotFoundErr returns true if error means that object is not found. Relevant to Get operations.

			( Bucket) Iter(ctx context.Context, dir string, f func(string) error, options ...objstore.IterOption) error
				Iter calls f for each entry in the given directory (not recursive.). The argument to f is the full
				object name including the prefix of the inspected directory.
				Entries are passed to function in sorted order.

			( Bucket) Name() string
				Name returns the bucket name for the provider.

			( Bucket) Upload(ctx context.Context, name string, r io.Reader) error
				Upload the contents of the reader as an object into the bucket.
				Upload should be idempotent.

		Implemented By (at least 2)
			*BucketReaderAt
			 github.com/polarsignals/frostdb.DefaultObjstoreBucket
		Implements (at least 5)
			 Bucket : github.com/polarsignals/frostdb/query/logicalplan.Named
			 Bucket : github.com/prometheus/common/expfmt.Closer
			 Bucket : github.com/thanos-io/objstore.Bucket
			 Bucket : github.com/thanos-io/objstore.BucketReader
			 Bucket : io.Closer
		As Inputs Of (at least one exported)
			func github.com/polarsignals/frostdb.NewDefaultBucket(b Bucket, options ...frostdb.DefaultObjstoreBucketOption) *frostdb.DefaultObjstoreBucket

	 type BucketReaderAt (struct)
		BucketReaderAt implements the Bucket interface.

		Fields (only one)
			Bucket objstore.Bucket
		Methods (total 12)
			( BucketReaderAt) Attributes(ctx context.Context, name string) (objstore.ObjectAttributes, error)
				Attributes returns information about the specified object.

			( BucketReaderAt) Close() error
			( BucketReaderAt) Delete(ctx context.Context, name string) error
				Delete removes the object with the given name.
				If object does not exist in the moment of deletion, Delete should throw error.

			( BucketReaderAt) Exists(ctx context.Context, name string) (bool, error)
				Exists checks if the given object exists in the bucket.

			( BucketReaderAt) Get(ctx context.Context, name string) (io.ReadCloser, error)
				Get returns a reader for the given object name.

			( BucketReaderAt) GetRange(ctx context.Context, name string, off, length int64) (io.ReadCloser, error)
				GetRange returns a new range reader for the given object name and range.

			(*BucketReaderAt) GetReaderAt(ctx context.Context, name string) (io.ReaderAt, error)
				GetReaderAt returns a io.ReaderAt for the given filename.

			( BucketReaderAt) IsAccessDeniedErr(err error) bool
				IsAccessDeniedErr returns true if access to object is denied.

			( BucketReaderAt) IsObjNotFoundErr(err error) bool
				IsObjNotFoundErr returns true if error means that object is not found. Relevant to Get operations.

			( BucketReaderAt) Iter(ctx context.Context, dir string, f func(string) error, options ...objstore.IterOption) error
				Iter calls f for each entry in the given directory (not recursive.). The argument to f is the full
				object name including the prefix of the inspected directory.
				Entries are passed to function in sorted order.

			( BucketReaderAt) Name() string
				Name returns the bucket name for the provider.

			( BucketReaderAt) Upload(ctx context.Context, name string, r io.Reader) error
				Upload the contents of the reader as an object into the bucket.
				Upload should be idempotent.

		Implements (at least 6)
			*BucketReaderAt : Bucket
			 BucketReaderAt : github.com/polarsignals/frostdb/query/logicalplan.Named
			 BucketReaderAt : github.com/prometheus/common/expfmt.Closer
			 BucketReaderAt : github.com/thanos-io/objstore.Bucket
			 BucketReaderAt : github.com/thanos-io/objstore.BucketReader
			 BucketReaderAt : io.Closer
		As Outputs Of (at least one exported)
			func NewBucketReaderAt(bucket objstore.Bucket) *BucketReaderAt

	 type FileReaderAt (struct)
		FileReaderAt is a wrapper around a objstore.Bucket that implements the ReaderAt interface.

		Fields (only one)
			Bucket objstore.Bucket
		Methods (total 12)
			( FileReaderAt) Attributes(ctx context.Context, name string) (objstore.ObjectAttributes, error)
				Attributes returns information about the specified object.

			( FileReaderAt) Close() error
			( FileReaderAt) Delete(ctx context.Context, name string) error
				Delete removes the object with the given name.
				If object does not exist in the moment of deletion, Delete should throw error.

			( FileReaderAt) Exists(ctx context.Context, name string) (bool, error)
				Exists checks if the given object exists in the bucket.

			( FileReaderAt) Get(ctx context.Context, name string) (io.ReadCloser, error)
				Get returns a reader for the given object name.

			( FileReaderAt) GetRange(ctx context.Context, name string, off, length int64) (io.ReadCloser, error)
				GetRange returns a new range reader for the given object name and range.

			( FileReaderAt) IsAccessDeniedErr(err error) bool
				IsAccessDeniedErr returns true if access to object is denied.

			( FileReaderAt) IsObjNotFoundErr(err error) bool
				IsObjNotFoundErr returns true if error means that object is not found. Relevant to Get operations.

			( FileReaderAt) Iter(ctx context.Context, dir string, f func(string) error, options ...objstore.IterOption) error
				Iter calls f for each entry in the given directory (not recursive.). The argument to f is the full
				object name including the prefix of the inspected directory.
				Entries are passed to function in sorted order.

			( FileReaderAt) Name() string
				Name returns the bucket name for the provider.

			(*FileReaderAt) ReadAt(p []byte, off int64) (n int, err error)
				ReadAt implements the io.ReaderAt interface.

			( FileReaderAt) Upload(ctx context.Context, name string, r io.Reader) error
				Upload the contents of the reader as an object into the bucket.
				Upload should be idempotent.

		Implements (at least 7)
			 FileReaderAt : github.com/polarsignals/frostdb/query/logicalplan.Named
			*FileReaderAt : github.com/polarsignals/wal/types.ReadableFile
			 FileReaderAt : github.com/prometheus/common/expfmt.Closer
			 FileReaderAt : github.com/thanos-io/objstore.Bucket
			 FileReaderAt : github.com/thanos-io/objstore.BucketReader
			 FileReaderAt : io.Closer
			*FileReaderAt : io.ReaderAt

	 type Iceberg (struct)
		Iceberg is an Apache Iceberg backed DataSink/DataSource.

		Methods (total 7)
			(*Iceberg) Close() error
			(*Iceberg) Delete(_ context.Context, _ string) error
			(*Iceberg) Maintenance(ctx context.Context) error
			(*Iceberg) Prefixes(ctx context.Context, prefix string) ([]string, error)
				Prefixes lists all the tables found in the warehouse for the given database(prefix).

			(*Iceberg) Scan(ctx context.Context, prefix string, _ *dynparquet.Schema, filter logicalplan.Expr, _ uint64, callback func(context.Context, any) error) error
				Scan will load the latest Iceberg table. It will filter out any manifests that do not contain useful data.
				Then it will read the manifests that may contain useful data. It will then filter out the data file that dot not contain useful data.
				Finally it has a set of data files that may contain useful data. It will then read the data files and apply the filter to each row group in the data file.

			(*Iceberg) String() string
			(*Iceberg) Upload(ctx context.Context, name string, r io.Reader) error
				Upload a parquet file into the Iceberg table.

		Implements (at least 7)
			*Iceberg : github.com/polarsignals/frostdb.DataSink
			*Iceberg : github.com/polarsignals/frostdb.DataSinkSource
			*Iceberg : github.com/polarsignals/frostdb.DataSource
			*Iceberg : github.com/prometheus/common/expfmt.Closer
			*Iceberg : expvar.Var
			*Iceberg : fmt.Stringer
			*Iceberg : io.Closer
		As Outputs Of (at least one exported)
			func NewIceberg(uri string, ctlg catalog.Catalog, bucket objstore.Bucket, options ...IcebergOption) (*Iceberg, error)

	 type IcebergOption (func)
		IcebergOption is a function that configures an Iceberg DataSink/DataSource.

		As Outputs Of (at least 4)
			func WithDataFileExpiry(maxAge time.Duration) IcebergOption
			func WithIcebergPartitionSpec(spec iceberg.PartitionSpec) IcebergOption
			func WithLogger(l log.Logger) IcebergOption
			func WithMaintenanceSchedule(schedule time.Duration) IcebergOption
		As Inputs Of (at least one exported)
			func NewIceberg(uri string, ctlg catalog.Catalog, bucket objstore.Bucket, options ...IcebergOption) (*Iceberg, error)


Package-Level Functions (total 6)

	 func NewBucketReaderAt(bucket objstore.Bucket) *BucketReaderAt
		NewBucketReaderAt returns a new Bucket.

	 func NewIceberg(uri string, ctlg catalog.Catalog, bucket objstore.Bucket, options ...IcebergOption) (*Iceberg, error)
		NewIceberg creates a new Iceberg DataSink/DataSource.
		You must provide the URI of the warehouse and the objstore.Bucket that points to that warehouse.

	 func WithDataFileExpiry(maxAge time.Duration) IcebergOption
		WithDataFileExpiry will a maxiumum age for data files. Data files older than the max age will be deleted from the table periodically according to the maintenance schedule.

	 func WithIcebergPartitionSpec(spec iceberg.PartitionSpec) IcebergOption
		WithIcebergPartitionSpec sets the partition spec for the Iceberg table. This is useful for pruning manifests during scans.
		note that at this time the Iceberg storage engine does not write data in a partition fashion. So this is only useful for setting the upper/lower bounds
		of columns in the manifest data.

	 func WithLogger(l log.Logger) IcebergOption
	 func WithMaintenanceSchedule(schedule time.Duration) IcebergOption
		WithMaintenanceSchedule sets the schedule for the maintenance of the Iceberg table.
		This will spawn a goroutine that will periodically expire data files if WithDataFileExpiry is set.
		And will delete orphanded files from the table.


Package-Level Constants (only one)

	const DefaultOrphanedFileAge time.Duration = 86400000000000

The pages are generated with Golds v0.8.2. (GOOS=linux GOARCH=amd64)
Golds is a Go 101 project developed by Tapir Liu.
PR and bug reports are welcome and can be submitted to the issue list.
Please follow @zigo_101 (reachable from the left QR code) to get the latest news of Golds.