package segment

Import Path
	github.com/polarsignals/wal/segment (on go.dev)

Dependency Relation
	imports 9 packages, and imported by one package



Package-Level Type Names (total 3)


	/* sort by:  |  */
	
		Filer implements the abstraction for managing a set of segment files in a
		directory. It uses a VFS to abstract actual file system operations for easier
		testing.

		
			
				Create adds a new segment with the given info and returns a writer or an
				error.

			
				Delete removes the segment with given baseIndex and id if it exists. Note
				that baseIndex is technically redundant since ID is unique on it's own. But
				in practice we name files (or keys) with both so that they sort correctly.
				This interface allows a  simpler implementation where we can just delete
				the file if it exists without having to scan the underlying storage for a.

			
				DumpLogs attempts to read all log entries from segment files in the directory
				for debugging purposes. It does _not_ use the metadata and so may output log
				entries that are uncommitted or already truncated as far as the writing
				process is concerned. As such it should not be used for replication of data.
				It is useful though to debug the contents of the log even while the writing
				application is still running. After and before if non-zero specify exclusive
				bounds on the logs that should be returned which may allow the implementation
				to skip reading entire segment files that are not in the range.

			
				DumpSegment attempts to read the segment file specified by the baseIndex and
				ID. It's intended purpose is for debugging the contents of segment files and
				unlike the SegmentFiler interface, it doesn't assume the caller has access to
				the correct metadata. This allows dumping log segments in a WAL that is still
				being written to by another process. Without metadata we don't know if the
				file is sealed so always recover by reading through the whole file. If after
				or before are non-zero, the specify a exclusive lower or upper bound on which
				log entries should be emitted. No error checking is done on the read data. fn
				is called for each entry passing the raft info read from the file header (so
				that the caller knows which codec to use for example) the raft index of the
				entry and the raw bytes of the entry itself. The callback must return true to
				continue reading. The data slice is only valid for the lifetime of the call.

			
				List returns the set of segment IDs currently stored. It's used by the WAL
				on recovery to find any segment files that need to be deleted following a
				unclean shutdown. The returned map is a map of ID -> BaseIndex. BaseIndex
				is returned to allow subsequent Delete calls to be made.

			
				Open an already sealed segment for reading. Open may validate the file's
				header and return an error if it doesn't match the expected info.

			
				RecoverTail is called on an unsealed segment when re-opening the WAL it will
				attempt to recover from a possible crash. It will either return an error, or
				return a valid segmentWriter that is ready for further appends. If the
				expected tail segment doesn't exist it must return an error wrapping
				os.ErrNotExist.

		
			*Filer : github.com/polarsignals/wal/types.SegmentFiler
		
			func NewFiler(dir string, vfs types.VFS) *Filer


	
		Reader allows reading logs from a segment file.

		
			
				Close implements io.Closer

			
				GetLog returns the raw log entry bytes associated with idx. If the log
				doesn't exist in this segment types.ErrNotFound must be returned.

		
			*Reader : github.com/polarsignals/wal/types.SegmentReader
			*Reader : github.com/prometheus/common/expfmt.Closer
			*Reader : io.Closer


	
		Writer allows appending logs to a segment file as well as reading them back.

		
			
				Append adds one or more entries. It must not return until the entries are
				durably stored otherwise raft's guarantees will be compromised.

			
				Close implements io.Closer

			
				GetLog implements types.SegmentReader

			
				LastIndex returns the most recently persisted index in the log. It must
				respond without blocking on append since it's needed frequently by read
				paths that may call it concurrently. Typically this will be loaded from an
				atomic int. If the segment is empty lastIndex should return zero.

			
				OffsetForFrame implements tailWriter and allows readers to lookup entry
				frames in the tail's in-memory index.

			
				Sealed returns whether the segment is sealed or not. If it is it returns
				true and the file offset that it's index array starts at to be saved in
				meta data. WAL will call this after every append so it should be relatively
				cheap in the common case. This design allows the final Append to write out
				the index or any additional data needed at seal time in the same fsync.

		
			*Writer : github.com/polarsignals/wal/types.SegmentReader
			*Writer : github.com/polarsignals/wal/types.SegmentWriter
			*Writer : github.com/prometheus/common/expfmt.Closer
			*Writer : io.Closer




Package-Level Functions (total 2)


	
		FileName returns the formatted file name expected for this segment.
		SegmentFiler implementations could choose to ignore this but it's here to


	
		NewFiler creates a Filer ready for use.




Package-Level Variables (only one)


	
		ErrTooBig indicates that the caller tried to write a logEntry with a
		payload that's larger than we are prepared to support.




Package-Level Constants (total 5)


	const FrameCommit uint8 = 3
	const FrameEntry uint8 = 1
	const FrameIndex uint8 = 2
	const FrameInvalid uint8 = 0
	
		MaxEntrySize is the largest we allow any single raft log entry to be. This
		is larger than our raft implementation ever allows so seems safe to encode
		statically for now. We could make this configurable. It's main purpose it
		to limit allocation when reading entries back if their lengths are
		corrupted.