package segmenter

Import Path
	github.com/go-text/typesetting/segmenter (on go.dev)

Dependency Relation
	imports one package, and imported by 3 packages


Involved Source Files

	
		Package segmenter implements Unicode rules used
		to segment a paragraph of text according to several criteria.
		In particular, it provides a way of delimiting line break opportunities.
		
		The API of the package follows the very nice iterator pattern proposed
		in github.com/npillmayer/uax,
		but use a somewhat simpler internal implementation, inspired by Pango.
		
		The reference documentation is at https://unicode.org/reports/tr14
		and https://unicode.org/reports/tr29.

	    unicode14_rules.go
	    unicode29_rules.go


Package-Level Type Names (total 7)


	/* sort by:  |  */
	
		Grapheme is the content of a grapheme delimited by the segmenter.

		
			
				Offset is the start of the grapheme in the input rune slice

			
				Text is a subslice of the original input slice, containing the delimited grapheme

		
			func (*GraphemeIterator).Grapheme() Grapheme


	
		GraphemeIterator provides a convenient way of
		iterating over the graphemes delimited by a `Segmenter`.

		
			
				Grapheme returns the current `Grapheme`

			
				Next returns true if there is still a grapheme to process,
				and advances the iterator; or return false.

		
			*GraphemeIterator : github.com/apache/arrow-go/v18/arrow/compute/exec.ArrayIter[bool]
		
			func (*Segmenter).GraphemeIterator() *GraphemeIterator


	
		Line is the content of a line delimited by the segmenter.

		
			
				IsMandatoryBreak is true if breaking (at the end of the line)
				is mandatory

			
				Offset is the start of the line in the input rune slice

			
				Text is a subslice of the original input slice, containing the delimited line

		
			func (*LineIterator).Line() Line


	
		LineIterator provides a convenient way of
		iterating over the lines delimited by a `Segmenter`.

		
			
				Line returns the current `Line`

			
				Next returns true if there is still a line to process,
				and advances the iterator; or return false.

		
			*LineIterator : github.com/apache/arrow-go/v18/arrow/compute/exec.ArrayIter[bool]
		
			func (*Segmenter).LineIterator() *LineIterator


	
		Segmenter is the entry point of the package.
		
		Usage :
		
			var seg Segmenter
			seg.Init(...)
			iter := seg.LineIterator()
			for iter.Next() {
			  ... // do something with iter.Line()
			}

		
			
				GraphemeIterator returns an iterator over the graphemes
				delimited in [Init].

			
				Init resets the segmenter storage with the given input,
				and computes the attributes required to segment the text.

			
				InitWithBytes resets the segmenter storage with the given byte slice input,
				and computes the attributes required to segment the text.
				
				If paragraph includes an invalid UTF-8 sequence, these are replaced with U+FFFD.
				
				InitWithBytes is more efficient than [Init] if the input is a byte slice.
				No allocation for the text is made if its internal buffer capacity is already large enough.

			
				InitWithString resets the segmenter storage with the given string input,
				and computes the attributes required to segment the text.
				
				If paragraph includes an invalid UTF-8 sequence, these are replaced with U+FFFD.
				
				InitWithString is more efficient than [Init] if the input is a string.
				No allocation for the text is made if its internal buffer capacity is already large enough.

			
				LineIterator returns an iterator on the lines
				delimited in [Init].

			
				WordIterator returns an iterator over the word
				delimited in [Init].



	
		Word is the content of a word delimited by the segmenter.
		
		More precisely, a word is formed by runes
		with the [Alphabetic] property, or with a General_Category of Number,
		delimited by the Word Boundary Unicode Property.
		
		See also https://unicode.org/reports/tr29/#Word_Boundary_Rules,
		http://unicode.org/reports/tr44/#Alphabetic and
		http://unicode.org/reports/tr44/#General_Category_Values

		
			
				Offset is the start of the grapheme in the input rune slice

			
				Text is a subslice of the original input slice, containing the delimited grapheme

		
			func (*WordIterator).Word() Word


	

		
			
				Next returns true if there is still a word to process,
				and advances the iterator; or return false.

			
				Word returns the current `Word`

		
			*WordIterator : github.com/apache/arrow-go/v18/arrow/compute/exec.ArrayIter[bool]
		
			func (*Segmenter).WordIterator() *WordIterator