post_processors_template_processing.Rd
Provides a way to specify templates in order to add the special tokens to each input sequence as relevant.
hftokenizers::post_processors
-> processors_template_processing
Inherited methods
new()
post_processors_template_processing$new( single = NULL, pair = NULL, special_tokens = NULL )
single
(Template) – The template used for single sequences
pair
(Template) – The template used when both sequences are specified
special_tokens
(Tokens) – The list of special tokens used in each sequences
clone()
The objects of this class are cloneable with this method.
post_processors_template_processing$clone(deep = FALSE)
deep
Whether to make a deep clone.