LLM Midterm Quiz 2

Along with the special tokens [UNK], [SEP], [PAD], [CLS], [MASK], the cased version of BERT has another special token [CASE]. (T/F)

False

[PAD]

ensure all texts have the same length (longest text in the batch)

[BOS]

start of text

[EOS}

end of text

Rotary positional encoding captures absolute and relative token position information. (T/F)

True

BPE tokenizer can handle any unknown word by representing it as a sequence of ______ or ______ tokens

subword, characters

BERT uses which tokenization method?

WordPiece

Using a batch size of one during data loadinghas two advantages 1) uses less memory and 2) smoother update of the model. (T/F)

False

Which of the following is an easier tokenizer but difficult for modeling?

Byte Tokens

Which one of the following models uses the special token <work> - for chain of thought reasoning?

Galactica

Having context length (max_length) and stride equal to each other while defining a dataloader avoids overfitting. (T/F)

True

Which one of the following tokens was used to prepend independent text sources?

<endoftext>