8 Commits (68e4af8b0e12d0dd64eefa5b061d9861f8ce3198)

Author	SHA1	Message	Date
Stanislas0	2147a88827	Update megatron tokenizer	2 years ago
Stanislas0	d65d72af9a	refactor	2 years ago
Stanislas0	d055d96812	Fix vocab error	2 years ago
Stanislas0	b7dde72832	Refactor encode_whitespaces	2 years ago
Semen Zhydenko	479aa44d0e	Fixed typo: thier -> their	2 years ago
Stanislas0	7b2ad293dd	Refactor tokenizer	2 years ago
Stanislas0	a2b6a15321	Add int8 quantization	2 years ago
Qinkai	d5d97ed024	Add HumanEval-X benchmark	2 years ago