5 Commits (68e4af8b0e12d0dd64eefa5b061d9861f8ce3198)

Author SHA1 Message Date
Stanislas0 2147a88827 Update megatron tokenizer 2 years ago
Stanislas0 d65d72af9a refactor 2 years ago
Stanislas0 b7dde72832 Refactor encode_whitespaces 2 years ago
Stanislas0 7b2ad293dd Refactor tokenizer 2 years ago
Stanislas0 a2b6a15321 Add int8 quantization 2 years ago