New AR checkpoint

The latest autoregressive module checkpoint, trained for another 1M steps. It has greater stability and naturalness compared to the prior model. We provide the latest AR model in both torch and safetensors format.

Please give it a try!

Latest checkpoints

The latest pytorch and safetensors checkpoints, with marked improved AR performance (better pronunciation) for deep cloning. Please give it a try!

Latest checkpoints

Checkpoints further trained to 1.7M steps for the AR model (with some fixes) and 1.75M steps for the NAR model.

Now available in both safetensors and pytorch .pt format. Both checkpoints have the tokenizers embedded in them.

Initial checkpoints

The AR and NAR model checkpoints for Mars 5, trained for 1.68M steps for the AR model and 1.26M steps for the NAR model, the number of updates is some fraction thereof with gradient accumulation steps.