This is an amazing work you have done! Congratulations. (I came here after seeing the paper)
I noticed that you are using two different frameworks (Llama factory for continuous pre-training and Axolotl for fine-tuning).
Any particular reasons for this choice ?
I am also interested in continuous pre-training and am looking for a good framework (which is not mono-GPU)
"Note: I have migrated to Llama-Factory for pretraining and Axolotl for finetuning.
This is an amazing work you have done! Congratulations. (I came here after seeing the paper)
I noticed that you are using two different frameworks (Llama factory for continuous pre-training and Axolotl for fine-tuning).
Any particular reasons for this choice ?
I am also interested in continuous pre-training and am looking for a good framework (which is not mono-GPU)
"Note: I have migrated to Llama-Factory for pretraining and Axolotl for finetuning.