Scripts and methods for reproducing small architecture in-domain LeJEPA pretraining in appendix Table 5

[kc-111]

Hi Team,

I really enjoyed reading this paper. I am currently trying to reproduce the results in appendix Table 5 using resnet18 and the Epps-Pulley statistical test for the distributional regularization. Unfortunately, I am encountering a major road block in trying to achieve 70% linear probe accuracy on CIFAR100. Is there any guidance on how to select the augmentations and hyperparameters? The max I got right now is 60% by varying regularization strength, number of views, projection dimension, and batch size. For the augmentations, I used what was suggested in the readme in this repository. Any help would be greatly appreciated!

Thanks!
Kevin