Hyperparameters to reproduce ConvNeXt-atto/femto/pico/nano

[yuweihao]

Hi @rwightman , many thanks for your great work. My recent papers are based on and cite timm. Thank you! I am trying to reproduce ConvNeXt-atto/femto/pico/nano, could you please share the hyperparameters? Thank you very much for your help.

[rwightman]

@yuweihao There are a set of nano that can be easily adapted to the others, need to adjust batch sizes, scale LR with batch size, generally decrease augmentation + regularization as you decrease size from nano...

https://gist.github.com/rwightman/ee0b02c1e99a0761264d1d1319e95e5b

[emrahnazif]

After more than 3 years, for anyone who is interested I have also converted them to v1 style:

https://github.com/datamarkin/ConvNeXt-v1.5

ConvNeXt-v1-atto/femto/pico/nano

[rwightman]

@emrahnazif I'm not following, I created v1 atto/femto/pico/nano models and trained them before they were used in the v2 repo and paper (paper has a reference for timm). The hparams that were asked about here were for training those (https://huggingface.co/timm/convnext_atto.d2_in1k).

[emrahnazif]

@rwightman You're completely right, and I apologize. This is quite embarrassing. I missed that convnext_atto/femto/pico/nano are already V1-style models (no GRN) at exactly these sizes existed and trained by you already. My project's premise (that no permissively-licensed weights existed at these sizes) was simply wrong.

I'm correcting my repo's README and I've pulled my write-up to fix its claims. Somehow I totally missed it, and thank you for the correction and great work.

[yuweihao]

Hi @rwightman , thank you very much!