Skip to content

README.md

Latest commit

 

History

History
74 lines (51 loc) · 2.53 KB

README.md

File metadata and controls

74 lines (51 loc) · 2.53 KB
Raw

RandOpt

Neural Thickets

Neural Thickets: Diverse Task Experts Are Dense Around Pretrained Weights

Yulu Gan, Phillip Isola

Paper | Project Page | Openreview | Starting with a 1D Experiment: Open In Colab

News

  • [2026-07] Iterative RandOpt is now on the iterative-randopt branch! Also support the pip install-able implementation that drops into existing verl or huggingface/trl setups.

Requirements

Option1: Python / Conda

(optional) conda activate your_env
pip install -r requirements.txt

Option2: Docker

From the directory containing RandOpt/:

Step Command
Build docker build -f RandOpt/docker/Dockerfile_vllm -t randopt-vllm:latest .
Run docker run -it --gpus all randopt-vllm:latest bash
Run (with data) docker run -it --gpus all -v /path/to/RandOpt/data:/workspace/data randopt-vllm:latest bash

Run RandOpt

Post-train on your own dataset

Please follow the instructions in CUSTOM_DATASET_GUIDE.md

Post-train on a standard dataset

First download the data here: data/README.md

Then, from the RandOpt directory:

Mode Command
Single node sbatch scripts/single_node.sh
Multiple nodes sbatch scripts/multiple_nodes.sh
Local (no Slurm) bash scripts/local_run.sh

Distill top-k models into a single model

Please follow the instructions in distillation/README.md.

Run Baselines

Please follow the instructions in baselines/README.md

Having questions?

Open an issue @ github.com/sunrainyg/RandOpt/issues.

Citation

@misc{gan2026neuralthickets,
      title={Neural Thickets: Diverse Task Experts Are Dense Around Pretrained Weights}, 
      author={Yulu Gan and Phillip Isola},
      year={2026},
      eprint={2603.12228},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2603.12228}, 
}