We provide:
- Pretrained Cutie model: https://github.com/hkchengrex/Cutie/releases/tag/v1.0 or https://drive.google.com/drive/folders/1E9ESHFlGU2KQkeRfH14kZzbdnwA0dH0f?usp=share_link
- Pre-computed outputs: https://drive.google.com/drive/folders/1x-jf5GHB4hypU9cDZ0VSkMKGm8MR0eQJ?usp=share_link
Note: the provided BURST visualizations were not done correctly. You can use scripts/convert_burst_to_vos_train.py to visualize from the prediction JSON instead.
-
Datasets should be placed out-of-source. See TRAINING.md for the directory structure.
-
For the LVOS validation set, pre-process it by keeping only the first annotations:
python scripts/data/preprocess_lvos.py ../LVOS/valid/Annotations ../LVOS/valid/Annotations_first_only- Download the models that you need and place them in
./output.
python cutie/eval_vos.py dataset=[dataset] weights=[path to model file] model=[small/base]- Possible options for
dataset: seecutie/config/eval_config.yaml. - To test on your own data, use
dataset=genericand directly specify the image/mask directories. Example:
python cutie/eval_vos.py dataset=generic image_directory=examples/images mask_directory=examples/masks size=480- To separate outputs from different models/settings, specify
exp_id=[some unique id]. - By default, the results are saved out-of-source in
../output/. - By default, we only use the first-frame annotation in the generic mode. Specify
--use_all_masksto incorporate new objects (as in the YouTubeVOS dataset). - To evaluate with the "plus" setting, specify
--config-name eval_plus_config.yamlimmediately afterpython cutie/eval_vos.pybefore other arguments.
To get quantitative results:
- DAVIS 2017 validation: davis2017-evaluation or vos-benchmark.
- DAVIS 2016 validation: vos-benchmark.
- DAVIS 2017 test-dev: CodaLab
- YouTubeVOS 2018 validation: CodaLab
- YouTubeVOS 2019 validation: CodaLab
- LVOS val: LVOS
- LVOS test: CodaLab
- MOSE val: CodaLab
- BURST: CodaLab
See https://github.com/hkchengrex/XMem/blob/main/docs/PALETTE.md.