Description
Since Argo supports looping over parameters with a single task configuration, we want to take advantage of that to process images from an images collection in parallel with just one job configuration and have all the resulting data going to the same output folder/collection.
Proposal
Add support for Argo loops:
- list of parameters to loop over will be generated by a task inserted between a job and its dependencies,
- add
withParam property in workflow task template,
- add
loopParams property support in plugin manifest
- create Docker image for generating param list
Additional context
Since behind the scene, Argo will create multiple jobs for the loop, this is breaking the changes proposed in #137, so moving this issue to milestone 3.1.0
cc: @Nicholas-Schaub
Description
Since Argo supports looping over parameters with a single task configuration, we want to take advantage of that to process images from an images collection in parallel with just one job configuration and have all the resulting data going to the same output folder/collection.
Proposal
Add support for Argo loops:
withParamproperty in workflow task template,loopParamsproperty support in plugin manifestAdditional context
Since behind the scene, Argo will create multiple jobs for the loop, this is breaking the changes proposed in #137, so moving this issue to milestone 3.1.0
cc: @Nicholas-Schaub