Previously the pod would only be deleted after all the steps are run. This did create an issue since the entrypoint is not reaping, it is just tail; and I would recommend switching to a reaping entrypoint if possible.
However, currently the pod is removed and rescheduled after every step. I don't think this is sound; it creates a lot of extra friction with the scheduler, When you are trying to fit containers efficiently, rescheduling the same pod constantly is not very efficient. If the point of that is maximum isolation, it might be ok to enable that through some environment variable.
This really works against the usage of services. Since I would expect services to stay up throughout the entire execution of the container job, not restarted after every step
(Misunderstood)
In the new implementation (0.8), for containerStep it uses a pod, while on the previous implementation (0.7) it was using a batch job, and not cleaning the jobs after it finishes. At least if there is an error they appear to remain there forever... Is there anything meant to clean these up over time?
Can we provide some documentation explaining how the container step hooks are actually placed
Previously the pod would only be deleted after all the steps are run. This did create an issue since the entrypoint is not reaping, it is justtail; and I would recommend switching to a reaping entrypoint if possible.However, currently the pod is removed and rescheduled after every step. I don't think this is sound; it creates a lot of extra friction with the scheduler, When you are trying to fit containers efficiently, rescheduling the same pod constantly is not very efficient. If the point of that is maximum isolation, it might be ok to enable that through some environment variable.This really works against the usage of services. Since I would expect services to stay up throughout the entire execution of the container job, not restarted after every step(Misunderstood)
In the new implementation (0.8), for containerStep it uses a pod, while on the previous implementation (0.7) it was using a batch job, and not cleaning the jobs after it finishes. At least if there is an error they appear to remain there forever... Is there anything meant to clean these up over time?
Can we provide some documentation explaining how the container step hooks are actually placed