Increasing efficiency during global optimisation of Wflow model

[AquaPore]

Feature type

Adding new functionality

Improvement Description

Dear Wflow developers,

Greeting from Ireland. I will be shortly working in performing global optimisation of Wflow model. For my experience the first run of the optimisation requires running the whole Wflow model by Main.Wflow.run(PathToml). This includes initializing the model for example the forcing and the Ldd data.

During the optimisation process, after modifying the input parameters it might not be necessary to initialize again the model. The question is which function should I call to bypass the initialisation of the Wflow model?

With kindness,
Joseph A.P. Pollacco

Implementation Description

From my experience, a function would need to be written to bypass the initialisation of Wflow ?

Additional Context

No response

[SouthEndMusic]

Hello @AquaPore, do I understand correctly that you want to run the same simulation multiple times, only changing certain parameters in each run? Then essentially you want to bring all the states (and the clock) back to their initial states. I don't think that is supported very well at the moment. But the question is: why do you want to do this, isn't the initialization time insignificant compared to the simulation time? You might also be interested in #885, depending on your optimization workflow.

[AquaPore]

Dear @SouthEndMusic,

A great thanks for your quick response. Yes, indeed as in my paper Multistep optimization of HyPix model for flexible vertical scaling of soil hydraulic parameters I would need to run Wflow model multiple times (e.g. 500 times) and only change certain parameters for example hydraulic parameters of a particular soil. Yes, indeed, this would require resetting all the states (and the clock) to their initial states, which, from my experience, is not a straightforward problem. I was getting different results with multiple runs although the parameters remained unchanged!

I guess that we would need to check if the initialisation time is insignificant compared to the simulation time? I found that initialising variables of the physically based model which I wrote was costly. Nevertheless, for the time being, at each run we will run Wflow from scratch, and in future development we can explore ways to increase the speed of optimisation.

I would be ever so grateful if you could provide further explanations how #885 could be of interest to the optimisation problem?

I will also need to investigate using of HPC.

Once again, a great thanks for your amazing support,
With kindness,
Joseph A.P. Pollacco

[SouthEndMusic]

Hi AquaPore,

At least make sure that you do successive runs in the same Julia session, then you don't have to wait for the precompile time each time.

#855 lets you run sub-areas of your model, which was precisely developed to speed up parameter tuning.

[AquaPore]

Dear @SouthEndMusic ,

A great thanks for providing explanations, it is great that we can select to optimize a parameter in a masked region of interest (e.g. soils which are sandy etc...).

With kindmess,
Joseph A.P. Pollacco