I'm using least_squares optimization to adjust the output of a numerical model to some measured data. At this stage I'm wondering how to determine appropriate values for the ftol, xtol and gtol parameters as they determine how and where the optimization will stop. To me it seems that these parameters fit nicely into the framework of the algorithms but I find it difficult to connect them to real world properties.
For example I have uncertainty estimates for my measured data, so once the optimizer reaches sufficient agreement between the model's output and the measured data within the limits of the uncertainty, it is reasonable to stop (i.e. np.all(np.abs(model_output - measured_data) < uncertainty)). However it doesn't seem this can be expressed with the ftol parameter. The termination condition is dF < ftol * F (where F is the sum of squared residuals and dF is its change), so even though I could compute ftol such as to prevent updates smaller than the uncertainty once F has been reached within that level, I also risk premature termination somewhere far off the desired solution. In the end it's up to the optimizer how large a step it wants to take at each iteration (and thus determines dF), so dF might be small compared to F even though it's far off the desired solution.
Another aspect is the change of parameters values. In the end, the resulting values obtained from the optimization will be used to adjust some real devices and these have a finite precision. So for example the devices won't distinguish values that differ by less than 1e-6. This means that again, once sufficient agreement between the model's output and the measured data has been achieved, any parameter update of less than 1e-6 is not meaningful. On the other hand, many small updates of < 1e-6 could sum up to a larger overall update > 1e-6 and I'm back at the same problem: it's up to the optimizer how large a step it wants to take and restricting this, I fear that I risk premature termination. In addition the xtol parameter again only describes a scaling factor between the parameter update and the current values. While I could use some value that reflects the precision of the devices around the expected final parameter values, I've seen the optimizer reaching intermediate parameter values two orders of magnitude larger than the final estimate, so this clearly risks premature termination.
While I find it difficult to choose appropriate values for the ftol, xtol and gtol parameters, it's equally unsatisfying to leave them at their default values without good arguments, since that implies agreement with their default values being reasonable.
