How can I access the cost at each iteration when using the solve function in the cvxpy library?

Question

If you want to solve a convex optimization problem using the cvxpy library, only the last iteration cost can be calculated and returned, while other intermediate costs and related values are printed during the process. Take a look at the following code, which is obtained from the cvxpy library itself:

import cvxpy

x = cvxpy.Variable(2)
objective = cvxpy.Minimize(x[0] + cvxpy.norm(x, 1))
constraints = [x >= 2]
problem = cvxpy.Problem(objective, constraints)

problem.solve(solver=cvxpy.ECOS, verbose=True)

And the printed result:

ECOS 2.0.10 - (C) embotech GmbH, Zurich Switzerland, 2012-15. Web: www.embotech.com/ECOS

It     pcost       dcost      gap   pres   dres    k/t    mu     step   sigma     IR    |   BT
 0  +6.667e-01  +7.067e-01  +6e+00  6e-01  1e-02  1e+00  9e-01    ---    ---    1  1  - |  -  - 
 1  +3.500e+00  +3.925e+00  +1e+00  3e-01  4e-03  8e-01  2e-01  0.9890  2e-01   1  1  1 |  0  0
 2  +5.716e+00  +5.825e+00  +2e-01  6e-02  8e-04  2e-01  4e-02  0.9091  8e-02   1  1  1 |  0  0
 3  +5.997e+00  +5.998e+00  +3e-03  7e-04  1e-05  2e-03  5e-04  0.9881  1e-04   1  1  1 |  0  0
 4  +6.000e+00  +6.000e+00  +3e-05  8e-06  1e-07  3e-05  5e-06  0.9890  1e-04   1  1  1 |  0  0
 5  +6.000e+00  +6.000e+00  +3e-07  9e-08  1e-09  3e-07  6e-08  0.9890  1e-04   1  0  0 |  0  0
 6  +6.000e+00  +6.000e+00  +4e-09  1e-09  1e-11  3e-09  6e-10  0.9890  1e-04   1  0  0 |  0  0

OPTIMAL (within feastol=9.9e-10, reltol=6.2e-10, abstol=3.7e-09).
Runtime: 0.000061 seconds.

We can only evaluate the final cost value of the last iteration using the objective.value or problem.value.

There is a similar question here but I don't need the other variables, I only need the costs which are printed by the library itself.

Answer 1

After an extensive search through cvxpy internals, I discovered that it utilizes an underlying library (such as ecos) for solving, but unfortunately, it lacks any data on the intermediate iterations. I also confirmed that the printed log is generated by the underlying library and not by cvxpy itself. Subsequently, my investigation of the ecos library revealed that it also does not store the intermediate information. Instead, it maintains a simple info struct that gets updated after each iteration.

Finally, I made the decision to capture the output stream and subsequently parse the result using the regex to extract the required information. I am aware that this approach is somewhat tricky and fragile, but it effectively solves my problem:

import re

import cvxpy
import wurlitzer

# The convex problem definition ...

with wurlitzer.pipes() as (out, err):
    problem.solve(solver=cvxpy.ECOS, verbose=True)

matches = re.findall(
    "^\s*(?P<iteration>\d+)\s+(?P<pcost>\S+)\s+.*$",
    out.getvalue(),
    re.MULTILINE
)

data = []
for iteration, pcost in matches:
    data.append(float(pcost))

print(data)

And I get:

[0.6667, 3.5, 5.716, 5.997, 6.0, 6.0, 6.0]

I only need the pcost value, but feel free to improve the regex pattern to suit your specific requirements.

I had to use wurlitzer because the table is printed at the C-level, and the common approaches, such as contextlib.redirect_stdout, cannot be used.