Relaxation of bilinear term in objective function - using CVXR and McCormick envelopes

Question

I have an objective function

objective function

with some constraints

constraints

which I want to minimize.

I want to use the R package CVXR and the McCormick envelopes.

Let's check the code:

library(CVXR)  # if necessary
x <- Variable(1)
y <- Variable(1)
w <- Variable(1)
objective <- Minimize(5*x^2 + 14*w + 10*y^2 -76*x -108*y +292)
constraints <- list(x >= 0, x <= 100,
                    y >= 0, y <= 100,
                    x+2*y <= 10, 
                    x+y <= 6,
                    w >= 0, w >= 100*x + 100*y - 10000,  # constraints according to McCormick envelopes
                    w <= 100*y, w <= 100*x)  # constraints according to McCormick envelopes
prob_OF <- Problem(objective, constraints)
solution_OF <- solve(prob_OF)
solution_OF$value
## -125.0667
solution_OF$getValue(x)                  
## 2.933333
solution_OF$getValue(y)
## 3.066667
solution_OF$getValue(w)
## 1.000135e-30

But sadly here is one drawback:

the values for and are only approximations because the McCormick is a relaxation of the contraints.
These are the true values:

To overcome this drawback you can use this approach (see the answer of josliber) where you split up the interval on one of the variables. Like josliber I split up on the x variable...

In the following part my problems are arising - but first have a look on my code:

library(CVXR)  # if necessary

# the two variables for the objective function
x <- Variable(1)
y <- Variable(1)

# five variables for w - substitute of the bilinear part 
w1 <- Variable(1)
w2 <- Variable(1)
w3 <- Variable(1)
w4 <- Variable(1)
w5 <- Variable(1)

# five boolean variables to know which range of the x-axis is used
z1 <- Variable(1, boolean=TRUE)
z2 <- Variable(1, boolean=TRUE)
z3 <- Variable(1, boolean=TRUE)
z4 <- Variable(1, boolean=TRUE)
z5 <- Variable(1, boolean=TRUE)

# objective function with the five linear substitutes for xy
objective <- Minimize(5*x^2 + 14*w1 + 14*w2 + 14*w3 + 14*w4 + 14*w5 + 10*y^2 -76*x -108*y +292)

# for convenience - to build up the constraints easier
x_Range=matrix(rep(seq(from=0,to=100,by=20),each=2)[-c(1,12)],nrow=2,byrow=FALSE)
##     [,1] [,2] [,3] [,4] [,5]
##[1,]    0   20   40   60   80
##[2,]   20   40   60   80  100
y_Range=matrix(c(0,100),nrow=2,byrow=FALSE)
##     [,1]
##[1,]    0
##[2,]  100

# FIRST alternative of the constraints
constraints <- list(x >= 0, x <= 100,
                    y >= 0, y <= 100,
                    x+2*y <= 10,  
                    x+y <= 6,
                    ## w1
                    w1 <= x_Range[2,1]*y + x1*y_Range[1,1] - x_Range[2,1]*y_Range[1,1]*z1,  #w1 <= xU1*y + x1*yL - xU1*yL*z1,
                    w1 <= x1*y_Range[2,1] + x_Range[1,1]*y - x_Range[1,1]*y_Range[2,1]*z1,  #w1 <= x1*yU + xL1*y - xL1*yU*z1,
                    x1 >= x_Range[1,1]*z1, x1 <= x_Range[2,1]*z1,                           #xL1*z1 <= x1 <= xU1*z1,
                    ## w2
                    w2 <= x_Range[2,2]*y + x2*y_Range[1,1] - x_Range[2,2]*y_Range[1,1]*z2,  #w2 <= xU2*y + x2*yL - xU2*yL*z2,
                    w2 <= x2*y_Range[2,1] + x_Range[1,2]*y - x_Range[1,2]*y_Range[2,1]*z2,  #w2 <= x2*yU + xL2*y - xL2*yU*z2,
                    x2 >= x_Range[1,2]*z2, x2 <= x_Range[2,2]*z2,                           #xL2*z2 <= x2 <= xU2*z2,
                    ## w3
                    w3 <= x_Range[2,3]*y + x3*y_Range[1,1] - x_Range[2,3]*y_Range[1,1]*z3,  #w3 <= xU3*y + x3*yL - xU3*yL*z3,
                    w3 <= x3*y_Range[2,1] + x_Range[1,3]*y - x_Range[1,3]*y_Range[2,1]*z3,  #w3 <= x3*yU + xL3*y - xL3*yU*z3,
                    x3 >= x_Range[1,3]*z3, x3 <= x_Range[2,3]*z3,                           #xL3*z3 <= x3 <= xU3*z3,
                    ## w4
                    w4 <= x_Range[2,4]*y + x4*y_Range[1,1] - x_Range[2,4]*y_Range[1,1]*z4,  #w4 <= xU4*y + x4*yL - xU4*yL*z4,
                    w4 <= x4*y_Range[2,1] + x_Range[1,4]*y - x_Range[1,4]*y_Range[2,1]*z4,  #w4 <= x4*yU + xL4*y - xL4*yU*z4,
                    x4 >= x_Range[1,4]*z4, x4 <= x_Range[2,4]*z4,                           #xL4*z4 <= x4 <= xU4*z4,
                    ## w5
                    w5 <= x_Range[2,5]*y + x5*y_Range[1,1] - x_Range[2,5]*y_Range[1,1]*z5,  #w5 <= xU5*y + x5*yL - xU5*yL*z5,
                    w5 <= x5*y_Range[2,1] + x_Range[1,5]*y - x_Range[1,5]*y_Range[2,1]*z5,  #w5 <= x5*yU + xL5*y - xL5*yU*z5,
                    x5 >= x_Range[1,5]*z5, x5 <= x_Range[2,5]*z5                            #xL5*z5 <= x5 <= xU5*z5
                    )

# SECOND alternative of the constraints
constraints <- list(x >= 0, x <= 100,
                    y >= 0, y <= 100,
                    x+2*y <= 10,  
                    x+y <= 6, 
                    ## w1
                    w1 >= x_Range[1,1]*y + x1*y_Range[1,1] - x_Range[1,1]*y_Range[1,1]*z1,  #w1 >= xL1*y + x1*yL - xL1*yL*z1,
                    w1 >= x_Range[2,1]*y + x1*y_Range[2,1] - x_Range[2,1]*y_Range[2,1]*z1,  #w1 >= xU1*y + x1*yU - xU1*yU*z1,
                    w1 <= x_Range[2,1]*y + x1*y_Range[1,1] - x_Range[2,1]*y_Range[1,1]*z1,  #w1 <= xU1*y + x1*yL - xU1*yL*z1,
                    w1 <= x1*y_Range[2,1] + x_Range[1,1]*y - x_Range[1,1]*y_Range[2,1]*z1,  #w1 <= x1*yU + xL1*y - xL1*yU*z1,
                    x1 >= x_Range[1,1]*z1, x1 <= x_Range[2,1]*z1,                           #xL1*z1 <= x1 <= xU1*z1,
                    ## w2
                    w2 >= x_Range[1,2]*y + x2*y_Range[1,1] - x_Range[1,2]*y_Range[1,1]*z2,  #w2 >= xL2*y + x2*yL - xL2*yL*z2,
                    w2 >= x_Range[2,2]*y + x2*y_Range[2,1] - x_Range[2,2]*y_Range[2,1]*z2,  #w2 >= xU2*y + x2*yU - xU2*yU*z2,
                    w2 <= x_Range[2,2]*y + x2*y_Range[1,1] - x_Range[2,2]*y_Range[1,1]*z2,  #w2 <= xU2*y + x2*yL - xU2*yL*z2,
                    w2 <= x2*y_Range[2,1] + x_Range[1,2]*y - x_Range[1,2]*y_Range[2,1]*z2,  #w2 <= x2*yU + xL2*y - xL2*yU*z2,
                    x2 >= x_Range[1,2]*z2, x2 <= x_Range[2,2]*z2,                           #xL2*z2 <= x2 <= xU2*z2,
                    ## w3
                    w3 >= x_Range[1,3]*y + x3*y_Range[1,1] - x_Range[1,3]*y_Range[1,1]*z3,  #w3 >= xL3*y + x3*yL - xL3*yL*z3,
                    w3 >= x_Range[2,3]*y + x3*y_Range[2,1] - x_Range[2,3]*y_Range[2,1]*z3,  #w3 >= xU3*y + x3*yU - xU3*yU*z3,
                    w3 <= x_Range[2,3]*y + x3*y_Range[1,1] - x_Range[2,3]*y_Range[1,1]*z3,  #w3 <= xU3*y + x3*yL - xU3*yL*z3,
                    w3 <= x3*y_Range[2,1] + x_Range[1,3]*y - x_Range[1,3]*y_Range[2,1]*z3,  #w3 <= x3*yU + xL3*y - xL3*yU*z3,
                    x3 >= x_Range[1,3]*z3, x3 <= x_Range[2,3]*z3,                           #xL3*z3 <= x3 <= xU3*z3,
                    ## w4
                    w4 >= x_Range[1,4]*y + x4*y_Range[1,1] - x_Range[1,4]*y_Range[1,1]*z4,  #w4 >= xL4*y + x4*yL - xL4*yL*z4,
                    w4 >= x_Range[2,4]*y + x4*y_Range[2,1] - x_Range[2,4]*y_Range[2,1]*z4,  #w4 >= xU4*y + x4*yU - xU4*yU*z4,
                    w4 <= x_Range[2,4]*y + x4*y_Range[1,1] - x_Range[2,4]*y_Range[1,1]*z4,  #w4 <= xU4*y + x4*yL - xU4*yL*z4,
                    w4 <= x4*y_Range[2,1] + x_Range[1,4]*y - x_Range[1,4]*y_Range[2,1]*z4,  #w4 <= x4*yU + xL4*y - xL4*yU*z4,
                    x4 >= x_Range[1,4]*z4, x4 <= x_Range[2,4]*z4,                           #xL4*z4 <= x4 <= xU4*z4,
                    ## w5
                    w5 >= x_Range[1,5]*y + x5*y_Range[1,1] - x_Range[1,5]*y_Range[1,1]*z5,  #w5 >= xL5*y + x5*yL - xL5*yL*z5,
                    w5 >= x_Range[2,5]*y + x5*y_Range[2,1] - x_Range[2,5]*y_Range[2,1]*z5,  #w5 >= xU5*y + x5*yU - xU5*yU*z5,
                    w5 <= x_Range[2,5]*y + x5*y_Range[1,1] - x_Range[2,5]*y_Range[1,1]*z5,  #w5 <= xU5*y + x5*yL - xU5*yL*z5,
                    w5 <= x5*y_Range[2,1] + x_Range[1,5]*y - x_Range[1,5]*y_Range[2,1]*z5,  #w5 <= x5*yU + xL5*y - xL5*yU*z5,
                    x5 >= x_Range[1,5]*z5, x5 <= x_Range[2,5]*z5                            #xL5*z5 <= x5 <= xU5*z5
                    )

# Now the final calculations
prob_OF <- Problem(objective, constraints)
solution_OF <- solve(prob_OF)

# results
### of FIRST constraint alternative
solution_OF$value
## NA
solution_OF$getValue(x1)
## NA
solution_OF$getValue(x2)
## NA
solution_OF$getValue(x3)
## NA
solution_OF$getValue(x4)
## NA
solution_OF$getValue(x5)
## NA

# results
### of SECOND constraint alternative
solution_OF$value
## 15.99776
solution_OF$getValue(x1)
## 4.206985
solution_OF$getValue(x2)
## 28.49989
solution_OF$getValue(x3)
## 49.34965
solution_OF$getValue(x4)
## 69.59733
solution_OF$getValue(x5)
## 89.71508

Questions:

Why is josliber only using two of the four inequalities?
- Therefore also my attempt of the second constraint alternative in the code

Can you please help me to solve my issue?!
- I think I have to use second constraint alternative but here the values of solution_OF$getValue(x'ses) are so high that like josliber mentioned to sum them up won't get me the expected result of x=2.
- In both alternatives there are the two constraints x+2*y <= 10 and x+y <= 6. Must I convert them due to I splitted up the x range in five subparts?
Is it also possible to split up on more than one variable? Referring to the answer of josliber.

Just FYI: there are global solvers that do this automatically for you. I solve a lot of non-convex problems, and never, ever write down these McCormick envelopes. — Erwin Kalvelagen, Jun 17 '21 at 20:51
@Erwin Kalvelagen: And can you please name such an solver to the McCormick envelopes - best applicable in the CVXR environment. I'm new to this topic and therefore I'm writing this down. For every convenience I'm thankful. — tueftla, Jun 17 '21 at 21:41
Not CVXR but using `optim` in base R: `g <- function(z, x = z[1], y = z[2]) 5*x^2 + 14*x*y + 10*y^2 -76*x -108*y +292 + max(x+2*y-10, 0) + max(x+y-6, 0) + max(-x, 0) + max(-y, 0); ans <- optim(c(1, 1), g); ans$par` — G. Grothendieck, Jun 21 '21 at 15:16
@G. Grothendieck: Thanks a lot for showing the formulation in `optim`. But I'm really interessted in the usage of the McCormick envelopes like [josliber](https://stackoverflow.com/questions/30774270/how-to-convert-quadratic-to-linear-program) explained in his answer. The reason why I'm aking this is because I failed in the implementation... — tueftla, Jun 22 '21 at 06:56

Erwin Kalvelagen · Answer 1 · 2021-06-18T15:58:57.950

3

We can make the problem convex:

min 5z^2+(1/5)y^2-76x-108*y+292
    z = x+(7/5)y
    x+2y <= 10
    x+y <= 6

This can be fed into CVXR and solved with any QP solver. (Of course, check my math).

edited Jun 18 '21 at 15:58

answered Jun 18 '21 at 15:18

Erwin Kalvelagen

15,677
2
14
39

That's also an awesome hint to reformulate the equation to get a convex problem. But I was rather interessted in the rigth usage of the McCormick envelopes - although you commented above that you will never ever program these McCormick envelopes by hand. [Here](https://stackoverflow.com/questions/30774270/how-to-convert-quadratic-to-linear-program) josliber showed the procedure but I was not able to implement it. Therefore I asked this question... – tueftla Jun 21 '21 at 08:23

Relaxation of bilinear term in objective function - using CVXR and McCormick envelopes

1 Answers1