CVXR using Mosek for a quadratic minimization problem

Question

I'm trying to solve a Quadratic optimization problem with linear constrains using the R package CVXR. Although the default solver is able to solve the optimization, Mosek solver is not. The reason I'm looking to use Mosek is because I need to solve a bigger problem with more than 250 constrains and the default solver gives an inaccurate solution, so I'm looking to solve the bigger problem with Mosek. Here's a simple example were Mosek is not working:

suppressMessages(suppressWarnings(library(CVXR)))

Problem data

set.seed(10)
n <- 10
SAMPLES <- 100
mu <- matrix(abs(rnorm(n)), nrow = n)
Sigma <- matrix(rnorm(n^2), nrow = n, ncol = n)
Sigma <- t(Sigma) %*% Sigma

Form problem

w <- Variable(n)
ret <- t(mu) %*% w
risk <- quad_form(w, Sigma)
constraints <- list(w >= 0, sum(w) == 1,ret==mean(mu))

Risk aversion parameters

prob <- Problem(Minimize(risk), constraints)
result <- solve(prob,solver='MOSEK')

It gives the following error.

 Error in py_call_impl(callable, dots$args, dots$keywords) : 
  TypeError: 'int' object is not iterable 
10.stop(structure(list(message = "TypeError: 'int' object is not iterable", 
    call = py_call_impl(callable, dots$args, dots$keywords), 
    cppstack = structure(list(file = "", line = -1L, stack = c("1   reticulate.so                       0x000000010d278f9b _ZN4Rcpp9exceptionC2EPKcb + 219", 
    "2   reticulate.so                       0x000000010d27fa35 _ZN4Rcpp4stopERKNSt3__112basic_stringIcNS0_11char_traitsIcEENS0_9allocatorIcEEEE + 53",  ... 
9.mosek_intf at mosekglue.py#51
8.get_mosekglue()$mosek_intf(reticulate::r_to_py(A), b, reticulate::r_to_py(G), 
    h, c, dims, offset, reticulate::dict(solver_opts), verbose) 
7.Solver.solve(solver, objective, constraints, object@.cached_data, 
    warm_start, verbose, ...) 
6.Solver.solve(solver, objective, constraints, object@.cached_data, 
    warm_start, verbose, ...) 
5.CVXR::psolve(a, b, ...) 
4.CVXR::psolve(a, b, ...) 
3.solve.Problem(prob, solver = "MOSEK") 
2.solve(prob, solver = "MOSEK") 
1.solve(prob, solver = "MOSEK")

Somebody knows how to solve it, may be re expressing the problem?

My session info is the following:

R version 3.5.2 (2018-12-20)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS Mojave 10.14.1

Matrix products: default
BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] reticulate_1.10  Matrix_1.2-15    CVXR_0.99-2      e1071_1.7-0.1    rstudioapi_0.9.0
[6] openxlsx_4.1.0  

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.0        lattice_0.20-38   class_7.3-14      gmp_0.5-13.2      R.methodsS3_1.7.1
 [6] grid_3.5.2        R6_2.3.0          jsonlite_1.6      zip_1.0.0         Rmpfr_0.7-2      
[11] R.oo_1.22.0       R.utils_2.7.0     tools_3.5.2       bit64_0.9-7       bit_1.1-14       
[16] compiler_3.5.2    scs_1.1-1         ECOSolveR_0.4

Thanks

This looks like a bug in the cvxr interface to Mosek rather than Mosek not being able to solve the problem — Michal Adamaszek, Feb 05 '19 at 20:32
@JonnyCrunch this is a link where you can find how to do it https://cvxr.rbind.io/cvxr_examples/cvxr_using-other-solvers/. — Alejandro Andrade, Feb 05 '19 at 21:13
According to "mosek_intf at mosekglue.py#51" it is in the part that sets up the problem, in fact quite early. Maybe it gets incorrect data. — Michal Adamaszek, Feb 06 '19 at 06:23

Henrik Alsing Friberg · Answer 1 · 2019-02-06T14:55:57.017

The Python error

TypeError: 'int' object is not iterable'

shows that the call to mosek_intf in mosekglue.py expected a list to iterate over (somewhere unspecified), but received a scalar. This might be caused by the fact that -- since everything is a list in R -- reticulate handles single-element and multi-element lists different (see its type conversions).

Having only read the source code, my best guess is that it is mosekglue.py (line 102) that fails because your problem only has one second-order cone (specifically, I believe reticulate sends dims=dict(q: n) instead of dims=dict(q: [n])).

Your options are to submit a bug report to the CVXR project on GitHub and wait, and/or work to confirm my suspicion (maybe even propose a fix and contribute it to the project), and/or modify your problem with dummy material until it slips through the interface.

score 1 · Accepted Answer · answered Mar 19 '19 at 01:45

This was indeed a problem in mosekglue.py that was addressed with version CVXR-0.99-3 and up; see changelog

Here is your example worked out:

> suppressMessages(suppressWarnings(library(CVXR)))
> set.seed(10)
> n <- 10
> SAMPLES <- 100
> mu <- matrix(abs(rnorm(n)), nrow = n)
> Sigma <- matrix(rnorm(n^2), nrow = n, ncol = n)
> Sigma <- t(Sigma) %*% Sigma
> w <- Variable(n)
> ret <- t(mu) %*% w
> risk <- quad_form(w, Sigma)
> constraints <- list(w >= 0, sum(w) == 1,ret==mean(mu))
> prob <- Problem(Minimize(risk), constraints)
> result <- solve(prob,solver='MOSEK', verbose=TRUE)
Problem
  Name                   :                 
  Objective sense        : min             
  Type                   : CONIC (conic optimization problem)
  Constraints            : 24              
  Cones                  : 1               
  Scalar variables       : 23              
  Matrix variables       : 0               
  Integer variables      : 0               

Optimizer started.
Presolve started.
Linear dependency checker started.
Linear dependency checker terminated.
Eliminator - tries                  : 0                 time                   : 0.00            
Lin. dep.  - tries                  : 1                 time                   : 0.00            
Lin. dep.  - number                 : 0               
Presolve terminated. Time: 0.00    
Problem
  Name                   :                 
  Objective sense        : min             
  Type                   : CONIC (conic optimization problem)
  Constraints            : 24              
  Cones                  : 1               
  Scalar variables       : 23              
  Matrix variables       : 0               
  Integer variables      : 0               

Optimizer  - threads                : 8               
Optimizer  - solved problem         : the primal      
Optimizer  - Constraints            : 13
Optimizer  - Cones                  : 1
Optimizer  - Scalar variables       : 22                conic                  : 12              
Optimizer  - Semi-definite variables: 0                 scalarized             : 0               
Factor     - setup time             : 0.00              dense det. time        : 0.00            
Factor     - ML order time          : 0.00              GP order time          : 0.00            
Factor     - nonzeros before factor : 89                after factor           : 91              
Factor     - dense dim.             : 0                 flops                  : 2.54e+03        
ITE PFEAS    DFEAS    GFEAS    PRSTATUS   POBJ              DOBJ              MU       TIME  
0   1.0e+00  6.9e+00  1.0e+00  0.00e+00   2.756709152e+01   2.756709152e+01   1.0e+00  0.00  
1   2.7e-01  1.8e+00  2.2e-01  -6.02e-01  -9.936262607e+00  4.762263760e+00   2.7e-01  0.00  
2   9.9e-02  6.8e-01  4.3e-01  1.23e+00   9.201188225e-01   -6.807855573e-01  9.9e-02  0.00  
3   3.6e-02  2.5e-01  4.4e-01  1.97e+00   -1.264256365e-01  -5.225584219e-01  3.6e-02  0.00  
4   9.2e-03  6.3e-02  1.8e-01  2.33e+00   2.868592743e-01   2.542157558e-01   9.2e-03  0.00  
5   2.0e-03  1.4e-02  8.8e-02  1.17e+00   2.840102253e-01   2.769232377e-01   2.0e-03  0.00  
6   3.7e-04  2.5e-03  3.9e-02  1.04e+00   2.879204145e-01   2.866174921e-01   3.7e-04  0.00  
7   6.3e-05  4.3e-04  1.6e-02  1.02e+00   2.901039465e-01   2.898751147e-01   6.3e-05  0.00  
8   1.3e-05  8.8e-05  7.4e-03  1.00e+00   2.902381000e-01   2.901914751e-01   1.3e-05  0.00  
9   3.6e-06  2.5e-05  4.0e-03  1.00e+00   2.902501323e-01   2.902366424e-01   3.6e-06  0.00  
10  1.1e-06  7.4e-06  2.2e-03  1.00e+00   2.902666158e-01   2.902625442e-01   1.1e-06  0.00  
11  3.6e-07  2.5e-06  1.3e-03  1.00e+00   2.902689540e-01   2.902675850e-01   3.6e-07  0.00  
12  5.7e-08  3.9e-07  5.1e-04  1.00e+00   2.902716546e-01   2.902714343e-01   5.7e-08  0.00  
13  6.6e-09  4.6e-08  1.7e-04  1.00e+00   2.902720234e-01   2.902719974e-01   6.6e-09  0.00  
Optimizer terminated. Time: 0.00    


Interior-point solution summary
  Problem status  : PRIMAL_AND_DUAL_FEASIBLE
  Solution status : OPTIMAL
  Primal.  obj: 2.9027202344e-01    nrm: 1e+00    Viol.  con: 6e-09    var: 0e+00    cones: 0e+00  
  Dual.    obj: 2.9027199740e-01    nrm: 1e+01    Viol.  con: 0e+00    var: 8e-08    cones: 0e+00  
> result$value
[1] 0.290272
>

CVXR using Mosek for a quadratic minimization problem

Problem data

Form problem

Risk aversion parameters

2 Answers2