pyomo + reticulate error 6 the handle is invalid

Question

I'm trying to run a pyomo optimization and I get the error message [Error 6] The handle is invalid. Not sure how to interpret it, looking around it seems to have something to do with privileges but I don't really understand it.

Find below the complete error trace and also a toy example to reproduce it.

Full error trace:

Error in py_run_file_impl(file, local, convert) : ApplicationError: Could not execute the command: 'C:\Users\xxx\AppData\Local\Continuum\anaconda3\envs\lucy\Library\bin\ipopt.exe c:\users\xxx\appdata\local\temp\tmpp2hmid.pyomo.nl -AMPL' Error message: [Error 6] The handle is invalid

Detailed traceback: File "", line 46, in File "C:\Users\xxx\AppData\Local\CONTIN~1\ANACON~1\envs\lucy\lib\site-packages\pyomo\opt\base\solvers.py", line 578, in solve _status = self._apply_solver() File "C:\Users\xxx\AppData\Local\CONTIN~1\ANACON~1\envs\lucy\lib\site-packages\pyomo\opt\solver\shellcmd.py", line 246, in _apply_solver self._rc, self._log = self._execute_command(self._command) File "C:\Users\xxx\AppData\Local\CONTIN~1\ANACON~1\envs\lucy\lib\site-packages\pyomo\opt\solver\shellcmd.py", line 309, in _execute_command tee = self._tee File "C:\Users\xxx\AppData\Local\CONTIN~1\ANACON~1\envs\lucy\lib\site-packages\pyutilib\subprocess\processmngr.py", line 660, in run_command

Reproducible example based on this.

Pure python code (it works when I run it in python, in the conda environment called "lucy"):

from pyomo.environ import *
infinity = float('inf')

model = AbstractModel()

# Foods
model.F = Set()
# Nutrients
model.N = Set()

# Cost of each food
model.c    = Param(model.F, within=PositiveReals)
# Amount of nutrient in each food
model.a    = Param(model.F, model.N, within=NonNegativeReals)
# Lower and upper bound on each nutrient
model.Nmin = Param(model.N, within=NonNegativeReals, default=0.0)
model.Nmax = Param(model.N, within=NonNegativeReals, default=infinity)
# Volume per serving of food
model.V    = Param(model.F, within=PositiveReals)
# Maximum volume of food consumed
model.Vmax = Param(within=PositiveReals)

# Number of servings consumed of each food
model.x = Var(model.F, within=NonNegativeIntegers)

# Minimize the cost of food that is consumed
def cost_rule(model):
    return sum(model.c[i]*model.x[i] for i in model.F)
model.cost = Objective(rule=cost_rule)

# Limit nutrient consumption for each nutrient
def nutrient_rule(model, j):
    value = sum(model.a[i,j]*model.x[i] for i in model.F)
    return model.Nmin[j] <= value <= model.Nmax[j]
model.nutrient_limit = Constraint(model.N, rule=nutrient_rule)

# Limit the volume of food consumed
def volume_rule(model):
    return sum(model.V[i]*model.x[i] for i in model.F) <= model.Vmax
model.volume = Constraint(rule=volume_rule)

opt = SolverFactory('ipopt')
instance = model.create_instance('diet.dat')
results = opt.solve(instance, tee=False)
results

The code to run it in R with reticulate is pretty straightforward:

library(reticulate)
use_condaenv(condaenv = "lucy")
py_run_file("../pyomo_scripts/test.py")

And finally for completeness this is the diet.dat file (must be at the same path as the python/R files):

param:  F:                          c     V  :=
  "Cheeseburger"                 1.84   4.0  
  "Ham Sandwich"                 2.19   7.5  
  "Hamburger"                    1.84   3.5  
  "Fish Sandwich"                1.44   5.0  
  "Chicken Sandwich"             2.29   7.3  
  "Fries"                         .77   2.6  
  "Sausage Biscuit"              1.29   4.1  
  "Lowfat Milk"                   .60   8.0 
  "Orange Juice"                  .72  12.0 ;

param Vmax := 75.0;

param:  N:       Nmin   Nmax :=
        Cal      2000      .
        Carbo     350    375
        Protein    55      .
        VitA      100      .
        VitC      100      .
        Calc      100      .
        Iron      100      . ;

param a:
                               Cal  Carbo Protein   VitA   VitC  Calc  Iron :=
  "Cheeseburger"               510     34     28     15      6    30    20
  "Ham Sandwich"               370     35     24     15     10    20    20
  "Hamburger"                  500     42     25      6      2    25    20
  "Fish Sandwich"              370     38     14      2      0    15    10
  "Chicken Sandwich"           400     42     31      8     15    15     8
  "Fries"                      220     26      3      0     15     0     2
  "Sausage Biscuit"            345     27     15      4      0    20    15
  "Lowfat Milk"                110     12      9     10      4    30     0
  "Orange Juice"                80     20      1      2    120     2     2 ;

edit after comments:

These are the versions for pyomo and ipopt

pyomo                     5.6.4                    py36_0    conda-forge
pyomo.extras              3.3                 py36_182212    conda-forge
ipopt                     3.11.1                        2    conda-forge

I have inherited loads of code in R with the optimization done in pyomo through system calls. I'm trying to improve it by using reticulate so that I avoid writing and reading files and I have more control... if I still have do system calls within python, I will gain very little by using reticulate.

Thanks.

What version of Pyomo are you using? How are you using Pyomo to solve the model, using the `pyomo` command line interface or using a Python script? — Bethany Nicholson, May 30 '19 at 19:20

score 3 · Answer 1 · answered Jun 07 '19 at 09:23

3

I can not say I understand this problem entirely, however it is a very interesting one to research, mainly because I got a different error message

TypeError: signal handler must be signal.SIG_IGN, signal.SIG_DFL, or a callable object

and while I got the error every time I ran py_run_file("test.py") in a new r session, by the second run there was no error.

That being said I believe it is related to this issue: https://github.com/PyUtilib/pyutilib/issues/31

I didn't face any problem after adding the two lines :

import pyutilib.subprocess.GlobalData
pyutilib.subprocess.GlobalData.DEFINE_SIGNAL_HANDLERS_DEFAULT = False

in the python script before invoking the solver.

Hope this helps

answered Jun 07 '19 at 09:23

DS_UNI

2,600
2
11
22

Thank you so much for your answer. However I did a quick test and I get the same error message... I'll take a slower look at the issue you link and I'll let you know. – lrnzcig Jun 07 '19 at 09:32
Although this answer does not really work for me, the bounty goes automatically since it is the only one.. And anyway it was helpful, thank you for that. However, since the error message is not really the same, I'd appreciate details on your machine and software versions, so that I can maybe get a hint on how to solve it. Thanks! – lrnzcig Jun 13 '19 at 10:35
Sure! The pyomo and ipopt were the same as yours, system wise I was on a MacOS Mojave, with the preview version of Rstudio v1.2 and reticulate 1.10 – DS_UNI Jun 17 '19 at 07:21
As I don't have access to that machine anymore I will be trying again on a windows 10 and I'l get back to you on that – DS_UNI Jun 17 '19 at 07:22

score 1 · Accepted Answer · answered Jul 31 '19 at 15:11

1

If you can execute the python version, try to r session with administrative right with the following code

library("reticulate")


##-- your directory containing 'diet.py' and 'diet.dat'
setwd("D:/project/Dropbox/lectures/2104xxx scg_opt/src/02"")



##-- execute code
a <- py_run_file("diet.py",local=T)
a$results

answered Jul 31 '19 at 15:11

oran

26
1

Thanks for your answer. It seems to be working! The key is the `setwd` so that the file `diet.dat` may be read. Administrator rights don't seem to be needed. Again thanks! – lrnzcig Aug 01 '19 at 12:27

pyomo + reticulate error 6 the handle is invalid

2 Answers2