"Failed to import pydot" throws in kerasR

Question

I use the package keras under R and I would like to know if there was a command like python with plot_model () which allows to display its neuron network

library(keras)

for example I would like to display this neural network under R

  model <- keras_model_sequential()
  model %>%
    layer_dense(units = 5, input_shape = 2) %>%
    layer_activation("relu") %>%
    layer_dense(units = 1)

I install package kerasR for use the function plot_model(), but i have this error.

> library(kerasR)
> plot_model(model)

Error in py_call_impl(callable, dots$args, dots$keywords) : 
  ImportError: Failed to import pydot. You must install pydot and graphviz for `pydotprint` to work.

Detailed traceback: 
  File "C:\Users\Idriss\ANACON~1\envs\R-TENS~1\lib\site-packages\keras\utils\vis_utils.py", line 131, in plot_model
    dot = model_to_dot(model, show_shapes, show_layer_names, rankdir)
  File "C:\Users\Idriss\ANACON~1\envs\R-TENS~1\lib\site-packages\keras\utils\vis_utils.py", line 52, in model_to_dot
    _check_pydot()
  File "C:\Users\Idriss\ANACON~1\envs\R-TENS~1\lib\site-packages\keras\utils\vis_utils.py", line 27, in _check_pydot
    raise ImportError('Failed to import pydot. You must install pydot'

I'm use windows 10 64 bits, i use RStudio with Anaconda

---

In [4] pydot.Dot.create(pydot.Dot())

Out[4]: b"%!PS-Adobe-3.0\r\n%%Creator: graphviz version 2.38.0 (20140413.2041)\r\n%%Title: G\r\n%%Pages: (atend)\r\n%%BoundingBox: (atend)\r\n%%EndComments\r\nsave\r\n%%BeginProlog\r\n/DotDict 200 dict def\r\nDotDict begin\r\n\r\n/setupLatin1 {\r\nmark\r\n/EncodingVector 256 array def\r\n EncodingVector 0\r\n\r\nISOLatin1Encoding 0 255 getinterval putinterval\r\nEncodingVector 45 /hyphen put\r\n\r\n% Set up ISO Latin 1 character encoding\r\n/starnetISO {\r\n        dup dup findfont dup length dict begin\r\n        { 1 index /FID ne { def }{ pop pop } ifelse\r\n        } forall\r\n        /Encoding EncodingVector def\r\n        currentdict end definefont\r\n} def\r\n/Times-Roman starnetISO def\r\n/Times-Italic starnetISO def\r\n/Times-Bold starnetISO def\r\n/Times-BoldItalic starnetISO def\r\n/Helvetica starnetISO def\r\n/Helvetica-Oblique starnetISO def\r\n/Helvetica-Bold starnetISO def\r\n/Helvetica-BoldOblique starnetISO def\r\n/Courier starnetISO def\r\n/Courier-Oblique starnetISO def\r\n/Courier-Bold starnetISO def\r\n/Courier-BoldOblique starnetISO def\r\ncleartomark\r\n} bind def\r\n\r\n%%BeginResource: procset graphviz 0 0\r\n/coord-font-family /Times-Roman def\r\n/default-font-family /Times-Roman def\r\n/coordfont coord-font-family findfont 8 scalefont def\r\n\r\n/InvScaleFactor 1.0 def\r\n/set_scale {\r\n       dup 1 exch div /InvScaleFactor exch def\r\n       scale\r\n} bind def\r\n\r\n% styles\r\n/solid { [] 0 setdash } bind def\r\n/dashed { [9 InvScaleFactor mul dup ] 0 setdash } bind def\r\n/dotted { [1 InvScaleFactor mul 6 InvScaleFactor mul] 0 setdash } bind def\r\n/invis {/fill {newpath} def /stroke {newpath} def /show {pop newpath} def} bind def\r\n/bold { 2 setlinewidth } bind def\r\n/filled { } bind def\r\n/unfilled { } bind def\r\n/rounded { } bind def\r\n/diagonals { } bind def\r\n/tapered { } bind def\r\n\r\n% hooks for setting color \r\n/nodecolor { sethsbcolor } bind def\r\n/edgecolor { sethsbcolor } bind def\r\n/graphcolor { sethsbcolor } bind def\r\n/nopcolor {pop pop pop} bind def\r\n\r\n/beginpage {\t% i j npages\r\n\t/npages exch def\r\n\t/j exch def\r\n\t/i exch def\r\n\t/str 10 string def\r\n\tnpages 1 gt {\r\n\t\tgsave\r\n\t\t\tcoordfont setfont\r\n\t\t\t0 0 moveto\r\n\t\t\t(\\() show i str cvs show (,) show j str cvs show (\\)) show\r\n\t\tgrestore\r\n\t} if\r\n} bind def\r\n\r\n/set_font {\r\n\tfindfont exch\r\n\tscalefont setfont\r\n} def\r\n\r\n% draw text fitted to its expected width\r\n/alignedtext {\t\t\t% width text\r\n\t/text exch def\r\n\t/width exch def\r\n\tgsave\r\n\t\twidth 0 gt {\r\n\t\t\t[] 0 setdash\r\n\t\t\ttext stringwidth pop width exch sub text length div 0 text ashow\r\n\t\t} if\r\n\tgrestore\r\n} def\r\n\r\n/boxprim {\t\t\t\t% xcorner ycorner xsize ysize\r\n\t\t4 2 roll\r\n\t\tmoveto\r\n\t\t2 copy\r\n\t\texch 0 rlineto\r\n\t\t0 exch rlineto\r\n\t\tpop neg 0 rlineto\r\n\t\tclosepath\r\n} bind def\r\n\r\n/ellipse_path {\r\n\t/ry exch def\r\n\t/rx exch def\r\n\t/y exch def\r\n\t/x exch def\r\n\tmatrix currentmatrix\r\n\tnewpath\r\n\tx y translate\r\n\trx ry scale\r\n\t0 0 1 0 360 arc\r\n\tsetmatrix\r\n} bind def\r\n\r\n/endpage { showpage } bind def\r\n/showpage { } def\r\n\r\n/layercolorseq\r\n\t[\t% layer color sequence - darkest to lightest\r\n\t\t[0 0 0]\r\n\t\t[.2 .8 .8]\r\n\t\t[.4 .8 .8]\r\n\t\t[.6 .8 .8]\r\n\t\t[.8 .8 .8]\r\n\t]\r\ndef\r\n\r\n/layerlen layercolorseq length def\r\n\r\n/setlayer {/maxlayer exch def /curlayer exch def\r\n\tlayercolorseq curlayer 1 sub layerlen mod get\r\n\taload pop sethsbcolor\r\n\t/nodecolor {nopcolor} def\r\n\t/edgecolor {nopcolor} def\r\n\t/graphcolor {nopcolor} def\r\n} bind def\r\n\r\n/onlayer { curlayer ne {invis} if } def\r\n\r\n/onlayers {\r\n\t/myupper exch def\r\n\t/mylower exch def\r\n\tcurlayer mylower lt\r\n\tcurlayer myupper gt\r\n\tor\r\n\t{invis} if\r\n} def\r\n\r\n/curlayer 0 def\r\n\r\n%%EndResource\r\n%%EndProlog\r\n%%BeginSetup\r\n14 default-font-family set_font\r\n1 setmiterlimit\r\n% /arrowlength 10 def\r\n% /arrowwidth 5 def\r\n\r\n% make sure pdfmark is harmless for PS-interpreters other than Distiller\r\n/pdfmark where {pop} {userdict /pdfmark /cleartomark load put} ifelse\r\n% make '<<' and '>>' safe on PS Level 1 devices\r\n/languagelevel where {pop languagelevel}{1} ifelse\r\n2 lt {\r\n    userdict (<<) cvn ([) cvn load put\r\n    userdict (>>) cvn ([) cvn load put\r\n} if\r\n\r\n%%EndSetup\r\nsetupLatin1\r\n%%Page: 1 1\r\n%%PageBoundingBox: 36 36 44 44\r\n%%PageOrientation: Portrait\r\n0 0 1 beginpage\r\ngsave\r\n36 36 8 8 boxprim clip newpath\r\n1 1 set_scale 0 rotate 40 40 translate\r\nendpage\r\nshowpage\r\ngrestore\r\n%%PageTrailer\r\n%%EndPage: 1\r\n%%Trailer\r\n%%Pages: 1\r\n%%BoundingBox: 36 36 44 44\r\nend\r\nrestore\r\n%%EOF\r\n"

Answer 1

First thing first, the error:

  File "C:\Users\Idriss\ANACON~1\envs\R-TENS~1\lib\site-packages\keras\utils\vis_utils.py", line 52, in model_to_dot
    _check_pydot()

If we check the file C:\Users\Idriss\ANACON~1\envs\R-TENS~1\lib\site-packages\keras\utils\vis_utils.py and search for the function _check_pydot():

def _check_pydot():
    try:
        # Attempt to create an image of a blank graph
        # to check the pydot/graphviz installation.
        pydot.Dot.create(pydot.Dot())
    except Exception:
        # pydot raises a generic Exception here,
        # so no specific class can be caught.
        raise ImportError('Failed to import pydot. You must install pydot'
                          ' and graphviz for `pydotprint` to work.')

This error message is lack of information since it catch ALL exception instead of specific execption and raise hard-coded error ImportError(Failed to import blah blah).

To ensure it import the relevant pydot, we should also check import part in that file (Rerun R and library(kerasR) to test):

import os
print("hole 0")
try:
    # pydot-ng is a fork of pydot that is better maintained.
    import pydot_ng as pydot
    print("hole 1")

except ImportError:
    # pydotplus is an improved version of pydot
    try:
        print("hole 1.2")
        import pydotplus as pydot
        print("hole 2")

    except ImportError:
        # Fall back on pydot if necessary.
        try:
            print("hole 3")
            import pydot
        except ImportError:
            print("hole 4")
            pydot = None
print("hole -1: " + str(locals())) #alternative way to debug
...

Tips: The safer way to debug is userepr instead of str.

If you manually run python in interactive mode and do pydot.Dot.create(pydot.Dot()), you will find out the exact exception (below is my Linux sample):

xb@dnxb:~/anaconda3/envs/r-tensorflow/bin$ ./python
Python 3.6.3 |Anaconda, Inc.| (default, Nov 20 2017, 20:41:42) 
[GCC 7.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pydot
>>> pydot.Dot()
<pydot.Dot object at 0x7f7d045cdb38>
>>> pydot.Dot
<class 'pydot.Dot'>
>>> pydot.Dot.create(pydot.Dot())
Traceback (most recent call last):
  File "/home/xiaobai/anaconda3/envs/r-tensorflow/lib/python3.6/site-packages/pydot.py", line 1878, in create
    stderr=subprocess.PIPE, stdout=subprocess.PIPE)
  File "/home/xiaobai/anaconda3/envs/r-tensorflow/lib/python3.6/subprocess.py", line 709, in __init__
    restore_signals, start_new_session)
  File "/home/xiaobai/anaconda3/envs/r-tensorflow/lib/python3.6/subprocess.py", line 1344, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'dot': 'dot'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/xiaobai/anaconda3/envs/r-tensorflow/lib/python3.6/site-packages/pydot.py", line 1883, in create
    prog=prog))
Exception: "dot" not found in path.
>>>

Let's print some variables used in the file /home/xiaobai/anaconda3/envs/r-tensorflow/lib/python3.6/site-packages/pydot.py before the line 1878:

try:
    print("env: " + str(env))
    print("cmdline: " + str(cmdline))
    print("tmp_dir: " + str(tmp_dir))
    p = subprocess.Popen(
        cmdline,
        env=env,
        cwd=tmp_dir,
        shell=False,
        stderr=subprocess.PIPE, stdout=subprocess.PIPE)
except OSError as e:
    if e.errno == os.errno.ENOENT:
        raise Exception(
            '"{prog}" not found in path.'.format(
                prog=prog))
    else:
        raise

Restart your python interpreter, rerun the import pydot and pydot.Dot.create(pydot.Dot()), it will shows:

xb@dnxb:~/anaconda3/envs/r-tensorflow/bin$ ./python
Python 3.6.3 |Anaconda, Inc.| (default, Nov 20 2017, 20:41:42) 
[GCC 7.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pydot
>>> pydot.Dot.create(pydot.Dot())
env: {'PATH': '/home/xiaobai/anaconda3/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:...<My other PATH>'}
cmdline: ['dot', '-Tps', '/tmp/tmpffo17gx5']
tmp_dir: /tmp
Traceback (most recent call last):
  File "/home/xiaobai/anaconda3/envs/r-tensorflow/lib/python3.6/site-packages/pydot.py", line 1881, in create
    stderr=subprocess.PIPE, stdout=subprocess.PIPE)

So basically what it does is run the command dot -Tps /tmp/tmpffo17gx5 but failed due to dot command not found.

In Linux, it will suggest run sudo apt install graphviz to install dot if I run the command manually in terminal:

xb@dnxb:~/anaconda3/envs/r-tensorflow/bin$ dot
The program 'dot' is currently not installed. You can install it by typing:
sudo apt install graphviz
xb@dnxb:~/anaconda3/envs/r-tensorflow/bin$ sudo apt install graphviz
...

Run dot -Tps /tmp/tmpffo17gx5 will success now:

xb@dnxb:~/anaconda3/envs/r-tensorflow/bin$ dot -Tps /tmp/tmpffo17gx5 
%!PS-Adobe-3.0
%%Creator: graphviz version 2.38.0 (20140413.2041)
%%Title: G
%%Pages: (atend)
%%BoundingBox: (atend)
%%EndComments
save
%%BeginProlog
/DotDict 200 dict def
DotDict begin
...

Restart R session, no more error:

> plot_model(model)
> 

This sudo apt install graphviz is for Linux, but I hope this answer help you debug the error in Windows.

Answer 2

You should install the Python libraries:

pip install pydot graphviz 

And also you need to download the graphviz binaries, and these are not installed with Python.

On Ubuntu you can install them with apt:

apt-get install -y graphviz libgraphviz-dev

On osX with brew:

brew install graphviz

For Windows and other operating systems, the instructions can be found at http://www.graphviz.org/