0

How do I construct a function foo() such that

pycode = '''
from matplotlib import pyplot as plt
plt.plot([1, 2, 3])
plt.show()
'''

out = foo(pycode)

where out is the rich text HTML output that ipython or jupyter notebook would output

Is there some inbuilt function in ipython that can do this? I would guess there is but I couldn't find it anywhere googling.

Edit: I want the function to exist in my python script, not when I am running ipython shell

Edit: What I want to do is to essentially recreate something like a jupyter notebook cell where I give it the code and It return the rich outputs visually (perhaps in HTML form) like a jupyter notebook cell would.

Julian Chu
  • 1,790
  • 2
  • 7
  • 12

1 Answers1

2

I think you are looking for Python's exec() method?

In a JupyterLab where matplotlib is installed, you can run this in a cell and see the output below it:

pycode = '''
from matplotlib import pyplot as plt
plt.plot([1, 2, 3])
plt.show()
'''
out = exec(pycode)

Keep in mind that use of exec(), and the related eval(), are generally discouraged. Their use can potentially enable others, or even you unwittingly, execute code you didn't mean to run that can cause severe problems. Often, their use is unnecessary if you use proper methods, see here. (Seems you can even do it using WebAssembly-based Python now, see PyScript: is it possible to execute an uploaded Python script?.)

UPDATE IN RESPONSE TO FIRST COMMENT:

As written, out wasn't collecting anything despite the code working to make the plot because exec(pycode) happened to be on last line. There is a way to collect the RichOutput when running it using IPython's capture utilities. And because it works via imports, it should work in Python, if you have the Jupyter ecosystem installed there as well. In other words although I am demonstrate it in a notebook, you can make a function to use return captured.outputs[0] and it will return the RichOutput from the plotting.

See the underlying reasoning more fleshed out in this notebook here.

Note because I deal with RichOutput and making images, you'll also want to see the references therein. There's a lot more in there that I'm not touching on.

Making a simple function & not using exec()

See programmatically making a .py file in the example notebook here. Normally, you could such a file in your text editor by writing saving the text by hand, such as the following saved as plot_func.py:

def my_plot_func():
    from matplotlib import pyplot as plt
    plt.plot([1, 2, 3])
    plt.savefig('my_plot_via_func.png')

Then that is imported and run. Part of the process saves an image of the file. In the notebook, that image file my_plot_via_func.png is displayed.

This is the simplistic version. The importing and executing the function will work in pure Python run in a terminal, or elsewhere, in the same way, up to displaying the image file. If you returned the plot as a plot object (figure) you'd need some way to display. That works in Jupyter, but not in pure Python.

I added this because I am still unclear what you want to do. I have a way more advanced version of that where you supply a specially organized Pandas dataframe and get a specialized plot back. To try it go here and press launch binder and work through the example notebook that comes up when the sessions starts. Perhaps that will help you better formulate asking about what you seek. Especially since it takes something much like this plotting script with everything hardcoded and generalizes it to make a version where it is a command line script or a function you can call. You'll see the infrastructure I was saying is necessary in my comment below. The command line script runs with pure python, too. You'll see I use %run generally in the notebook because that is more full-featured; however, in the notebook you can generally substitute !python for %run if you can sacrifice better output handling and don't need to run the script in the notebook namespace like %run -i allows.

For an example of using matplotlib plotting to make code that returns a specialized plot by another person, see here.

Generating HTML

The OP wanted HTML, and I've added a section where HTML is generated to the bottom of the main example notebook here, that was in response to the first comment. In fact, this post 'Imbed [sic] matplotlib figure into iPython HTML' may have covered much of what the OP wanted?

Wayne
  • 6,607
  • 8
  • 36
  • 93
  • But exec doesn't return the rich text output that ipython has. For example, in python, calling matplotlib will create images of the plot and calling pandas frames will return interactive tables – Julian Chu Dec 06 '22 at 07:43
  • Ideally, you'd provide code or a tdescription or text code of what you are actually looking to see your included code block make. I provided a way in a Jupyter notebook to get the "output ...jupyter notebook would output". I cannot tell if your 'Edit:' line was after I already replied. Anyhoo...`exec()` was definitely what you are seeking with the post as you originally posted it. I think though what you are really seeking is to make a script that is going to do plotting right? Your post came across that you wanted your 'pycode' doc string as it was and just `foo()` to work. ... – Wayne Dec 06 '22 at 15:55
  • ... If you want Python to do plotting... I generally make a script that will return the rich output when I use the core function from it in a Jupyter notebook. And then when I run the script on the command line, it makes an image file of the plot. There's some infrastucture necessary to do this that is separate from the underlying plotting ability. It can be something you can build towards. But what specifically do you need now? I don't work in IPython directly enough to know what it makes when you run matplotlib there. I thought to get an image, I needed to save the figure? ... – Wayne Dec 06 '22 at 15:55
  • And out of the blue you bring up Pandas. Unless I'm missing something that should be a separate post. But I'll try to address some of that here because maybe I am really missing something. You write, "For example, in python,... calling pandas frames will return interactive tables". With basic Python on a computer & the Pandas package also installed, when Python is run in a terminal calling a Pandas dataframes returns nothing interactive. They are text based versions of the dataframes rendered in text. The rendering in vanilla Jupyter is much nicer but still not interactive. – Wayne Dec 06 '22 at 15:59
  • I added a notebook that shows capturing the RichOutput. This wouldn't be the way you'd write anything other than a quick hack. See my comments where I think you are just asking to write pure Python that handles returning what you want for a plot. – Wayne Dec 06 '22 at 18:31
  • I added to the bottom of the main notebook how to get out HTML code from all this. I updated the answer to point to a similar question for that part, too. It's all expanded on with examples in the HTML section of [the main notebook](https://nbviewer.org/gist/fomightez/405e049fa082cca9b7a0ef0bee904b4f) – Wayne Dec 07 '22 at 21:29
  • Note the part of my comment above that says, "... If you want Python to do plotting... I generally make a script that will return the rich output when I use the core function from it in a Jupyter notebook" would be better written as, "... If you want Python to do plotting... I generally make a script that will return the matplotlib figure object that can be viewed as the rich output when I use the core function from it in a Jupyter notebook." – Wayne Dec 07 '22 at 21:39
  • I expanded on the making of the HTML via nbconvert by adding an example to the [main notebook](https://nbviewer.org/gist/fomightez/405e049fa082cca9b7a0ef0bee904b4f) where a notebook with a single cell, very similar to the code block I posted at the top of this answer is used. I then use Python to run that simple notebook and `jupyter nbconvert` is used to convert that HTML, leaving out the input source code cells as well. – Wayne Dec 08 '22 at 21:26
  • Related, may be [jupyter_capture_output](https://github.com/Octoframes/jupyter_capture_output), see [description here](https://discourse.jupyter.org/t/is-there-a-way-to-save-the-output-of-a-cell/2489/12?u=fomightez). – Wayne Dec 12 '22 at 19:51
  • Thanks for your reply, that is really helpful. What I want to do is to essentially recreate something like a jupyter notebook cell where I give it the code and It return the rich outputs visually (perhaps in HTML form) like a jupyter notebook cell would. Any Idea how it could be done? – Julian Chu Feb 01 '23 at 20:02
  • See this post https://stackoverflow.com/questions/75315748/how-do-i-build-something-similar-to-a-jupyter-notebook-code-cell-with-rich-outpu for more info – Julian Chu Feb 01 '23 at 20:19
  • Yes, it seems like you want what used to be Papermill's record functionality. It was split out to its own development project called 'scrapbook'. (See [here](https://github.com/nteract/papermill/issues/467). [scrapbook](https://nteract-scrapbook.readthedocs.io/en/latest/) - 'scrapbook is a library for recording a notebook’s data values and generated visual content as “scraps”.' Emphasis on "generated visual content". I note though since it used to be in Papermill that it may still need it, perhaps? I haven't used it myself. See also... – Wayne Feb 01 '23 at 20:23
  • 1
    [here](https://discourse.jupyter.org/t/is-there-a-way-to-save-the-output-of-a-cell/2489/8?u=fomightez). But you may just want the HTML? So you could run the code in a notebook using nbformat to inject it and then use jupyter nbconvert with the `--no-input`. Like I suggested for here but you could have one notebook and not even put the command in it unless you wanted. Also see [here](https://discourse.jupyter.org/t/executenb-monitor-cell-execution-stdout-stream-asynchronously/3069?u=fomightez). – Wayne Feb 01 '23 at 20:29
  • I think the nbformat & nbconvert solution works! thanks – Julian Chu Feb 01 '23 at 22:15
  • Although I also wonder if there is a way to get the partial notebook cell outputs while running it instead of having to wait until the whole notebook is executed to get the result of the whole notebook. For example, if I want to display a progress bar, the current method will simply run the whole notebook without displaying anything and then return the whole output – Julian Chu Feb 01 '23 at 22:17
  • How are you running the notebook? I believe Papermill allows that. See what I say about [this here](https://discourse.jupyter.org/t/auto-save-periodically-during-nbconvert/17361/2?u=fomightez). The idea being you can then query the notebook being produced in real-time every so often using nbformat and give accurate percentage of the cells run already back to your progress bar. You could use Papermill in the middle then before you use nbconvert and after nbformat. – Wayne Feb 01 '23 at 22:20