0

I have a Notebook which I call it as Template . I need to pass a variable called ID = 'EF12345'(string value) as an argument based on which my notebook will run and create a new notebook

These are few posts which I referred , I am missing something where I am not to figure out Argparse in Jupyter Notebook throws a TypeError

I cannot do pip install (blocked -- no internet)

Passing command line arguments to argv in jupyter/ipython notebook

Original Code cold_mill_id = 'E126644'

  strip_stats = hive_ctx.sql(query).toPandas()
  strip_stats[strip_stats.cold_mill_id == cold_mill_id]

  sns.set(font_scale=1.2, font='DejaVu Sans')
  fig, ax = plt.subplots(1, 1, figsize=(15, 6))
  xp = df.position
  yp = df.fm_entry_temperature
  p = ax.plot(xp, yp, linestyle='-', label='fm_entry_temperature')
  yp = df.fm_exit_temperature
  p = ax.plot(xp, yp, linestyle='-', label='fm_exit_temperature')
  p = ax.set_title('finishing mill temperature profiles for cold_mill_id=' + 
  cold_mill_id)
  p = ax.set_xlabel('position')
  p = ax.set_ylabel('temperature    (F)')
  p = ax.set_ylim(1650, 2100)
  p = plt.legend()

I added argparse to get value for my cold mill id 

   parser.add_argument('--id')
   args = parser.parse_args([])
   args, cold_mill_id = parser.parse_known_args()

ID which I am passing is not working , I am not able to pass ID - which is string and then my code sucks This is how I need

nbconvert --to notebook --execute Template.ipynb 'E12664'

ID which I am passing from Parser is not working , neither can I debug from Command line what is happening fails at certain point in execution when TYPE of ID mismatches with what is expected

Added this piece of code after argparse to check what is getting inside but won't print it in command line 

import sys
print(sys.argv)

I am doing something horribly wrong I am clueless What I need is I need to somehow pass this ID to cold_mill_id in jupyter notebook and save those results for new ID as one more notebook

nithin
  • 753
  • 3
  • 7
  • 21
  • As your links show, it is hard, if not impossible to provide command line values to a notebook. The ones it sees via `sys.argv` will include the ones provided to the `jupyter` server. I'm not familiar with this use of `nbconvert`, but it appears to run the notebook 'blind'. If your notebook can't run as intended with a regular server, there's no way it will run with `nbconvert`. Does `nbconvert` complain about unrecognized command line values like the ID string? – hpaulj Jul 17 '19 at 16:51
  • What I am saying is My Notebook runs fine with cold_mill_id I provide when I pass the argument from command line it fails and I am unable to debug what the arg parse is doing when I pass the command line argument , otherwise my code from notebook works fine. I have 50 id's and I need to create 50 notebooks coz I need to send them to concerned person .Problem is that I need to do it manually . So I am thinking of passing command line parameters . My first line of notebook takes that id and then process. – nithin Jul 17 '19 at 16:56
  • Your description is still confusing. I can't tell exactly what code (and invocation) runs and what doesn't. Looking at the `jupyter nbconvert -h` help, I suspect `nbconvert` views E12664 as just another notebook that it's supposed to execute. – hpaulj Jul 17 '19 at 17:09
  • Have you explore the logging parameters for `nbconvert`? – hpaulj Jul 17 '19 at 22:20

1 Answers1

1

When I try to run a commandline like yours I get:

1509:~$ jupyter nbconvert --to notebook --execute text.ipynb IPYQX
[NbConvertApp] WARNING | pattern 'IPYQX' matched no files
[NbConvertApp] Converting notebook text.ipynb to notebook
[NbConvertApp] Executing notebook with kernel: python3
[NbConvertApp] Writing 921 bytes to text.nbconvert.ipynb

In other words, it view 'IPYQX' as another file that it is supposed to convert. It is not an argument to be passed on to a notebook

With a notebook cell like:

import sys
with open('test.txt', 'w') as f:
    f.write(str(sys.argv))
    f.write('\n')

I get a file like:

1510:~$ cat test.txt
['/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py', '-f', '/tmp/tmpis4wjqdn.json']

This is the sys.argv that was passed to the launcher script to run the notebook. It looks nothing like the command line given to nbconvert.

As has been observed in other SO, we cannot pass command line values to a notebook. Can't you run this code as a python script instead?

hpaulj
  • 221,503
  • 14
  • 230
  • 353