1

How can I programmatically determine if the python code in my notebook is running under DSX?

I'd like to be able to do different things under a local Jupyter notebook vs. DSX.

Kendrick Wong
  • 394
  • 1
  • 12

2 Answers2

4

While the method presented in another answer (look for specific environment variables) works today, it may stop working in the future. This is not an official API that DSX exposes. It will obviously also not work if somebody decides to set these environment variables on their non-DSX system.

My take on this is that "No, there is no way to reliably determine whether the notebook is running on DSX".

In general, (in my opinion) notebooks are not really designed as artifacts that you can arbitrarily deploy anywhere; there will always need to be someone wearing the "application developer" hat and transform them - how to do that, you could put into a markdown cell inside the notebook.

Philipp Langer
  • 328
  • 1
  • 8
  • The `RUNTIME_ENV_*` variables are kind of an official API to distinguish between different contexts of Spark as a Service in Bluemix. But in general, I agree with your assessment. – Roland Weber Jan 25 '17 at 09:22
1

You can print your environment or look for some specific environment variable. I am sure you will find some differences.

For example:

import os
if os.environ.get('SERVICE_CALLER'):
  print ('In DSX')    
else:
  print ('Not in DSX')
Kendrick Wong
  • 394
  • 1
  • 12
Sumit Goyal
  • 575
  • 3
  • 16
  • Which environment variable in particular? I scanned through the ones currently defined in DSX, and while there are many that are different from a local Jupyter install, there is only one that seemed DSX specific: APP_ENV_CDSX_NOTEBOOKS_API. Is this a good one to rely on? – Kendrick Wong Jan 24 '17 at 21:42
  • For example: USER – Sumit Goyal Jan 24 '17 at 21:53
  • I could be using USER in my own local machine for a different purpose. It isn't very 'DSX' specific. – Kendrick Wong Jan 24 '17 at 21:56
  • Then you can go for "SERVICE_CALLER". – Sumit Goyal Jan 24 '17 at 21:59
  • I recommend RUNTIME_ENV_SPARK or RUNTIME_ENV_STOREFRONT. Unlike SERVICE_CALLER, those are part of an API to distinguish between different Spark as a Service environments. – Roland Weber Jan 25 '17 at 09:25