Access GlobalParameters in Azure ML Python script

Question

How can one access the global parameters ("GlobalParameters") sent from a web service in a Python script on Azure ML?

I tried:

if 'GlobalParameters' in globals():
    myparam = GlobalParameters['myparam']

but with no success.

EDIT: Example

In my case, I'm sending a sound file over the web service (as a list of samples). I would also like to send a sample rate and the number of bits per sample. I've successfully configured the web service (I think) to take these parameters, so the GlobalParameters now look like:

"GlobalParameters": {
     "sampleRate": "44100",
     "bitsPerSample": "16",
}

However, I cannot access these variables from the Python script, neither as GlobalParameters["sampleRate"] nor as sampleRate. Is it possible? Where are they stored?

What do you want to access? What do you look for? Can you tell me some details? — Peter Pan, Aug 07 '15 at 09:30
@PeterPan-MSFT I've edited the question to provide an example — gozzilli, Aug 07 '15 at 17:05

score 1 · Answer 1 · answered Aug 13 '15 at 06:39

based on our understanding of your question, here may has a miss conception that Azure ML parameters are not “Global Parameters”, as a matter of fact they are just parameter substitution tied to a particular module. So in affect there are no global parameters that are accessible throughout the experiment you have mentioned. Such being the case, we think the experiment below accomplishes what you are asking for:

Please add an “Enter Data” module to the experiment and add Data in csv format. Then for the Data click the parameter to create a web service parameter. Add in the CSV data which will be substituted from data passed by the client application. I.e.
Please add an “Execute Python” module and hook up the “Enter Data” output to the “Execute Python” input1. Add the python code to take the dataframe1 and add it to a python list. Once you have it in a list you can use it anywhere in your python code.
Python code snippet

def azureml_main(dataframe1 = None, dataframe2 = None):

import pandas as pd
global_list = []

for g in dataframe1["Col3"]:
    global_list.append(g)

df_global = pd.DataFrame(global_list)        
print('Input pandas.DataFrame:\r\n\r\n{0}'.format(df_global))        
return [df_global]

Once you publish your experiment, you can add in new values in the “Data”: “”, section below with the new values that you was substituted for the “Enter Data” values in the experiment.

data =  {

        "Inputs": {

                "input1":
                {
                    "ColumnNames": ["Col1", "Col2", "Col3"],
                    "Values": [ [ "0", "value", "0" ], [ "0", "value", "0" ], ]
                },        },
            "GlobalParameters": {
        "Data": "1,sampleRate,44500\\n2,bitsPerSample,20",
}
}

Please feel free to let us know if this makes sense.

In your example, the "Data" entry in `GlobalParameters` is redundant, because the python script already receives `input1` as soon as `input1` is connected through the "Web Service Input" and the "Enter Data" boxes. Moreover, we are already using that input to send a sound file as a list of samples (of unknown length), so having another column of length 2 in the same `pd.DataFrame` wouldn't work. — gozzilli, Aug 13 '15 at 09:58
A bit confused, are you trying to figure out http://stackoverflow.com/questions/2835559/parsing-values-from-a-json-file-in-python? — , Aug 14 '15 at 00:54
Apologize if I still have not fully understood the issue, actually the "Execute Python Script" has two inputs, you can use other input for a second "Enter Data" module. What you need to do is to modify the python script appropriately, here is an article that points out the details on how to use more than one input: https://azure.microsoft.com/en-us/documentation/articles/machine-learning-execute-python-scripts/#basic-usage-scenarios-in-machine-learning-for-python-scripts, please feel free to let me know if it helps. — Ming Xu - MSFT, Aug 18 '15 at 03:25

score 0 · Answer 2 · answered Jan 29 '19 at 13:31

The GlobalParameters parameter can not be used in a Python script. It is used to override certain parameters in other modules.

If you, for example, take the 'Split Data' module, you'll find an option to turn a parameter into a web service parameter:

Once you click that, a new section appears titled "Web Service Parameters". There you can change the default parameter name to one of your choosing.

If you deploy your project as a web service, you can override that parameter by putting it in the GlobalParameters parameter:

"GlobalParameters": {
     "myFraction": 0.7
}

I hope that clears things up a bit.

score 0 · Answer 3 · answered Jan 29 '19 at 14:28

Although it is not possible to use GlobalParameters in the Python script (see my previous answer), you can however hack/abuse the second input of the Python script to pass in other parameters. In my example I call them metadata parameters.

To start, I added:

a Web service input module with name: "realdata" (for your real data off course)
a Web service input module with name: "metadata" (we will abuse this one to pass parameters to our Python).
a Web service output module with name: "computedMetadata"

Connect the modules as follows:

As you can see, I also added a real data set (Restaurant ratings) as wel as a dummy metadata csv (the Enter Data Manually) module.

In this manual data you will have to predefine your metadata parameters as if they were a csv with a header and a only a single row to hold the data:

In the example both sampleRate and bitsPerSample are set to 0.

My Python scripts then takes in that fake csv as metadata, does some dummy calculation with it and returns it as column name:

import pandas as pd

def azureml_main(realdata = None, metadata = None):

    theSum = metadata["sampleRate"][0] + metadata["bitsPerSample"][0]

    outputString = "The sum of the sampleRate and the bitsPerSecond is " + str(theSum)

    print(outputString)

    return pd.DataFrame([outputString])

I then published this as a web service and called it using Node.js like this:

httpreq.post('https://ussouthcentral.services.azureml.net/workspaces/xxx/services/xxx', {
    headers: {
        Authorization: 'Bearer xxx'
    },
    json: {
    "Inputs": {
        "realdata": {
            "ColumnNames": [
                "userID",
                "placeID",
                "rating"
            ],
            "Values": [
                [
                    "100",
                    "101",
                    "102"
                ],
                [
                    "200",
                    "201",
                    "202"
                ]
            ]
        },
        "metadata": {
            "ColumnNames": [
                "sampleRate",
                "bitsPerSample"
            ],
            "Values": [
                [
                    44100,
                    16
                ]
            ]
        }
    },
    "GlobalParameters": {}
}

}, (err, res) => {
    if(err) return console.log(err);
    console.log(JSON.parse(res.body));
});

The output was as expected:

{ Results:
   { computedMetadata:
      { type: 'table',
        value:
         { ColumnNames: [ '0' ],
           ColumnTypes: [ 'String' ],
           Values:
            [ [ 'The sum of the sampleRate and the bitsPerSecond is 44116' ] ] } } } }

Good luck!

Access GlobalParameters in Azure ML Python script

EDIT: Example

3 Answers3