1

I have a need to create global deques in a main program and use them in functions called from that program. In my example, I create a deque, "questions", and add two lists to the deque. I print the deque from the main program to show that it was successfully created and populated.

In the function I attempt to pop the items off the deque "questions" and into a variable, "question" but I get a "NameError: global name 'questions' is not defined"

How can I access the deque in the function? The commented out "global questions" and "from collections import deque" have also been tried with the same result and all the permutations of including or excluding those lines.

The function is in a module, and the file structure is as follows:

├── problemmain.py
├── modules
│   ├── problemchild.pyc
│   ├── problemchild.py
│   ├── __init__.pyc
│   └── __init__.py

problemmain.py:

#!/usr/bin/python

# Send them to the id connector
from modules.problemchild import problem_child
from collections import deque
#idconnector_send(questions,\
#                 rmdconfig.idconnector['send_dst_hostname'],\
#                 rmdconfig.idconnector['send_dst_port'])

questions=deque()

questions.append(['item1','item2','item3',['inneritem1','inneritem2','inneritem3','inneritem4','inneritem5','inneritem6']])
questions.append(['item1','item2','item3',['inneritem1','inneritem2','inneritem3','inneritem4','inneritem5','inneritem6']])

print questions

problem_child()

#idconnector_receive(rmdconfig.idconnector['receive_dst_hostname']\
#                    rmdconfig.idconnector['receive_dst_port'])

modules/__init__.py:

#!/usr/bin/python
#__all__ = ["readconf",]

modules/problemchild.py:

#!/usr/bin/python

def problem_child():

    """Determine what's wrong with modules scope and deque
    """
#    from collections import deque
#    global questions
    while True:
            #question = questions.pop()
            question = questions.pop()
            print "Question is: "
            print question
            print "----------------------------------------------------"

Output:

./problemmain.py


deque([['item1', 'item2', 'item3', ['inneritem1', 'inneritem2', 'inneritem3', 'inneritem4', 'inneritem5', 'inneritem6']], ['item1', 'item2', 'item3', ['inneritem1', 'inneritem2', 'inneritem3', 'inneritem4', 'inneritem5', 'inneritem6']]])
Traceback (most recent call last):
  File "./problemmain.py", line 17, in <module>
    problem_child()
  File "/home/justin.h.haynes/rmd/modules/problemchild.py", line 11, in problem_child
    question = questions.pop()
NameError: global name 'questions' is not defined

Trying Aaron's suggestion, and replacing "questions_queue" I get the following:

$ ./problemmain.py
Traceback (most recent call last):
  File "./problemmain.py", line 13, in <module>
    modules.questions_queue=deque()
NameError: name 'modules' is not defined
Justin Haynes
  • 79
  • 1
  • 7
  • Why not make `questions` an argument to the function? Global variables are generally not a good idea. – jonrsharpe Mar 20 '14 at 17:46
  • Passing the deque to this function might solve this particular problem, and I did try that (and ran into another problem beyond the scope of this problem). However, architecturally I need to have the deque be global unless there is another way to accomplish my intent: I need to have a queue which can be appended to and popped from while other threads or processes are using that data structure as well. – Justin Haynes Mar 20 '14 at 17:48
  • Justin, how does my suggestion work for you? – Russia Must Remove Putin Mar 20 '14 at 18:03
  • @AaronHall, see my comment to your answer below. I'm not sure where you would want to add modules.questions to get it into the namespace.. – Justin Haynes Mar 20 '14 at 19:31
  • @jonrsharpe I'll try passing it again. I'm reading " For example, passing an object is cheap since only a pointer is passed by the implementation; and if a function modifies an object passed as an argument, the caller will see the change — this eliminates the need for two different argument passing mechanisms as in Pascal." (http://docs.python.org/2/tutorial/classes.html). This was not my understanding before, or maybe I just didn't believe it. I'll play with that now. – Justin Haynes Mar 20 '14 at 21:39
  • @jonrsharpe, that worked. Would you like to propose that as an answer so that I can vote on it? I am not far enough along in the code I need to write to try this with threads just yet, but I would expect I can be appending things to this queue in one thread from a function, while I am poplefting things off the other side of it in another thread. Either that or I will cause some serious problems. That will be another question at some point soon probably. – Justin Haynes Mar 20 '14 at 21:52
  • I tried answering my own question with the example I just created to demonstrate, but alas with less than 10 reputation I cannot do that until 8 hours have elapsed. :-). Go ahead if you like @jonrsharpe, and if not I'll paste it in later. – Justin Haynes Mar 20 '14 at 22:07

3 Answers3

2

@jonrsharpe has answered the question, but I thought it would be constructive to go into a bit more on the thought process for why this was the answer, and I did not think it appropriate to edit the question as a user who may try to do the same thing would not find the original example.

The spirit of this question is really "how do I access a deque globally", but here I will explain what was wrong with my assumptions when I asked my more specific question above. The question was still valid - just based on wrong assumptions and understanding.

Functions should not be modifying global variables (or global anything really). There aren't very many good reasons to be doing this. With variables, lists and "most other types" mentioned in section 9.1 here (http://docs.python.org/2/tutorial/classes.html) this is true.

However, this is not true of all objects. when one passes a deque, only the reference is passed (someone may correct me if I am oversimplifying). The way I understand this is that the reference is simply another pointerlike construct in a namespace refering to the same object. In the case of passing a deque, there is a reference it it now inside the function and in the global space.

So I can create a deque, add some elements, pass it to a function, add elements to the deque I passed within the function, and then outside the function look at the contents and prove that I was operating on the same object all along:

#!/usr/bin/python


print "-------- strings --------"
a = "this is something"

print a

def scopetest(b):
    print b + ":was passed to me"
    print "I will change it now"
    b="This is something else"
    print "now it is:"
    print b


scopetest(a)

print a


print "------- deque ----------"

from collections import deque

my_deque = deque()

print "I just made a deque called my_deque"
print "it has:"
print my_deque

my_deque.append("this is the first thing in my deque")
my_deque.append("this is the second thing in my deque")

print "I just appended a couple of lines"
print "it has now:"
print my_deque


print "now I will define a new function called scopetest_deque()"

def scopetest_deque(a):
    a.append("this is the third thing in my deque")
    a.append("this is the fourth thing in my deque")
    print "I just appended a couple of lines"
    print "it has now:"
    print a


print "Calling it now:"
scopetest_deque(my_deque)

print "it has now:"
print my_deque

And in the output we see the strings are local to the function while in the deque we simply have a different name for the same instance of deque which we have modified in both the main part of the program and from within the function:

./test.py

-------- strings --------
this is something
this is something:was passed to me
I will change it now
now it is:
This is something else
this is something
------- deque ----------
I just made a deque called my_deque
it has:
deque([])
I just appended a couple of lines
it has now:
deque(['this is the first thing in my deque', 'this is the second thing in my deque'])
now I will define a new function called scopetest_deque()
Calling it now:
I just appended a couple of lines
it has now:
deque(['this is the first thing in my deque', 'this is the second thing in my deque', 'this is the third thing in my deque', 'this is the fourth thing in my deque'])
it has now:
deque(['this is the first thing in my deque', 'this is the second thing in my deque', 'this is the third thing in my deque', 'this is the fourth thing in my deque'])
Justin Haynes
  • 79
  • 1
  • 7
0

Why not stick it in a common namespace?

modules.questions_queue=deque()

And then access it like this:

modules.questions_queue.append(['item1'...

Edit

You'll need to either

import modules

or

import modules.something
Russia Must Remove Putin
  • 374,368
  • 89
  • 403
  • 331
  • One way I could create modules.questions might be to create a questions.py in the modules directory, and then create a deque, in questions.py, so that perhaps we would have modules.questions.questions. Is that what you had in mind? I could "import modules.problemchild" instead of "from modules.problemchild import problem_child". then I can do this: "modules.problemchild.questions=deque()". Or were you suggesting I should create classes? At any rate, I'm unclear on how this will make the data structure more global. I dont' understand why I can't access questions from the function. – Justin Haynes Mar 20 '14 at 19:30
  • @JustinHaynes I considered you might want to do that, so to avoid namespace collisions, I suggest using questions_queue (as I now give above) – Russia Must Remove Putin Mar 20 '14 at 19:33
  • $ ./problemmain.py Traceback (most recent call last): File "./problemmain.py", line 13, in modules.questions_queue=deque() NameError: name 'modules' is not defined – Justin Haynes Mar 20 '14 at 19:45
  • @JustinHaynes You do have to import it somewhere, whereever you want to use it, see my addendum. – Russia Must Remove Putin Mar 20 '14 at 19:46
  • I'll re-read the bit on python namespaces and modules again and come back to this. Thanks. I assume I am missing something fundamental. – Justin Haynes Mar 20 '14 at 20:07
  • Maybe some big guns could help here, maybe SuperCoder @MartijnPieters could help? – Russia Must Remove Putin Mar 20 '14 at 20:10
  • I got what you suggested basically working. I think I am conflating scope, modules and some other concepts. This example is an attempt to abstract the problem I'm seeing in my code and it does that. From a design perspective I like including functions from modules. I'm not making much use of object oriented design at all yet (because I do not yet understand it well) and I am instead carefully writing mostly procedural code. The spirit of my question is really how can I share a deque such that I can operate on it from inside many functions. – Justin Haynes Mar 20 '14 at 21:37
  • Review the Python scoping rules, nice post on it here: http://stackoverflow.com/questions/291978/short-description-of-python-scoping-rules – Russia Must Remove Putin Mar 20 '14 at 21:41
0

I would suggest you make questions an explicit argument:

def problem_child(questions):

and pass it from the caller:

print questions
problem_child(questions)

It is generally a bad sign that you feel the need to use global variables.

jonrsharpe
  • 115,751
  • 26
  • 228
  • 437