3

My query is on the below program with respect to symbols that are storing values and functions, when ran on http://pythontutor.com/.

memory model

My question is:

  1. How does python execution model look for above program on memory before start interpreting the python program? How do i visualise that memory layout? for example c executable has code/stack/heap/extra/data segments, just as an example, am not comparing

  2. Is 'const' a name of 32/64 bit memory area storing the value 2 with type assigned as integer?

  3. add()/sub()/other functions are shown in Objects column as per the diagram, So, How do i perceive functions being stored as Objects? How do i visualise it?

  4. As per the diagram, Is op a function pointer pointing to function sub()?

overexchange
  • 15,768
  • 30
  • 152
  • 347
  • 7
    Most of the details you're asking about, you should completely ignore for Python programming. Python isn't defined in terms of memory or pointers or low-level things like that. (The implementation needs to worry about that stuff, but you don't need to worry about the implementation.) – user2357112 Mar 24 '14 at 09:11
  • @user2357112 For your statement"you should completely ignore for Python programming", One cannot avoid such questions from people coming from languages of OOP paradigm like Java or imperative paradigm like C. As Noel is saying below that python is dynamically & strongly typed, so it make sense to understand type things applied on literals & functions(in runtime). After i start use python for 2-3 years and i do not know whether op is function pointer or just a reference variable of some class, I feel this is not proper approach in learning a language. – overexchange Mar 25 '14 at 06:55
  • I feel, declarative paradigm languages(like SQL) does not encourage user to learn inner details. – overexchange Mar 25 '14 at 07:03
  • 1
    When you program in C, you think in terms of memory and pointers, because those things are fundamental to C. Unless you really need to, you don't think about which register stores which variable or which layer of cache you're hitting. Those issues are handled for you by the implementation. Similarly, in Python, you think in terms of objects and their behavior. You don't need to worry about memory and pointers, because at Python level, those are implementation details. – user2357112 Mar 25 '14 at 07:48
  • 4
    I removed the Java tag, it has no relevance to Java. It doesn't really have relevance to C either, but I kept the tag because you're trying to compare Python's memory model to C's. Still, I gave your question a -1 because it is not useful to try to understand Python programs in terms of how they map to C's memory model. Instead, try to learn about how to write a Python interpreter in C (e.g. by getting the source code from python.org). – Guido van Rossum Mar 25 '14 at 15:09
  • @GuidovanRossum My intention was not to compare two memory layouts. Do you think it make sense to ask 2nd/3rd/4th question above? if yes, please help me understand or confirm any below answer(as correct one). – overexchange Mar 25 '14 at 16:39

3 Answers3

3

  1. Dictionaries on dictionaries. Dictionaries are the number 1 most important structure in Python.

  2. It is the key of an entry in the current scope's dictionary. The value is the object 2.

  3. It is not that functions are objects, but that some objects are functions. Or numbers. Or dictionaries.

  4. It is the key of an entry in the current scope's dictionary. The value is sub.

Ignacio Vazquez-Abrams
  • 776,304
  • 153
  • 1,341
  • 1,358
  • Ignacio, What is the meaning of these 2 statements: a) 'The value is the object 2' b) 'some objects are functions. Or numbers. Or dictionaries.'??---How do you define word 'object' here? Because, In OOP world, object has state & behaviour. – overexchange Mar 24 '14 at 10:42
  • Why do you believe that objects in Python don't have state and behavior? – Ignacio Vazquez-Abrams Mar 24 '14 at 10:44
  • I didn't say, objects in python don't have state & behaviour, But As per [link](http://docs.oracle.com/javase/tutorial/java/concepts/object.html) I understand that, Software objects stores its state in fields (variables in some programming languages) and exposes its behaviour through methods (functions in some programming languages). ------But in your update, In point 2, You say that an integer literal(2) is an object, In point 3, You say a function add()/sub() is an object. – overexchange Mar 24 '14 at 10:56
  • That's because they are. – Ignacio Vazquez-Abrams Mar 24 '14 at 10:56
  • As i said above, Object has state(fields/property) and behaviour(methods/functions). Now, integer literal(2), I can think of defining behaviour(like abs()/square()/..) on this literal, but how can i think of a state for this integer literal, when **2** itself is one state----For, function add()/sub() is itself a behaviour but on what field? So how can add() be called as an object? – overexchange Mar 24 '14 at 11:55
  • @Sham In the case of functions, e. g., you have an object which has certain fields, which contain e. g. the code in terms of bytecode, the number of parameters it has, etc. And it even has methods itself, for example `__get__()` which is called internally for binding a method to an object, or `__call__()` for the real function call, besides `__format__()`, `__repr__()` etc. – glglgl Mar 24 '14 at 12:30
  • @glglgl and what about integer literal 2, how is that an object? – overexchange Mar 24 '14 at 12:33
  • 1
    Same way everything else is an object. `>>> (2).__add__(3)` `5` – Ignacio Vazquez-Abrams Mar 24 '14 at 12:35
  • Somewhere deep inside, it as well has a place for storing the actual value. In the case of a `long()` in Py2, or any `int()` in Py3, it stores the data to hold the value 2 and as well the length needed for it. Besides, it has a number of methods. Have a look at the output of `dir(2)` to see it having a bunch of methods as well, such as for formatting, for arithmetics, etc. – glglgl Mar 24 '14 at 12:35
  • @Sham Have a look at the expression `a % b`. Simply spoken, it calls `a.__mod__(b)`. It depends on `a` what that does: if `a` is an int, the modulo is actually called; if it is a string, the formatting is invoked. – glglgl Mar 24 '14 at 12:37
  • @glglgl Please let me know, if my observation is correct wrt your answer below. – overexchange Mar 25 '14 at 01:50
  • @glglgl I had one query wrt your latest update, which i posted below. Another point is as you said that literals are autoboxed to corresponding class objects, Now, what about operators(like + * / %), Are these similar to primitive operators in Java or are they also considered as objects in python? I get this feel because every operand(including literals) is object in python, so operator cannot be primitive. like for example >>>2+3 – overexchange Mar 25 '14 at 08:36
  • @Sham As already was written, `2+3` gets translated to a call of `.__add__(3)` on the `2` object. It returns a reference to a `5` object. You should read the respective tutorials for Python. – glglgl Mar 25 '14 at 11:20
  • BTW, why do you comment on Ignacio's answer if you have a question to my answer? – glglgl Mar 25 '14 at 11:21
  • @glglgl Example 2+3 was from me, If it is mentioned by him as well that is coincidence, nothing more than that. – overexchange Mar 25 '14 at 11:40
  • @Sham Yes, but you put it into the comment field of *his* answer, although you want to address *me*. That's why I was confused... – glglgl Mar 25 '14 at 11:51
  • @glglgl Wrt to my Q3 I got this info from [link](http://www.linuxtopia.org/online_books/programming_books/python_programming/python_ch10s04.html), which says ' function is an object of the class function, a subclass of callable. The common feature that all callables share is that they have a very simple interface: they can be called.' ---- so that means op is a reference variable of class type function pointing to an instance of class function which is sub in the above diagram, Is my understanding correct? – overexchange Mar 26 '14 at 06:43
  • @Sham Nearly, yes. `op` is a reference variable *of no special type* pointing to an instance of class (or type) `function` defined as `sub`. And so is `sub` itself. (And you again used the wrong comment field :-) ) And, function does not *subclass* callable, but it implements the needed interface to be called. That is a tiny difference, covered by the so-called duck typing: you don't need to declare that you implement a given interface, as you do in Java, but you just do it. – glglgl Mar 26 '14 at 08:16
  • ignacio when you say dictionaries on dictionaries, do you mean each dictionary is a module? for example __main__ module is a dictionary? – overexchange Apr 15 '14 at 07:06
  • @Sham: Modules have a dictionary, which contains the global-level names. Scopes have a dictionary, which contains the local names. Python classes have a dictionary, which contains the methods, descriptors, and class variables. Python class instances have a dictionary, which contains the instance variables. – Ignacio Vazquez-Abrams Apr 15 '14 at 07:15
  • @IgnacioVazquez-Abrams As you said, any object is dictionary, which understand, but, do you want me to expect `isintance(operator, dict)` or `isintance(any_object, dict)` to be `True`? – overexchange Apr 30 '17 at 22:02
3

  1. In Python, don't worry so much about memory segments and what goes on behind the scenes. Rather, environments (scopes) are more important. The block-and-pointer diagram you included is a reasonable way to visualize the memory. The white portion shows what the global environment looks like. When the function is called, a new (blue) environment is created.

  2. const is a variable. Variables in Python are weakly dynamically typed, and can store anything. In fact, Python integers don't overflow, and can store numbers exceeding 264. In this case, const is a variable (with a confusing name) that contains the number 2.

  3. A function is an abstract notion of a callable blob of code. You can assign it to a variable just like any other value.

  4. You could consider it a function pointer if it makes you feel comfortable, but then you would be outing yourself as a C programmer. A Python programmer would just say that op has the function sub as a value.

Community
  • 1
  • 1
200_success
  • 7,286
  • 1
  • 43
  • 74
  • *"Variables in Python are weakly typed"* - I'd rephrase it. Variables aren't typed at all, but values are typed (strongly typed). – Kos Mar 24 '14 at 12:16
  • 2
    Python is *not* weakly-typed like Perl or C are. It's *dynamically*, but *strongly*-typed. – Max Noel Mar 24 '14 at 12:17
  • @Kos Do you meant, language agnostically, one should not say, variables are weakly/strongly typed, as it does not make sense? Do we need to say, values are weakly/strongly typed? – overexchange Apr 29 '17 at 22:59
2

Every C (compiled language) program has code/data/stack/extra/heap segments loaded in memory before execution. Does python interpreter create any memory layout for every python program before start interpreting the python program? If yes, How do i visualise that memory layout?

It has a kind of layout, but here the heap is the most important part, as every object is put to the heap. The code segment is merely the interpreter, the data segment is as well internal state of the interpreter, and the stack as well.

What is relevant for the Python program is only the heap. But the layout itself is like any other program.

Is const a name of 32/64 bit memory area storing the value 2 with type assigned as integer?

It is a name in the current working space, (here: in the module's namespace), which is essentially a dict which makes assignments between strings and arbitrary objects. In this case, it makes the string const refer to an integer object which holds the value 2. (This object can be created newly or re-used depending on the circumstances; this makes no difference, as it is immutable.)

add()/sub()/other functions are shown in Objects column as per the diagram, So, How do i perceive functions being stored as Objects? How do I visualise it?

As written in my comments to Ignacio's answer:

In the case of functions, you have an object which has certain fields, which contain e. g. the code in terms of bytecode, the number of parameters it has, etc. And it even has methods itself, for example __get__() which is called internally for binding a method to an object, or __call__() for the real function call, besides __format__(), __repr__() etc.

An integer object has, somewhere deep inside, a place for storing the actual value. In the case of a long() in Py2, or any int() in Py3, it stores the data to hold the value (e. g. 2) and as well the length needed for it. Besides, it has a number of methods. Have a look at the output of dir(2) to see it having a bunch of methods as well, such as for formatting, for arithmetics, etc.

As per the diagram, Is op a function pointer pointing to function sub()?

Kind of, yes.

There is a function object, which internally knows that its original name was sub. But this knowledge is only for displaying purposes.

In your case, it is referred from two names, op and sub. So referring to either of them has the same result.

Note that there are no "function pointers" per se, there are just references, or names, which refer to an object of any type. The type of an object is fixed, but not the "type of a reference" (as there is no such thing).

Community
  • 1
  • 1
glglgl
  • 89,107
  • 13
  • 149
  • 217
  • glglgl Wrt your second point, i see a point, i think, integer literal(2) is auto boxed to integer class(at runtime)and const **will be** of type integer class in runtime. In general, any literal "abc" , 3.2 will be autoboxed to corresponding class in runtime during the assignment time of 'const = 2', so const **will be** of type integer class in runtime. – overexchange Mar 25 '14 at 01:36
  • glglgl Wrt your third point, i dont agree with you, i feel a function(say sub() in above example) is also wrapped within a class and then pointed by a reference variable(say op in above example), where op will be of that class type in runtime. for 3rd point, we do this in java as well 'interface Sample { int sub(int x, int y); }' or 'class Sample { int sub(int x, int y){return x-y;} }' so i think const is a reference variable of type integer class above(in runtime) & op is a reference variable(not a function pointer) of type class that sub() is wrapped into. – overexchange Mar 25 '14 at 01:39
  • glglgl and when i say reference variable in python, i think the behaviour of this reference variable is same as in java(in runtime) except that in java type of this reference variable is known at compile time(which is X) and Y in runtime as per example 'X x = new Y();' where X & Y are classes Y is child of X and x is a reference variable – overexchange Mar 25 '14 at 01:46
  • 1. Maybe you call it autoboxing if you come from Java, I'd rather call it "automatic object creation (and interning)". But `const` just *refers* to an object of class `int`. 2. Yes, in general everything is a reference in Python, but I don't see a contradiction. (Besides, a function *may be* "wrapped in a class" (bound to it), but doesn't have to.) 3. This seems to be right. – glglgl Mar 25 '14 at 08:08
  • glglgl Wrt your statement in point 2 above: 'Besides, a function may be "wrapped in a class" (bound to it), but doesn't have to.'----- If you don't wrap into a class, then what is the type of reference variable op in runtime? – overexchange Mar 25 '14 at 08:23
  • @Sham `op` refers to the function exactly the same way as `sub` does. There is no type connected to the variable, only to the object, which is a function object. – glglgl Mar 25 '14 at 11:22
  • glglgl Oh now i understood what mistake am doing, i need to update here. After all this dicussion, Do you think you need to edit your answer so that it will help in future? – overexchange Mar 26 '14 at 08:23
  • @Sham I already did so yesterday where I thought it would help. – glglgl Mar 26 '14 at 08:24
  • glglgl op is a reference variable of **no special type**? i think python is strongly typed in runtime. op should be of some type and i feel it is of function type as sub is object of class function. – overexchange Mar 26 '14 at 08:26
  • @Sham The object it points to is strongly typed. But you can happily do in succession `op = sub; print(op); op = 4.0; print(op); op = []; print(op);` and so on. After each step, `op` happily refers to an object of different type. So references are not typed, but objects are. – glglgl Mar 26 '14 at 08:32
  • glglgl to check the type of op am trying to run this code, but i get indentation error **from operator import add, sub def a_plus_abs_b(a, b): if b < 0: op = sub print(type(op)) else: op = add print(type(op)) return op(a, b)** – overexchange Mar 26 '14 at 08:56
  • glglgl i test some sample code 'from operator import add, sub def abc(): return 0 op = abc' and then ran print(type(op)) it says [class 'function'] so variables are typed but weakly typed in python and objects are strongly typed, am i correct? – overexchange Mar 26 '14 at 09:04
  • glglgl i want to edit your answer, If you feel OK you can keep that as it is. – overexchange Mar 26 '14 at 09:06
  • glglgl i think out of 4 questions, last 3 are answered, first one is still a vague one will now – overexchange Mar 26 '14 at 09:34
  • glglgl If i say 'x = 1 y = x' i see that y is creating a new object which is different from x which is pointing to. How do you see this? – overexchange Mar 26 '14 at 11:37
  • @Sham Try it: `x = 1; y = x; print (id(x), id(y), x is y);` – glglgl Mar 26 '14 at 12:37
  • glglgl what i mean is, if i say 'x=1 then y=x then x = x + 1 then print y' value of y is 1 , why id(x) is same as id(y), no idea. – overexchange Mar 26 '14 at 13:41
  • @Sham No, then of course not. `y` still points to the `1` object, but `x` to a different one. – glglgl Mar 26 '14 at 14:14
  • glglgl I dont know, what u mean, when you say, 'No ofcourse not', another point is, if i say 'ref1 = class Die().. then ref2 = class Die() then print(id(ref1), id(ref2))' then ref2 =ref1 then print(id(ref1), id(ref2))' this is working as expected. So, that means, despite python is considering everything as an object internally, at presentation layer, it shows primitives as primitives and user defined class objects as objects. – overexchange Mar 27 '14 at 01:37
  • @Sham I don't think a comment ping pong is the right way to answer your questions, which are evolving and hopping from subject to subject and keep on enhancing. It would be better if you open another question. Feel free to reference this one and also to put link the new question here so that I get notified. – glglgl Mar 27 '14 at 07:52
  • To the question above: after `x = 1; y = x`, they point both to the same object, as you can verify with the `is` operator and by comparing their `id()`. After `x = x + 1` `x` now refers a different object, as it now represents even a different value. – glglgl Mar 27 '14 at 07:55
  • glglgl i raised the seperate query here [link](http://stackoverflow.com/questions/22705945/python-function-stored-as-object) – overexchange Mar 28 '14 at 06:45