7

I like to figure out the myth behind Python's namespace packages by setuptools, and here is what I did test.

  • Make a virtual environment by virtualenv.
  • Find a namespaced package on PyPI.
  • Install that package by pip install.
  • Check the installed file hierarchy.

The package I played with is zope.interface and it worked well with the following file hierarchy on my virtualenv:

~virenv/.../site-packages/zope.interface-3.8.0-py2.6-nspkg.pth
                         /zope.interface-3.8.0-py2.6.egg-info/
                         /zope/
                              /interface/
                                        /...

Everything looked fine and I love the way zope.interface got installed as a real namespaced package (under folder zope).

Then, I did another test and that's the question I would like to ask for your help. I downloaded the tared zope.interface source file. I liked to play it manually again

  • Make a virtual environment by virtualenv.
  • Untar the zope.interface into somewhere.
  • Install the package by python setup.py install.
  • Go check what happened in site-packages.

The site-packages looks like this:

~virenv/../site-packages/zope.interface-...egg/
                                              /zope/
                                                   /__init__.py
                                                   /interface/
                                              /EGG-INFO/

Q. How come I can't get the exactly result to pip install by manually python setup.py install?

Drake Guan
  • 14,514
  • 15
  • 67
  • 94

2 Answers2

15

pip uses setup.py internally. It just passes additional option to it. To reproduce what pip is doing, execute 

python setup.py install --single-version-externally-managed

You can also run pip -vv to see exactly which commands are run.

vartec
  • 131,205
  • 36
  • 218
  • 244
  • In order to make this work, it seems I need to provide *--root* or *--record* then~ It looks to me that I have to prepare something like *~zope.interface-3.8.0-py2.6.egg-info/installed-files.txt* for *--record*. Am I right? – Drake Guan Apr 17 '12 at 04:05
  • 1
    exact command it runs: `python -c "import setuptools; __file__='/path/to/setup.py'; execfile('/path/to/setup.py')" install --single-version-externally-managed --record /tmp/pip-NM9sBK-record/install-record.txt --install-headers /site-packages/path` – vartec Apr 17 '12 at 10:12
-8

Q. How come I can't get the exactly result to pip install by manually python setup.py install?

Because pip and setup.py are two different pieces of software.

pip is not advertised as providing identical behaviour to setup.py, nor vice versa.

If you want the behaviour of pip, use pip; if you want the behaviour of setup.py, use setup.py.

Marcin
  • 48,559
  • 18
  • 128
  • 201
  • 2
    Wow! Really? Never thought about this! – Drake Guan Apr 16 '12 at 10:31
  • still looking for possible description or even solution to this cause I do need that clean file hierarchy by *pip* but with *setup.py* way. – Drake Guan Apr 16 '12 at 10:44
  • @DrakeGuan: You should change the title of your question to reflect your last comment. What's the difference between pip and setup.py isn't much of a question, it seems like you're actually interested in something a bit more specific. – Rik Poggi Apr 16 '12 at 10:48
  • @DrakeGuan If you want the behaviour of `pip`, use `pip`. – Marcin Apr 16 '12 at 11:03
  • That's somewhat misleading, though - `pip` does use `setup.py`. I'm sure if you get the right options, or possibly use a wrapper script, it's possible to get the same results with setup.py. – Thomas K Apr 16 '12 at 12:09
  • 3
    This is incorrect. In most cases, pip does download the package from PyPI, unzip it and run the setup.py script (albeit with some extra options). If you try to have pip install from a tarball for example without a setup.py, you'll get an error saying so. – Noufal Ibrahim Apr 16 '12 at 12:11
  • @ThomasK Sure, but that's not the question - the question is why the one does not give identical results to the other. – Marcin Apr 16 '12 at 12:17
  • @NoufalIbrahim In what way is this incorrect? Just because pip expects a setup.py it does not mean that pip does not provide any other behaviour. – Marcin Apr 16 '12 at 12:18
  • 1
    dude, you answer is just plain wrong. And downvoting my answer won't change that. – vartec Apr 16 '12 at 12:20
  • @vartec How is my answer wrong? The reason OP does not get exactly the same behaviour from setup.py as from pip is because they are not the same programme, nor are they supposed to be. Whether or not he could use some combination of switches to achieve that is irrelevant. – Marcin Apr 16 '12 at 12:25
  • @Marcin: Obviously they're not the same program: they have different names. But your answer fails to highlight the connection (one calls the other), or explain how to get `setup.py` to do what the OP wants. So it's technically correct, but not useful. @vartec has provided what the OP was almost certainly looking for. – Thomas K Apr 16 '12 at 12:32
  • @vartec Are you? I'm answering the question that was asked. – Marcin Apr 16 '12 at 12:34
  • @ThomasK That still does not make my answer "misleading" as you so charmingly put it. – Marcin Apr 16 '12 at 12:35
  • @Marcin: Implying that they're entirely separate when `pip` calls `setup.py` is misleading. – Thomas K Apr 16 '12 at 12:39
  • @ThomasK Not really. `pip` probably also uses `list`, but it doesn't follow that they are connected. The question does not ask "how can I re-implement `pip` myself". – Marcin Apr 16 '12 at 12:42
  • @Marcin: In the context of installing a package, pip and setup.py are clearly connected. – Thomas K Apr 16 '12 at 12:49
  • @ThomasK They may be "connected" but the question doesn't ask "are they connected" - it asks why they don't have the exact same behaviour, and the answer is because they are not supposed to. That's all there is to it; how they achieve those effects is not relevant. – Marcin Apr 16 '12 at 12:51
  • @Marcin: By strict literalism, yes, you have correctly answered the question. But the details are relevant: it's pretty clear that the OP wants some understanding of why they behave differently, or a way to modify the behaviour of one. It's like answering "why do I perceive red differently from blue" with "because they're different colours". Correct, but useless. – Thomas K Apr 16 '12 at 12:57
  • @ThomasK What other answer is there? There is no reason whatsoever to expect them to have the same behaviour. You might as well complain that I haven't dumped the whole code for pip in my answer. – Marcin Apr 16 '12 at 12:59
  • @Marcin: Of course you can expect them to have similar behaviour: they both serve to install a Python package. It's different behaviour that needs explanation. As for another answer: @vartec's answer is a good start - providing a single option to `setup.py` makes it behave like `pip`. That's relevant and likely useful. – Thomas K Apr 16 '12 at 13:04
  • @ThomasK Why does different behaviour need explanation? `pip` does not advertise that it provides identical behaviour (nor vice versa). In so far as they are supposed to be similar, they are similar: they install a python package. To expect a different explanation is like asking why you and I don't use the same writing style: neither of us has indicated that we would. – Marcin Apr 16 '12 at 13:07
  • @Marcin: Different behaviour requires explanation because the question is about the difference in behaviour. Where would humanity be if we never asked how or why? – Thomas K Apr 16 '12 at 13:26
  • @ThomasK Where would we be if we constantly asked trivial questions? Different things are different. If OP wants to know about the implementation, he is free to look at the source. – Marcin Apr 16 '12 at 13:29
  • @Marcin: Almost all code questions could be answered by reading enough source, but asking can give more useful information more quickly. Re different things: see red/blue example above. – Thomas K Apr 16 '12 at 13:34
  • @ThomasK The reason why you don't perceive blue and red as exactly the same thing is because they are not exactly the same thing. There is no deeper explanation than that. If you want to inquire into the mechanisms of perception, that is a different question. – Marcin Apr 16 '12 at 13:37
  • 3
    @Marcin: You're trolling, aren't you? – Thomas K Apr 16 '12 at 13:39
  • @ThomasK Certainly not. Are you trolling? You are the one who is defending a question that amounts to "why are two distinct things not the same?" – Marcin Apr 16 '12 at 13:41
  • 2
    @Marcin: In both the OP's question and my red/blue example, saying "because they're different" does not convey any information that the asker did not already know. Assuming the question is posed in order to gain information, an answer that does not provide information has no value. A useful answer to the OP's question might discuss how the difference is implemented, or why developers chose different mechanisms. For the red/blue conundrum, a useful answer might cover wavelengths of light, pigments, and colour receptors in the eye. – Thomas K Apr 16 '12 at 13:49
  • @ThomasK In neither case does the mechanism answer the "why" question: it amounts to a restatement of the fact that the programme/eye works in a particular way. You are essentially complaining that I didn't make up a question that like more, and choose to answer that. – Marcin Apr 16 '12 at 13:51
  • @Marcin: The details of the way that the programme/eye works can be informative and interesting. Given that the question is asked to gain information, if the literal answer to the question contains no information, we can infer that we have misunderstood what the asker wants. In that case, we can attempt to infer the question they meant to ask, and provide useful information. This process does not always work perfectly, but it's better than an answer with no information. – Thomas K Apr 16 '12 at 14:02
  • @ThomasK You may doubt your powers of comprehension, but I am confident in my own. When someone asks a nonsensical question, I assume they don't know that what they are asking is nonsensical, rather than inferring that it is an oblique way of soliciting a general disquisition on the topics mentioned. In that case, informing them that their question is nonsensical is new information. – Marcin Apr 16 '12 at 14:08
  • @Marcin: No, the question can be clearly understood to have a rational answer, more specific than a general discussion. Possible readings are "how can I make them behave in the same way" (which vartec answered), "how do they achieve different results" (similar), or "why did the developers choose different mechanisms?" (we can presume the `pip` developers were aware of the other mechanism). – Thomas K Apr 16 '12 at 14:15
  • @ThomasK the very fact that you identify three possible topics shows that all that can be provided is general information of some kind. SO is not about making up your own better question and answering it. – Marcin Apr 16 '12 at 14:19
  • 3
    @Marcin: SO is not about providing unhelpful overly-literal answers. To provide useful information, we often have to infer a 'better question'. The vote counts on these answers suggest other users think that providing some useful information is better than a literal but unhelpful answer. – Thomas K Apr 16 '12 at 14:26
  • @ThomasK Leave the votes of others out of it - you are still arguing that I should make up a new question. If you want this to be a different question, go ahead and edit it. – Marcin Apr 16 '12 at 14:31
  • 3
    @Marcin: The votes of others became relevant when the discussion moved to what SO is about. In as much as SO is a community, what it is about is determined by the opinions of its members. – Thomas K Apr 16 '12 at 14:34
  • @ThomasK Well, if you really want to argue about that, it turns out that SO users don't have to leave a reasoned explanation for their downvotes. For example, I am getting a bunch of downvotes on old questions, which I assume come from you (for obvious reasons) and vartec (whenever he sees my handle he is reminded that I once pointed out an error in one of his answers). So no, they don't really convey much information. – Marcin Apr 16 '12 at 14:38
  • @Marcin: I have not gone back to downvote your other answers, and I do not intend to. – Thomas K Apr 16 '12 at 14:40
  • @ThomasK Perhaps vartec is feeling extra insecure today. – Marcin Apr 16 '12 at 14:44
  • @Marcin: *bzzzt*, that's an ad hominem. Thanks for playing. – Thomas K Apr 16 '12 at 14:46
  • @ThomasK No, it's not an *ad hominem*. Look it up. OTOH, your comment *is* a childish attempt to provide an excuse to withdraw from the discussion. – Marcin Apr 16 '12 at 14:48
  • I **will** downvote any answer that is clearly wrong. @Marcin: if you can name any of your **correct** answers that I've allegedly downvoted, don't be shy, mention it. – vartec Apr 16 '12 at 14:48
  • @vartec You know that I can't tell who is downvoting my answers, especially as you never leave comments. One presumes that is because you have no valid criticisms. – Marcin Apr 16 '12 at 14:50
  • @ThomasK: he means his great answers like: http://stackoverflow.com/questions/10129080/analysis-and-design-for-functional-programming/10129172#10129172 – vartec Apr 16 '12 at 14:50
  • @vartec I note that you haven't left a comment on that. I defy you to locate any inaccuracies in that answer. – Marcin Apr 16 '12 at 14:51
  • 1
    I'm not going to try to arbitrate on other answers. Withdrawing from the discussion, on the other hand, sounds like an excellent idea. @Marcin, good luck trying to argue the case for this answer. – Thomas K Apr 16 '12 at 14:53
  • 1
    @Marcin: I don't have to. There are plenty of comments already. FYI I've also flagged that for moderator attention as *"not an answer"*. You keep using SO answers to boost your ego w/o any benefit to the community. You **will** get downvoted. – vartec Apr 16 '12 at 14:54
  • @ThomasK There's no case to be argued. You have spent several hours complaining that I have provided an answer to the question asked, not to your preferred question. You could complain about every other question on SO on that basis. – Marcin Apr 16 '12 at 14:54
  • 2
    @Marcin: any FYI, your revenge downvotes **will** get revoked. – vartec Apr 16 '12 at 14:56
  • @vartec Just because you string some words together, it doesn't follow that they make sense. You're the one whose fragile ego has lead you to engage in a campaign of downvoting my answers willy-nilly. – Marcin Apr 16 '12 at 14:56
  • @vartec Unlikely. It's good to know that you don't understand the downvote-reversing system. – Marcin Apr 16 '12 at 14:57
  • I really appreciate your work and comments on this thread. Actually those comments run beyond my expectation. I might need to re-check my question statements for clarity, or I should fire another question then~ – Drake Guan Apr 17 '12 at 03:58