How to roll my own pypi?

Question

I would like to run my own internal pypi server, for egg distribution within my organization.

I have found a few projects, such as:

As I understand it, pypi.python.org uses software called Cheese Shop.

My questions:

Why can't I use cheeseshop itself? (I can't find it, not sure it exists)
How do other people solve this problem? (Currently we use blush svn to distribute eggs)

*edit: This seems canonical http://wiki.python.org/moin/PyPiImplementations. Still, I'm interested in feedback.

The main reason for the clones is that the pypi code is horrible :) — Jorge Vargas, Nov 17 '09 at 22:04
We tried to assemble all dependencies in a buildout script @ [Buildout crate.io](https://github.com/InQuant/buildout_crate.io) — altipard, May 02 '13 at 19:47

score 22 · Answer 1 · edited Jun 20 '20 at 09:12

22

For light-weight solution, use pypiserver.

edited Jun 20 '20 at 09:12

Community

1
1

answered Oct 13 '12 at 15:46

wangeek

266
2
4

score 16 · Accepted Answer · edited Jan 16 '20 at 16:28

Update: PyPi is now powered by Warehouse, which is the replacement for Cheese Shop.

The source to Cheese Shop can be downloaded from https://bitbucket.org/pypa/pypi/src. There is also an example, from the page you linked to, of using Apache as a "dumb" Python package repository:

# Mount pypi repositories into URI space
Alias /pypi   /var/pypi

# /pypi/dev: Redirect for unknown packages (fallback to pypi)
RewriteCond   /var/pypi/dev/$1 !-d
RewriteCond   /var/pypi/dev/$1 !-f
RewriteRule   ^/pypi/dev/([^/]+)/?$ http://pypi.python.org/pypi/$1/ [R,L]

RewriteCond   /var/pypi/dev/$1/$2 !-f
RewriteRule   ^/pypi/dev/([^/]+)/([^/]+)$ http://pypi.python.org/pypi/$1/$2 [R,L]

# /pypi/stable: Redirect for unknown packages (fallback to pypi)
RewriteCond   /var/pypi/stable/$1 !-d
RewriteCond   /var/pypi/stable/$1 !-f
RewriteRule   ^/pypi/stable/([^/]+)/?$ http://pypi.python.org/pypi/$1/ [R,L]

RewriteCond   /var/pypi/stable/$1/$2 !-f
RewriteRule   ^/pypi/stable/([^/]+)/([^/]+)$ http://pypi.python.org/pypi/$1/$2 [R,L]

After investigating all of the available options, I am not satisfied with any of them. However, the apache solution is the simplest, and is what I ended up using, even if it is far from ideal. — drue, Sep 01 '09 at 16:15

score 9 · Answer 3 · answered Sep 05 '17 at 20:34

Warehouse

Warehouse would be your best bet in 2017. From the project's README:

Warehouse is a next generation Python Package Repository designed to replace the legacy code base that currently powers PyPI

...

You can run Warehouse locally using docker and docker-compose. See Getting started in the documentation for instructions on how to set it up.

It is maintained by The Python Packaging Authority (PyPA) who work in cooperation with members of the Python core development team, and there is a live version running at https://pypi.org/ which mirrors everything in the legacy PyPI (https://pypi.python.org/).

I started a small EC2 instance to use as pypi server with 15GB of hard drive. All the containers needed by warehouse didn't fit in that drive! — Marco Lavagnino, May 10 '18 at 13:07

score 6 · Answer 4 · answered Apr 05 '18 at 15:53

6

devpi

We are using it in a corporate environment and are pretty satisfied. It supports replication, private indexes and index inheritance.

answered Apr 05 '18 at 15:53

David

9,635
5
62
68

How do you guys add more packages to devpi ? – Ciasto piekarz Apr 16 '18 at 08:27
The interface is the same as pypi. You can use twine to upload packages. There are some access settings that need to be enabled in the devpi server though. – David Apr 17 '18 at 12:18

juliocesar · Answer 5 · 2013-10-07T23:30:56.230

5

There is a fork of djangopypi named djangopypi2 you can get it from https://github.com/popen2/djangopypi2/, I installed it and works for me, this option is what I had choose from a list of about 24 alternatives that I have found in a recently search, you can see the list here: http://uyeya.blogspot.com/2013/10/list-of-local-alternatives-of-pypi.html

edited Oct 07 '13 at 23:30

answered Oct 07 '13 at 22:46

juliocesar

5,706
8
44
63

tilgovi · Answer 6 · 2015-12-10T22:33:59.123

2

Updated: crate.io has shut down and the domain is now something else entirely.

One project that hasn't been mentioned is https://crate.io/, which seems very active. It claims to be a "Next Generation Python Packaging Index", but they have their repositories split nicely into pieces that seem to welcome customization and remixing to your purposes.

edited Dec 10 '15 at 22:33

answered Sep 28 '12 at 00:40

tilgovi

306
2
11

It looks as though the crate.io domain has now been repurposed? – Phil P Apr 01 '14 at 00:42
And now that lin is dead :p – Mark Dec 09 '15 at 22:16
2

@Mark Think I found the GitHub, at least: https://github.com/crateio/crate.io. Superseded by [Warehouse](https://github.com/pypa/warehouse), according to [Reddit](https://www.reddit.com/r/Python/comments/1wcp93/what_happened_to_crateio/). – jpmc26 Dec 11 '15 at 00:01
1

This project is no longer under active development. – astrojuanlu Jun 03 '16 at 14:17

score 1 · Answer 7 · answered Sep 23 '09 at 20:14

1

If you would like a lighter solution then deploying an entire pypi server, you could try using a server index generated by basketweaver.

answered Sep 23 '09 at 20:14

Taras Mankovski

1,639
3
16
30

1

Important to note that BasketWeaver **doesn't have a clue about wheels** and that its latest release was in 2009! – sorin May 02 '16 at 12:15

score -1 · Answer 8 · answered Jan 23 '13 at 08:57

-1

Another missing from this (oldish) list:

djangopypi

Django based, which might be a slight overkill, but I love django and it makes it extremely simple to modify it to your need should it not be satisfying.

answered Jan 23 '13 at 08:57

Stefano

18,083
13
64
79

score -1 · Answer 9 · answered Jan 23 '13 at 09:04

And crate source code is available, though documentation is, least that can be said, not-existent:

Crate.Web

It's a Django Application providing a Python Package Index. Uses a couple other packages from https://github.com/crateio so you might be able to roll out your own version without django.

I'm specifically thinking about a static one, I always thought there should be a very easy way to go explore directly some [pre-configured] repositories and shop cheese directly from my github/bitbucket public and private repos, with just a simple (gunicorn) process running.

Sorry just notice that @tilgovi answer was also speaking about the crate source repository, not only the live service at http://crate.io, though that's the one which is linked in the answer. — Stefano, Jan 23 '13 at 09:05

How to roll my own pypi?

9 Answers9

Warehouse

devpi

djangopypi

Linked

Related