19

I'd like to snapshot the global Hackage database into a frozen, smaller one for my company's deploys. How can one most easily copy out some segment of Hackage onto a private server?

J. Abrahamson
  • 72,246
  • 9
  • 135
  • 180
  • Check out [yackage](http://hackage.haskell.org/package/yackage) which is a simple hackage like server. The [Hackage trac](http://hackage.haskell.org/trac/hackage) is where the instructions were to copy all of hackage but it is down currently and I do not remember explicit direction on how to do partial mirroring. – Davorak Feb 07 '13 at 19:03
  • There's a link to download an archive of the most recent releases of all packages on Hackage right on the front page. Is that enough? – Daniel Wagner Feb 07 '13 at 20:11
  • 1
    I'd like effectively to have a snapshot of just the libraries I want to use in production so that developer and build boxes can both install from that source. Yackage may work. – J. Abrahamson Feb 07 '13 at 20:40

4 Answers4

6

Here's one script that does it in just about the simplest way possible: https://github.com/jamwt/mirror-hackage

You can also use the MirrorClient directly from the hackage2 repo: http://code.haskell.org/hackage-server/

sclv
  • 38,665
  • 7
  • 99
  • 204
  • 1
    Very cool. `mirror-hackage` may be enough, but I never even thought of picking through the Hackage2 code to try to use it. What's the stability of Hackage2 these days? – J. Abrahamson Feb 07 '13 at 21:59
3

This is not an answer two the question in the title but an answer to my interpretation of what the OP wish to achieve.

Depending of what you want for level of stability in your production circle you can approach the problem in several ways.

I have split the dependencies in two parts, things that I can use that are in the haskell platform (keep every platform used in production) and then only use a small number of packages outside that and don't let anyone (including yourself) add more packages into your dependency tree just because of laziness (as developer). These extra packages you use some kind of script for and collect from hackage (lock to version) by using cabal fetch. Keep them safe. Create a install script that uses your safe packages and if a new machine (developer) are added to your team, use that script.

yackage is great but it all comes down to how you ship your product. If you have older versions in production you need to have a yackage setup for every version and that could be quiet annoying after a couple of years.

Jonke
  • 6,525
  • 2
  • 25
  • 40
2

You can download Hackage with Voker57's hackage-mirror.sh. You'll need 'curl' for it to run. If you're using a Debian based Linux distribution, you can install curl by typing apt-get install curl.

subtlearray
  • 1,251
  • 1
  • 11
  • 23
1

Though it's not a segment of Hackage, I've written a bash script, that downloads the whole Hackage, what can be further easily set up as a mirror using an HTTP server. Also, it downloads all required stuff like GHC compilers ready to be used with Stack.

Currently, a complete Hackage mirror occupies ~10GiB (~100000 packages of all versions) and Stack related stuff like GHC compilers ~21GiB (~200 files). Consequent runs of the script skip already downloaded stuff, so it downloads only new one. So it's a pretty convenient way to "live offline" and sync up to date when online.

AleXoundOS
  • 341
  • 1
  • 4
  • 13