3

I just tried installing and running scrapy on my PC at work, which runs Windows XP.

If I run scrapy startproject myproject I will get the following error:

ImportError: No module named w3lib.html

Wining: It's really troublesome running Python / Scrapy on windows XP. On linux I just run pip install Scrapy and it's ok lol.

samayo
  • 16,163
  • 12
  • 91
  • 106
Sam Stoelinga
  • 4,881
  • 7
  • 39
  • 54
  • You can use pip and easy_install on Windows. Otherwise you'll have to get all the dependencies separately. – Eryk Sun Nov 28 '11 at 07:12
  • See the [docs](http://doc.scrapy.org/en/latest/intro/install.html#windows) for links to the dependencies. – Eryk Sun Nov 28 '11 at 07:16
  • I already installed all dependencies, even installed some stuff that was not in the dependies like VC 2008 c++ redistritubalt And I could not use easy install and pip because it said my python is compiled with vc 2003 but could not find the compiler and asked me to use some other compiler.. – Sam Stoelinga Nov 28 '11 at 07:19

3 Answers3

5

It appears they forgot to list w3lib and simplejson. The latter is only required for Python versions before 2.6. Here's an installer for Distribute, in case you don't have easy_install and pip. Then you can use one of the latter to get the remaining dependencies.

Eryk Sun
  • 33,190
  • 5
  • 92
  • 111
  • 1
    This was exactly what was needed! Thanks a lot, seemed kinda obvious it said it was missing w3lib :P I got easy_install working now also. – Sam Stoelinga Nov 28 '11 at 08:03
4

use easy_install w3lib worked for me

samayo
  • 16,163
  • 12
  • 91
  • 106
Navi
  • 449
  • 1
  • 6
  • 16
3

If you have pip installed, on Mac & Linux:

try:

pip install w3lib

if you have installed it with root before:

sudo pip install w3lib

user1767754
  • 23,311
  • 18
  • 141
  • 164