Multithreading in python - urllib2

Question

This is a simple script for doing request using multiple threads. I got it from http://www.ibm.com/developerworks/aix/library/au-threadingpython/.

How to put all the html of each requested page into a varible. I tried with a global variable and it doesn't work.

Instead of print soup.findAll(['title']) I need html. – Brana Mar 17 '14 at 17:06 — Brana, Mar 17 '14 at 17:06

score 0 · Answer 1 · answered Mar 17 '14 at 17:18

0

This is not the answer on your question, but i want to give you a little advice. You can use gevent library for your task. Your code will be simplier.

And first of all you should read this patterns. Threads in python are evil. :)

answered Mar 17 '14 at 17:18

Ellochka Cannibal

1,750
2
19
31

I tried few hours ago but i couldn't install the gevent library. I got some strange error. – Brana Mar 17 '14 at 17:21
Windows ...unable to find vcvarsall.bat – Brana Mar 17 '14 at 17:24
did you try this http://stackoverflow.com/questions/2817869/error-unable-to-find-vcvarsall-bat can't help you with your problem, cause i use linux and mac os :( – Ellochka Cannibal Mar 17 '14 at 17:28
with threads you can put your results (prints) in separate queue and after all get them :) – Ellochka Cannibal Mar 17 '14 at 17:31
how to do this? Why global varible cannot get the value? – Brana Mar 17 '14 at 17:32
pass results_queue arg to threads __init__'s; i don't see global variable in your code – Ellochka Cannibal Mar 17 '14 at 17:35
I do not know how to pass pass results_queue arg to threads init's. you could write an example if you know how to do it. – Brana Mar 17 '14 at 17:46
let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/49981/discussion-between-ellochka-cannibal-and-brana) – Ellochka Cannibal Mar 18 '14 at 21:26
Never need, i have solved the problem another way. Thanks anyway. – Brana Mar 19 '14 at 01:02

Multithreading in python - urllib2

1 Answers1