2

I use Queue.Queue in my program, but it would be better if there was a type like set. I need the container to be thread-safe, but I do not want it to be ordered.

def init_query_pool(self):
    self._pool = Queue.Queue() // todo: replace Queue.Queue with set
    self._new_query_pool()

def _create_threads(self):
    sessions = session_util.load_sessions()

    for s in sessions:
        t = threading.Thread(target=self.gen_qs, args=(s,))
        t.start()

def gen_qs(self, session):
    params = self._init_params(session)
    while True:
        qs = self._pool.get()
        self._search(session, params, qs)
roger
  • 9,063
  • 20
  • 72
  • 119
  • I'm not sure I understand the question. You can simply use `set`, right? I can't really see how you relate queues with sets, seem to be different concepts? – freakish Aug 26 '15 at 09:40
  • 2
    I think the question may be 'are sets thread-safe'? Possibly [a duplicate](http://stackoverflow.com/questions/2227169/are-python-built-in-containers-thread-safe) – Pynchia Aug 26 '15 at 09:44
  • @freakish with my code, can you understand me ? – roger Aug 26 '15 at 09:47
  • So is the reason for using set to avoid duplicates? Because otherwise set is quite inefficient compared to queue. – freakish Aug 26 '15 at 09:48
  • @freakish that is not the main reason, I just want to be unordered – roger Aug 26 '15 at 09:50
  • @freakish and why set is inefficient? – roger Aug 26 '15 at 09:51
  • OK, but what's the benefit? You can simply use `set` with `.pop()` instead of `.get()`. Sets are thread safe but `.pop()` is inefficient since searching and removing from sets (which `.pop()` does) requires hashing. While in queues `.get()` simply takes the entry from the head. Well, I'm not 100% sure that it performs worse, you should check this. Still, the conceptual difference is very important. Looking at such code I would not understand why you use set instead of queue. – freakish Aug 26 '15 at 09:53
  • @freakish thanks, I am downloading a set of url(these url have some order in them), may be an ordered url can be easily recognized – roger Aug 26 '15 at 09:59

0 Answers0