0

I am trying to execute different methods in a pool object from the python multiprocessing library. I've tried too many ways but all of them get stuck when I call any of the methods .get() or .join(). I've googled a lot and none of the topics nor tutorials worked for me. My code is next:

def get_profile(artist_id):
    buckets = ['years_active', 'genre', 'images']
    artist = Artist(artist_id)
    return artist.get_profile(buckets=buckets)

def get_songs(artist_id):
    from echonest.playlist import Playlist
    return Playlist().static(artist_ids=[artist_id])

def get_similar(artist_id):
    artist = Artist(artist_id)
    return artist.get_similar(min_familiarity=0.5, buckets=['images'])


def get_news(artist_id):
    artist = Artist(artist_id)
    print "Executing get_news"
    return artist.get_news(high_relevance='true')


def search_artist(request, artist_name, artist_id):
    from multiprocessing import Pool, Process

    requests = [
        dict(func=get_profile, args=(artist_id,)),
        dict(func=get_songs, args=(artist_id,)),
        dict(func=get_similar, args=(artist_id,)),
        dict(func=get_news, args=(artist_id,))
    ]

    pool = Pool(processes=2)

    for req in requests:
        result = pool.apply_async(req['func'], req['args'])
    pool.close()
    pool.join()
    print "HERE IT STOPS AND NOTHING HAPPENS."
    output = [p.get() for p in results]

Hope someone could help because I've been stuck with this for too long. Thank you in advance.

pypy
  • 443
  • 5
  • 19
  • Sure it's not crashing without you knowing? Seems like you have a typo iin `results`/`result`, I don't see where you assign `results` – unddoch Aug 02 '15 at 18:26
  • Error handling in multithreaded programs is not the most intuitive thing. http://stackoverflow.com/questions/6728236/exception-thrown-in-multiprocessing-pool-not-detected http://stackoverflow.com/questions/19924104/python-multiprocessing-handling-child-errors-in-parent – unddoch Aug 02 '15 at 18:52
  • @lolopop Sorry that's not a real typo. Is just because I have edited the code wile posting to remove some parts that are not related to the issue and make the code clearer. Still trying to handle errors, currently following the first link. – pypy Aug 02 '15 at 20:18
  • @pypy: please see if you are having the same issue I answered [here](http://stackoverflow.com/questions/31708646/process-join-and-queue-dont-work-with-large-numbers/31709993#31709993) – Patrick Maupin Aug 03 '15 at 02:35

0 Answers0