Python: Newspaper Module - Downloading from multiple URLs

Question

I hate to start a new post but I am trying to accomplish the exact thing described in this question: Python: Newspaper Module - Any way to pool getting articles straight from URLs?

In attempting to implement the solution, though, I am getting the following error:

NameError   Traceback (most recent call last)
<ipython-input-38-2707f1416873> in <module>()
----> 1 sources = [SingleSource(articleURL=u) for u in urls]
      2 
      3 newspaper.news_pool.set(sources)
      4 newspaper.news_pool.join()
      5 

<ipython-input-38-2707f1416873> in <listcomp>(.0)
----> 1 sources = [SingleSource(articleURL=u) for u in urls]
      2 
      3 newspaper.news_pool.set(sources)
      4 newspaper.news_pool.join()
      5 

<ipython-input-37-4949a9e51da5> in __init__(self, articleURL)
      1 class SingleSource(newspaper.Source):
      2     def __init__(self, articleURL):
----> 3         super(StubSource, self).__init__("http://localhost")
      4         self.articles = [newspaper.Article(url=url)]

NameError: name 'StubSource' is not defined

Would very much appreciate a push in the right direction.

score 0 · Accepted Answer · answered Apr 03 '17 at 04:16

Looks like a typo in the linked answer as StubSource is not defined

class SingleSource(newspaper.Source):
    def __init__(self, articleURL):
        super(StubSource, self).__init__("http://localhost")
        self.articles = [newspaper.Article(url=url)]

It should probably be:

class SingleSource(newspaper.Source):
    def __init__(self, articleURL):
        super(SingleSource, self).__init__("http://localhost")
        self.articles = [newspaper.Article(url=url)]

Python: Newspaper Module - Downloading from multiple URLs

1 Answers1