Import error with Ubuntu script using newspaper module

Question

I have a script that will run locally, but not on my Ubuntu server. Other scripts work fine on both platforms, but this specific one throws a import error when I attempt to run it on Ubuntu.

root@ip-xxx-xx-xx-xxx:~# /usr/bin/python3.5 /opt/script_folder/main.py
Traceback (most recent call last):
  File "/opt/script_folder/main.py", line 2, in <module>
    import process
  File "/opt/script_folder/process.py", line 25, in <module>
    from tools.getImages import getImages
  File "/opt/script_folder/tools/getImages.py", line 5, in <module>
    from newspaper import Article
  File "/usr/local/lib/python3.5/site-packages/newspaper/__init__.py", line 10, in <module>
    from .api import (build, build_article, fulltext, hot, languages,
  File "/usr/local/lib/python3.5/site-packages/newspaper/api.py", line 14, in <module>
    from .article import Article
  File "/usr/local/lib/python3.5/site-packages/newspaper/article.py", line 15, in <module>
    from . import network
  File "/usr/local/lib/python3.5/site-packages/newspaper/network.py", line 14, in <module>
    from .configuration import Configuration
  File "/usr/local/lib/python3.5/site-packages/newspaper/configuration.py", line 15, in <module>
    from .parsers import Parser
  File "/usr/local/lib/python3.5/site-packages/newspaper/parsers.py", line 10, in <module>
    import lxml.etree
ImportError: No module named 'lxml.etree'

Other scripts work fine and I didn't have this problem until I tried using the newspaper library. I have attempted the following:

pip3 install --upgrade newspaper3k
pip3 install --upgrade lxml
adding /usr/local/lib/python3.5/site-packages to $PATH
running on and off of a virtual environment
importing lxml.etree directly within the script
running on python3.5 terminal - no errors thrown
apt-get remove python3.5, pip3 uninstall newspaper3k, pip3 install newspaper3k, apt-get install python3.5
checked permissions of site-packages folder
checked the default library path that python3.5 uses
ensured the hashbang #!/usr/bin/env python3.5 is at the top of all files
sudo apt-get install python-lxml

I'm stumped here. Any guidance would be much appreciated.

EDIT: I have also followed the instructions listed here: https://newspaper.readthedocs.io/en/latest/ Everything downloaded correctly except the following:

root@ip-xxx-xx-xx-xxx:~# curl https://raw.githubusercontent.com/codelucas/newspaper/master/download_corpora.py | python3
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     100   657  100   657    0     0   9387      0 --:--:-- --:--:-- --:--:-- 10265
Traceback (most recent call last):
  File "<stdin>", line 6, in <module>
  File "/usr/local/lib/python3.5/site-packages/nltk/__init__.py", line 137, in <module>
    from nltk.stem import *
  File "/usr/local/lib/python3.5/site-packages/nltk/stem/__init__.py", line 29, in <module>
    from nltk.stem.snowball import SnowballStemmer
  File "/usr/local/lib/python3.5/site-packages/nltk/stem/snowball.py", line 32, in <module>
    from nltk.corpus import stopwords
  File "/usr/local/lib/python3.5/site-packages/nltk/corpus/__init__.py", line 66, in <module>
    from nltk.corpus.reader import *
  File "/usr/local/lib/python3.5/site-packages/nltk/corpus/reader/__init__.py", line 105, in <module>
    from nltk.corpus.reader.panlex_lite import *
  File "/usr/local/lib/python3.5/site-packages/nltk/corpus/reader/panlex_lite.py", line 15, in <module>
    import sqlite3
  File "/usr/local/lib/python3.5/sqlite3/__init__.py", line 23, in <module>
    from sqlite3.dbapi2 import *
  File "/usr/local/lib/python3.5/sqlite3/dbapi2.py", line 27, in <module>
    from _sqlite3 import *
ImportError: No module named '_sqlite3'

Possible duplicate of [Installing lxml module in python](https://stackoverflow.com/questions/4598229/installing-lxml-module-in-python) — Dominik Stańczak, Oct 24 '18 at 16:47
I've attempted all solutions in the above question, and still no luck — JLB, Oct 24 '18 at 17:45

score 0 · Answer 1 · answered Dec 23 '18 at 04:50

based on your post, it seems like the problem can be with ubuntu environment differing from your other (mac os?) local environments. I assume that you've gotten newspaper3k lib to install properly on Mac OS environments but not Ubuntu?

Perhaps try the guidance in this post first, ImportError: No module named '_sqlite3' in python3.3

I think the core issue is that your ubuntu env is missing some important underlying packages that these python libs require.

Or maybe: apt-get install sqlite3 sqlite3 -version apt-get install python-pysqlite2 apt-get install python-pysqlite2-dbg apt-get install libsqlite3-dev apt-get install sqlite pip install pysqlite

Import error with Ubuntu script using newspaper module

1 Answers1