Syntax for importing scipy and sklearn modules

Question

I use (just the standards) Win10, Anaconda-2018.12, Python-3.7, MKL-2019.1, mkl-service-1.1.2, Jupyter ipython-7.2. see here e.g. I"m wondering why the following syntax works for import statements with the numpy modules but does not work for scipy or sklearn modules:

import scipy as sp
import numpy as np
A = np.random.random_sample((3, 3)) + np.identity(3)
b = np.random.rand((3))
x = sp.sparse.linalg.bicgstab(A,b)

> AttributeError                            Traceback (most recent call
> last) <ipython-input-1-35204bb7c2bd> in <module>()
>       3 A = np.random.random_sample((3, 3)) + np.identity(3)
>       4 b = np.random.rand((3))
> ----> 5 x = sp.sparse.linalg.bicgstab(A,b)
> AttributeError: module 'scipy' has no attribute 'sparse'

or with sklearn

import sklearn as sk
iris = sk.datasets.load_iris()

> AttributeError                            Traceback (most recent call
> last) <ipython-input-2-f62557c44a49> in <module>()
>       2 import sklearn as sk
> ----> 3 iris = sk.datasets.load_iris() AttributeError: module 'sklearn' has no attribute 'datasets

This syntax however does work (but are for rare commands not really lean):

import sklearn.datasets as datasets
iris = datasets.load_iris()

and

from scipy.sparse.linalg import bicgstab as bicgstab
x = bicgstab(A,b)
x[0]

array([ 0.44420803, -0.0877137 , 0.54352507])

What type of problem is that ? Could it be eliminated with reasonable effort ?

Answer 1

The "problem"

The behavior you're running into is actually a feature of Scipy, though it may seem like a bug at first glance. Some of the subpackages of scipy are quite large and have many members. Thus, in order to avoid lag when running import scipy (as well as to save on usage of system memory), scipy is structured so that most subpackages are not automatically imported. You can read all about it in the docs right here.

The fix

You can work around the apparent problem by exercising the standard Python import syntax/semantics a bit:

import numpy as np

A = np.random.random_sample((3, 3)) + np.identity(3)
b = np.random.rand((3))

import scipy as sp

# this won't work, raises AttributeError
# x = sp.sparse.linalg.bicgstab(A,b)

import scipy.sparse.linalg

# now that same line will work
x = sp.sparse.linalg.bicgstab(A,b)
print(x)
# output: (array([ 0.28173264,  0.13826848, -0.13044883]), 0)

Basically, if a call to sp.pkg_x.func_y is raising an AttributeError, then you can fix it by adding a line before it like:

import scipy.pkg_x

Of course, this assumes that scipy.pkg_x is a valid scipy subpackage to begin with.