Module not imported, yet still works

Question

I am testing pyodbc, and wrote an extremely simple script. The database table has some numeric data types, and a datetime field. If I open an interactive python window, and import pyodbc, the following fails:

>>> import pyodbc
>>> print(Decimal('0.3'))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'Decimal' is not defined
>>>

Clearly, pyodbc does not implicitly import decimal.

However, the script collects Decimal fields from the database, and prints them correctly. At no time did I import decimal, nor did I import datetime. What is happening? Can I rely on this?

The full script:

#! /usr/bin/env python
"""Tests odbc connections
"""

import pyodbc

def getData():
    #returns a row set from the ODBC_Test database
    conn_str="DSN=ODBC_Test"
    cnx=pyodbc.connect(conn_str)
    crsr=cnx.cursor()
    sql="select id_num,entry_date,amt1,amt2,amt3,note from tbl3"
    crsr.execute(sql)
    columns = [column[0] for column in crsr.description]
    print(columns) #a list of column names
    rows=crsr.fetchall() #a list of tuples
    results={}
    n=0
    for row in rows:
        n+=1
        key=row[0]
        print('{} key:{}'.format(n,key))
        results[key]=dict(zip(columns,row))
    crsr.close()
    cnx.close()
    return results

data=getData()
print('data: {}'.format(data))
for k in data.keys():
    for field in data[k].keys():
        print("'{}': {}".format(field,data[k][field]))

all_modules=dir()
print(all_modules)

The output:

['id_num', 'entry_date', 'amt1', 'amt2', 'amt3', 'note']
1 key:1512075160
2 key:1512075027
data: {'1512075160': {'id_num': '1512075160', 'entry_date': datetime.datetime(2022, 3, 13, 11, 22), 'amt1': Decimal('0.18'), 'amt2': Decimal('0.35'), 'amt3': Decimal('0.02'), 'note': '$0.05 NL (6 max)'}, '1512075027': {'id_num': '1512075027', 'entry_date': datetime.datetime(2022, 3, 13, 11, 21), 'amt1': Decimal('0.00'), 'amt2': Decimal('1.17'), 'amt3': Decimal('0.06'), 'note': '$0.05 NL (6 max)'}}
'id_num': 1512075160
'entry_date': 2022-03-13 11:22:00
'amt1': 0.18
'amt2': 0.35
'amt3': 0.02
'note': $0.05 NL (6 max)
'id_num': 1512075027
'entry_date': 2022-03-13 11:21:00
'amt1': 0.00
'amt2': 1.17
'amt3': 0.06
'note': $0.05 NL (6 max)
['__annotations__', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', 'data', 'field', 'getData', 'k', 'pyodbc']

table create script:

CREATE TABLE IF NOT EXISTS public.tbl3
(
    id_row bigint NOT NULL,
    id_num character varying(32) COLLATE pg_catalog."default" NOT NULL,
    entry_date timestamp without time zone NOT NULL,
    amt1 numeric(14,2) DEFAULT 0,
    amt2 numeric(14,2) DEFAULT 0,
    amt3 numeric(14,2) DEFAULT 0,
    note character varying(32) COLLATE pg_catalog."default",
    CONSTRAINT tbl3_pkey PRIMARY KEY (id_row)
)
WITH (
    OIDS = FALSE
)
TABLESPACE pg_default;

Note: I looked at What is the best way of listing all imported modules in python? the output of the instructions in that question showed definitely that decimal had not been imported.

I'm using

Python 3.9.5
Windows 10
postgresql 10

That your code does not import a module does not mean that other code you are calling cannot use that module. — mkrieger1, Mar 14 '22 at 19:51
`dir()` gives you all names that are defined in your script, not all modules that have been imported somewhere. To get all imported modules, look at `sys.modules` (as shown in the other question you have linked). — mkrieger1, Mar 14 '22 at 19:54
You can easily do your own experiment. Make a function in it's own module which returns an instance of a class from another module. You will still be able to call members of that class without importing it yourself. — quamrana, Mar 14 '22 at 19:56
`import pyodbc` adds *exactly one* new name to your namespace, which is `pyodbc`. — jasonharper, Mar 14 '22 at 20:02
As I mentioned sys.modules does *not* include either of the _missing_ modules. I'm rather surprised print() can print it though. I'm not complaining :) and I will still include datetime and decimal in the script. — Vorpal Swordsman, Mar 14 '22 at 20:14

score 1 · Answer 1 · answered Mar 14 '22 at 20:27

There's a difference between a module being loaded and it being available in your namespace. It is entirely possible for one module to get access to objects of types it cannot directly access from its namespace. You can do this yourself quite easily:

file1.py:

def foo():
    import decimal
    return decimal.Decimal('0.1')

file2.py

import file1
print(repr(file1.foo())) # prints Decimal('0.1')
print(Decimal("0.2"))    # raises a NameError because Decimal is not in our namespace

Note that before the foo() function is called, the decimal module has not yet been loaded. It gets imported only into the local namespace of the function, so there's no way for any code in file2 to access it directly (without doing its own import).

I'd note that you also haven't tested the types of the objects in question, you're relying on their repr, which matches a standard library type. It's possible (if perhaps unlikely) that the database module has its own versions of those types that it is using, and it just chooses to have them replicate the repr of builtin types, since that's easier than documenting the reimplementations in detail.

It is beginning to make sense now. Decimal('0.3') is itself a representation of the PostgreSQL numeric type. The conversions are explicit in the pyodbc source. I had previously been scrupulously importing the needed modules. This time I forgot, but was surprised when it "worked anyway" — Vorpal Swordsman, Mar 14 '22 at 21:03

Module not imported, yet still works

1 Answers1