python: dynamic type casting - unicode object to python object conversion

Question

This is a problem I am facing for a data pipeline project. I have 2 data sources. One contains all the user data, the other contains meta data of what all columns we have to process from the user data to output.

So python is good with dynamic type casting, like if I say

a = float
b = "25.123"
c = a(b)
print(c)
>> 25.123

This is what I want to do, I want to dynamically type cast values in order to process them correctly. The type is retrieved from the meta-data data source. The problem is when I do a django model query on the meta-data, I get unicode objects.

a = model.objects.filter(id = 'id') # get the type variable from the meta-data
a = a[0]['type']
print(a)
>> u'float'
a("123.123")
>> TypeError: 'unicode' object is not callable

How do I convert this u'float' to float ? Does this approach have any better alternatives ? I checked out this, but it does not work

Open to all suggestions

I would create an explicit mapping from strings to types. – juanpa.arrivillaga May 03 '18 at 06:50 — juanpa.arrivillaga, May 03 '18 at 06:50

Taku · Accepted Answer · 2018-05-03T07:14:10.857

In your first example where a = float, a is a built in function, but in your second example, a = u"float", a is an unicode string. If you want complete "dynamicness" of the built in type to convert to without the need of creating a mapping, you can do this:

# for Python 2
a = u"float"
b = "123.123"
import __builtin__
print getattr(__builtin__, a.decode())(b) 
# 123.123

# for Python 3+
a = u"float"
b = "123.123"
import builtins
print(getattr(builtins, a)(b))
# 123.123

I would advise you not to use eval() (as suggested by another answer) because it could lead to major security risks. This is why I used the __builtin__/builtins module and getattr() to retrieve the float(...) function.

You may also create a mapping (ie. a dict), to map the unicode string to its corresponding function (suggested by this comment):

# both Python 2 and 3
a = u"float"
b = "123.123"
mapping = {u"float": float, u"int": int, u"str": str, u"list": list}
print(mapping[a](b))
# 123.123

Using a mapping is the safest way to go, but it limits your "dynamicness" to only the types listed in the mapping.

how do we do this with date or datetime objects ? suppose a = u"date", getattr will not work there right — skybunk, May 10 '18 at 09:54
understood, so instead of using __builtin__, you use datetime — skybunk, May 10 '18 at 09:57

score 0 · Answer 2 · answered May 03 '18 at 06:51

0

You may use the eval() function to evaluate the code, but you need to be careful with that function! Otherwise, you might have a look at that post. Another solution is to pre-define allowed types and gather them inside a dictionary mapping typeName with typeConstructor.

answered May 03 '18 at 06:51

picnix_

377
2
7

1

better to have a predefined dict mapped, eval() may lead to problems in the future – skybunk May 03 '18 at 06:57
Yes, that's why I emphasize the fact that it is needed to be careful. Many security flaws may appear, so I actually won't recommend `eval()` but that was the first that came to my mind. – picnix_ May 03 '18 at 07:09

score 0 · Answer 3 · answered May 03 '18 at 06:59

0

You could use astype from the numpy library:

import numpy as np
np.array(('213.123')).astype(a)

admittedly, this needs to go via an array, because astype() works on numpy-arrays - but it can compute a string as type.

answered May 03 '18 at 06:59

SpghttCd

10,510
2
20
25

python: dynamic type casting - unicode object to python object conversion

3 Answers3