-1

This error has a bit of a story to give further context: I made this bit of code because if I create a SQLAlchemy Relationship with a lambda function inside, it will only initialize the Relationship after all the models have been created. What it does is basically fill all models in a dictionary (self._all_models) and the lambda function will then later get the model by accessing (self._all_models[table_name]['model']). So when I do a session.add(Model), this lambda function will be called and try to get the model from the dictionary.

I executed it just fine, but my colleagues had a weird problem: The self instance was "old", and what I mean by that is that the dictionary self._all_models had only the models that were created so far, and not all the models. For example, if it had to create 3 models (a,b,c), when the lambda inside b's Relationship were to be called, the self._all_models would only have {'a': ...} instead of having all 3 ({'a': ..., 'b': ..., 'c': ...}).

The only difference between me and my colleagues is that I run Python 3.9, and they run Python 3.8. And when one of my colleagues – "Jack" – upgraded his Python to 3.9, it ran smoothly (even after changing back to his 3.8 venv); but my other colleague – "Mark" – didn't have as much luck: it still didn't work for him even after upgrading.

Comparing their libs with pip freeze didn't reveal anything different, and it's very unlikely that the problem lies in SQLAlchemy.

Code:

class ORM:
    def __init__(self, source_name: str, ..., address_http_protocol: str):
        self._all_models = defaultdict(lambda: {})
        ...
        self._create_all_models()
        ...
    def _create_all_models(self) -> None:
        base = declarative_base()
        base.metadata.reflect(self._get_engine())
        all_tables = base.metadata.tables
        for table in all_tables.values():
            current_table_name = table.name
            relationships = self._get_parents(table)
            obj = type(current_table_name, (object,), dict())
            mapper(obj, table,
                   properties={table_name: relationship(lambda table_name=table_name:
                                                        self._get_model_by_table_name(table_name), lazy='select')
                               for table_name in relationships})
            self._all_models[current_table_name]['model'] = obj
            self._all_models[current_table_name]['table'] = table
    @staticmethod
    def _get_parents(table: Table) -> set:
        parents = set()
        if table.foreign_keys:
            for fk in table.foreign_keys:
                parent_name = fk.column.table.name
                parents.add(parent_name) if parent_name != table.name else None
        return parents
    def _get_model_by_table_name(self, table_name: str) -> type:
        if isinstance(table_name, str):
            return self._all_models[table_name]['model']

I'm talking about the lambda function inside _create_all_models: lambda table_name=table_name: self._get_model_by_table_name(table_name).

Vinicius F.
  • 1
  • 1
  • 2
  • If your colleagues are all running the same code on their machines, then the obvious troubleshooting questions is: Are they all connected to the same database? Or if they're using local databases, do they all have the same tables in their local databases? I doubt this is a "code problem". – shadowtalker Jul 17 '21 at 18:17
  • @shadowtalker Yes, although each uses a local database, we have the same tables, columns, etc. It seems to be a "code problem" because only when **inside** the lambda the `self._all_models` dictionary seems to have fewer items, but when **outside** it has every single model. – Vinicius F. Jul 17 '21 at 18:43
  • Are you talking about a `lambda` somewhere in Python code that you haven't shown? Or an AWS "Lambda Function"? You showed how your `ORM` class is defined, but not how it's used. – shadowtalker Jul 17 '21 at 18:54
  • @shadowtalker I'm sorry, I'm talking about the lambda function inside `_create_all_models`. `lambda table_name=table_name: self._get_model_by_table_name(table_name)`. – Vinicius F. Jul 17 '21 at 19:06
  • And why do you think that this is getting an "old version" of `self`? It only looks like `_create_all_models` is run once, when the `ORM` object is initialized. Were you expecting this action with `mapper` and `relationship` to be taken automatically upon `session.add`? I didn't see any mention of auto-updating behavior in the SQLAlchemy docs, but I am not very familiar with that library so maybe I missed it. – shadowtalker Jul 17 '21 at 20:05
  • I was looking at the docs for [`sqlalchemy.orm.mapper`](https://docs.sqlalchemy.org/en/14/orm/mapping_api.html#sqlalchemy.orm.mapper) and [`sqlalchemy.orm.relationship`](https://docs.sqlalchemy.org/en/14/orm/relationship_api.html#sqlalchemy.orm.relationship). My hypothesis is not that `self` is somehow "old" (in violation of basic Python semantics), but that the `ORM` object is not being updated when you think it's being updated. – shadowtalker Jul 17 '21 at 20:05
  • @shadowtalker I think it's an old `self` because it seems like the lambda function is remembering the state of `self._all_models` at the time when the lambda function was defined. The model creation is done iteratively, table by table, so it may happen that when setting the `mapper` `properties` parameter, the `lambda` inside is pointing to the `self` available at that time, thus being an "old" `self`. But I still can't wrap my mind around the fact that this doesn't happen with me but happens with someone else. – Vinicius F. Jul 17 '21 at 20:51
  • Admittedly I still feel like I am lacking the big picture here. `lambda` is indeed a closure, but it closes over a *reference* to `self`. If you only ever have one instance of `ORM`, then this closure should always be pointing to the same thing. I think maybe you should write an instance method on `ORM` instead of using a `lambda` anyway. – shadowtalker Jul 17 '21 at 21:37
  • @shadowtalker Just wanted to update you on the matter since you might've gotten a bit curious. See my answer below. Thanks. – Vinicius F. Jul 20 '21 at 20:33

1 Answers1

0

So apparently the problem was that the self._all_models dictionary was bound to the __init__ function, which made it "local" and not have "access" to the right self.

More at: What is the difference between a function, an unbound method and a bound method?

Vinicius F.
  • 1
  • 1
  • 2
  • That still doesn't make much sense... are you subclassing this? Calling init from multiple places? Making shallow copies of this object? – shadowtalker Jul 21 '21 at 23:20