Dependency from a part table, but not its master table, to an upstream table suddenly fails

Question

I have a table Downstream with a part table DownstreamPart. DownstreamPart, but not Downstream, has a dependency to an upstream table Upstream (the only other additional dependency is to Downstream). This setup has worked so far, correctly populating and also cascading deletes from Upstream to DownstreamPart, but now suddenly fails. The error I get is:

---------------------------------------------------------------------------
DataJointError                            Traceback (most recent call last)
<ipython-input-6-17abf9cc6c8e> in <module>
----> 1 (TrainedModel() & dict(dataset_hash="464e47555aae42ee0ee6edd980dd66ad")).delete()

~/.local/lib/python3.7/site-packages/datajoint/table.py in delete(self, verbose)
    415         delete_list = OrderedDict(
    416             (name, _RenameMap(next(iter(graph.parents(name).items()))) if name.isdigit() else FreeTable(conn, name))
--> 417             for name in graph.descendants(self.full_table_name))
    418 
    419         # construct restrictions for each relation

~/.local/lib/python3.7/site-packages/datajoint/dependencies.py in descendants(self, full_table_name)
    147             nx.algorithms.dag.descendants(self, full_table_name))
    148         return unite_master_parts([full_table_name] + list(
--> 149             nx.algorithms.dag.topological_sort(nodes)))
    150 
    151     def ancestors(self, full_table_name):

~/.local/lib/python3.7/site-packages/datajoint/dependencies.py in unite_master_parts(lst)
     28                     break
     29             else:
---> 30                 raise DataJointError("Found a part table {name} without its master table.".format(name=name))
     31     return lst
     32 

DataJointError: Found a part table `my_schema`.`downstream__part` without its master table.

I have DJ version 0.12.8 and python version 3.7.5. My colleague, using the same versions, and same datajoint schema, does not get this error. The part table B_part correctly shows up as a descendant of table A, without its master table, and without throwing an error. Which of the two behaviours is intended, and what can I do to resolve my error?

Edit I am showing below the table definitions and have adjusted the references in the text above accordingly Table definitions

@my_schema
class Upstream(dj.Computed):
    definition = """
    -> further_upstream
    ---
    upstream_attribute: int
    """
    class UpstreamStorage(dj.Part):
        definition = """
        -> master
        ---
        stored_attrib:   attach@store
        """
@my_schema
class Downstream(dj.Manual):
    definition = """
    -> other_dependency
    """
    class DownstreamPart(dj.Part):
        definition = """
        -> master
        -> Upstream
        """

I also found out that this sometimes fails, and sometimes works, depending on the order the tables show up in the unite_master_part function (as the docstring says, "The input list must be topologically sorted."; but I don't know why it sometimes is, and sometimes is not topologically sorted).

I should also note, that the schema is wrapped in a custom schema class like this:

class CustomSchema(Schema):
    def __call__(self, cls, *, context=None):
        context = context or self.context or inspect.currentframe().f_back.f_locals
        # Process all part tables and replace with a subclass
        for attr in dir(cls):
            if attr[0].isupper():
                part = getattr(cls, attr)
                if inspect.isclass(part) and issubclass(part, dj.Part):

                    class WrappedPartTable(part):
                        pass

                    WrappedPartTable.__name__ = attr
                    setattr(cls, attr, WrappedPartTable)
        return super().__call__(cls, context=context)

Answer 1

Hmm, looks like there could be an issue with the table reference. Seems a bit strange that your table actually exists as `my_schema`.`B__part`. In DataJoint Python, table classes are supposed to be named in CamelCase format. Since database (MySQL) support is only for lowercase, it is then translated to snake_case format.

@lara Would you update your post with the definitions for the tables involved? Feel free to use a simplified version if you like.

Here's what happens when using the below example (DataJoint 0.12.8, python 3.7.3)

import datajoint as dj

schema = dj.Schema('rguzman_my_schema')

@schema
class Upstream(dj.Lookup):
    definition = """
    upstream_id: int
    ---
    upstream_name: varchar(30)
    """
    contents = [(0,'joe')]

@schema
class B(dj.Manual):
    definition = """
    b_id: int
    ---
    b_name: varchar(30)
    """
    
    class Part(dj.Part):
        definition = """
        -> Upstream
        ---
        b_part_name: varchar(30)
        """

# Display how the tables are actually represented in the database.
# In DataJoint Python `0.13.0`, it can now be easily called using
# `schema.list_tables()`
print([t['table_name'] for t in dj.conn().query(
    """
    SELECT CONCAT('`', table_schema, '`.`', table_name, '`') AS table_name
    FROM information_schema.tables
    WHERE table_schema = %s""",
    args=('rguzman_my_schema',), as_dict=True).fetchall()])

Output:

['`rguzman_my_schema`.`#upstream`', '`rguzman_my_schema`.`~log`', '`rguzman_my_schema`.`b`', '`rguzman_my_schema`.`b__part`']