3

In CPython, the current implementation of the datetime module defines all the classes and methods of datetime in Python, then imports all that same stuff from a faster C implementation (the _datetime module), overriding or deleting all the Python implementations if the C implementation is available. This happens every time you import datetime, but that's pretty wasteful. We're hoping to re-organize the code so that import datetime first checks for the C implementation and, if it fails to import that, instead import from a _pydatetime module.

This is a simple enough change, but it comes at a maintenance cost — because we support older versions for several years, we generally work on the main branch and backport appropriate changes to older branches. We have a bot that uses git to automatically cherry-pick new PRs onto old branches.

The problem we're having is that when we re-organize the code in this way, git no longer seems to know that when I make changes to Lib/_pydatetime.py, I want the changes backported on to Lib/datetime.py in older branches. In this branch, I have tried using two commits, one that moves Lib/datetime.py to Lib/_pydatetime.py and a second that re-introduces Lib/datetime.py, then introduced a change in Lib/pydatetime.py. When I try git cherry-pick <commit-id> (where commit-id is the head of the move_pydatetime branch), I get a merge conflict because main doesnt have a Lib/pydatetime branch.

My question for y'all git experts out there is: is there a strategy that will make it possible to cleanly cherry-pick changes in Lib/_pydatetime.py directly into the backport branches, or is there some reason that this cannot work in principle?

Ezio Melotti did some investigations on strategies here, and could not find one that works.

Note: I am really only looking for answers that involve how I structure the commits when making this change, since I'm looking to avoid special-casing Lib/_pydatetime.py / Lib/datetime.py in the bot (if I can even do this). If there's no way to get git to understand natively that Lib/_pydatetime.py in the post-change world should map to Lib/datetime.py` in the pre-change world, I just want to know that that's true, and I will figure out workarounds (or manually backport all changes until the pre-change branches are all out of support — there are only a few of these each year).

Edit: The answers to Is it possible to move/rename files in Git and maintain their history? hint that this may not be possible, but some say "yes" and other say "no", some involve rewriting history (which we cannot do here) and others focus on how to follow the history using git log --folow. They certainly suggest that this may not be possible, but do not specifically address how git cherry-pick works in such a way that I am confident saying, "Ah yes, there is no way to structure these changes to the repository in such a way that git cherry-pick cannot follow them."

I think the biggest additional wrinkle that this particular question adds is the fact that Lib/datetime.py will exist both before and after the change, but it will represent two fundamentally different files. I'm OK with changes to Lib/datetime.py not being backportable to before the change, but changes to Lib/pydatetime.py need to map to pre-change Lib/datetime.py.

Paul
  • 10,381
  • 13
  • 48
  • 86
  • Basically, no... you need two commits to preserve the history of the move/edit. If git can figure out the history, then it will usually handle cherry-picks just fine. The linked Question explains in more detail. – JDB Mar 23 '23 at 15:55
  • @JDB I'm fine with two commits (the linked branch has move and edit as two separate commits), but `cherry-pick` doesn't seem to see in the history the fact that `Lib/_pydatetime.py` had a commit where it was just moved from `Lib/datetime.py`. – Paul Mar 23 '23 at 16:00
  • 1
    You may need to try a different merge strategy as recommended in this post: [git cherry-pick with target file renamed](https://stackoverflow.com/questions/32843857/git-cherry-pick-with-target-file-renamed) – JDB Mar 23 '23 at 16:03
  • @JDB I'm still getting the same problem with `-Xrename-threshold=10%` (though honesty `Lib/_pydatetime.py` should be >95% similar to `Lib/_datetime.py` anyway), plus, this question is "What is the best way to structure the commits to make this change", and for the purposes of the question I cannot change what the cherry-pick command is. – Paul Mar 23 '23 at 16:12
  • 1
    Ok... sounds like a more complex scenario than is normal. Apologies for the knee-jerk reaction. I'll read through more carefully. – JDB Mar 23 '23 at 16:18
  • For clarification... is it `Lib/_datetime.py` or `Lib/datetime.py`? You seem to be using them interchangeably in your question. I think it's supposed to be `Lib/datetime.py` – JDB Mar 23 '23 at 16:29
  • @JDB `Lib/datetime.py`, sorry the `_datetime.py` ones are just me being confused. I've edited the post to remove references to `Lib/_datetime.py`, which is not a file on either branch (the `_datetime` module comes from `Mod/datetimemodule.c` I think). – Paul Mar 23 '23 at 16:48

0 Answers0