-1

I have made a few Python projects for work that all revolve around extracting data, performing Pandas manipulations, and exporting to Excel. Obviously, there are common functions I've been reusing. I've saved these into utils.py, and I copy paste utils.py into each new project.

Whenever I change utils.py, I need to ensure that I change it in my other project, which is an error-prone process

What would you suggest?

Currently, I create a new directory for each project, so

/PyCharm Projects
--/CollegeBoard
----/venv
----/CollegeBoard.py
----/Utils.py
----/Paths.py
--/BoxTracking
----/venv
----/BoxTracking.py
----/Utils.py
----/Paths.py

I'm wondering if this is the most effective way to structure/version control my work. Since I have many imports in common, too, would a directory like this be better?

/Projects
--/Reporting
----/venv
----/Collegeboard
------/Collegeboard.py
------/paths.py
----/BoxTracking
------/BoxTracking.py
------/paths.py
----/Utils.py

I would appreciate any related resources.

Christina Zhou
  • 1,483
  • 1
  • 7
  • 17

1 Answers1

1

Instead of putting a copy of utils.py into each of your projects, make utils.py into a package with it's own dedicated repository/folder somewhere. I'd recommend renaming it to something less generic, such as "zhous_utils".

In that dedicated repository for zhous_utils, you can create a setup.py file and you can use that setup.py file to install the current version of the zhous_utils into your python install. That way you can import zhous_utils into any other python script on your PC, just like you would import pandas or any other package you've installed to your computer.

Check out this stackoverflow thread: What is setup.py?

When you understand setup.py, then you will understand how to make and install your own packages so that you can import those installed packages to any python script on your PC. That way all source code for zhous_utils is centralized to just one folder on your PC, which you can update whenever you want and re-install the package.

Now, of course, there are some potential challenges/downsides to this. If you install zhous_utils to your computer and then import and use zhous_utils in one of your other projects, then you've just made zhous_utils into a dependency of that project. That means that if you want to share that project with other people and let them work on it as well or use it in some way, then they will need to install zhous_utils. Just be aware of that. This won't be an issue if you're the only one interacting/developing the source code of the projects you intend to import zhous_utils into.

LetEpsilonBeLessThanZero
  • 2,395
  • 2
  • 12
  • 22
  • Thanks. I just have a question about "you can use that setup.py file to install the current version of the zhous_utils into your python install." Is this the recommended way to install my own package: https://kb.iu.edu/d/acey#setup? It seems kind of complex to do each time. – Christina Zhou Aug 12 '19 at 19:17
  • @ChristinaZhou I just open a command prompt in the directory where setup.py is saved and then I run `pip install .` (note the period there). pip should then look for your setup.py file and use it to install your package. That's the pip solution. There's also other solutions, like just running something like `python setup.py build` followed by `python setup.py install`. You can see more here: https://pythonhosted.org/an_example_pypi_project/setuptools.html#using-setup-py – LetEpsilonBeLessThanZero Aug 12 '19 at 20:21