PyCharm/GitHub: Managing the directories of multiple repositories for the same project

Question

Background: I'm currently working on a bioinformatics project. I'm using PyCharm as an IDE and GitHub for version control. I was wondering about my project structure...

Setup: We have 2 repositories:

1. A main repo that houses all of our files and directories
2. And then we have a side repo that hosts our R-Shiny app/files/directores

The GitHub repo for the R-shiny app (2) is a mirror image of one of the directories in the main repo (1). We will be doing constant maintenance on the R-Shiny app (hence the reason for 2 repos).

I will reference the PyCharm project AND the GitHub repo as (1) and (2) from here forward.

Question: What is the best way to deal with this situation?

1. In my main project (1) directory is it safe to create a new PyCharm project (2)?
And then push/pull main project (1) + side project (2) to main project repo(1). And then push/pull side project (2) to side project repo (2)

2. Create separate directories for both PyCharm Project.
I would have to copy/paste my directories from (2) and paste them appropriately into (1).
And then push/pull project (1) and project (2) separately.

3. Or should I just consolidate everything into one project/repository, and then create a separate branch for the R-Shiny app/directories? I have no idea if this is possible or practical.

4. Other..

I want to find a better/standard/practical way to deal with this situation. I'm not a novice programmer by any means. But I've taught myself a lot by trial and error, so sometimes my methods are a bit unconventional.

Answer 1

I often work with projects where I need to reference other directory trees that are also PyCharm projects. Usually I just add the reference as a Content Root. Open project 1, then go to Preferences > Project > Project Structure > Add Content Root and add the path to project 2.

You should now see both project 1 and 2 listed in View > Tool Window > Project.

Exclude the sub directory of project 1 (right-click Mark Directory As > Excluded), this will prevent classes, files and symbols in that location from showing up as duplicates in the Navigate searches.

PyCharm is smart about tracking multiple git repos in the same project, see Preferences > Version Control. Just don't nest git repos inside each other.

To compare your latest changes between project 1 and 2 select the folders in the Project Tool Window, right-click and choose Compare Directories.

To keep project 1 and 2 in sync you have a couple of options

• Use git submodules as suggested by @Pockets answer, note that casual git users might struggle with this setup. If you go this route then all the previous steps do not apply.

• Keep project 1 and 2 in separate repos. Don't duplicate project 2 inside project 1. Add path to project 2 subfolder to the .gitignore in project 1. Push from the project 2 content root and pull into the subfolder.

• Duplicate project 2 files inside 1. Use PyCharm Preferences > Build, Execution, Deployment > Deployment with Type > Local or mounted folder

Answer 2

The correct way to handle this is with Git submodules, which allow you to include a reference to a specific commit of another repository in the parent repository. This has the added benefit of being able to version your dependencies: if you update something in R-shiny, those changes do not propagate up until you manually bring in those changes and know that the code in the parent repository supports the changes in the R-shiny stuff. tl;dr: your setup should have both as their own repos, with R-shiny included as a submodule of the other.

See my answer here for a more thorough explanation about how to use submodules.

Be warned: you say it's a bioinformatics project, so chances seem to be that the people who need to be able to view your code may not necessarily be the most proficient Git users - it might help to include scripts for them to get around.