33

I have a Zend Framework 2 application. It contains some library code containing business logic and some other utilities that will be common to other applications that will be created later.

My intention is to share it across projects using Composer. The question is, how do I do this properly and streamline the development? I will almost certainly need to make changes and additions to the library, from within the other project.

I tried setting up vendor/stuff as a git submodule containing the package needed, and referencing it in the primary composer.json like this (ref):

"repositories": [
    {
        "type": "git",
        "url": "vendor/stuff"
    }
],
"require": {
    "stuff/library": "master"
},

Composer isn't able to load it in this way. It complains that the package could not be found, presumably because it's ignoring the fact that the URL is both local and relative. Technically, it doesn't need to; the vendor/stuff folder was initialised separately through git submodule commands.

Rubens Mariuzzo
  • 28,358
  • 27
  • 121
  • 148
bcmcfc
  • 25,966
  • 29
  • 109
  • 181

4 Answers4

51

Unfortunately* Composer doesn't support Git submodules, as the main aim of Composer is to provide a similar inter-project dependency functionality and it would be pointless to try to replicate submodules in Composer.

I have the same problem that you are trying to solve, of developing a library while simultaneously developing the application that uses that library. There are a couple of ways to solve that problem just using composer.

Make a symbolic link for the library directory

This is the quickest and dirtiest way of doing it. Just do a composer update to create the appropriate directory for the library in your vendors directory, then replace it with symbolic link from your the directory that contains your library.

Obviously this is not great as you can accidentally overwrite code that you may have edited by running composer update.

Use Composer prefer source option

Composer has the option to download the source via a Git clone (--prefer-src) rather than downloading a zipball (--prefer-dist) which is the default. This allows you to edit the source code inside the vendors directory and then commit it through Git.

e.g. Say you have project that requires amongst other libraries symfony/yaml which you want to fix a bug in. What you could do is:

  1. composer update - This will download all of dependencies of the project.

  2. composer update symfony/yaml --prefer-source - This will now update just the symfony/yaml directory in the vendors directory.

  3. Fix the bug and then commit it through git.

Use Composer local repository

The way I ---actually--- used to work when I'm developing a project and it's requirement in tandem, is to use Composers ability to set explicitly set a repository to use to resolve dependencies. e.g. if your code is in:

/projects/library/
/projects/project/

In the composer file for your project add the repository entry:

"repositories": [
    {
        "type": "vcs",
        "url": "/projects/library/"
    }
]

Running composer update will now look at the Git entries in /projects/library/ to resolve any dependencies on the library in preference to those listed in Packagist or other repository.

This does mean that when you want to test a change in the library code you need to:

  1. Commit it, so that it has a Git entry.

  2. Run Composer update in the project directory to get the latest version.

But you avoid having to push the commit to an external repository, which is good as it means you aren't pushing code that may not work, and it means you can work offline as Git commits don't require an internet connection.


Although that is apparently the best way of working, it's still kind of dangerous as it is too easy to accidentally check in a version of your composer.json that has references to local directories, which obviously will break the project for everyone else.

To avoid this I made a couple of small scripts that i) backup my real composer.json file, ii) Add in some local repositories, iii) run composer update iv) Restore the real composer.json file.

localupdate.sh

cp -f composer.json composer.json.bak
php composerLocal.php
composer update
cp -f composer.json.bak composer.json

composerLocal.php

<?php

$srcFile = file_get_contents("composer.json");
$hackFile = file_get_contents("composer.local");
$finalString = str_replace('"LOCALHACK",', $hackFile, $srcFile);
file_put_contents("composer.json", $finalString);

?>

composer.local

"LOCALHACK",

"repositories": [
    {
        "type": "vcs",
        "url": "/projects/library1"
    },
    {
        "type": "vcs",
        "url": "/projects/library2"
    }   
],

And then place "//": "LOCALHACK", somewhere in your projects composer.json file. Running localupdate.sh now safely does a composer update against local repositories without any chance of committing a bad version of composer.json.

Just clone it yourself with Git

This is how I tend to work now:

i) Composer update in the project ii) Go into the vendors directory and delete the library that I want to be developing at the same time. iii) Git clone from whichever repo you are developing the library in, into the appropriate vendors directory.

Composer understands git repos, so won't overwrite a git cloned directory(, though it does seem to get a little confused about editing the composer.json of the library).

Doing the git clone yourself, gives you complete control over what gets installed, and allows you to install from a repo that composer doesn't know about, or an untagged version, without having to edit the composer.json in the project.

That's the key feature of doing the git clone yourself; by not touching the composer.json of the project, it's completely safe, with no possibility of checking in a composer.json that has been modified to use local/custom repos.

  • Edit - 6th Sept 2014

The validation for composer.json files has been tightened and it is no longer possible to have a "//": "LOCALHACK" entry in the file. Which is another reason why the Composer guys not having versioning for the Composer project is nuts.

* I actually think Git Submodules are a dumb, dumb, dumb implementation to 'solve' a difficult problem in a way that only causes more problems down the road, and so Composer not supporting them is way more 'fortunate' than 'unfortunate'. Obviously other people do use them, and are happy with them, so that is just my opinion, man, but if you're using Composer you shouldn't have a need for submodules.

Danack
  • 24,939
  • 16
  • 90
  • 122
  • 2
    This point "Use Composer prefer source option" removed my dilemma in switching to composer. Thanks for answering the question extensively! – JohnnyQ Jun 24 '14 at 03:11
  • +1 to "If you're using composer you shouldn't need submodules." – Tek Jul 03 '14 at 15:21
  • I had trouble editing dependencies installed by composer using the [global installation](https://getcomposer.org/doc/00-intro.md#globally) because it was setting ownership of files as root and my IDE didn't have permissions. I solved this by moving composer to `~/bin/composer` and including `~/bin` in my PATH. – Corie Slate Mar 18 '15 at 17:06
  • I ended up doing something similar to LOCALHACK, except that it's [a script which merges the JSON and triggers composer against a temporary file](https://github.com/DHager/composer-haydn). – Darien Jun 12 '15 at 17:56
  • `Just clone it yourself with Git` still works, but if you run `compser update` with latest release, it won't ignore the package - instead it [will warn you if it overwrites files](https://i.stack.imgur.com/Aluzb.png) – Adam Nov 22 '19 at 19:22
  • @Danack how do you get a complete picture of changes for a feature if it touches several packages? e.g. 5 or more... – bilogic Sep 19 '22 at 02:41
  • @bilogic I would cry for a good twenty minutes, and then switch to using a mono-repo. Feel free to ask for more details in https://chat.stackoverflow.com/rooms/11/php , but that sounds like a nasty problem to have. – Danack Sep 19 '22 at 19:55
  • @Danack actually after posting yesterday, I found a pretty good solution: I wrote a scanner that scans the whole `vendor` folder for git repos that have (1)modified, (2)staged, (3)untracked files and get my Git client to bring up all repos+the project itself, quite effective I must say, but I will chat with you thanks :) – bilogic Sep 20 '22 at 08:56
2

Composer has an autoload mapping function that is very useful to do what you are asking for if your library follow the PSR-x standards. It is also possible to autoload non-psr-x libraries following a similar procedure.

Here the reference: https://getcomposer.org/doc/04-schema.md#autoload

Let's make an example (assuming your library follows the PSR-4 standard)

Suppose you have cloned the submodule in the folder lib/your-private-git-repo.

All you have to do is to add to your composer.json file autoload section:

{
    "name": "YourNamespace/YourApplicationName",
    "description": "Describe your library",
    "license": "Your licen",
    "keywords": ["some","keywords"],
    "authors": [
        {
            "name": "Adamo Aerendir Crespi",
            "email": "hello@aerendir.me"
        }
    ],
    "require": {
        "joomla/framework": "*@stable"
    },
    "require-dev": {
        "phpunit/phpunit": "*@stable"
    },
    "autoload": {
        "psr-4": {
            "YourNamespace\\YourSubmodule\\": "lib/your-private-git-repo/src"
        }
    }
}

Note the lines form the sixth line from the bottom of the code

    "autoload": {
        "psr-4": {
            "YourNamespace\\YourSubmodule\\": "lib/your-private-git-repo/src"
        }
    }

Now update Composer

php composer.phar update

In this way Composer will update the autoload file including the submodule too.

All done: now your submodule is autoloaded by Composer.

Hope this will help.

Rubens Mariuzzo
  • 28,358
  • 27
  • 121
  • 148
Aerendir
  • 6,152
  • 9
  • 55
  • 108
  • 1
    This was not the question of OP. The problem is that if you run `composer install` it won't install possible dependencies mentioned in `composer.json` of your submodule. – Adam Nov 22 '19 at 17:39
2

Desired behavior:

  • You have 2 Zend projects: project_A and project_B
  • both project use the same common library named library_C
  • library_C is required from project_A and project_B trough the composer
  • library_C should be editable in vendor folder in project_A and project_B and you should be able to push those changes
  • your git URL of library is https://github.com/stuff/library_C.git

How to do it:

  1. Put you library inside composer.json of project_A and project_B
"require": {
    "stuff/library_C": "master"
},

2.run composer install. Now your library is in folder vendor/stuff/library_C

3.Now when you want to edit something inside library_C in project_A or project_B, delete folder vendor/stuff/library_C and run this command

git submodule add -f https://github.com/stuff/library_C.git vendor/stuff/library_C

this command will recreate this folder vendor/stuff/library_C, but this folder is now another git repo (submodule)

4.Go to folder vendor/stuff/library_C change anything commit it and push, AND THAT IS ALL!!!

After that when you run composer update stuff/library_C in other project the code will be updated.

Note: after you run git submodule add you will be on a branch master so you must checkout to proper branch or tag and do a change there.

fico7489
  • 7,931
  • 7
  • 55
  • 89
0

If you have a project A including a git submodule , then calling composer install/update.. from project A will ignore all the composer dependencies from your submodule.

In short, do not use a git submodule if it contains composer dependencies.

This is how I would rather do it:

1) If its not an official package (like a fork or a private repository) add the repository to composer.json:

"repositories": [
    {
        "type": "vcs",
        "url": "https://github.com/vendor_name/package_name.git"
    }
],

2) Require the package with --prefere-source:

composer require vendor_name/package_name --prefer-source

Note: If you fork official/mango into your own repository ralf/mango, then you need to point to ralf/mango in composer.json, but call composer require official/mango ..` You can read details in the docs.

Now you may access files from within vendor and push pull etc. The --prefere-source command already setup the git connection inside the vendor file.

If you want to access the package form your root folder like package/mango instead of vendor/official/mango you could set up a symlink.

Adam
  • 25,960
  • 22
  • 158
  • 247