214

Composer has the option to load several dependencies only while being in development, so the tools will not be installed in production (on the live server). This is (in theory) very handy for scripts that only make sense in development, like tests, fake-data-tools, debugger, etc.

The way to go is to add an additional require-dev block with the tools you need in dev:

"require-dev": {
    "codeception/codeception": "1.6.0.3"
}

and then (theoretically) load these dependencies via

composer install --dev

Problem & Question:

Composer has changed the behaviour of install and update dramatically in 2013, require-dev-dependencies are now installed by default (!), feel free to create a composer.json with a require-dev block and perform an composer install to reproduce.

As the most accepted way to deploy is to push the composer.lock (that holds your current composer setup) and then do an composer install on the production server, this will also install the development stuff.

What's the correct way to deploy this without installing the -dev dependencies ?

Note: I'm trying to create a canonical Q/A here to clarify the weird Composer deployment. Feel free to edit this question.

Sliq
  • 15,937
  • 27
  • 110
  • 143
  • @all: Don't know where the bounty is :( I'll start another approach. – Sliq Feb 23 '14 at 10:16
  • 1
    If you don't actively award it, and no answer gets accepted or gets enough upvotes, nobody gets the bounty. – Sven Feb 23 '14 at 10:59
  • @JasperN.Brouwer Sorry, didn't looked into my account for some time. There's just a very short time window for that. I'll request an option on meta.stackflow.com for this issue! Sorry... – Sliq May 22 '14 at 12:42
  • 2
    I personally don't like this approach at all. The `composer.lock` should never be added to the Git repo, NEVER. The right approach is to use composer update on staging and then synch the file into production (if everything works, of course). Staging must be the exact copy of a production environment. `composer.lock` should be part of `.gitignore`. – noun Jan 19 '17 at 17:48
  • 22
    composer.lock has definitly to be included in your CSV!!! How else you make sure everybody uses the same version?? So NEVER exclude composer.lock from your CSV!!! – Tobias Gaertner Dec 06 '17 at 10:59
  • 8
    @TobiasGaertner I think you mean VCS (version control software), but otherwise you're correct and in-line with [the project's official recommendations](https://getcomposer.org/doc/01-basic-usage.md#commit-your-composer-lock-file-to-version-control). – Xiong Chiamiov Aug 14 '18 at 18:09
  • @ Xiong Chiamiov: you are absolutly right! It has to be VCS of caurse... :-) – Tobias Gaertner Aug 15 '18 at 11:01
  • @noun it would be interesting if you could add an answer to clarify **why** you want to avoid putting `composer.lock` under version control – Nico Haase Apr 08 '21 at 09:56
  • @NicoHaase because it holds the current composer setup, of one developer, of his local env, and it might be outdated. You can even commit an older version by mistake. If you are working with docker, and you have a pipeline that runs a test suite when you do a pull request, then you want your code to be updated with the last version of any library and run the tests against it. E.g. you have v2.1.* in composer.json, you want 2.1.4 which includes fixes. You don't want 3.*, but you want 2.1.4. If you use composer.lock you are screwed. Commit composer.lock if you work alone and you deploy manually. – noun Apr 08 '21 at 16:08
  • No, that's exactly what you don't want: why should the test pipeline run with other dependencies than your local development? How would you even start to check for errors if each run possible uses other dependencies? I want to control when updates are used - do them on purpose, do this on all used machines in the same manner – Nico Haase Apr 08 '21 at 19:53

5 Answers5

404

Why

There is IMHO a good reason why Composer will use the --dev flag by default (on install and update) nowadays. Composer is mostly run in scenario's where this is desired behavior:

The basic Composer workflow is as follows:

  • A new project is started: composer.phar install --dev, json and lock files are commited to VCS.
  • Other developers start working on the project: checkout of VCS and composer.phar install --dev.
  • A developer adds dependancies: composer.phar require <package>, add --dev if you want the package in the require-dev section (and commit).
  • Others go along: (checkout and) composer.phar install --dev.
  • A developer wants newer versions of dependencies: composer.phar update --dev <package> (and commit).
  • Others go along: (checkout and) composer.phar install --dev.
  • Project is deployed: composer.phar install --no-dev

As you can see the --dev flag is used (far) more than the --no-dev flag, especially when the number of developers working on the project grows.

Production deploy

What's the correct way to deploy this without installing the "dev" dependencies?

Well, the composer.json and composer.lock file should be committed to VCS. Don't omit composer.lock because it contains important information on package-versions that should be used.

When performing a production deploy, you can pass the --no-dev flag to Composer:

composer.phar install --no-dev

The composer.lock file might contain information about dev-packages. This doesn't matter. The --no-dev flag will make sure those dev-packages are not installed.

When I say "production deploy", I mean a deploy that's aimed at being used in production. I'm not arguing whether a composer.phar install should be done on a production server, or on a staging server where things can be reviewed. That is not the scope of this answer. I'm merely pointing out how to composer.phar install without installing "dev" dependencies.

Offtopic

The --optimize-autoloader flag might also be desirable on production (it generates a class-map which will speed up autoloading in your application):

composer.phar install --no-dev --optimize-autoloader

Or when automated deployment is done:

composer.phar install --no-ansi --no-dev --no-interaction --no-plugins --no-progress --no-scripts --optimize-autoloader

If your codebase supports it, you could swap out --optimize-autoloader for --classmap-authoritative. More info here

Luciano Nascimento
  • 2,600
  • 2
  • 44
  • 80
Jasper N. Brouwer
  • 21,517
  • 4
  • 52
  • 76
  • 8
    I agree with most of what is said with one exception. "composer install --no-dev" should be executed only in a staging environment and that environment should be considered immutable. I wouldn't want to have any dependency downloaded directly at my production server and without going through preview/staging. That's just an extra bit of caution. – Scalable Mar 25 '15 at 01:14
  • 4
    @Scalable: Although I agree with you (and Sven covers this nicely in his answer), that's not the scope of my answer, and not what I meant with "production deploy". I've added a paragraph to make that clear. – Jasper N. Brouwer Mar 25 '15 at 08:16
  • 11
    Actually I think that the default should be the less dangerous option. Making --dev the default and accidentally doing a composer install in production could be fatal. – Hector Ordonez Jul 17 '15 at 00:11
  • 4
    Good point in the `--optimize-autoloader`. Consider also `--classmap-authoritative` - From the documentation here https://getcomposer.org/doc/03-cli.md you can see this: "Autoload classes from the classmap only. Implicitly enables --optimize-autoloader" so you can use if you know the classes "are there", which probably should happen in your prod environment unless you generate classes dynamically. – Xavi Montero Mar 24 '17 at 23:02
  • 8
    Great answer, I would suggest adding `optimize-autoloader` directly in the `composer.json`: `{"config": { "optimize-autoloader": true } }` – Yvan Jul 04 '17 at 14:57
  • 1
    For the automated deployment I would add --no-plugins for security reasons here: https://getcomposer.org/doc/faqs/how-to-install-untrusted-packages-safely.md#how-do-i-install-untrusted-packages-safely-is-it-safe-to-run-composer-as-superuser-or-root- – ddelrio1986 Apr 04 '18 at 18:10
  • What about --no-suggest in case you are using the output and want a cleaner parsing? – NaturalBornCamper Jun 03 '19 at 13:58
  • Why would you use `--no-scripts` during automated deployment ? – Cid Sep 26 '19 at 08:18
  • 1
    @Cid In my experience there's often some sort of build/deploy pipeline/street in place, which provides a more granular way to ensure the correct steps are taken. In that context having Composer install dependencies is one of those steps, running scripts are separate steps. There's nothing wrong with letting Composer handle this if it's sufficient for your case. – Jasper N. Brouwer Sep 26 '19 at 10:56
  • @JasperN.Brouwer In fact, I couldn't find any other way than the scripts in composer.json to automate the clearing and warming up of the cache of a symfony application using Azure DevOps pipelines. – Cid Sep 26 '19 at 11:43
  • This answer is outdated, because Composer 2 automatically installs dev dependencies. Composer 3 will remove the dev flag. – mbomb007 Mar 23 '21 at 18:56
95

Actually, I would highly recommend AGAINST installing dependencies on the production server.

My recommendation is to checkout the code on a deployment machine, install dependencies as needed (this includes NOT installing dev dependencies if the code goes to production), and then move all the files to the target machine.

Why?

  • on shared hosting, you might not be able to get to a command line
  • even if you did, PHP might be restricted there in terms of commands, memory or network access
  • repository CLI tools (Git, Svn) are likely to not be installed, which would fail if your lock file has recorded a dependency to checkout a certain commit instead of downloading that commit as ZIP (you used --prefer-source, or Composer had no other way to get that version)
  • if your production machine is more like a small test server (think Amazon EC2 micro instance) there is probably not even enough memory installed to execute composer install
  • while composer tries to no break things, how do you feel about ending with a partially broken production website because some random dependency could not be loaded during Composers install phase

Long story short: Use Composer in an environment you can control. Your development machine does qualify because you already have all the things that are needed to operate Composer.

What's the correct way to deploy this without installing the -dev dependencies?

The command to use is

composer install --no-dev

This will work in any environment, be it the production server itself, or a deployment machine, or the development machine that is supposed to do a last check to find whether any dev requirement is incorrectly used for the real software.

The command will not install, or actively uninstall, the dev requirements declared in the composer.lock file.

If you don't mind deploying development software components on a production server, running composer install would do the same job, but simply increase the amount of bytes moved around, and also create a bigger autoloader declaration.

Community
  • 1
  • 1
Sven
  • 69,403
  • 10
  • 107
  • 109
  • 20
    Interesting workflow, but there's a big **con**: Repositories should never contain the vendor folder/contents itself (official statements on Composer page), so they will never be directly pushed to production in a git-based deployment (which is a common standard afaik, correct me if i'm wrong). So basically the above solution only works with "old-school" FTP-deployment !? Please let's discuss this further... – Sliq Feb 20 '14 at 16:39
  • 24
    My suggested workflow does not include pushing the code via GIT to the production server. In fact, I would recommend against, because doing so will force you to install Composer dependencies on the production server, which can bring up any number of issues. If you want your deployment to run smoothly, you have to assemble all the code needed to run the application before you destroy the current version and replace it. Don't like FTP? RSync via SSH, then switch versions by flipping a symlink. But you can also push, checkout and composer install in prod if you want to. – Sven Feb 20 '14 at 22:06
  • 3
    @Panique: I've just seen that part of your comment and I have to answer: "pushed to production in a git-based deployment (which is a common standard afaik, correct me if i'm wrong)" - No, this is not common standard. It is just one way to do it. – Sven Feb 22 '14 at 11:28
  • 2
    The team I'm on has incorporated this into their workflow with great success. We have a build machine (Jenkins, of course) which: 1) checks out from SC 2) runs composer install/update 3) runs unit tests 4) removes dev dependencies 5) generates a phar file (`app-1.34.phar` etc). There's a separate mechanism that is notified and decides when to grab that file, where to transfer it to, and then what to do with it. Some teams choose to have the phar unpacked once it's on the server and some teams run it as-is. It's lent a lot of confidence to the stability and reproducibility of our deploys. – Josh Johnson Dec 14 '16 at 19:24
  • In most cases the "install" strategy works fine but with a larger server I'm doing a build cycle that generates "Docker image" which is then synced and installed on servers. Obviously it's easy to roll-back to previous image, also switchover between images happens instantly so no downtime occurs. – romaninsh Jan 17 '17 at 10:43
  • 3
    I agree 100% with this answer. Composer should not be installed on the deployment server, nor git. Continuous Deployment/Intergration servers are exactly supposed to manage the source and dependencies fetching : git pull > composer install > deploy – Eric MORAND Nov 15 '17 at 13:35
  • I agree with Eric. You should use some deploy-Server or script (whatever) to do the deploy. get the code -> composer install --no-dev -> deploy. You can write a script or better use automated Tools like Jenkins, deploy.hq, bitbucket-pipelines, and there are many more... – Tobias Gaertner Dec 06 '17 at 11:05
  • Do you think is ok if within my team we directly publish the whole `vendor` directory instead of installing the dependencies server-side on the production machine? I am wondering what is the best practice to deploy composer packages used by the application. – tonix Aug 16 '18 at 09:54
  • From bashing my head against memory issues in VMs I 100% agree with you. `composer` struggles. `yarn` / `npm` / `webpack` often die in those constrained scenarios. – datashaman May 13 '19 at 04:33
  • `composer install` is very lightweight if you have `composer.lock` in your project, I highly doubt that you will get any memory-related problems on any usable server. – rob006 May 13 '19 at 08:09
  • @Sven That's a nice discussion. Most sites I develop use shared hosts, which has many restrictions. I maintain a server in AWS and I [never precompile assets in production](https://stackoverflow.com/questions/44990957/rails-is-precompile-assets-after-deploy-amazon-ec2), because it does server down (many assets). With Ruby on Rails, I precompile assets locally and send to GIT. With PHP, I use Gulp to precompile local production assets and send to GIT. I don't send development assets to GIT (nor PHP nor Ruby). Depending your app, `composer install/update` will not down the server (most cases). – Diego Somar Feb 26 '20 at 19:48
  • Here is another set of rules 1) Don't use shared-hosting for serious projects that needs "no downtime deployments". 2) Don't rely on a single php server (instance) - use them in an auto scaling group with a load balancer. 3) Don't deploy on an instance that is an active part of load balancer, block traffic first. 4) Don't allow traffic if deployment is failed or if health checks are failing. 5) Make your autoscaling group bring another instance up if the deployment has failed 6) Hire me, so I could make it working for you in a few hours. :) – Yevgeniy Afanasyev Sep 22 '20 at 23:54
5

Now require-dev is enabled by default, for local development you can do composer install and composer update without the --dev option.

When you want to deploy to production, you'll need to make sure composer.lock doesn't have any packages that came from require-dev.

You can do this with

composer update --no-dev

Once you've tested locally with --no-dev you can deploy everything to production and install based on the composer.lock. You need the --no-dev option again here, otherwise composer will say "The lock file does not contain require-dev information".

composer install --no-dev

Note: Be careful with anything that has the potential to introduce differences between dev and production! I generally try to avoid require-dev wherever possible, as including dev tools isn't a big overhead.

dave1010
  • 15,135
  • 7
  • 67
  • 64
  • 2
    This is actually incorrect in the details. There is no need to check `composer.lock` for dev dependencies. You'd simply run `composer install --no-dev`, and you'll get only the regular dependencies installed - in fact, Composer will also remove any dev dependencies in this step. – Sven Feb 19 '14 at 15:07
  • If my local `composer.lock` had dev dependencies in it (and potentially affected the versions of non-dev packages) then I'd want to update it to reflect how it would be in production. This also forces you to run `composer install --no-dev` in production, as `composer install` will error. Technically I think you're right; this isn't required, but it is an extra level of safety, which I like. – dave1010 Feb 20 '14 at 10:23
  • Ok, demo scenario: Your app requires `dev/tool` and `prod/lib:~1.0`. The newest prod/lib is 1.3, but dev/tool also requires `prod/lib:1.1.*`. Result: You will install version 1.1.9 (newest of 1.1.x branch) and use it during your development. I would say it is NOT safe to just update `--no-dev`, thus include the newest prod/lib 1.3 and assume everything works without testing. And maybe testing is then impossible because of the lack of dev/tool. I would assume that because dev/tool is not needed in production, it should not be rolled out, but the software must use prod/lib 1.1.9 then. – Sven Feb 20 '14 at 21:58
  • If you're using `--no-dev` then you need to test it locally, as I mentioned in the answer. I'd still recommend not using `--no-dev` at all though. – dave1010 Feb 25 '14 at 14:14
  • So basically you suggest this: `composer update`, then do some development, then do `composer update --no-dev`, then do the release testing, then push to production and do `composer install --no-dev`. Two problems: 1. I cannot test the release without dev dependencies, and 2. I cannot install with for example Git in production. – Sven Feb 25 '14 at 19:57
  • To test properly you need to make sure you you have the same dependencies locally as you would in production. If you can't test the release properly without dev dependencies then IMO you should require dev dependencies in production. Being 100% sure your dev dependencies aren't affecting the rest of the code is difficult. – dave1010 Feb 26 '14 at 13:57
  • I am pretty sure that PHPUnit is no production part of my software, so it does not belong to the regular `require` list - and testing without the dev requirements would also be impossible. – Sven Mar 04 '14 at 21:08
3

I think is better automate the process:

Add the composer.lock file in your git repository, make sure you use composer.phar install --no-dev when you release, but in you dev machine you could use any composer command without concerns, this will no go to production, the production will base its dependencies in the lock file.

On the server you checkout this specific version or label, and run all the tests before replace the app, if the tests pass you continue the deployment.

If the test depend on dev dependencies, as composer do not have a test scope dependency, a not much elegant solution could be run the test with the dev dependencies (composer.phar install), remove the vendor library, run composer.phar install --no-dev again, this will use cached dependencies so is faster. But that is a hack if you know the concept of scopes in other build tools

Automate this and forget the rest, go drink a beer :-)

PS.: As in the @Sven comment bellow, is not a good idea not checkout the composer.lock file, because this will make composer install work as composer update.

You could do that automation with http://deployer.org/ it is a simple tool.

datashaman
  • 8,301
  • 3
  • 22
  • 29
Giovanni Silva
  • 705
  • 5
  • 17
  • 3
    Not committing and checking out `composer.lock` will make `composer install` act like `composer update`. So the versions you deploy are not the ones you developed with. This is likely to generate trouble (and more so in the light of the only recently solved security issue with "replace" in Composer). You should NEVER run `composer update` unattended without verifying it did not break anything. – Sven Feb 22 '14 at 11:22
  • 1
    @Sven this is way a suggest in the same comment to run Unit tests automatically before the deploy. But you are right, it's better keep the composer.lock file anyway. – Giovanni Silva Feb 23 '14 at 12:04
  • Now the only thing you'd have to explain: How do you run the tests on the server without the dev dependencies like PHPUnit? – Sven Feb 23 '14 at 12:46
  • Would be very nice, if dependencies, tests and deployment were placed together in one tool, like Java Gradle or SBT or even Maven (maven is not so good). One PHP tool that make composer phpunit and deployment work together. Or even a Gradle or Scala SBT plugin to make this things, as they are agnostic build tools, the plugin could even work with assets like minimising javascript and compiling sass, minimising css. Someone know something? – Giovanni Silva Feb 23 '14 at 12:47
  • If the test depend on dev dependencies, as composer do not have a test scope dependency, a not much elegant solution could be run the test with the dev dependencies, remove the vendor library, run composer.phar --install --no-dev again this will use cached dependencies. But that is a hack if you know the concept of scopes in other build tools. – Giovanni Silva Feb 23 '14 at 12:55
  • 1
    Of course this is done in the server to test the real environment, but not direct in the site vhost, you could to this in a separated temporary folder and move the result to the vhost when succeed – Giovanni Silva Feb 23 '14 at 12:59
  • @GiovanniSilva url appears to be broken. is the tool you referenced the same as http://deployer.org/ ? – Rob Olmos Apr 26 '16 at 05:02
2

On production servers I rename vendor to vendor-<datetime>, and during deployment will have two vendor dirs.

A HTTP cookie causes my system to choose the new vendor autoload.php, and after testing I do a fully atomic/instant switch between them to disable the old vendor dir for all future requests, then I delete the previous dir a few days later.

This avoids any problem caused by filesystem caches I'm using in apache/php, and also allows any active PHP code to continue using the previous vendor dir.

Despite other answers recommending against it, I personally run composer install on the server, since this is faster than rsync from my staging area (a VM on my laptop).

I use --no-dev --no-scripts --optimize-autoloader. You should read the docs for each one to check if this is appropriate on your environment.

Abhi Beckert
  • 32,787
  • 12
  • 83
  • 110