2

So, I am setting up a new site and my project's folder structure looks like this now.

foo.com/
       index.php
       assets/
          css/
          img/
          js/
          vendor/

I have added vendor/ for js/css libraries that I must install to keep them separate, since I want anyone who installs my project to install those in vendor from package.json - most libraries contain too many files 99% which I don't want to push to production.

Now once the project is finished, I would like to push the code to production with only the necessary js/css files.

This is where the problem comes. For example, if I install bulma css using:

yarn add bulma --modules-folder ./assets/vendor

It will dump all bulma-related files which are almost 70 into /vendor/bulma/ but I will only be needing one or two css files afterwards, since I will compiles the sass file to css as:

sass  vendors/bulmna/style.scss assets/css/style.css 

So my questions is: I am assuming this is how every developer does it and there are no documentations I can find that suggest how to do it. Is it safe to ignore the /vendor directory? What if I install vue, font-awesome, bootstrap .. how can I only fetch the files I need but not everything in /vendors folder?

tk421
  • 5,775
  • 6
  • 23
  • 34
tikimti-mvrht
  • 232
  • 2
  • 12
  • 1
    By "commit" and "push" you mean doing that to your code into your repository? you can do that by adding the files you *dont* want to commit to a `.gitignore` file in that case. – DarkCygnus Jun 28 '17 at 23:22
  • @GrayCygnus Yes. I can use `.gitignore` to ignore for example the `vendor/`, this would work for bulma because with the command `sass vendors/bulmna/style.scss assets/css/style.css ` I have transfered everything I need from bulma from vendor to my css dir. But what about for vuejs for example? There is no way to transfer the library, there are many other fat libraries that I don't want to push to prod if I am not using 99% of them. I understand the question may not be clear, but I am asking more from a work-flow point of view. as in how to keep you project clean – tikimti-mvrht Jun 29 '17 at 17:57
  • Then you could use more something like Python `virtualenv` (not on PC now to google for its equivallent on node), so you have isolated environments each with its own dependencies and packages. Does it sound like something that you could use? – DarkCygnus Jun 29 '17 at 20:55
  • you can also have a `dist/` folder (which you `.gitignore`) - however you may need to add documentation to notify future developers that a build step is required to output the desired production files from that folder – Denis Tsoi Jun 30 '17 at 04:31
  • @GrayCygnus I am not using Python. This is a JS/HTML project so I can't see how that helps – tikimti-mvrht Jun 30 '17 at 06:54
  • @DenisTsoi This is the closest answer to my question I have gotten. But why does the process sound new or outside the norm? How do all the front-end developers maintain their libraries? I thought they ignored the vendors and uploaded the lib and the dependancies in package.json then people would install the main library and later the dependancies. – tikimti-mvrht Jun 30 '17 at 06:56
  • `npm install --dev library` - then when you're ready to release, you have a script in your package.json like `npm build` which bundles everything and puts it all to `dist/` from a `.gitignore` - you don't want to include `node_modules` or `dist/` – Denis Tsoi Jun 30 '17 at 07:08
  • @DenisTsoi Can you please provide an answer for this? I thought you understood my question well. What if you were building a project from scratch and needed to include vuejs, jquery, fontawesome .... But you probably need just a couple of files. I think you are on the right track with the answer, can you please expand it a bit – tikimti-mvrht Jun 30 '17 at 09:41
  • @tikimti-mvrht *points below* – Denis Tsoi Jun 30 '17 at 10:42

4 Answers4

2

Your question is actually quite broad - however, I'll try to list as much as possible.


Lets say you're building a project from scratch and needed to include vuejs, jquery, fontawesome but only need to include a couple of files.

The issue you're hitting here is module dependency with respect to npm modules. (and there are many different tools that you can use to manage versions on your library dependencies as well as ensuring they are included into your project, i.e. package managers).

Ok - now from here, you may say to yourself

but I only need say, one icon from fontawesome in your final build (dist) and I don't want to commit all my modules into source control

Again, this is where you omit node_modules and other dependent libraries from source control (i.e. include node_modules your .gitignore)

To reiterate

  1. You can install the required library,
  2. add node_modules to .gitignore ,
  3. bundle those libraries into a vendor single file to be consumed by your users (can be via browserify/webpack/rollup/gulp/grunt/yarn etc).
  4. generate bundle within npm script

Now you may ask further -

Why would I use any of those tools? - they're distracting me from simply copy/pasting vendor libaries into my source control.

Build tools were created to

  1. streamline the developer pipeline so that you DONT have to copy/paste vendor libaries into a vendor folder.
  2. ensures that all files are bundled to the client automatically
  3. allows you to keep track/restrict library version updates/ when required via package.json
  4. allows you to add other build steps (such as minification, md5hash versioning, compression, code splitting, asset management to name a few).

Now lets break down the original question here:

  1. How to ensure other developers get everything they need when cloning the repository
  2. how do I ensure I can provide only the necessary files to the end user (if I only use specific parts of vendor libaries?)

1. How to ensure developers get what they need

Again, to reiterate above, adding devDependancies and .gitignoring allows you to only add the necessary files to your project.

2. How can I ensure clients get what they need without bloating request files?

This is where build tools such as webpack, browserify, gulp, grunt, rollup, attempt to achieve. This is because with some libraries that exceed in file size of 200kb minified, you may need to separate these files into different client requests (and as such, not demand the user to request one large file, which was symtomatic of browserify projects).

The second technique you will need to be aware of, is with specific libraries, you can use import mdn link where you can require one function/class from a dependant library (which further reduces file size).

Another technique is using less import statements (which can extract less functions/styles similar to above, but this isn't currently supported in SCSS). for SCSS, you're basically left with copy/pasting the necessary styles into your base scss and that'll save you space as well.

EDIT How to create a bundle from npm install libaries

From the comments you've mentioned above (about not wanting to include a tool into your workflow, there's no point outlining any one particular strategy - you can find answers/tutorials online about how to setup gulp/browserify/webpack for your particular needs).

However, As you are currently using yarn - I'll go into details about that.

Firstly, yarn is a package manager (like npm). All it does with the --modules-folder is install the package into the specified folder, that's all. So we don't really care about that (since it's doing the same thing as npm). (i.e. your vendor folder is the same as node_modules in many respects).

We could use

  1. webpack
  2. gulp
  3. grunt
  4. rollup
  5. browserify
  6. brunch

(All build tools that essentially allow you to bundle all those packages into a single entry point to be packaged to the client).

I won't go into how, because that is a process you can find online, and from the above comments, I can tell you don't particularly care either.

What you are looking for is a zero-config javascript build tool. (Extremely out of the scope of your original question, and i'll only answer that in a separate Q&A).

Denis Tsoi
  • 9,428
  • 8
  • 37
  • 56
  • So far so good, but you missed out on the important stuff, which is how is the building from modules done? Personally, I download vue, bootstrap, fontawesome using `yarn add vue --modules-folder ./assets/vendor/` now where do I got from here? Because you can see in my question I have folders {css, js, images} in my `assets/` dir. So, how do I build those libraries from /vendor to their respect destination in css/js/images? For bulma it's super simple, just get what you need from vendors with `sass vendors/bulmna/style.scss assets/css/style.css` for other libraries, you say build .. but how? – tikimti-mvrht Jun 30 '17 at 12:33
  • Fair enough. When I heard things like webpack, gulp, grunt ... I never had an idea what they were used for or how. Since I used yarn/npm I assumed they were just variants of those. So now the mystery to my answer lies in those tools? I guess I better check one out. Problem is that my vbox does not support symlinks and I can't user webpack unless I install it globally (i.e. symlinks) but I will give it another shot – tikimti-mvrht Jun 30 '17 at 16:37
  • re symlinks: This post might be helpful https://stackoverflow.com/questions/18641864/git-bash-shell-fails-to-create-symbolic-links – JGFMK Jul 01 '17 at 07:31
1

I'd try Googling on "tree shaking CSS" to see if that gives you something. Maybe something like: https://github.com/jacobp100/es-css-modules

JGFMK
  • 8,425
  • 4
  • 58
  • 92
  • This means I have yet to add another library just to keep what I need from vendors. I will look if there are better alternatives. – tikimti-mvrht Jun 30 '17 at 09:37
1

Rollup plugin may be useful. It works for js, with postcss, the link says it works with css also.

https://code.lengstorf.com/learn-rollup-css

Amit Kumar Singh
  • 4,393
  • 2
  • 9
  • 22
  • This means I have yet to add another library just to keep what I need from vendors. I will look if there are better alternatives. – tikimti-mvrht Jun 30 '17 at 09:37
1

Have a look at Pancake. It has been built specifically for the purpose of moving only those files out of the node_modules folder that you need. I wrote an article about it here: https://medium.com/dailyjs/npm-and-the-front-end-950c79fc22ce

(probably not an answer but a good tip)

PS: I am the author of both, the article and the tool so with clear bias :)

Dominik
  • 6,078
  • 8
  • 37
  • 61
  • This involves installing yet another tool to handle a problem which is less inconvenient than adding library. I'm sure there are better way to handle this, or let's see – tikimti-mvrht Jun 30 '17 at 12:41