615

Is it possible to use jQuery selectors/DOM manipulation on the server-side using Node.js?

hexacyanide
  • 88,222
  • 31
  • 159
  • 162
John
  • 6,167
  • 3
  • 16
  • 3
  • 4
    I wonder: Why use on the server-side while you can do it on the client-side? – Inanc Gumus May 13 '14 at 12:12
  • 40
    Perhaps you might want to create a web scrapper that scraps specific information at regular intervals and stores the results in a database? This would not be as practical from the client-side. – Trevor Jun 05 '14 at 03:59
  • 3
    You should also take a look at phantomjs, which allow you to emulate a browser server side with V8 engine. – Dimitri Kopriwa Jan 06 '15 at 15:28
  • 2
    @deeperx DOM manipulation on the server side could be useful when making a crawler. See [this answer](http://stackoverflow.com/a/31361044/778272). – Lucio Paiva Jul 11 '15 at 19:45
  • YES - have a look at [this answer](https://stackoverflow.com/a/50231309/814620) - I prefer this over using cheerio, since you get the full power of jQuery selector. – monika mevenkamp Jan 22 '20 at 20:29

21 Answers21

585

Yes. First, install the dependencies

npm install jquery jsdom

then

var jsdom = require('jsdom');
const { JSDOM } = jsdom;
const { window } = new JSDOM();
const { document } = (new JSDOM('')).window;
global.document = document;

var $ = jQuery = require('jquery')(window);

See this answer.

Boris Verkhovskiy
  • 14,854
  • 11
  • 100
  • 103
Philippe Rathé
  • 8,888
  • 3
  • 22
  • 13
  • 45
    Is it possible to use jQuery ajax from node.js with that npm module? – ajsie Mar 01 '11 at 21:02
  • where should we add that code index.js ? – mercury May 30 '22 at 21:16
  • @ajsie I was able to do `$.ajax('http://127.0.0.1:8888')` after starting a server on port 8888 with `nc -kl 8888`. Trying to query `$.ajax('http://example.com')` fails with `Error: Cross origin null forbidden` – Boris Verkhovskiy May 29 '23 at 20:39
  • 1
    @mercury add that code to index.js and the run `node index.js`. Or you can name it any other filename and pass that filename to `node` – Boris Verkhovskiy May 29 '23 at 20:47
59

Yes you can, using a library I created called nodeQuery

var Express = require('express')
    , dnode = require('dnode')
    , nQuery = require('nodeQuery')
    , express = Express.createServer();

var app = function ($) {
    $.on('ready', function () {
        // do some stuff to the dom in real-time
        $('body').append('Hello World');
        $('body').append('<input type="text" />');
        $('input').live('click', function () {
            console.log('input clicked');
            // ...
        });
    });
};

nQuery
    .use(app);

express
    .use(nQuery.middleware)
    .use(Express.static(__dirname + '/public'))
    .listen(3000);

dnode(nQuery.middleware).listen(express);
Tien Duong
  • 2,517
  • 1
  • 10
  • 27
Thomas Blobaum
  • 3,680
  • 1
  • 17
  • 16
  • 21
    Note that nodeQuery is actually changing the page of the user in real time, so it's even cooler than one might expect. – alessioalex Nov 04 '11 at 20:01
  • I was searching something like this when I stumbled here... I have just looked at nQuery and jquery node packages and nQuery was updated a year ago where jquery was yesterday... Is nQuery no longer developed? and does jquery affect client side as nQuery does? Has anybody tried them both maybe? – Logan Dec 02 '12 at 08:23
  • 3
    @Logan nQuery is basically just jquery. the difference is the code is run on the server and, rather than delivering the jquery code to the browser, it runs the code on the server and remotely executes dom manipulation to connected browsers. Also note that nQuery was an experimental project, and while I will accept pull requests to fix bugs, it was never created for any specific purpose or project so it has not had many commits – Thomas Blobaum Dec 06 '12 at 15:13
  • @ThomasBlobaum not working for me, error : `, express = Express.createServer();` and `TypeError: Express.createServer is not a function` any idea ? – questionasker Aug 08 '17 at 11:02
  • @ThomasBlobaum looks like you haven't got the latest version of Express. Try `npm install --save express` in your command prompt. – gilbert-v May 31 '19 at 09:52
57

At the time of writing there also is the maintained Cheerio.

Fast, flexible, and lean implementation of core jQuery designed specifically for the server.

Alfred
  • 60,935
  • 33
  • 147
  • 186
38

A simple crawler using Cheerio

This is my formula to make a simple crawler in Node.js. It is the main reason for wanting to do DOM manipulation on the server side and probably it's the reason why you got here.

First, use request to download the page to be parsed. When the download is complete, handle it to cheerio and begin DOM manipulation just like using jQuery.

Working example:

var
    request = require('request'),
    cheerio = require('cheerio');

function parse(url) {
    request(url, function (error, response, body) {
        var
            $ = cheerio.load(body);

        $('.question-summary .question-hyperlink').each(function () {
            console.info($(this).text());
        });
    })
}

parse('http://stackoverflow.com/');

This example will print to the console all top questions showing on SO home page. This is why I love Node.js and its community. It couldn't get easier than that :-)

Install dependencies:

npm install request cheerio

And run (assuming the script above is in file crawler.js):

node crawler.js


Encoding

Some pages will have non-english content in a certain encoding and you will need to decode it to UTF-8. For instance, a page in brazilian portuguese (or any other language of latin origin) will likely be encoded in ISO-8859-1 (a.k.a. "latin1"). When decoding is needed, I tell request not to interpret the content in any way and instead use iconv-lite to do the job.

Working example:

var
    request = require('request'),
    iconv = require('iconv-lite'),
    cheerio = require('cheerio');

var
    PAGE_ENCODING = 'utf-8'; // change to match page encoding

function parse(url) {
    request({
        url: url,
        encoding: null  // do not interpret content yet
    }, function (error, response, body) {
        var
            $ = cheerio.load(iconv.decode(body, PAGE_ENCODING));

        $('.question-summary .question-hyperlink').each(function () {
            console.info($(this).text());
        });
    })
}

parse('http://stackoverflow.com/');

Before running, install dependencies:

npm install request iconv-lite cheerio

And then finally:

node crawler.js


Following links

The next step would be to follow links. Say you want to list all posters from each top question on SO. You have to first list all top questions (example above) and then enter each link, parsing each question's page to get the list of involved users.

When you start following links, a callback hell can begin. To avoid that, you should use some kind of promises, futures or whatever. I always keep async in my toolbelt. So, here is a full example of a crawler using async:

var
    url = require('url'),
    request = require('request'),
    async = require('async'),
    cheerio = require('cheerio');

var
    baseUrl = 'http://stackoverflow.com/';

// Gets a page and returns a callback with a $ object
function getPage(url, parseFn) {
    request({
        url: url
    }, function (error, response, body) {
        parseFn(cheerio.load(body))
    });
}

getPage(baseUrl, function ($) {
    var
        questions;

    // Get list of questions
    questions = $('.question-summary .question-hyperlink').map(function () {
        return {
            title: $(this).text(),
            url: url.resolve(baseUrl, $(this).attr('href'))
        };
    }).get().slice(0, 5); // limit to the top 5 questions

    // For each question
    async.map(questions, function (question, questionDone) {

        getPage(question.url, function ($$) {

            // Get list of users
            question.users = $$('.post-signature .user-details a').map(function () {
                return $$(this).text();
            }).get();

            questionDone(null, question);
        });

    }, function (err, questionsWithPosters) {

        // This function is called by async when all questions have been parsed

        questionsWithPosters.forEach(function (question) {

            // Prints each question along with its user list
            console.info(question.title);
            question.users.forEach(function (user) {
                console.info('\t%s', user);
            });
        });
    });
});

Before running:

npm install request async cheerio

Run a test:

node crawler.js

Sample output:

Is it possible to pause a Docker image build?
    conradk
    Thomasleveil
PHP Image Crop Issue
    Elyor
    Houston Molinar
Add two object in rails
    user1670773
    Makoto
    max
Asymmetric encryption discrepancy - Android vs Java
    Cookie Monster
    Wand Maker
Objective-C: Adding 10 seconds to timer in SpriteKit
    Christian K Rider

And that's the basic you should know to start making your own crawlers :-)


Libraries used

Lucio Paiva
  • 19,015
  • 11
  • 82
  • 104
22

in 2016 things are way easier. install jquery to node.js with your console:

npm install jquery

bind it to the variable $ (for example - i am used to it) in your node.js code:

var $ = require("jquery");

do stuff:

$.ajax({
    url: 'gimme_json.php',
    dataType: 'json',
    method: 'GET',
    data: { "now" : true }
});

also works for gulp as it is based on node.js.

low_rents
  • 4,481
  • 3
  • 27
  • 55
  • What version of node are you using? On Mac, Node 6.10.2, jquery 2.2.4, `var $ = require("jquery"); $.ajax // undefined` (Downvoted for the moment). – AJP May 09 '17 at 14:18
  • @AJP and you are sure you did `npm install jquery` first? – low_rents May 10 '17 at 07:41
  • 2
    Yes. `> console.log(require("jquery").toString());` gives me the factory function: `function ( w ) { if ( !w.document ) { throw new Error( "jQuery requires a window with a document" ); } return factory( w ); }` I had to use the answer above with jsdom: http://stackoverflow.com/a/4129032/539490 – AJP May 10 '17 at 09:35
  • @AJP ok, that's strange. – low_rents May 10 '17 at 12:04
  • I get exactly the same factory function as @AJP does. What version of jquery did you use, @low_rents? – Boris Burkov Feb 15 '18 at 00:55
20

I believe the answer to this is now yes.
https://github.com/tmpvar/jsdom

var navigator = { userAgent: "node-js" };  
var jQuery = require("./node-jquery").jQueryInit(window, navigator);
Matt
  • 74,352
  • 26
  • 153
  • 180
rdzah
  • 233
  • 2
  • 2
  • 10
    I'm sorry to report that it is going to take more work to get jQuery running on jsdom. Sizzle however does work! I really want to keep jsdom as light as possible, so adding in full browser emulation like env.js is not really a priority at this time. – tmpvar May 04 '10 at 14:34
  • never mind, i found the modified copy that's bundled with jsdom. – drewish Feb 27 '12 at 21:18
  • FYI node-jquery is deprecated now in favor of jquery – Ruslan López Apr 25 '18 at 06:52
  • 1
    ReferenceError: window is not defined – Bonn Aug 07 '19 at 10:39
17

npm install jquery --save #note ALL LOWERCASE

npm install jsdom --save

const jsdom = require("jsdom");
const dom = new jsdom.JSDOM(`<!DOCTYPE html>`);
var $ = require("jquery")(dom.window);


$.getJSON('https://api.github.com/users/nhambayi',function(data) {
  console.log(data);
});
Edgard Leal
  • 2,592
  • 26
  • 30
Noah
  • 481
  • 5
  • 15
10

jQuery module can be installed using:

npm install jquery

Example:

var $ = require('jquery');
var http = require('http');

var options = {
    host: 'jquery.com',
    port: 80,
    path: '/'
};

var html = '';
http.get(options, function(res) {
res.on('data', function(data) {
    // collect the data chunks to the variable named "html"
    html += data;
}).on('end', function() {
    // the whole of webpage data has been collected. parsing time!
    var title = $(html).find('title').text();
    console.log(title);
 });
});

References of jQuery in Node.js** :

marc_s
  • 732,580
  • 175
  • 1,330
  • 1,459
SUNDARRAJAN K
  • 2,237
  • 2
  • 22
  • 38
  • 3
    Doesn't work for me... C:\...\\node_modules\jquery\dist\jquery.js:31 throw new Error( "jQuery requires a window with a document" ); ^ Error: jQuery requires a window with a document at module.exports (C:\...\WebContent\resources\js\node_modules\jquery\dist\jquery.js:31:12) – Jose Manuel Gomez Alvarez Oct 20 '16 at 17:59
  • var jsdom = require("jsdom"); var window = jsdom.jsdom().defaultView; jsdom.jQueryify(window, "http://code.jquery.com/jquery.js", function () { var $ = window.$; $("body").prepend("

    The title

    "); console.log($("h1").html()); });
    – SUNDARRAJAN K Oct 20 '16 at 18:06
8

You have to get the window using the new JSDOM API.

const jsdom = require("jsdom");
const { window } = new jsdom.JSDOM(`...`);
var $ = require("jquery")(window);
asp
  • 743
  • 2
  • 9
  • 29
webcpu
  • 3,174
  • 2
  • 24
  • 17
6

First of all install it

npm install jquery -S

After installing it, you can use it as below

import $ from 'jquery';
window.jQuery = window.$ = $;
$(selector).hide();

You can check out a full tutorial that I wrote here: https://medium.com/fbdevclagos/how-to-use-jquery-on-node-df731bd6abc7

dbc
  • 104,963
  • 20
  • 228
  • 340
Oyetoke Tobi
  • 148
  • 3
  • 9
2

WARNING

This solution, as mentioned by Golo Roden is not correct. It is just a quick fix to help people to have their actual jQuery code running using a Node app structure, but it's not Node philosophy because the jQuery is still running on the client side instead of on the server side. I'm sorry for giving a wrong answer.


You can also render Jade with node and put your jQuery code inside. Here is the code of the jade file:

!!! 5
html(lang="en")
  head
    title Holamundo!
    script(type='text/javascript', src='http://code.jquery.com/jquery-1.9.1.js')
  body
    h1#headTitle Hello, World
    p#content This is an example of Jade.
    script
      $('#headTitle').click(function() {
        $(this).hide();
      });
      $('#content').click(function() {
        $(this).hide();
      });
Community
  • 1
  • 1
Timbergus
  • 3,167
  • 2
  • 36
  • 35
2

My working code is:

npm install jquery

and then:

global.jQuery   = require('jquery');
global.$        = global.jQuery;

or if the window is present, then:

typeof window !== "undefined" ? window : this;
window.jQuery   = require('jquery');
window.$        = window.jQuery;
Roman
  • 19,236
  • 15
  • 93
  • 97
2

None of these solutions has helped me in my Electron App.

My solution (workaround):

npm install jquery

In your index.js file:

var jQuery = $ = require('jquery');

In your .js files write yours jQuery functions in this way:

jQuery(document).ready(function() {
Elio Osés
  • 29
  • 3
1

The module jsdom is a great tool. But if you want to evaluate entire pages and do some funky stuff on them server side I suggest running them in their own context:

vm.runInContext

So things like require / CommonJS on site will not blow your Node process itself.

You can find documentation here. Cheers!

hexacyanide
  • 88,222
  • 31
  • 159
  • 162
Jakub Oboza
  • 5,291
  • 1
  • 19
  • 10
1

As of jsdom v10, .env() function is deprecated. I did it like below after trying a lot of things to require jquery:

var jsdom = require('jsdom');
const { JSDOM } = jsdom;
const { window } = new JSDOM();
const { document } = (new JSDOM('')).window;
global.document = document;

var $ = jQuery = require('jquery')(window);

Hope this helps you or anyone who has been facing these types of issues.

Plabon Dutta
  • 6,819
  • 3
  • 29
  • 33
  • `TypeError: JSDOM is not a constructor` – Nathan Hawks Dec 30 '19 at 20:37
  • If you're running jQuery at Node side, firstly, install jquery and jsdom using npm install. Then, add above lines in the file you're trying to use jquery selector in. For instance, I used a `$.each`. I just included these lines and then did it like below: `$.each(errors, function (ind,error) { res.send(error.msg);console.log(error.msg); });` Hope this helps !! – Plabon Dutta Dec 31 '19 at 11:10
  • Somehow jsdom had decided not to install at all. I guess I'm still figuring npm out. Thanks@ – Nathan Hawks Dec 31 '19 at 15:35
1

Yes, jQuery can be used with Node.js.

Steps to include jQuery in node project:-

npm i jquery --save Include jquery in codes

import jQuery from 'jquery';

const $ = jQuery;

I do use jquery in node.js projects all the time specifically in the chrome extension's project.

e.g. https://github.com/fxnoob/gesture-control-chrome-extension/blob/master/src/default_plugins/tab.js

fxnoob
  • 166
  • 8
1

I did it manually easy way without any additional packages or code.

npm i jquery

then I copy the jquery.min.js file from node_modules/jquery/dist directory to public/js

<script type='text/javascript' src='/js/jquery.min.js'></script>
<script>
  $(document).ready(function() { console.log( "ready!" ); });
</script>

And it will work. TEST IT

Note copy/pasting the file is not the ideal thing, you could enable the file as a static file by enabling it as a static so expressJS could read it. But it is easier for me to just copy it to the static public directory.

mercury
  • 2,390
  • 3
  • 30
  • 39
-1

No. It's going to be quite a big effort to port a browser environment to node.

Another approach, that I'm currently investigating for unit testing, is to create "Mock" version of jQuery that provides callbacks whenever a selector is called.

This way you could unit test your jQuery plugins without actually having a DOM. You'll still have to test in real browsers to see if your code works in the wild, but if you discover browser specific issues, you can easily "mock" those in your unit tests as well.

I'll push something to github.com/felixge once it's ready to show.

Felix Geisendörfer
  • 2,902
  • 5
  • 27
  • 36
-2

You can use Electron, it allows hybrid browserjs and nodejs.

Before, I tried to use canvas2d in nodejs, but finally I gave up. It's not supported by nodejs default, and too hard to install it (many many ... dependeces). Until I use Electron, I can easily use all my previous browserjs code, even WebGL, and pass the result value(eg. result base64 image data) to nodejs code.

gonnavis
  • 336
  • 1
  • 7
  • 16
-8

Not that I know of. The DOM is a client side thing (jQuery doesn't parse the HTML, but the DOM).

Here are some current Node.js projects:

https://github.com/ry/node/wiki (https://github.com/nodejs/node)

And SimonW's djangode is pretty damn cool...

RamC
  • 1,287
  • 1
  • 11
  • 15
Nosredna
  • 83,000
  • 15
  • 95
  • 122
  • I wish it was possible. I already tried including jquery on a node.js project and of course it didn't work. jQuery is based on document/window. Rhino is capable of running jQuery server side: http://ejohn.org/blog/bringing-the-browser-to-the-server/ I'm going to look for more parsers. Maybe there is one that doesn't depend on the browser. – John Nov 26 '09 at 02:46
  • @John: the only reason jQuery can run on Rhino is because of this project: http://github.com/jeresig/env-js/blob/master/src/env.js It simulates a small portion of the DOM and the JavaScript runtime. It relies on Java apis so is a no-go for Node.js (which uses V8/C++). – Crescent Fresh Nov 26 '09 at 04:13
  • 2
    @Nosredna While this may have been true when you wrote it, it is clearly not true any more. I suggest that you delete your answer now. – Keith Pinson Mar 08 '14 at 15:25
-19

An alternative is to use Underscore.js. It should provide what you might have wanted server-side from JQuery.

Matt
  • 74,352
  • 26
  • 153
  • 180
John Wright
  • 2,418
  • 4
  • 29
  • 34