Anyway I can get the html body in node.js

Question

I try to send a request to an Apache server and use the returned body.

Follow the manual of nodejs http://nodejs.org/api/https.html#https_https_request_options_callback

and related SO content How to make external HTTP requests with Node.js

My src is

callback = function(response) {
  body='';
  console.log('HEADERS: ' + JSON.stringify(response.headers));
  response.on('data', function (chunk) {
    body+= chunk;
  }); 

  response.on('end', function () {
    console.log(body);
  }); 
}

My question is that the body here will include all the HTML header tag, such as <!DOCTYPE html>...etc, which cannot be parsed by JSON object. Because all my data in the HTML body is JSOn, I want to get only the HTML body. Are there anyway to achieve this goal? Thanks in advance.

I am confused. Whatever returned is already HTML only, right? — thefourtheye, Dec 24 '13 at 04:42
response body is not HTML page body. You would have to scrape the page to extract contents and then JSON.parse it. — user568109, Dec 24 '13 at 05:00
There are some package for html query in node.js: cherio or jquery. https://github.com/MatthewMueller/cheerio — damphat, Dec 24 '13 at 05:31
@thefourtheye You are right, but I am trying to get the 'body' of HTML — StevenR, Dec 24 '13 at 06:04
@StevenR So, you are only interested in everything insider ``? — thefourtheye, Dec 24 '13 at 06:07
In the end, I can get the json only by set the HTTP header "Accept" to "application/json" — StevenR, Dec 24 '13 at 09:31

thefourtheye · Accepted Answer · 2013-12-24T09:42:32.070

4

I am not giving complete code here.

You can use modules like htmlparser2 to extract the data from HTML's body (Online demo for the same).
And then you can use JSON.parse to parse the extracted string to a JSON Object.

edited Dec 24 '13 at 09:42

answered Dec 24 '13 at 04:40

thefourtheye

233,700
52
457
497

Thanks! I will also try this one. – StevenR Dec 24 '13 at 10:09
@StevenR This is the way to go. There may be some regex based solutions, but they are not reliable. If you want to extract data from HTML, you just have to parse it. – thefourtheye Dec 24 '13 at 10:12

Anyway I can get the html body in node.js

1 Answers1