NodeJS - Retrieving Post Data Without Browser

Question

I am having an issue attempting to receive post data without first running the html page from a web browser. I was wondering if there is a way using NodeJS that someone could run a given html document silently retrieving only the Post data and the console output. I need this in order to allow development in a cloud or remote server environment.

Current Usage Server.js :

var spawn = require('child_process').spawn; // Will not do for the situation of development within something similar to an Ubuntu Server
spawn(command, ["http://localhost:80/Develop/Client.html"]);
http.createServer(mainArgs).listen(options); // mainArgs is the function (req,res) callback

function mainArgs(req,res){
    if (req.method == 'POST') { // NodeJS is Getting what is posted...
        console.log("POST");

        var url_parts = url.parse(req.url, true);
        currentQuery = url_parts.query;

        req.on('end', function () {
            console.log("Body: " + currentQuery['stubScript']);
            writeHTML(currentQuery['stubScript']);
        });
    }

    ..... // HTML, JavaScript, & CSS Handled here
}

Current Usage Client.html :

<html>
     <head>
          //Posts data via XMLHttpRequest()
          <script src=devScript.js></script>
     </head>
     <body>
          <script>
               // Access Devscript functions and build page using javascript
          </script>
     </body>
</html>

Current Usage devScript.js :

function postIt(varname, variable) {

    var req = new XMLHttpRequest();
    var getParms = '?'+ varname + '=' + variable;

    req.open('POST', document.URL + getParms, true);

    req.onreadystatechange = function(){
        if (req.readyState == 4 && req.status == 200)
        {
            alert('PHPcall() : JSObject =' + varname );
        }
    };

    req.send(getParms);


} // Client is posting...

score 1 · Answer 1 · edited May 23 '17 at 12:34

You are looking to send an html request.

You can do this with a module such as request or with terminal programs such as wget or curl.

However if you need to do it without any modules (I advise using a module, they are sane in many ways the following isn't) you can do something like what is discussed in this SO thread by using the native http and querystring modules.

The following is the code from that thread:

// We need this to build our post string
var querystring = require('querystring');
var http = require('http');
var fs = require('fs');

function PostCode(codestring) {
  // Build the post string from an object
  var post_data = querystring.stringify({
      'compilation_level' : 'ADVANCED_OPTIMIZATIONS',
      'output_format': 'json',
      'output_info': 'compiled_code',
      'warning_level' : 'QUIET',
      'js_code' : codestring
  });

  // An object of options to indicate where to post to
  var post_options = {
      host: 'closure-compiler.appspot.com',
      port: '80',
      path: '/compile',
      method: 'POST',
      headers: {
          'Content-Type': 'application/x-www-form-urlencoded',
          'Content-Length': Buffer.byteLength(post_data)
      }
  };

  // Set up the request
  var post_req = http.request(post_options, function(res) {
      res.setEncoding('utf8');
      res.on('data', function (chunk) {
          console.log('Response: ' + chunk);
      });
  });

  // post the data
  post_req.write(post_data);
  post_req.end();

}

// This is an async file read
fs.readFile('LinkedList.js', 'utf-8', function (err, data) {
  if (err) {
    // If this were just a small part of the application, you would
    // want to handle this differently, maybe throwing an exception
    // for the caller to handle. Since the file is absolutely essential
    // to the program's functionality, we're going to exit with a fatal
    // error instead.
    console.log("FATAL An error occurred trying to read in the file: " + err);
    process.exit(-2);
  }
  // Make sure there's data before we post it
  if(data) {
    PostCode(data);
  }
  else {
    console.log("No data to post");
    process.exit(-1);
  }
});

However I strongly recommend using request / wget / curl as the error handling is centralised, things like forms (multipart, urlencoded etc.), json data, and Headers are correctly implemented on the 'client' side, drastically reducing the number of bugs you will encounter.

I am looking for strictly a way to do what I need without using middleware. I am sure there is a way of doing the same thing using blanket NodeJS, but I am having difficulty finding it. — StoneAgeCoder, Dec 26 '15 at 23:38
@StoneAgeCoder I have edited my answer to include a reference to doing it with "pure" nodejs. It demonstrates POSTing to a google service, you will of course want to POST to some other address. — egoate, Dec 27 '15 at 00:50
The request module is a good tool I agree, but it does not provide what I am looking for neither does the above code. I may be trying to do something that is impossible. I would figure that node would have functionality which allows for execution of javascript code and html without needing a browser. Strictly for server side testing purposes for developers. I am using an Ubuntu Server and enjoy the console environment. I can still test the client by opening a browser on my client and navigating to the url, but it would make sense to embed this functionality into Node. — StoneAgeCoder, Dec 27 '15 at 18:59
@StoneAgeCoder you mean you want to fake being a browser so you can run js on the front end? Look at some of the things in this list: https://github.com/dhamaniasad/HeadlessBrowsers — egoate, Dec 28 '15 at 22:53

score 1 · Answer 2 · edited May 23 '17 at 10:27

1

So you want to run a process and POST the stdout to a URL? Your current usage would pass the URL as an argument to the command that your spawning. So that process would be responsible for sending the POST data to the URL.

To get the output from the spawn process, see: 15339379

To make a POST request, see: 6158933

If those answers are helpful make sure to upvote them.

Update: Sounds like you want to accept a POST request; see this blog post.

edited May 23 '17 at 10:27

Community

1
1

answered Dec 26 '15 at 21:09

cynicaljoy

2,047
1
18
25

I have edited my question to better explain the problem with more code usage within my program. I am actually trying to GET a POST variable from the client without the spawn method actually opening a web browser. I only want the post data, nothing visual. – StoneAgeCoder Dec 26 '15 at 23:36

NodeJS - Retrieving Post Data Without Browser

2 Answers2