Partial POSTED Data in sails.js (From time to time)

Question

I have a production server(ubuntu), where a user fills and sends a form (multipart), to either create or update a record. Now sometimes when I try to use req.allParams() I get an object with incomplete data.

Lets say this is what I usually get:

data: {
    id: '58155',
    name: 'The Gallery Name',
    email: 'gallery@gmail.com',
    phone: '(+54911)68460005',
    url: 'www.theurl.com',
    workingHours: 'Tuesday - Friday 3 - 8 pm',
    artists: ['58350', '15503', '58346', '58347', '58348', '58349'],
    locations: [{ 
      country: 1,
      state: null,
      city: 45,
      zip: '33175',
      address: '' }
    ],
    showOnGuide: true,
    preferredLanguage: 'en',
    events: [1, 4, 5]
  }
};

But from time to time i get:

   data: {
        id: '58155',
        name: 'The Gallery Name',
        email: 'gallery@gmail.com',
        phone: '(+54911)68460005',
        url: 'www.theurl.com',
        workingHours: 'Tuesday - Friday 3 - 8 pm',
        artists: ['58350', '15503', '58346', '58347', '58348', '58349']
      }
    };

I've checked the forms and they are correct, so the only thing that's obvious is that only the last keys (form fields) are missing, so I'm assuming it might be related to some POST timeout?

Just to be clear, this is how I get the data, prior to any formatting done on my part, and even placing a log in sails/node_modules/skipper/index.js

/// .. some code
MultipartBodyParser(req, res, function(err) {
    if (err) return next(err);
    console.log(req.body);
/// ... some code
}

shows partial information from time to time. I haven't seen this behavior on my local machine, but as I said it's rare, like 1 every 30 times but still pretty disastrous for the client to loose sent data.

This seems like an issue with sails, express or skipper.

I've been trying to debug this for a couple of days now and that's the closest I've gotten, since at the beginning I knew nothing of why or when it was happening, now I know that the form is sent correctly, yet sails either receives it incomplete or parses it incorrectly.

Have anyone come across with a similar issue? Any thoughts that lead to resolving this is most welcome.

Versions:

Node 0.12
Sails 0.11.3

Note: This happens with or without uploading a file

EDIT: By using a network manager (NetBalancer) I've tried reducing the upload speed as much as possible 1 Byte/s but I'm not getting the weird behavior, so it doesn't seem that a slow network upload is related, at least from the manual testing, I'm not sure how reliable NetBalancer is either.

Update: Updated sails to 0.12.1 and node to 5.8.0, but the problem persists.

Thanks

They are usually sent, just a few times they get chopped empty or not. And only happens in reverse order, so the last 2 or 3 fields don't get send or parsed — Lu Roman, Mar 14 '16 at 14:52

Lu Roman · Accepted Answer · 2016-05-05T21:46:46.903

I realized what the issue was. It pretty much presented only on connections with low upload speed and specially when uploading images, the problem is how skipper handles form data when it detects a file input (it seems that it happens even if the input file is empty) it closes the text input streams, so they never get captured. Now, the solution is then format your form in such a way that file inputs are placed after all of other inputs (Since the html spec dictates that browsers must send inputs in order), but it's not always possible to do this without making some heavy style changes, so the solution i came up is using this little snippet:

(function () {
    var $form = $('form[enctype="multipart/form-data"]');
    var $fileInputs = $($form.find('input[type="file"]'));
    $form.submit(function() {
      $fileInputs.detach();
      $form.append($fileInputs);
    })
  })();

It detaches the file inputs from the form and append them at the end, so they are always sent last.

I believe sails have a note about the importance of the input ordering, but its quite easy to miss, they should place a huge warning on their homepage about this...

Partial POSTED Data in sails.js (From time to time)

1 Answers1