PHP some $_POST values missing but are present in php://input

Question

I have a very big html form (containing table with rows, which contain multiple inputs), which i need to submit to PHP script via POST request. The problem is some values don't come through and are absent in PHP's $_POST superglobal.

I checked (using Firebug extension) that the values are actually sent to server by the browser.

$_POST gets populated, but some values are just missing.

I checked what is raw request using:

$raw_post = file_get_contents('php://input');

and the string returned has the values. They are just not parsed into $_POST array. The strange thing i noticed is, it seems that the php://input values are cut after some length, and rest of the string does not come through to $_POST.

I thought about post_max_size and memory_limit and set them to large values:

memory_limit = 256M
post_max_size = 150M

but according to php documentation $_POST should not contain any values if request made is bigger than post_max_size.

Due to big size of form and request I cannot post it here, but i can post php script i used to debug the problem:

var_dump($file = file_get_contents('php://input'));
var_dump($_POST);
//... then i parsed the $file

Server version: Apache/2.2.9 (Debian)
PHP version: PHP 5.3.2-0.dotdeb.2

Can enyone explain reason of such strange PHP behaviour, and what should i do (change php settings, code?) to use $_POST array while processing form?

EDIT: To be clear: not only the values are missing. $_POST does not contain these keys either.

e.x. fragment of raw post:

t_dodparam%5B198%5D=&t_dodparam2%5B198%5D=&t_kolejnosc%5B198%5D=199&n_indeks=201&n_wartosc=testtesttest

Key 't_dodparam' is in post and it has key 198. The rest of parameters are missing (e.x. t_dodparam2 is in post, but it has no such key as 198, and there is no such key as n_wartosc in $_POST)

Answer 1

PHP modifies fields containing the characters space, dot, open square bracket and others to be compatible with with the deprecated register_globals

you can find a lot of workarounds in the comments here: PHP: Variables From External Sources

For Exampe (comment by POSTer):

<?php
//Function to fix up PHP's messing up POST input containing dots, etc.
function getRealPOST() {
    $pairs = explode("&", file_get_contents("php://input"));
    $vars = array();
    foreach ($pairs as $pair) {
        $nv = explode("=", $pair);
        $name = urldecode($nv[0]);
        $value = urldecode($nv[1]);
        $vars[$name] = $value;
    }
    return $vars;
}
?>

Answer 2

I just fixed this issue by adding a value to max_input_vars in my PHP configuration file. According to this. it was introduced in 5.3.9, yet after some package upgrades I experienced the issue in 5.3.2.

The default for max_input_vars is 1000, which was too small for my form.

Answer 3

There are many different things that could be causing this. Best to check your error log. Many of the things that cause this symptom will put messages in the error log, but not display errors in your PHP application.

Some of the possible causes:

Suhosin

Suhosin is an extension for PHP designed to protect servers and users from known and unknown flaws in PHP applications and the PHP core. One of the things that it does is limit the size of $_POST, $_GET, $_REQUEST, and $_COOKIE. If your problem is Suhosin you will get an error in your log such as

ALERT - configured POST variable limit exceeded - dropped variable 'foo'

The solution is simple, just increase the maximum number of allowed variables in php.ini. If you don’t have a suhosin section, just create one. Like such:

[suhosin]
suhosin.request.max_vars = 1000 # Default is 200
suhosin.post.max_vars = 1000 # Default is 200

There are other suhosin settings that can cause this, but these are the most likely candidates.

Invalid form field names

Back in the old days PHP had a setting called register_globals (now depricated) that automatically converted GET and POST variables into PHP variables. So if your form had fields 'foo' and 'bar', when that form is submitted the variables $foo and $bar would be automatically created. However there are several characters that are not valid for use in PHP variable names (space, dot, open square bracket and others). Depending on what characters you used, the variable may have the invalid characters stripped, be missing its value, or be unset. Despite the fact that register_globals is no longer used, PHP still strips these characters when building $_POST, $_GET, $_REQUEST, and $_COOKIE. If you are unable to fix the values of the form fields you might try something like:

<?php
/**
 * Converts raw POST data into an array.
 * 
 * Does not work for hierarchical POST data.
 *
 * @return array
 */
function real_post() {
  static $post;
  if (!isset($post)) {
    $pairs = explode("&", file_get_contents("php://input"));
    $post = array();
    foreach ($pairs as $pair) {
      $x = explode("=", $pair);
      $post[rawurldecode($x[0])] = rawurldecode($x[1]);
    }
  }
  return $post;
}
?>

Answer 4

What about using "parse_str" to convert the query string into php structures? This funcion is the inverse of http_build_query.

    $b = array();
    parse_str(file_get_contents("php://input"), $b);

Answer 5

I found this answer via a search and feel I should offer an alternative issue. In my case, my post wasn't too large, and yet the value I was submitting in the field was not showing up in the $_POST array. As it turned out, I accidentally had another field further down in my form with the same name. So:

<form>
   <input type="text" name="field1">
   <input type="text" name="field2">
   <input type="text" name="field3">
   <input type="text" name="field1">
   <input type="submit">
</form>

When the $_POST variable is populate with data from that form, the value in your first field will be overwritten with the value in that last field with the same name. If the first field is required and you fill in a value, but the last field is not required and gets submitted empty, you will see similar symptoms as this question because the value $_POST['field1'] will show the value of the last element in you form which is empty.

TLDR:

Make sure to check for duplicate field names in your form!

Answer 6

I am posting Jquery's .ajax() function. I was trying to retrieve my data from $_POST but it was incomplete. Then I found this post which put me on the right track. However, the getRealPOST() method described above wasn't working for me - it can't handle multi-dimentional and nested arrays very well. Instead I used PHP's parse_str() method which did the trick and was a bit cleaner:

$rawdata = file_get_contents('php://input');
$urldecoded = urldecode($rawdata);
parse_str($urldecoded, $parsed);

$data = $parsed['data'];

(In my JS I'm posting an object where the payload is in the data property. Yours would be likely be different.)

I also went into my php.ini and cranked up max_memory and max_input_vars, but it didn't seem to solve my issue. I also spent a while chasing a red herring because I was using my error log to print the raw data, and I forgot that error_log has a limit on how many characters it will print unless you remember to increase it.

Anyway, hope this helps to anyone who finds themselves battling this issue.

Answer 7

use this function for getting true data

function get_real_post() {

    function set_nested_value(&$arr, &$keys, &$value) {
        $key = array_shift($keys);
        if (count($keys)) {
            // Got deeper to go
            if (!array_key_exists($key, $arr)) {
                // Make sure we can get deeper if we've not hit this key before
                $arr[$key] = array();
            } elseif (!is_array($arr[$key])) {
                // This should never be relevant for well formed input data
                throw new Exception("Setting a value and an array with the same key: $key");
            }
            set_nested_value($arr[$key], $keys, $value);
        } elseif (empty($key)) {
            // Setting an Array
            $arr[] = $value;
        } else {
            // Setting an Object
            $arr[$key] = $value;
        }
    }

    $input = array();
    $parts = array();

    $rawdata=file_get_contents("php://input");
    $rawdata=urldecode($rawdata);
    $pairs = explode("&", $rawdata);
    foreach ($pairs as $pair) {
        $key_value = explode("=", $pair, 2);
        preg_match_all("/([a-zA-Z0-9_]*)(?:\[([^\[\]]*(?:(?R)[^\[\]]*)*)\])?/", urldecode($key_value[0]), $parts);
        $keys = array($parts[1][0]);
        if (isset($parts[2][0])&&$parts[2][0]!="") {
            array_pop($parts[2]); // Remove the blank one on the end
            $keys = array_merge($keys, $parts[2]);
        }
        $value = urldecode($key_value[1]);
        if ($value == "true") {
            $value = true;
        } else if ($value == "false") {
            $value = false;
        } else if (is_numeric($value)) {
            if (strpos($value, ".") !== false) {
                $num = floatval($value);
            } else {
                $num = intval($value);
            }
            if (strval($num) === $value) {
                $value = $num;
            }
        }
        set_nested_value($input, $keys, $value);
    }
    return $input;
}

Answer 8

For future reference: On an Ubuntu box, I've been struggling with this issue since a relatively long time and used to adopt a workaround similar as described above to save the day. I now tracked the issue down in my php.ini and finally found it in line with max_input_nesting_level = 0 I commented the line above, restarted apache and all's fixed.

Answer 9

I had this same issue and it turned out that I was using AJAX to dynamically modify the input field based on other inputs in the form. The ajax function re-created the input and failed to include an input name.

Answer 10

Came across this (admittedly old) post while trying to find a fix for the bug of Javascript Object keys including square brackets in their names and then PHP getting confused and acting like it's never even heard of nesting. @Dalin has a good basic response, but it doesn't make a nested array, nor does it convert value types to boolean / number - hence my version (get_real_post() is also on GitHub).

Despite the name, this should work identically for _GET (ie, anything that php://input grabs).

/**
 * Gets the _POST data with correct handling of nested brackets:
 * "path[to][data[nested]]=value"
 * "path"
 *    -> "to"
 *       -> "data[nested]" = value
 * @return array
 */
function get_real_post() {

    function set_nested_value(&$arr, &$keys, &$value) {
        $key = array_shift($keys);
        if (count($keys)) {
            // Got deeper to go
            if (!array_key_exists($key, $arr)) {
                // Make sure we can get deeper if we've not hit this key before
                $arr[$key] = array();
            } elseif (!is_array($arr[$key])) {
                // This should never be relevant for well formed input data
                throw new Exception("Setting a value and an array with the same key: $key");
            }
            set_nested_value($arr[$key], $keys, $value);
        } elseif (empty($key)) {
            // Setting an Array
            $arr[] = $value;
        } else {
            // Setting an Object
            $arr[$key] = $value;
        }
    }

    $input = array();
    $parts = array();
    $pairs = explode("&", file_get_contents("php://input"));
    foreach ($pairs as $pair) {
        $key_value = explode("=", $pair, 2);
        preg_match_all("/([a-zA-Z0-9]*)(?:\[([^\[\]]*(?:(?R)[^\[\]]*)*)\])?/", urldecode($key_value[0]), $parts);
        $keys = array($parts[1][0]);
        if (!empty($parts[2][0])) {
            array_pop($parts[2]); // Remove the blank one on the end
            $keys = array_merge($keys, $parts[2]);
        }
        $value = urldecode($key_value[1]);
        if ($value == "true") {
            $value = true;
        } else if ($value == "false") {
            $value = false;
        } else if (is_numeric($value)) {
            if (strpos($value, ".") !== false) {
                $num = floatval($value);
            } else {
                $num = intval($value);
            }
            if (strval($num) === $value) {
                $value = $num;
            }
        }
        set_nested_value($input, $keys, $value);
    }
    return $input;
}

Answer 11

I faced the same problem. My form was creating the input elements in a loop. Upto 1000 input elements were getting posted and print_r($_POST); showed all the posted values.

When the input elements in the form numbered above 1000, print_r($_POST) just vanished. As if the form was not getting posted at all. I reached your page of stackoverflow.com by google search.

Yes increasing post_max_size, memory_limit etc did not solve anything. I have practical experience that increasing memory_limit beyond 128M can be dangerous too. Once our live Apache hosting server got hung up. I was the experimenter :-)

max_input_vars was the only limiting factor.

echo ini_get('max_input_vars');

showed that it was 1000 by default. Strangely max_input_vars was absent in the php.ini. Anyway I added

max_input_vars=10000 

in my php.ini and restarted Apache.

Now echo ini_get('max_input_vars'); showed that it had increased to 10000 and my large form data could be trapped after posting. Problem solved.

I see that the max_input_vars is PHP_INI_PERDIR type. Which means I cannot change its value for the individual php page where I need a higher value using ini_set('max_input_vars', 10000);

Answer 12

I had a similar issue and modifying the MAX_INPUT_VARS fixed it.

It's worth checking what you send, and what is received. In my case, the AJAX call contained a formatted array of all the values but the VAR_DUMP($this->input->post()); revealed missing values.

So, the obvious answer was as someone here said - max_input_vars.

Answer 13

You need to change the max_input_vars value of the php.ini file

step 1: Locate the php.ini file
step 1(a): Print php info values to find out all the PHP predefined configurations.

echo phpinfo(); //will print the php info values

step 1(b): Make a note of the php.ini file location from the phpinfo values printed in the previous step as shown in the image below

step 2: Edit the php.ini file.
step 2(a): Find the variable max_input_vars and remove the semicolon before it and set the value to 10000 like below

max_input_vars = 10000

step 3: Restart apache

sudo service apache2 restart

Answer 14

I faced same problem and i found a very valuable solution without increasing any limit or size in .ini. We know that max_input_vars = 1000 by default and this is prefect. Just do one thing concatenate array values and try to precise your number of variables under 1000. Each variable name count one not their values. For example:

<code>
$i = 10; //counts 1
$i = array(1,2,3,4,5); //counts 5
</code>

You can do this:

<code>
$i = 1_2_3_4_5; // counts 1
</code>

Hope understanding.