3

Because MYSQL's "SELECT" selects integers and floats as strings, and I need every response I get (from JS) to be in a correct data model -

  • 1 not "1",
  • 53.2 not "53.2",

I created this recursive function that works on mixed type - array/object:

private function cast_number(&$mixed) {
    if(is_array($mixed)) {
        foreach ($mixed as $key => $val)
            if (is_numeric($val))
                $mixed[$key] = (double)$val;
            else if (is_array($val) || is_object($val))
                $mixed[$key] = $this->cast_number($val);
    } else if(is_object($mixed)) {
        foreach ($mixed as $key => $val)
            if (is_numeric($val))
                $mixed->$key = (double)$val;
            else if (is_array($val) || is_object($val))
                $mixed->$key = $this->cast_number($val);
    }
    return $mixed;
}

Pretty simple function - if is number, cast double, if is array or object, go over recursivly.

Everything here is in place.

I have two problems with this: - On 6MB of data, of mostly numbers that are represented as strings it took 0.5 seconds - On 200MB of data (yes, I need it, please don't focus on it), it failed after a couple of minutes (usually seconds), saying it needs more than 4GB of memory..

  1. How can I improve this function? (Speed, Memory)
  2. Why is it taking so long? even json_encode, that I would think is a bigger function takes much less time..
mickmackusa
  • 43,625
  • 12
  • 83
  • 136
Amit
  • 5,924
  • 7
  • 46
  • 94
  • Did you try type juggling instead : `$mixed[$key] = 0 + $val;` ? – Ivan Gabriele Jun 19 '16 at 15:06
  • @IvanGabriele Not yet. I don't understand how it would help the speed? Sure, the memory will be pretty much 4 bytes instead of 8 most of the time. Am trying now – Amit Jun 19 '16 at 15:16
  • There is another solution to your problem. It's to update mysqlnd on the server so PHP gets the correct datatype from MySQL... [Here's more info](http://stackoverflow.com/questions/1197005/how-to-get-numeric-types-from-mysql-using-pdo) – M. Eriksson Jun 19 '16 at 15:33
  • you might also want to check with the folks over at http://codereview.stackexchange.com/ – Jeff Puckett Jun 19 '16 at 15:39
  • @Amit I gave you a full answer to show you different timings, as you can see it's quite lowering the execution time if combined with variables by reference. – Ivan Gabriele Jun 19 '16 at 16:14
  • I'm confused as to where the problem is. I mix strings and numbers in MySQL in PHP all the time, yet rarely get into trouble. An `INT` is happy to take `"123"`. PHP converts strings to number on the fly in _most_ cases. Etc. Please provide a short code snippet that illustrates the issue. – Rick James Jun 27 '16 at 15:39

2 Answers2

2

Because coercion used to be faster than casting, I ran this code to calculate timings on PHP 7 :

function getTime($start) {
    return round((microtime(true) - $start) * 1000000) / 1000;
}

function mockData($length) {
    $data = [];
    $i = -1;

    while ($i++ < $length) {
        $data[$i] = strval(rand(1, 10000) / 100);
    }

    return $data;
}

$data = mockData(100000);

// Let's check that they are string before
echo gettype($data[0]) . '<br><br>';

$start = microtime(true);
$convertedData = [];
foreach ($data as $key => $value) {
    $convertedData[$key] = (double) $value;
}
echo '(double) cast took ' . getTime($start) . ' ms.<br>';

$start = microtime(true);
$convertedData = [];
foreach ($data as $key => $value) {
    $convertedData[$key] = 0 + $value;
}
echo 'Coercion took ' . getTime($start) . ' ms.<br>';

And my results are :

(double) cast took 27.508 ms.
Coercion took 28.789 ms.

CONCLUSION

Since using floatval (a third way to achieve string to double conversion) is even longer, you can't do better than that with PHP. What you're trying to achieve is a scripting operation, it shouldn't be used as a normal back-end operation for a web application.

But if you still want to do that, you can higher your memory_limit inside your php.ini file, as long as you don't use the -1 workaround.

UPDATE

I forgot one possible optimization, you should pass your variable by reference to at least execute immediate assignation :

$start = microtime(true);
foreach ($data as $key => $value) {
    $data[$key] = (double) $value;
}
echo getTime($start) . ' ms.<br>';

=> 34.018 ms.

$start = microtime(true);
foreach ($data as &$value) {
    $value = (double) $value;
}
echo getTime($start) . ' ms.<br>';

=> 17.081 ms.

And apparently using coercion with by reference gives even better results :

$start = microtime(true);
foreach ($data as &$value) {
    $value = 0 + $value;
}
echo getTime($start) . ' ms.<br>';

=> 13.1 ms.

Community
  • 1
  • 1
Ivan Gabriele
  • 6,433
  • 5
  • 39
  • 60
  • These are very nice comparisons! But it is probably on PHP7.. I forgot to mention, I use 5.5.34 (can't upgrade), and here is an example: http://sandbox.onlinephpfunctions.com/code/5b261993efbc969cfd7c4df9f03a1b5bb0d6b460 - it takes 0.35 seconds and looks very optimized.. it looks like it does not take too much memory now which is great – Amit Jun 19 '16 at 16:38
  • @Amit that's exactly the same function than the one I found for 13.1 ms... Anyway it won't improve your memory problems which are linked with the fact that your original data size is way too big. You'll still have to higher your `memory_limit`. You should [read that](http://stackoverflow.com/a/7993775/2736233) to understand how memory assignation work in PHP. – Ivan Gabriele Jun 19 '16 at 16:53
1

This should improve the speed and memory:

private function cast_number(&$mixed)
{
    foreach ($mixed as &$val) {
        if (is_numeric($val)) {
            $val = (double) $val;
        } else if (is_array($val) || is_object($val)) {
            $this->cast_number($val);
        }
    }
    // do not return $mixed
}

but perhaps you could achieve greater improvements using array_walk_recursive function.

godvsdeity
  • 969
  • 8
  • 15
  • Although this works, it does not change the execution time by a lot. Also, array walk recursive cannot handle objects – Amit Jun 19 '16 at 16:36
  • this is why i didnt provide a solution with array_walk_recursive; it actually partially can, if your object implements `ArrayAccess` it can; – godvsdeity Jun 19 '16 at 16:42