PHP Tokenizer multiline issue

Question

I am using token_get_all to develop a tool. I am stuck in a situation where I have following query in php code

$sql = "UPDATE `key_values` SET
                `Value_Content` = '" . $this->db->escape($revisionValues['value']) . "',
                `Comments` = '" . $this->db->escape($revisionValues['comment']) . "',
                `Is_Active` = '" . $this->db->escape($revisionValues['actstate']) . "',
                `Is_Modified`='1'
                WHERE
                `Key_Value`='" . $candidateKey['key'] . "'
                AND `Email_Template`='" . $candidateKey['template'] . "'
                AND `Locale_ID`='" . $candidateKey['locale'] . "'";

and another code

$array = array(
    "foo" => "bar",
    "bar" => "foo",
);

I want to treat this as one line. Am not able to detect end of line in multiline code as mentioned above. Is there any way to detect it. I need some identifier which tells me that this multiline sql query is one single line for php.

Not 100% sure about your question, but if you just need something to make it more clear for you to read why not use `<< — PhearOfRayne, Dec 18 '12 at 07:41

score 0 · Answer 1 · edited May 23 '17 at 12:12

You are making a single statement in which you declare newlines yourself. So your variable contains newlines because you put them there. Now you have two options:

1: Dont put the newlines in

$sql = "UPDATE `key_values` SET ".
            "`Value_Content` = '" . $this->db->escape($revisionValues['value']) . "', ".
            "`Comments` = '" . $this->db->escape($revisionValues['comment']) . "', ".
            "`Is_Active` = '" . $this->db->escape($revisionValues['actstate']) . "', ".
            "`Is_Modified`='1' ".
            "WHERE ".
            "`Key_Value`='" . $candidateKey['key'] . "' ".
            "AND `Email_Template`='" . $candidateKey['template'] . "' ".
            "AND `Locale_ID`='" . $candidateKey['locale'] . "'";

2: remove them after

$sql = "UPDATE `key_values` SET
            `Value_Content` = '" . $this->db->escape($revisionValues['value']) . "',
            `Comments` = '" . $this->db->escape($revisionValues['comment']) . "',
            `Is_Active` = '" . $this->db->escape($revisionValues['actstate']) . "',
            `Is_Modified`='1'
            WHERE
            `Key_Value`='" . $candidateKey['key'] . "'
            AND `Email_Template`='" . $candidateKey['template'] . "'
            AND `Locale_ID`='" . $candidateKey['locale'] . "'";
$sql = str_replace(array(chr(10), chr(13)), '', $sql);

So detecting a a newline is checking for chr(10) or chr(13). Depending on what system your on it could be either one of them or both. See: Is a new line = \n OR \r\n? (\r=chr(13) & \n=chr(10))

UPDATE

If you want to return a single line string from token_get_all() you could use:

<?php
$c = str_replace(array("\n","\r"), '', print_r(token_get_all('<?php echo; ?>'), true));
print $c;
// token_get_all() returns an array
// print_r(array, true) prints the array and the true param makes it return the output as a string
// replace the newline chars with nothing to make it single line

//single line output:
//Array( [0] => Array ( [0] => 372 [1] => 1 ) [1] => Array ( [0] => 316 [1] => echo [2] => 1 ) [2] => ; [3] => Array ( [0] => 375 [1] => [2] => 1 ) [4] => Array ( [0] => 374 [1] => ?> [2] => 1 ))
?>

Its not only my code. There are many devs so first option is not a solution for me. For second one, actually this is my point, how do I identify that this line is multiline, so that I can eliminate chr(10), chr(13). — user1635914, Dec 18 '12 at 09:34
If you want to identify it you can use substr_count or strpos to see if it returns >0 (or >=0 for strpos) on chr(10) or chr(13). But you could also just replace it straight away. If it already was a single line it wont alter anything and if it was multiline it will change it to single line — Hugo Delsing, Dec 18 '12 at 09:45
Hugo how will you get complete line via token_get_all. Am not getting this part ...... — user1635914, Dec 18 '12 at 10:01
Thanks,I have a file which has php code including this query and array . I ran same on it and still getting query in multiple token , not as a single line. Your solution will work once I have a complete query in hand as a single line. And this is what not happening . moreover $c in your code must be sting, where I need array . — user1635914, Dec 18 '12 at 12:03

PHP Tokenizer multiline issue

1 Answers1