Highlight match result in subject string from preg_match_all()

Question

I am trying to highlight the subject string with the returned $matches array from preg_match_all(). Let me start off with an example:

preg_match_all("/(.)/", "abc", $matches, PREG_OFFSET_CAPTURE | PREG_SET_ORDER);

This will return:

Array
(
    [0] => Array
        (
            [0] => Array
                (
                    [0] => a
                    [1] => 0
                )

            [1] => Array
                (
                    [0] => a
                    [1] => 0
                )

        )

    [1] => Array
        (
            [0] => Array
                (
                    [0] => b
                    [1] => 1
                )

            [1] => Array
                (
                    [0] => b
                    [1] => 1
                )

        )

    [2] => Array
        (
            [0] => Array
                (
                    [0] => c
                    [1] => 2
                )

            [1] => Array
                (
                    [0] => c
                    [1] => 2
                )

        )

)

What I want to do in this case is to highlight the overall consumed data AND each backreference.

Output should look like this:

<span class="match0">
    <span class="match1">a</span>
</span>
<span class="match0">
    <span class="match1">b</span>
</span>
<span class="match0">
    <span class="match1">c</span>
</span>

Another example:

preg_match_all("/(abc)/", "abc", $matches, PREG_OFFSET_CAPTURE | PREG_SET_ORDER);

Should return:

<span class="match0"><span class="match1">abc</span></span>

I hope this is clear enough.

I want to highlight overall consumed data AND highlight each backreference.

Thanks in advance. If anything is unclear, please ask.

Note: It must not break html. The regex AND input string are both unknown by the code and completely dynamic. So the search string can be html and the matched data can contain html-like text and what not.

If when I get to a computer still no answer, I'll give it a try. — Madara's Ghost, Aug 13 '12 at 14:39
To clarify your note at the end, given the input "
an item
" and the regex "/(
The output of that should be something like this: `<ul><li>an item</li></ul>` — Firas Dib, Aug 14 '12 at 12:58

IMSoP · Accepted Answer · 2012-08-13T20:13:14.707

This seems to behave right for all the examples I've thrown at it so far. Note that I've broken the abstract highlighting part from the HTML-mangling part for reusability in other situations:

<?php

/**
 * Runs a regex against a string, and return a version of that string with matches highlighted
 * the outermost match is marked with [0]...[/0], the first sub-group with [1]...[/1] etc
 *
 * @param string $regex Regular expression ready to be passed to preg_match_all
 * @param string $input
 * @return string
 */
function highlight_regex_matches($regex, $input)
{
    $matches = array();
    preg_match_all($regex, $input, $matches, PREG_OFFSET_CAPTURE | PREG_SET_ORDER);

    // Arrange matches into groups based on their starting and ending offsets
    $matches_by_position = array();
    foreach ( $matches as $sub_matches )
    {
            foreach ( $sub_matches as $match_group => $match_data )
            {
                    $start_position = $match_data[1];
                    $end_position = $start_position + strlen($match_data[0]);

                    $matches_by_position[$start_position]['START'][] = $match_group;

                    $matches_by_position[$end_position]['END'][] = $match_group;
            }
    }

    // Now proceed through that array, annotoating the original string
    // Note that we have to pass through BACKWARDS, or we break the offset information
    $output = $input;
    krsort($matches_by_position);
    foreach ( $matches_by_position as $position => $matches )
    {
            $insertion = '';

            // First, assemble any ENDING groups, nested highest-group first
            if ( is_array($matches['END']) )
            {
                    krsort($matches['END']);
                    foreach ( $matches['END'] as $ending_group )
                    {
                            $insertion .= "[/$ending_group]";
                    }
            }

            // Then, any STARTING groups, nested lowest-group first
            if ( is_array($matches['START']) )
            {
                    ksort($matches['START']);
                    foreach ( $matches['START'] as $starting_group )
                    {
                            $insertion .= "[$starting_group]";
                    }
            }

            // Insert into output
            $output = substr_replace($output, $insertion, $position, 0);
    }

    return $output;
}

/**
 * Given a regex and a string containing unescaped HTML, return a blob of HTML
 * with the original string escaped, and matches highlighted using <span> tags
 *
 * @param string $regex Regular expression ready to be passed to preg_match_all
 * @param string $input
 * @return string HTML ready to display :)
 */
function highlight_regex_as_html($regex, $raw_html)
{
    // Add the (deliberately non-HTML) highlight tokens
    $highlighted = highlight_regex_matches($regex, $raw_html);

    // Escape the HTML from the input
    $highlighted = htmlspecialchars($highlighted);

    // Substitute the match tokens with desired HTML
    $highlighted = preg_replace('#\[([0-9]+)\]#', '<span class="match\\1">', $highlighted);
    $highlighted = preg_replace('#\[/([0-9]+)\]#', '</span>', $highlighted);

    return $highlighted;
}

NOTE: As hakra has pointed out to me on chat, if a sub-group in the regex can occur multiple times within one overall match (e.g. '/a(b|c)+/'), preg_match_all will only tell you about the last of those matches - so highlight_regex_matches('/a(b|c)+/', 'abc') returns '[0]ab[1]c[/1][/0]' not '[0]a[1]b[/1][1]c[/1][/0]' as you might expect/want. All matching groups outside that will still work correctly though, so highlight_regex_matches('/a((b|c)+)/', 'abc') gives '[0]a[1]b[2]c[/2][/1][/0]' which is still a pretty good indication of how the regex matched.

Yes, repeating subgroups are the problem, that's why I linked http://stackoverflow.com/q/6371226/367456 as well in my answer. I stumbled over that problem some time ago. As long as those are not there, it just "works". However I'm somewhat sure part of the problem that made OP ask are these repeating subgroups - known or not. Anyway +1 for this answer. — hakre, Aug 13 '12 at 20:49
Ah, didn't realise that's what that post was about - it somewhat evaded skim-reading. :| It's actually a distinction between __clustering__ and __capturing__ - you can specify that a "cluster" of your regex should repeat in a certain way, but each pair of brackets "captures" a single value. So the regex /(thing)+/ contains only one "capture group", which is assigned to backreference 1; the group matches multiple times, but each time backreference 1 is over-written. [This is the same in Perl, BTW] — IMSoP, Aug 13 '12 at 21:15
In short, my function will show you "what will be captured", but it cannot show you "how the regex engine processes the string" (I guess you could say it's a regex debugger, not a regex engine debugger :P) — IMSoP, Aug 13 '12 at 21:16
Some regex enines can return *all* matches. For example in Perl. In PHP that has been not been implemented so far. Probably one should finally open a feature request. — hakre, Aug 13 '12 at 21:24
I'd be interested to know how you can do that in Perl; as far as I'm aware it has the 1:1 relationship between capture groups and backrefs that I mentioned, and everything proceeds from there... — IMSoP, Aug 13 '12 at 21:27
I do not know how, but it's written here that it works: http://stackoverflow.com/a/6571139/367456 - also by reading the PCRE manpages I think I've seen the API for that in the library itself. http://pcre.org/ — hakre, Aug 13 '12 at 21:29
Most people talking about "Perl" will mean "Perl 5", so the mention of "Perl 6" in that answer is important. Perl 6 is a fairly radical new version of the language, with a completely rewritten regex engine (among other things). "The PCRE library is a set of functions that implement ... the same syntax and semantics as Perl 5" - so unlikely to include such a feature. — IMSoP, Aug 13 '12 at 21:41
I do not think that the underlying regex engine changes are totally synched with the language version switch and can imagine that PHP uses the same library as Perl 6. — hakre, Aug 13 '12 at 21:46
PCRE is "Perl Compatible" - it has nothing to do with Perl's actual regex engine other than an attempt to emulate it. The capturing semantics of Perl6 are pretty radically different - see http://perlcabal.org/syn/S05.html — IMSoP, Aug 13 '12 at 22:02
Ah well, that's an interesting link, didn't know that. I wish we could have that in PHP. — hakre, Aug 13 '12 at 22:06
Very nice! Thank you. I have myself come up with a similar solution. I am right now trying to optimize it and make it faster as it is quite slow. The limitation you explained is OK, as that is the expected output. My solution does the same. I will post it as soon as I am done. — Firas Dib, Aug 14 '12 at 12:16
I have posted my "solution" as a solution instead of a comment as it grew too long. Please comment on it. It's a bit slow right now which sucks. In cases of a string like `aaaaaa` (but much longer) and a regex like `/(.)/` its very slow. If you know how I can optimize it, please tell. — Firas Dib, Aug 14 '12 at 12:46

score 0 · Answer 2 · edited May 23 '17 at 12:12

0

Reading your comment under the first answer, I'm pretty sure you did not really formulated the question as you intended to. However following to what you ask for in concrete that is:

$pattern = "/(.)/";
$subject = "abc";

$callback = function($matches) {
    if ($matches[0] !== $matches[1]) {
        throw new InvalidArgumentException(
            sprintf('you do not match thee requirements, go away: %s'
                    , print_r($matches, 1))
        );
    }
    return sprintf('<span class="match0"><span class="match1">%s</span></span>'
                   , htmlspecialchars($matches[1]));
};
$result = preg_replace_callback($pattern, $callback, $subject);

Before you now start to complain, take a look first where your shortcoming in describing the problem is. I have the feeling you actually want to actually parse the result for matches. However you want to do sub-matches. That does not work unless you parse as well the regular expression to find out which groups are used. That is not the case so far, not in your question and also not in this answer.

So please this example only for one subgroup which must also be the whole pattern as an requirement. Apart from that, this is fully dynamic.

Community

1
1

answered Aug 13 '12 at 18:13

hakre

193,403
52
435
836

I'm confused, why would `$matches[0]` and `$matches[1]` need to match? The intent of the question was perfectly clear to me. While fiddly to use, I think all the data needed is there in the capture data returned by `preg_match_all` – IMSoP Aug 13 '12 at 18:20
They need to match because this is a requirement of this example. The shortcoming is explained in the answer. Everything else needs a PHP regex parser which is actually not really possible IMHO as an answer here as it does not yet exists. Also PCRE syntax is pretty heavy. – hakre Aug 13 '12 at 18:22
I don't see that requirement anywhere. And the list of captured groups is known, that's exactly what `preg_match_all` returns. – IMSoP Aug 13 '12 at 18:26
The requirements are given in the answer: *"So please this example only for one subgroup which must also be the whole pattern as an requirement. Apart from that, this is fully dynamic."* Read that. It is in the answer. - This one subgroup as the whole pattern exactly spares the parsing for subpattern I was talking about. – hakre Aug 13 '12 at 18:34
Oh, I see, I thought you meant it was a requirement of the question. I'm not convinced that you need to reverse-engineer the regex in the way you're suggesting, though - `preg_match_all` with the flags used gives you all the data you need. – IMSoP Aug 13 '12 at 18:39
@IMSoP: Try that and you will learn why it's not that easily. Especially try with grouped subgroups. PHP implementation of PCRE does not offer true subpattern match information. – hakre Aug 13 '12 at 18:40

Firas Dib · Answer 3 · 2012-08-14T13:00:28.107

0

I am not too familiar with posting on stackoverflow so I hope I don't mess this up. I do this in almost the same way as @IMSoP, however, slightly different:

I store the tags like this:

$tags[ $matched_pos ]['open'][$backref_nr] = "open tag";
$tags[ $matched_pos + $len ]['close'][$backref_nr] = "close tag";

As you can see, almost identical to @IMSoP.

Then I construct the string like this, instead of inserting and sorting like @IMSoP does:

$finalStr = "";
for ($i = 0; $i <= strlen($text); $i++) {
    if (isset($tags[$i])) {
        foreach ($tags[$i] as $tag) {
            foreach ($tag as $span) {
                $finalStr .= $span;
            }
        }
    }
    $finalStr .= $text[$i];
}

Where $text is the text used in preg_match_all()

I think my solution is slightly faster than @IMSoP's since he has to sort every time and what not. But I am not sure.

My main worry right now is performance. But it might just not be possible to make it work any faster than this?

I have been trying to get a recursive preg_replace_callback() thing going, but I've not been able to make it work so far. preg_replace_callback() seems to be very, very fast. Much faster than what I am currently doing anyway.

edited Aug 14 '12 at 13:00

answered Aug 14 '12 at 12:51

Firas Dib

2,743
19
38

As far as performance goes, string manipulation is one of the most expensive operations in PHP. Your solution requires more string concatenations as a factor of the original string length, whereas mine performs insertions relative to the number of matches. The `ksort` calls can possibly be left out of my solution. However, the `krsort` to reverse the end tags is vital, otherwise the output won't nest correctly. It's not clear that your version handles this - '[1]a[2]b[/1][/2]' should be '[1]a[2]b[/2][/1]' – IMSoP Aug 14 '12 at 13:03
Yes, you are correct. But in reality it makes no difference, as the closing tags are always the same (``). So your `[/2]` and `[/1]` are both `` in the real world, and thus it does not matter if they are in order or not. I will try to implement your version and see how it goes. I'll report once I get it working. – Firas Dib Aug 14 '12 at 13:09
Alright. I've tried it out. On a ~500 char long string consisting of 'a' run against `/(.)/` your code executed on my computer in roughly 7.81 secs (yes, slow computer). This was with all sorting removed except the initial `krsort`. My code took 5.7 secs. While they are both slow, mine is actually faster. I have not tested much larger strings as it takes very long for the code to execute in both cases on my computer. I'm thinking substr_replace() does some fishy/slow stuff in the background. I'm hoping for a recursive `preg_replace_callback()`-solution. – Firas Dib Aug 14 '12 at 13:20
Yes, I just realised that about the closing tags after I posted my comment. Your test isn't a particularly fair one, since it makes the number of matches twice the number of characters in the initial string. While not written for speed, I would expect my version to be quicker for a pattern that matched 2 or 3 times in a 500-char string. – IMSoP Aug 14 '12 at 13:27
~2300 chars. Yours: 9.19s. Mine: 5.54s. I also *think* yours should be faster, but for some reason it isnt... Do you have any idea whats up? For any plain old expression and subject string they both work fine, but for this "worst-case" scenario they are both quite bad. I hope someone comes up with a genious solution, haha. – Firas Dib Aug 14 '12 at 13:32
I think you may be misunderstanding - the speed of my solution will depend on the number of matches, yours on the length of input; as the number of matches approaches the number of input chars, yours will perform better, because there will be as many `substr_replace` calls in mine as concatenations in yours. These would also be incredibly small differences on modern server hardware; on my work's dev server (a hardly top-of-the-range 6-year-old dual 1.8GHz Xeon), your worst-case regex processed a string of 10000 'a's in <2 seconds (between 2 and 3 secs if I add in the HTML post-processing). – IMSoP Aug 14 '12 at 13:56
Hmm I see. I guess it's just my computer thats super slow. But what does `substr_replace` really do? Perhaps it does some work which we don't take into account which results in slowdowns? I also send this data pre-formatted from my server to the client via json which results in large chunks of data sent back and forth. Perhaps it would be better to let jquery handle the insertion of span-tags instead of php? Because im thinking that `$.jsonParse` has to do a lot of work when processing that large chunk of data. – Firas Dib Aug 15 '12 at 06:59
I'm sure `substr_replace` is slower than `$string1 . $string2`, but I'm assuming in most real-world cases, it won't have to be run nearly as many times, offsetting that cost. As for transmitting the data back, any reason it needs to be JSON-wrapped, rather than just returning it as plain HTML? Check out jQuery's `load()` function for inserting it directly into the calling page. – IMSoP Aug 15 '12 at 15:53

score -1 · Answer 4 · answered Aug 13 '12 at 14:36

-1

A quick mashup, why use regex?

$content = "abc";
$endcontent = "";

for($i = 0; $i > strlen($content); $i++)
{
    $endcontent .= "<span class=\"match0\"><span class=\"match1\">" . $content[$i] . "</span></span>";
}

echo $endcontent;

answered Aug 13 '12 at 14:36

Ogelami

366
1
14

1

I am creating a regex service. The users regex and subject string will be used and run through preg_match_all(). I want to highlight the match result in the subject string based on the returned $matches array. Do you understand? – Firas Dib Aug 13 '12 at 14:38

Highlight match result in subject string from preg_match_all()

4 Answers4