RegEx to extract all matches from string using RegExp.exec

Question

I'm trying to parse the following kind of string:

[key:"val" key2:"val2"]

where there are arbitrary key:"val" pairs inside. I want to grab the key name and the value. For those curious I'm trying to parse the database format of task warrior.

Here is my test string:

[description:"aoeu" uuid:"123sth"]

which is meant to highlight that anything can be in a key or value aside from space, no spaces around the colons, and values are always in double quotes.

In node, this is my output:

[deuteronomy][gatlin][~]$ node
> var re = /^\[(?:(.+?):"(.+?)"\s*)+\]$/g
> re.exec('[description:"aoeu" uuid:"123sth"]');
[ '[description:"aoeu" uuid:"123sth"]',
  'uuid',
  '123sth',
  index: 0,
  input: '[description:"aoeu" uuid:"123sth"]' ]

But description:"aoeu" also matches this pattern. How can I get all matches back?

Answer 1

Continue calling re.exec(s) in a loop to obtain all the matches:

var re = /\s*([^[:]+):\"([^"]+)"/g;
var s = '[description:"aoeu" uuid:"123sth"]';
var m;

do {
    m = re.exec(s);
    if (m) {
        console.log(m[1], m[2]);
    }
} while (m);

Try it with this JSFiddle: https://jsfiddle.net/7yS2V/

Answer 2

str.match(pattern), if pattern has the global flag g, will return all the matches as an array.

For example:

const str = 'All of us except @Emran, @Raju and @Noman were there';
console.log(
  str.match(/@\w*/g)
);
// Will log ["@Emran", "@Raju", "@Noman"]

Answer 3

To loop through all matches, you can use the replace function:

var re = /\s*([^[:]+):\"([^"]+)"/g;
var s = '[description:"aoeu" uuid:"123sth"]';

s.replace(re, function(match, g1, g2) { console.log(g1, g2); });

Answer 4

This is a solution

var s = '[description:"aoeu" uuid:"123sth"]';

var re = /\s*([^[:]+):\"([^"]+)"/g;
var m;
while (m = re.exec(s)) {
  console.log(m[1], m[2]);
}

This is based on lawnsea's answer, but shorter.

Notice that the `g' flag must be set to move the internal pointer forward across invocations.

Answer 5

str.match(/regex/g)

returns all matches as an array.

If, for some mysterious reason, you need the additional information comes with exec, as an alternative to previous answers, you could do it with a recursive function instead of a loop as follows (which also looks cooler :).

function findMatches(regex, str, matches = []) {
   const res = regex.exec(str)
   res && matches.push(res) && findMatches(regex, str, matches)
   return matches
}

// Usage
const matches = findMatches(/regex/g, str)

as stated in the comments before, it's important to have g at the end of regex definition to move the pointer forward in each execution.

Answer 6

We are finally beginning to see a built-in matchAll function, see here for the description and compatibility table. It looks like as of May 2020, Chrome, Edge, Firefox, and Node.js (12+) are supported but not IE, Safari, and Opera. Seems like it was drafted in December 2018 so give it some time to reach all browsers, but I trust it will get there.

The built-in matchAll function is nice because it returns an iterable. It also returns capturing groups for every match! So you can do things like

// get the letters before and after "o"
let matches = "stackoverflow".matchAll(/(\w)o(\w)/g);

for (match of matches) {
    console.log("letter before:" + match[1]);
    console.log("letter after:" + match[2]);
}

arrayOfAllMatches = [...matches]; // you can also turn the iterable into an array

It also seem like every match object uses the same format as match(). So each object is an array of the match and capturing groups, along with three additional properties index, input, and groups. So it looks like:

[<match>, <group1>, <group2>, ..., index: <match offset>, input: <original string>, groups: <named capture groups>]

For more information about matchAll there is also a Google developers page. There are also polyfills/shims available.

Answer 7

If you have ES9

(Meaning if your system: Chrome, Node.js, Firefox, etc supports Ecmascript 2019 or later)

yourString.matchAll( /your-regex/g ) // dont forget the "g"

MDN Documentation

If you use NPM

You can use the official polyfill
npm install string.prototype.matchall

const matchAll = require('string.prototype.matchall')
console.log( [...  matchAll('blah1 blah2',/blah/g)  ] )
//[
//  [ 'blah', index: 0, input: 'blah1 blah2', groups: undefined ],
//  [ 'blah', index: 6, input: 'blah1 blah2', groups: undefined ]
//]

Otherwise

Here's some functionally similar copy-paste versions

// returns an array, works on super old javascript (ES3 -- 1999)
function findAll(regexPattern, sourceString) {
    var output = []
    var match
    // auto-add global flag while keeping others as-is
    var regexPatternWithGlobal = regexPattern.global ? regexPattern : RegExp(regexPattern, regexPattern.flags+"g")
    while (match = regexPatternWithGlobal.exec(sourceString)) {
        // store the match data
        output.push(match)
        // zero-length matches will end up in an infinite loop, so increment by one char after a zero-length match is found
        if (match[0].length == 0) {
            regexPatternWithGlobal.lastIndex += 1
        }
    }
    return output
}

// this version returns an iterator, which is good for large results
// note: iterators require ES6 - 2015 standard
function* findAll(regexPattern, sourceString) {
    var match
    // auto-add global flag while keeping others as-is
    const regexPatternWithGlobal = regexPattern.global ? regexPattern : RegExp(regexPattern, regexPattern.flags+"g")
    while (match = regexPatternWithGlobal.exec(sourceString)) {
        // store the match data
        yield match
        // zero-length matches will end up in an infinite loop, so increment by one char after a zero-length match is found
        if (match[0].length == 0) {
            regexPatternWithGlobal.lastIndex += 1
        }
    }
    return output
}

example usage:

console.log(   findAll(/blah/g,'blah1 blah2')   ) 

outputs:

[ [ 'blah', index: 0 ], [ 'blah', index: 6 ] ]

Answer 8

Based on Agus's function, but I prefer return just the match values:

var bob = "> bob <";
function matchAll(str, regex) {
    var res = [];
    var m;
    if (regex.global) {
        while (m = regex.exec(str)) {
            res.push(m[1]);
        }
    } else {
        if (m = regex.exec(str)) {
            res.push(m[1]);
        }
    }
    return res;
}
var Amatch = matchAll(bob, /(&.*?;)/g);
console.log(Amatch);  // yeilds: [>, <]

Answer 9

Iterables are nicer:

const matches = (text, pattern) => ({
  [Symbol.iterator]: function * () {
    const clone = new RegExp(pattern.source, pattern.flags);
    let match = null;
    do {
      match = clone.exec(text);
      if (match) {
        yield match;
      }
    } while (match);
  }
});

Usage in a loop:

for (const match of matches('abcdefabcdef', /ab/g)) {
  console.log(match);
}

Or if you want an array:

[ ...matches('abcdefabcdef', /ab/g) ]

Answer 10

Here is my function to get the matches :

function getAllMatches(regex, text) {
    if (regex.constructor !== RegExp) {
        throw new Error('not RegExp');
    }

    var res = [];
    var match = null;

    if (regex.global) {
        while (match = regex.exec(text)) {
            res.push(match);
        }
    }
    else {
        if (match = regex.exec(text)) {
            res.push(match);
        }
    }

    return res;
}

// Example:

var regex = /abc|def|ghi/g;
var res = getAllMatches(regex, 'abcdefghi');

res.forEach(function (item) {
    console.log(item[0]);
});

Answer 11

If you're able to use matchAll here's a trick:

Array.From has a 'selector' parameter so instead of ending up with an array of awkward 'match' results you can project it to what you really need:

Array.from(str.matchAll(regexp), m => m[0]);

If you have named groups eg. (/(?<firstname>[a-z][A-Z]+)/g) you could do this:

Array.from(str.matchAll(regexp), m => m.groups.firstName);

Answer 12

Since ES9, there's now a simpler, better way of getting all the matches, together with information about the capture groups, and their index:

const string = 'Mice like to dice rice';
const regex = /.ice/gu;
for(const match of string.matchAll(regex)) {
    console.log(match);
}

// ["mice", index: 0, input: "mice like to dice rice", groups: undefined]

// ["dice", index: 13, input: "mice like to dice rice", groups: undefined]

// ["rice", index: 18, input: "mice like to dice rice", groups: undefined]

It is currently supported in Chrome, Firefox, Opera. Depending on when you read this, check this link to see its current support.

Answer 13

Use this...

var all_matches = your_string.match(re);
console.log(all_matches)

It will return an array of all matches...That would work just fine.... But remember it won't take groups in account..It will just return the full matches...

Answer 14

I would definatly recommend using the String.match() function, and creating a relevant RegEx for it. My example is with a list of strings, which is often necessary when scanning user inputs for keywords and phrases.

    // 1) Define keywords
    var keywords = ['apple', 'orange', 'banana'];

    // 2) Create regex, pass "i" for case-insensitive and "g" for global search
    regex = new RegExp("(" + keywords.join('|') + ")", "ig");
    => /(apple|orange|banana)/gi

    // 3) Match it against any string to get all matches 
    "Test string for ORANGE's or apples were mentioned".match(regex);
    => ["ORANGE", "apple"]

Hope this helps!

Answer 15

This isn't really going to help with your more complex issue but I'm posting this anyway because it is a simple solution for people that aren't doing a global search like you are.

I've simplified the regex in the answer to be clearer (this is not a solution to your exact problem).

var re = /^(.+?):"(.+)"$/
var regExResult = re.exec('description:"aoeu"');
var purifiedResult = purify_regex(regExResult);

// We only want the group matches in the array
function purify_regex(reResult){

  // Removes the Regex specific values and clones the array to prevent mutation
  let purifiedArray = [...reResult];

  // Removes the full match value at position 0
  purifiedArray.shift();

  // Returns a pure array without mutating the original regex result
  return purifiedArray;
}

// purifiedResult= ["description", "aoeu"]

That looks more verbose than it is because of the comments, this is what it looks like without comments

var re = /^(.+?):"(.+)"$/
var regExResult = re.exec('description:"aoeu"');
var purifiedResult = purify_regex(regExResult);

function purify_regex(reResult){
  let purifiedArray = [...reResult];
  purifiedArray.shift();
  return purifiedArray;
}

Note that any groups that do not match will be listed in the array as undefined values.

This solution uses the ES6 spread operator to purify the array of regex specific values. You will need to run your code through Babel if you want IE11 support.

Answer 16

Here's a one line solution without a while loop.

The order is preserved in the resulting list.

The potential downsides are

1. It clones the regex for every match.
2. The result is in a different form than expected solutions. You'll need to process them one more time.

let re = /\s*([^[:]+):\"([^"]+)"/g
let str = '[description:"aoeu" uuid:"123sth"]'

(str.match(re) || []).map(e => RegExp(re.source, re.flags).exec(e))

[ [ 'description:"aoeu"',
    'description',
    'aoeu',
    index: 0,
    input: 'description:"aoeu"',
    groups: undefined ],
  [ ' uuid:"123sth"',
    'uuid',
    '123sth',
    index: 0,
    input: ' uuid:"123sth"',
    groups: undefined ] ]

Answer 17

My guess is that if there would be edge cases such as extra or missing spaces, this expression with less boundaries might also be an option:

^\s*\[\s*([^\s\r\n:]+)\s*:\s*"([^"]*)"\s*([^\s\r\n:]+)\s*:\s*"([^"]*)"\s*\]\s*$

---

If you wish to explore/simplify/modify the expression, it's been explained on the top right panel of regex101.com. If you'd like, you can also watch in this link, how it would match against some sample inputs.

---

Test

const regex = /^\s*\[\s*([^\s\r\n:]+)\s*:\s*"([^"]*)"\s*([^\s\r\n:]+)\s*:\s*"([^"]*)"\s*\]\s*$/gm;
const str = `[description:"aoeu" uuid:"123sth"]
[description : "aoeu" uuid: "123sth"]
[ description : "aoeu" uuid: "123sth" ]
 [ description : "aoeu"   uuid : "123sth" ]
 [ description : "aoeu"uuid  : "123sth" ] `;
let m;

while ((m = regex.exec(str)) !== null) {
    // This is necessary to avoid infinite loops with zero-width matches
    if (m.index === regex.lastIndex) {
        regex.lastIndex++;
    }
    
    // The result can be accessed through the `m`-variable.
    m.forEach((match, groupIndex) => {
        console.log(`Found match, group ${groupIndex}: ${match}`);
    });
}

RegEx Circuit

jex.im visualizes regular expressions:

Answer 18

const re = /^\[(?:(.+?):"(.+?)"\s*)+\]$/g
const matches = [...re.exec('[description:"aoeu" uuid:"123sth"]').entries()]
console.log(matches)

Basically, this is ES6 way to convert Iterator returned by exec to a regular Array

Answer 19

Here is my answer:

var str = '[me nombre es] : My name is. [Yo puedo] is the right word'; 

var reg = /\[(.*?)\]/g;

var a = str.match(reg);

a = a.toString().replace(/[\[\]]/g, "").split(','));