Javascript extract tokens from string

Question

I want to pull the tokens out of this string and build an array (this is a simplified example):

SELECT * FROM {token1} WHERE {token2}

So I have an array of two elements:

myArray[0] = "token1"

myArray[1] = "token2"

Is there a clever way to do this without iterating through the string?

I also don't want duplicates in the array, so

The quick {token1} dog {token2} over the {token1} wall

Will give;

myArray[0] = "token1"

myArray[1] = "token2"

(The real reason for this, is for using dates in the SQL statements.)

Mick

The clever way it to loop a regex. WIhtout any iteration, only if there's a fixed amount of tokens. — Shilly, Feb 17 '17 at 15:48
You don't have to parse the query. you have to parse everything between { and }, which I've never seen used in a sql query, so it might be simple. I use the same system in node and haven't had issues yet. — Shilly, Feb 17 '17 at 15:56
Well, the best would be using some library for this. For example, Dojo has String.substitute command that works like this: `var c = string.substitute("${replace} - ${me}", { replace: "foo", me: "bar" });`. There are other examples though. — Nelson Teixeira, Feb 17 '17 at 16:11
This page is relevant too: https://jsperf.com/string-replacement-iteration-vs-regexp — Nelson Teixeira, Feb 17 '17 at 16:22

score 7 · Accepted Answer · answered Feb 17 '17 at 15:52

7

You can use replace to have a regex loop run on the string and pull the tokens:

var sql = 'SELECT * FROM {token1} WHERE {token2}';

var tokens = [];

sql.replace(/\{(.*?)}/g, function(a, b) {
  tokens.push(b);
});

console.log(tokens);

answered Feb 17 '17 at 15:52

KevBot

17,900
5
50
68

That's not what `.replace()` is meant for... – Andreas Feb 17 '17 at 15:56
1

@Andreas, maybe not its most common use, but even MDN examples show using replace to avoid `for loops` in [their documentation](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String/replace#Use_an_inline_function_with_a_regular_expression_to_avoid_for_loops) – KevBot Feb 17 '17 at 16:01
1

OP asked for no iteration. So, `replace()` would possibly be a good option. (tho technically its iterating) – m87 Feb 17 '17 at 16:01
1

Love it, I wasn't expecting `.replace()` to be used in this context! Thanks, learning all the time :-) – Jan Molak Nov 22 '19 at 21:55
To get rid of duplicates, see also: https://stackoverflow.com/questions/1960473/get-all-unique-values-in-a-javascript-array-remove-duplicates – Shawn Eary Apr 09 '21 at 16:54

MaxZoom · Answer 2 · 2017-02-17T17:32:54.623

3

You can try this regex :

(?:{)([^}]*)(?:})

Example code:

var str = 'SELECT * FROM {token1} WHERE {token2}';
var regex = new RegExp('(?:{)([^}]*)(?:})', 'g');
var match, myArray = [];
while (match = regex.exec(str)) myArray.push(match[1]);
console.log(myArray);

edited Feb 17 '17 at 17:32

answered Feb 17 '17 at 15:56

MaxZoom

7,619
5
28
44

score 0 · Answer 3 · answered Feb 17 '17 at 15:52

0

function getTokens(str) {
  var results = [], re = /{([^}]+)}/g, text;

  while(text = re.exec(str)) {
    results.push(text[1]);
  }
  return results;
}

var myString = "SELECT * FROM {token1} WHERE {token2}";

console.log(getTokens(myString));

answered Feb 17 '17 at 15:52

AshBringer

2,614
2
20
42

Javascript extract tokens from string

3 Answers3