Javascript - How can I perform this string manipulation using regex more cleanly/effeciently?

Question

I have the following string:

[SM_g]This[SM_h][SM_g]is[SM_h][SM_g]a[SM_h][SM_g]sentence.[SM_h][SM_l][SM_g]Here[SM_h][SM_g]is[SM_h][SM_g]another[SM_h][SM_g]sentence.[SM_h][SM_1]

And I can turn that string into this string, which I then display within a <p> element:

This is a sentence.

Here is another sentence.

with the following code:

tokenResponseText_initial = "[SM_g]This[SM_h][SM_g]is[SM_h][SM_g]a[SM_h][SM_g]sentence.[SM_h][SM_l][SM_g]Here[SM_h][SM_g]is[SM_h][SM_g]another[SM_h][SM_g]sentence.[SM_h][SM_1]"

const newLineIndicator = "insert-double-new-line"

const tokenResponseText_fixedNewLines = tokenResponseText_initial.replace(/(\[SM_g].*?)(\[SM_h]\[SM_l])/g, "$1" + newLineIndicator + "$2");

const wordCompilationRegex = /\[SM_g](.*?)\[SM_h]/g;
var wordRegexResponse;

var summary = "";

do {
  wordRegexResponse = wordCompilationRegex.exec(tokenResponseText_fixedNewLines);
  if (wordRegexResponse) {
    if (wordRegexResponse[1].includes(newLineIndicator)) {
      summary += wordRegexResponse[1].replace(newLineIndicator, "") + "\n\n";
    } else {
      summary += wordRegexResponse[1] + " ";
    }
  }
} while (wordRegexResponse);
//The following is rough code, 
someParagraphElement.innerHTML = summary;

p {
  white-space: pre-line;
}

<p id="someParagraphElement"></p>

Where the paragraph element has the following attribute white-space: pre-line;

However, ideally, in order to create the double newline between the two sentences, I'd like to eliminate the use of newLineIndicator and simply do this:

But, this second method does not work. The end result does not end having the double new lines even though when I print tokenResponseText_fixedNewLines to the console, it seems like the double new lines have been inserted, like so:

[SM_g]This[SM_h][SM_g]is[SM_h][SM_g]a[SM_h][SM_g]sentence.

[SM_h][SM_l][SM_g]Here[SM_h][SM_g]is[SM_h][SM_g]another[SM_h][SM_g]sentence.[SM_h][SM_1]

tokenResponseText_initial = "[SM_g]This[SM_h][SM_g]is[SM_h][SM_g]a[SM_h][SM_g]sentence.[SM_h][SM_l][SM_g]Here[SM_h][SM_g]is[SM_h][SM_g]another[SM_h][SM_g]sentence.[SM_h][SM_1]"

const tokenResponseText_fixedNewLines = tokenResponseText_initial.replace(/(\[SM_g].*?)(\[SM_h]\[SM_l])/g, "$1\n\n$2");

const wordCompilationRegex = /\[SM_g](.*?)\[SM_h]/g;
var wordRegexResponse;

var summary = "";

do {
  wordRegexResponse = wordCompilationRegex.exec(tokenResponseText_fixedNewLines);
  if (wordRegexResponse) {
    summary += wordRegexResponse[1] + " ";

  }
} while (wordRegexResponse);
//The following is rough code, 
someParagraphElement.innerHTML = summary;

p {
  white-space: pre-line;
}

<p id="someParagraphElement"></p>

Why does the second method not work, although the first method does work? Does .*? not capture new lines?

Are all `[SM_x]` to be replaced with a single space or are some to be replaced with other characters? Can you help indicate what each of these codes should be replaced with? — ctwheels, Apr 18 '18 at 19:13
The [SM_x] are not actually replaced - rather, I use a do/while construct in order to compile all of the words in between each [SM_x] into a single string as shown in the example. Whenever there is a `[SM_g]word[SM_h][SM_l]`, the regex (in the second example) turns it into `[SM_g]word\n\n[SM_h][SM_l]` — Foobar, Apr 18 '18 at 19:16

Barmar · Accepted Answer · 2018-04-18T19:28:25.770

1

By default, . doesn't match newlines. So when you insert the newlines into the string with the first .replace(), some of the [SM_g]word[SM_h] sequences no longer match wordCompilationRegex.

You can use the s modifier to allow . to match newlines.

Oops This is a Chrome extension, not a Javascript standard flag. Use [\s\S] instead of ..

tokenResponseText_initial = "[SM_g]This[SM_h][SM_g]is[SM_h][SM_g]a[SM_h][SM_g]sentence.[SM_h][SM_l][SM_g]Here[SM_h][SM_g]is[SM_h][SM_g]another[SM_h][SM_g]sentence.[SM_h][SM_1]"

const tokenResponseText_fixedNewLines = tokenResponseText_initial.replace(/(\[SM_g].*?)(\[SM_h]\[SM_l])/g, "$1\n\n$2");

const wordCompilationRegex = /\[SM_g]([\s\S]*?)\[SM_h]/g;
var wordRegexResponse;

var summary = "";

do {
  wordRegexResponse = wordCompilationRegex.exec(tokenResponseText_fixedNewLines);
  if (wordRegexResponse) {
    summary += wordRegexResponse[1] + " ";

  }
} while (wordRegexResponse);
//The following is rough code, 
someParagraphElement.innerHTML = summary;

p {
  white-space: pre-line;
}

<p id="someParagraphElement"></p>

edited Apr 18 '18 at 19:28

answered Apr 18 '18 at 19:21

Barmar

741,623
53
500
612

Running the answer (in stack overflow at least) gives an error saying `"SyntaxError: invalid regular expression flag s"` – Foobar Apr 18 '18 at 19:25
Works fine for me in Chrome. – Barmar Apr 18 '18 at 19:26
Oh, huh, it failed to work in firefox quantam (but I checked and it worked in chrome). I don't know if that will be a problem, considering this code is for a web extension that should work in chrome or firefox. – Foobar Apr 18 '18 at 19:27
1

Apparently the `s` flag is not standard, it's a Chrome extension. – Barmar Apr 18 '18 at 19:27
Oh, that's problematic, do you know of ways to do this without a chrome specific extension? – Foobar Apr 18 '18 at 19:28
1

Updated the answer. Use `[\s\S]` instead of `.`. – Barmar Apr 18 '18 at 19:30

score 0 · Answer 2 · answered Apr 18 '18 at 19:24

Your new code isn't putting a newline between the sentences because there is nothing in your code that would do that.

What you want to do is 1) detect if there is a new sentence. 2) When there is a new sentence, insert two new lines between them.

In order to detect a new sentence, use let index = summary.indexOf('.') and if the index is not --1 and is not equal to the summary.length - 1 (that is, the last character of the string), then you have detected there is at least two sentences.
If you have detected that there are two sentences, then inserting the newlines between them is a simple matter of using array.splice().

Note: if you need to handle there possibly being more than two sentences, then I would recommend writing the code to handle two sentences (as I have outlined above) and then amending it to loop through all indexes as the first answer to this question explains.

If you get into writing your code and get stuck/have more questions, just comment on this answer and I'll respond.

Javascript - How can I perform this string manipulation using regex more cleanly/effeciently?

2 Answers2