2

My code is unable to detect the usage of operators along with non-english characters:

const OPERATOR_REGEX = new RegExp(
  /(?!\B"[^"|“|”]*)\b(and|or|not|exclude)(?=.*[\s])\b(?![^"|“|”]*"\B)/,
  'giu'
);

const query1 = '(Java or "化粧" or 化粧品)';
const query2 = '(Java or 化粧 or 化粧品)';

console.log(query1.split(OPERATOR_REGEX));
console.log(query2.split(OPERATOR_REGEX));

https://codepen.io/thewebtud/pen/vYraavd?editors=1111

Whereas the same code successfully detects all operators on regex101.com using the unicode flag: https://regex101.com/r/FC84BH/1

How can this be fixed for JS?

Wiktor Stribiżew
  • 607,720
  • 39
  • 448
  • 563
codingrohtak
  • 160
  • 8
  • 2
    Yes, only if you target ECMAScript 2018+ standard. It is impossible for old browsers like Safari. – Wiktor Stribiżew Nov 29 '22 at 09:37
  • I have just looked at the pattern again, and it seems you actually wanted to avoid matching the `or`, etc. if it is preceded with `"` (not after a word char) that is followed with zero or more chars other than `"`, `“` and `”`, right? Then your pattern must have looked like `/(?<!\B"[^"“”]*)\b(and|or|not|exclude)(?=.*\s)\b(?![^"“”]*"\B)/`. That is, the first lookaround must be a negative **lookbehind**. – Wiktor Stribiżew Dec 08 '22 at 11:43
  • Why not separate parenthesis and words? I would have thought the split result looks like this: `[ '(', 'Java', 'or', '"化粧"', 'or', '化粧品', ')']`, or possibly this with quotes removed: `[ '(', 'Java', 'or', '化粧', 'or', '化粧品', ')']` – Peter Thoeny Dec 14 '22 at 08:29

1 Answers1

3

Keeping in mind that

  • \b (word boundary) can be written as (?:(?<=^)(?=\w)|(?<=\w)(?=$)|(?<=\W)(?=\w)|(?<=\w)(?=\W)) and
  • \B (non-word boundary) can be written as (?:(?<=^)(?=\W)|(?<=\W)(?=$)|(?<=\W)(?=\W)|(?<=\w)(?=\w))

and that a Unicode-aware \w pattern is [\p{Alphabetic}\p{Mark}\p{Decimal_Number}\p{Connector_Punctuation}\p{Join_Control}] (see Replace certain arabic words in text string using Javascript), here is the ECMAScript 2018+ solution:

const w = String.raw`[\p{Alphabetic}\p{Mark}\p{Decimal_Number}\p{Connector_Punctuation}\p{Join_Control}]`;
const nw = String.raw`[^\p{Alphabetic}\p{Mark}\p{Decimal_Number}\p{Connector_Punctuation}\p{Join_Control}]`;
const uwb = String.raw`(?:(?<=^)(?=${w})|(?<=${w})(?=$)|(?<=${nw})(?=${w})|(?<=${w})(?=${nw}))`;
const unwb = String.raw`(?:(?<=^)(?=${nw})|(?<=${nw})(?=$)|(?<=${nw})(?=${nw})|(?<=${w})(?=${w}))`;

const OPERATOR_REGEX = new RegExp(
  String.raw`(?!${unwb}"[^"“”]*)${uwb}(and|or|not|exclude)(?=.*\s)${uwb}(?![^"“”]*"${unwb})`,
  'giu'
);

const query1 = '(Java or "化粧" or 化粧品)';
const query2 = '(Java or 化粧 or 化粧品)';

console.log(query1.split(OPERATOR_REGEX));
console.log(query2.split(OPERATOR_REGEX));
Wiktor Stribiżew
  • 607,720
  • 39
  • 448
  • 563