Difference between – and - or ' and ‘ or “ and "

Question

I’m trying to validate a string field to check for certain characters and raise an error if any of them are present. This is the regular expression I’m using:

var regex = /^[^'\\\/\:\*\?"<>\|-]*$/; // In ServiceNow

The JavaScript code in ServiceNow platform is as follows:

function onChange(control, oldValue, newValue, isLoading, isTemplate) {
   if (isLoading || newValue === '') {
      return;
   }

   var regex = /^[^'\\\/\:\*\?"<>\|-]*$/;
   var ans = g_form.getValue('u_native_renaming_suffix');
    if(!regex.test(ans)){
        alert('Please enter valid string');
        return;
    }
}

It is working perfectly for all the included characters. However, when elongated hyphen (–) or a curvy single or double quote (“, ”, ‘, ’) (which we find in apps like MS Word) are used, instead of the normal hyphen or quotes, this validation doesn’t work. That is, it is raising an error when normal hyphen or quotes are used, but not when elongated hyphen or curvy quotes are used.

I even tried copy-pasting the new characters into the regular expression, but they are just being replaced or treated as the normal ones, i.e.:

/^[^'’\\\/\:*\?"”<>\|-–]*$/

turns into

/^[^''\\\/\:*\?""<>\|-–]*$/

when I hit save.

How do I incorporate the validation for those 3 characters as well?

Please show the code of your proposed solution where you tried to copy-paste the characters. — Sebastian Simon, Jul 05 '16 at 19:45
Yes, please provide the code as without the context of where the regex is being used, it's hard to give a concrete answer — Taegost, Jul 05 '16 at 19:46
I bet, this is going to be a duplicate of [Including a hyphen in a regex character bracket?](http://stackoverflow.com/q/3697202/4642212). — Sebastian Simon, Jul 05 '16 at 19:48
This is a javascript code used in ServiceNow platform. function onChange(control, oldValue, newValue, isLoading, isTemplate) { if (isLoading || newValue === '') { return; } var regex = /^[^'\\\/\:\*\?"<>\|-]*$/; var ans = g_form.getValue('u_native_renaming_suffix'); if(!regex.test(ans)){ alert('Please enter valid string'); return; } } — Vee Na, Jul 05 '16 at 19:49
btw you don't need all those escapes: `var regex = /^[^'\/:*?"<>|-]*$/;` — Bohemian, Jul 05 '16 at 19:49
@VeeNa I don’t see the three characters included in that regex. You said, you tried to include the three characters in the regex. Show that code, please. — Sebastian Simon, Jul 05 '16 at 19:51
When I include the three characters, they are changing to normal ones. It's like a redundancy. Including same characters twice. — Vee Na, Jul 05 '16 at 19:53
@Xufox /^[^'’\\\/\:\*\?"”<>\|-–]*$/ That's the expression I tried to include. But when I hit save, it's turning into: /^[^''\\\/\:\*\?""<>\|-–]*$/ — Vee Na, Jul 05 '16 at 19:55
Okay, the first thing is: don’t put anything after the `-`. That means `–` needs to go before `-` or you need to escape `-` as `\-`. Secondly, the `“”` being converted to `""` is weird. What editor are you using? This shouldn’t happen. A workaround could be to use `\u201C\u201D` instead of `“”`. — Sebastian Simon, Jul 05 '16 at 20:02
Hey Xufox! Thank you so much. Using the unicode characters worked perfectly! :) — Vee Na, Jul 05 '16 at 21:00
My bet was on ascii tables (using range) and unicode characters (for special instances) as well... @Xufox you nailed it and make sure you add this as an answer. — Srini V, Jul 05 '16 at 21:31
@VeeNa When answering a comment make sure to type `@`, like this: `@xufox`. Otherwise I won’t get notified. — Sebastian Simon, Jul 05 '16 at 21:37

score 0 · Accepted Answer · edited May 23 '17 at 12:08

0

There are two parts to your regex not working:

First, |-– in a charcter class means a range from | to –. This covers 8087 different Unicode code points — clearly not what you want. As in Including a hyphen in a regex character bracket? and a few other related questions this is solved by moving - to the end or escaping it as \-.

Second, ” and ’ being converted to " and ' shouldn’t happen in any reasonable text editor. Use a better text editor, get better Unicode support or replace ” by \u201D and ’ by \u2019 — their respective Unicode hex codes.

Your final regex would look like this, then:

/^[^'\u2019\\\/\:*\?"\u201D<>\|–-]*$/

If you need them, the Unicode hex code for “ is \u201C, for ‘ it’s \u2018 and for – it’s \u2013.

edited May 23 '17 at 12:08

Community

1
1

answered Jul 05 '16 at 21:48

Sebastian Simon

18,263
7
55
75

Hi xufox! I ran into a similar problem again. That unicode characters thing was working perfectly but I need to change something. Right now I'm excluding all single quotes/apostrophes(both straight and curly) I excluded straight quote from the regex but it's still validating against it. When i tested the same with double quotes, it accepted straight quotes and validated against curly ones. Its not working similarly with single quotes. Only "curly" apostrophes should be considered invalid. So I want to be able to accept straight apostrophes. – Vee Na Jul 15 '16 at 19:41
This is the code I have and not working. (It doesn't have straight apostrophe but it still isnt accepting it) var regex = /^[^\\\/\:\*\?"\u02BC\u055A\u201C\u201D\u201E\u201F\u2012\u2013\u2014\u2015\u007c<>\|-]*$/; var ans = g_form.getValue('u_native_renaming_suffix'); if(!regex.test(ans)){ alert('Please enter valid string for "Native Renaming Suffix"'); //g_form.clearValue('u_native_renaming_suffix'); return; } – Vee Na Jul 15 '16 at 19:41

Difference between – and - or ' and ‘ or “ and "

1 Answers1