Javascript regular expression for punctuation (international)?

Question

I need a regular expression to match against all punctuation marks, such as the standard [,!@#$%^&*()], but including international marks like the upside-down Spanish question mark, Chinese periods, etc. My google-fu is coming up short. Does anyone have such a regular expression on hand that's compatible with Javascript?

Considering how bad Javascript's regex and unicode support are, I don't think there's anything other than listing all characters you want to include. — NullUserException, Sep 28 '11 at 00:03
ever since two eyes appeared started appearing at the end of a name :-) — RHT, Sep 28 '11 at 00:06
[This might be useful](http://stackoverflow.com/questions/280712/javascript-unicode) — Felix Kling, Sep 28 '11 at 00:08
You can try XRegExp plugin with unicode package http://xregexp.com/plugins/#unicode — Narendra Yadala, Sep 28 '11 at 07:18
@Stema's answer is excellent. But, if you want to avoid bloating up your project with another JS library see -http://stackoverflow.com/a/21396529/114140 — Chris Jacob, Jan 28 '14 at 03:59

score 9 · Answer 1 · edited May 23 '17 at 12:25

Adding to @stema's answer (https://stackoverflow.com/a/7578937/114140)... here is the regex as a string (so you don't need to bloat your project with XRegExp).

!-#%-\x2A,-/:;\x3F@\x5B-\x5D_\x7B}\u00A1\u00A7\u00AB\u00B6\u00B7\u00BB\u00BF\u037E\u0387\u055A-\u055F\u0589\u058A\u05BE\u05C0\u05C3\u05C6\u05F3\u05F4\u0609\u060A\u060C\u060D\u061B\u061E\u061F\u066A-\u066D\u06D4\u0700-\u070D\u07F7-\u07F9\u0830-\u083E\u085E\u0964\u0965\u0970\u0AF0\u0DF4\u0E4F\u0E5A\u0E5B\u0F04-\u0F12\u0F14\u0F3A-\u0F3D\u0F85\u0FD0-\u0FD4\u0FD9\u0FDA\u104A-\u104F\u10FB\u1360-\u1368\u1400\u166D\u166E\u169B\u169C\u16EB-\u16ED\u1735\u1736\u17D4-\u17D6\u17D8-\u17DA\u1800-\u180A\u1944\u1945\u1A1E\u1A1F\u1AA0-\u1AA6\u1AA8-\u1AAD\u1B5A-\u1B60\u1BFC-\u1BFF\u1C3B-\u1C3F\u1C7E\u1C7F\u1CC0-\u1CC7\u1CD3\u2010-\u2027\u2030-\u2043\u2045-\u2051\u2053-\u205E\u207D\u207E\u208D\u208E\u2329\u232A\u2768-\u2775\u27C5\u27C6\u27E6-\u27EF\u2983-\u2998\u29D8-\u29DB\u29FC\u29FD\u2CF9-\u2CFC\u2CFE\u2CFF\u2D70\u2E00-\u2E2E\u2E30-\u2E3B\u3001-\u3003\u3008-\u3011\u3014-\u301F\u3030\u303D\u30A0\u30FB\uA4FE\uA4FF\uA60D-\uA60F\uA673\uA67E\uA6F2-\uA6F7\uA874-\uA877\uA8CE\uA8CF\uA8F8-\uA8FA\uA92E\uA92F\uA95F\uA9C1-\uA9CD\uA9DE\uA9DF\uAA5C-\uAA5F\uAADE\uAADF\uAAF0\uAAF1\uABEB\uFD3E\uFD3F\uFE10-\uFE19\uFE30-\uFE52\uFE54-\uFE61\uFE63\uFE68\uFE6A\uFE6B\uFF01-\uFF03\uFF05-\uFF0A\uFF0C-\uFF0F\uFF1A\uFF1B\uFF1F\uFF20\uFF3B-\uFF3D\uFF3F\uFF5B\uFF5D\uFF5F-\uFF65

I used this in my own project with some additions...

    // any kind of punctuation character (including international e.g. Chinese and Spanish punctuation)
    // author: http://www.regular-expressions.info/unicode.html
    // source: https://github.com/slevithan/xregexp/blob/41f4cd3fc0a8540c3c71969a0f81d1f00e9056a9/src/addons/unicode/unicode-categories.js#L142
    // note: XRegExp unicode output taken from http://jsbin.com/uFiNeDOn/3/edit?js,console (see chrome console.log), then converted back to JS escaped unicode here http://rishida.net/tools/conversion/, then tested on http://regexpal.com/
    // suggested by: https://stackoverflow.com/a/7578937
    // added: extra characters like "$", "\uFFE5" [yen symbol], "^", "+", "=" which are not consider punctuation in the XRegExp regex (they are currency or mathmatical characters)
    // added: \u3000-\u303F Chinese Punctuation for good measure
    var regex_characters_to_remove = /[\$\uFFE5\^\+=`~<>{}\[\]|\u3000-\u303F!-#%-\x2A,-/:;\x3F@\x5B-\x5D_\x7B}\u00A1\u00A7\u00AB\u00B6\u00B7\u00BB\u00BF\u037E\u0387\u055A-\u055F\u0589\u058A\u05BE\u05C0\u05C3\u05C6\u05F3\u05F4\u0609\u060A\u060C\u060D\u061B\u061E\u061F\u066A-\u066D\u06D4\u0700-\u070D\u07F7-\u07F9\u0830-\u083E\u085E\u0964\u0965\u0970\u0AF0\u0DF4\u0E4F\u0E5A\u0E5B\u0F04-\u0F12\u0F14\u0F3A-\u0F3D\u0F85\u0FD0-\u0FD4\u0FD9\u0FDA\u104A-\u104F\u10FB\u1360-\u1368\u1400\u166D\u166E\u169B\u169C\u16EB-\u16ED\u1735\u1736\u17D4-\u17D6\u17D8-\u17DA\u1800-\u180A\u1944\u1945\u1A1E\u1A1F\u1AA0-\u1AA6\u1AA8-\u1AAD\u1B5A-\u1B60\u1BFC-\u1BFF\u1C3B-\u1C3F\u1C7E\u1C7F\u1CC0-\u1CC7\u1CD3\u2010-\u2027\u2030-\u2043\u2045-\u2051\u2053-\u205E\u207D\u207E\u208D\u208E\u2329\u232A\u2768-\u2775\u27C5\u27C6\u27E6-\u27EF\u2983-\u2998\u29D8-\u29DB\u29FC\u29FD\u2CF9-\u2CFC\u2CFE\u2CFF\u2D70\u2E00-\u2E2E\u2E30-\u2E3B\u3001-\u3003\u3008-\u3011\u3014-\u301F\u3030\u303D\u30A0\u30FB\uA4FE\uA4FF\uA60D-\uA60F\uA673\uA67E\uA6F2-\uA6F7\uA874-\uA877\uA8CE\uA8CF\uA8F8-\uA8FA\uA92E\uA92F\uA95F\uA9C1-\uA9CD\uA9DE\uA9DF\uAA5C-\uAA5F\uAADE\uAADF\uAAF0\uAAF1\uABEB\uFD3E\uFD3F\uFE10-\uFE19\uFE30-\uFE52\uFE54-\uFE61\uFE63\uFE68\uFE6A\uFE6B\uFF01-\uFF03\uFF05-\uFF0A\uFF0C-\uFF0F\uFF1A\uFF1B\uFF1F\uFF20\uFF3B-\uFF3D\uFF3F\uFF5B\uFF5D\uFF5F-\uFF65]+/g

score 8 · Accepted Answer · edited Jun 07 '12 at 09:35

If it's possible for you to use a plugin, there is a plugin for JavaScript: XRegExp Unicode plugins. That adds support for Unicode categories, scripts, and blocks (I personally have only read about it, I never used it).

With this plugin it should be possible to use Unicode categories like \p{P} as explained at regular-expressions.info.

Update: OK, I tested it, and it seems to work fine.

You need to get the lib from XRegExp and additionally the Unicode Base and Unicode Category plugins (linked above).

<script src="xregexp.js"></script>
<script src="addons/unicode-base.js"></script>
<script src="addons/unicode-categories.js"></script>
<script>
    var unicodePunctuation = XRegExp("^\\p{P}+$");

    alert(unicodePunctuation.test("?.,;!¡¿。、·")); // true
</script>

The above alerts true. I included some Spanish and Chinese punctuation in my test string, "?.,;!¡¿。、·".

This looks pretty promising, though I'm always hesitant to bloat up my Javascript app with yet another library. I'll check back in once I confirm it works for my purpose. — Ben Dilts, Sep 28 '11 at 14:55
Thank you. Here is a JSBin of the code (scroll to the bottom and also see Chrome console) http://jsbin.com/uFiNeDOn/3/edit?js,console — Chris Jacob, Jan 28 '14 at 03:48

score 4 · Answer 3 · answered Sep 18 '20 at 08:29

From ES 2018, Unicode property escapes are supported. You can use \p{Punctuation} or just \p{P} (the same as the XRegExp answer) to match any punctuation character (by the Unicode definition), or \P{Punctuation} to match any non-punctuation character.

If you want to match any "non-word" character, like a Unicode version of \W, you can try something like:

[^\p{Alphabetic}\p{Mark}\p{Decimal_Number}\p{Connector_Punctuation}\p{Join_Control}]

(as recommended in the proposal for the feature). You might want to remove \p{Connector_Punctuation}, since that includes underscores and similar.

Don't forget to add the u flag to your regular expression to make it Unicode-aware and enable this feature.

score 3 · Answer 4 · answered Sep 28 '11 at 00:03

3

Well... idk how extensive it would be, but you could use this:

[^\w\s\n\t]

answered Sep 28 '11 at 00:03

Joseph Marikle

76,418
17
112
129

1

The only thing about this is that the `_` character is included in `\w`. Also, would unicode letters (like `ö`) be allowed through, as false positives? – sdleihssirhc Sep 28 '11 at 00:06
2

`\s` includes both `\n` and `\t`, and this would also match non-ASCII word characters, control characters, currency characters, and assorted other non-punctuation characters. – Peter Boughton Sep 28 '11 at 00:11
2

Other unicode characters getting included as false positives is my current major problem--I guess an equivalent problem for me is finding a good regex that's the international equivalent of \w (and therefore \W) – Ben Dilts Sep 28 '11 at 04:42

alex · Answer 5 · 2011-09-28T00:11:57.020

2

Your regex would look something like...

/[,!@#$%^&*()\u9999]/

Where you replace each \u9999 with the Unicode codepoint for the other punctuation characters.

If you could find a bunch in a range, you could specify that with the - range operand, e.g. \u9990-\u9999.

As far as I know you can't use something like \pP in JavaScript regexes.

edited Sep 28 '11 at 00:11

answered Sep 28 '11 at 00:04

alex

479,566
201
878
984

You wouldn't need to do individual characters - you can still do ranges such as `[\u9000-\u9999]` (that's a syntax example, no idea what characters it might represent). – Peter Boughton Sep 28 '11 at 00:11

score 0 · Answer 6 · answered Mar 24 '15 at 06:59

For Python this regex to remove from the start and end any type of punctuation marks:

import re
def cleanspecialcharacters(str):   
    regex = re.compile((
    '^[/\"_\(\)&*\$￥\^\+=`~<>\{\}\[\]\|\-!#%\,\:;@¡§«¶·»¿;·՚-՟։֊؉،॥॰෴๏๚๛༄-༒༔༺-༽྅჻፠-፨᐀᙭᙮។-៖៘-៚‧‰-⁃⁅-⁑⁓-⁞⁽⁾₍₎、〃〈-【】〔-〟〰〽゠・﴾﴿︐-︙︰-﹒﹔-﹡﹣﹨﹪﹫！-＃％-＊，-／：；？＠［-］＿｛｝｟-･〔〕《》]*|'
    '([/\"_\(\)&*\$￥\^\+=`~<>\{\}\[\]\|\-!#%\,\:;@¡§«¶·»¿;·՚-՟։֊؉،॥॰෴๏๚๛༄-༒༔༺-༽྅჻፠-፨᐀᙭᙮។-៖៘-៚‧‰-⁃⁅-⁑⁓-⁞⁽⁾₍₎、〃〈-【】〔-〟〰〽゠・﴾﴿︐-︙︰-﹒﹔-﹡﹣﹨﹪﹫！-＃％-＊，-／：；？＠［-］＿｛｝｟-･〔〕《》])*$'))
    str = regex.sub('', str)
    return str

Javascript regular expression for punctuation (international)?

6 Answers6

Linked