1

According to CJK Unicode Ideographs, the Chinese, Japanese and Korean Unicode range is:

  • CJK Unified Ideographs (4E00-9FCC) [\u4E00-\u9FCC]
  • CJK Unified Ideographs Extension A (3400-4DB5) [\u3400-\u4DB5]
  • CJK Unified Ideographs Extension B (20000-2A6D6) [\ud840-\ud868][\udc00-\udfff]|\ud869[\udc00-\uded6]
  • CJK Unified Ideographs Extension C (2A700-2B734) \ud869[\udf00-\udfff]|[\ud86a-\ud86c][\udc00-\udfff]|\ud86d[\udc00-\udf34]
  • CJK Unified Ideographs Extension D (2B840-2B81D) \ud86d[\udf40-\udfff]|\ud86e[\udc00-\udc1d]
  • 12 characters within the CJK Compatibility Ideographs (F900-FA6D/FA70-FAD9) but which are actually CJK unified ideographs [\uFA0E\uFA0F\uFA11\uFA13\uFA14\uFA1F\uFA21\uFA23\uFA24\uFA27-\uFA29]

However, I checked the Chinese punctuation characters ？ (\uFF01) and ！(\uFF1F) are not in the range. May I know what is the Unicode range of all Chinese characters including the punctuations?

Actually, I am working on the RegEx of Python Chinese characters, but I cannot get the contents within the Chinese parathesis:

import re
sample3 = "「在看星星嗎？閣下！」"

result = re.search(r"\u300c([\u4e00-\u9fff\u31c0-\u31ef\u31f0-\u31ff\u3200-\u32ff\u3300-\u33ff\u3400-\u4dbf\u4dc0-\u4dff\u4e00-\u9fff]+)\u300d", sample3)
print(result)

which has no result. I want to extract the sentence:

在看星星嗎？閣下！

FYI, the meaning of the sentence above is: "Sir, are you looking into stars?"

Thanks.

Raptor
  • 53,206
  • 45
  • 230
  • 366

1 Answers1

4

That's not an easy question to answer, because the concept of a "Chinese character" is not rigidly definable. The "fullwidth" punctuation symbols you mention are certainly used in Chinese typography, but they are available for use in any context which requires wider characters. The Unicode block containing the fullwidth punctuation symbols also contains fullwidth versions of the 91 "ASCII" printable characters, which can be used in Chinese typography for words written in the Latin alphabet. Does that make them Chinese characters? It will depend on your application. That Unicode block also contains "halfwidth" versions of some CJK punctuation symbols, Katakana letters, and some Hangul forms. And so on.

Unicode definitely has a concept of "script", and every Unicode symbol is assigned to exactly one primary script in the Unicode Character Database (UCD). You can find the script property table here (for Unicode version 13.0.0). In that listing, the following characters are included in the "Han" script:

2E80..2E99    ; Han # So  [26] CJK RADICAL REPEAT..CJK RADICAL RAP
2E9B..2EF3    ; Han # So  [89] CJK RADICAL CHOKE..CJK RADICAL C-SIMPLIFIED TURTLE
2F00..2FD5    ; Han # So [214] KANGXI RADICAL ONE..KANGXI RADICAL FLUTE
3005          ; Han # Lm       IDEOGRAPHIC ITERATION MARK
3007          ; Han # Nl       IDEOGRAPHIC NUMBER ZERO
3021..3029    ; Han # Nl   [9] HANGZHOU NUMERAL ONE..HANGZHOU NUMERAL NINE
3038..303A    ; Han # Nl   [3] HANGZHOU NUMERAL TEN..HANGZHOU NUMERAL THIRTY
303B          ; Han # Lm       VERTICAL IDEOGRAPHIC ITERATION MARK
3400..4DBF    ; Han # Lo [6592] CJK UNIFIED IDEOGRAPH-3400..CJK UNIFIED IDEOGRAPH-4DBF
4E00..9FFC    ; Han # Lo [20989] CJK UNIFIED IDEOGRAPH-4E00..CJK UNIFIED IDEOGRAPH-9FFC
F900..FA6D    ; Han # Lo [366] CJK COMPATIBILITY IDEOGRAPH-F900..CJK COMPATIBILITY IDEOGRAPH-FA6D
FA70..FAD9    ; Han # Lo [106] CJK COMPATIBILITY IDEOGRAPH-FA70..CJK COMPATIBILITY IDEOGRAPH-FAD9
16FF0..16FF1  ; Han # Mc   [2] VIETNAMESE ALTERNATE READING MARK CA..VIETNAMESE ALTERNATE READING MARK NHAY
20000..2A6DD  ; Han # Lo [42718] CJK UNIFIED IDEOGRAPH-20000..CJK UNIFIED IDEOGRAPH-2A6DD
2A700..2B734  ; Han # Lo [4149] CJK UNIFIED IDEOGRAPH-2A700..CJK UNIFIED IDEOGRAPH-2B734
2B740..2B81D  ; Han # Lo [222] CJK UNIFIED IDEOGRAPH-2B740..CJK UNIFIED IDEOGRAPH-2B81D
2B820..2CEA1  ; Han # Lo [5762] CJK UNIFIED IDEOGRAPH-2B820..CJK UNIFIED IDEOGRAPH-2CEA1
2CEB0..2EBE0  ; Han # Lo [7473] CJK UNIFIED IDEOGRAPH-2CEB0..CJK UNIFIED IDEOGRAPH-2EBE0
2F800..2FA1D  ; Han # Lo [542] CJK COMPATIBILITY IDEOGRAPH-2F800..CJK COMPATIBILITY IDEOGRAPH-2FA1D
30000..3134A  ; Han # Lo [4939] CJK UNIFIED IDEOGRAPH-30000..CJK UNIFIED IDEOGRAPH-3134A

But many characters are classified as "Common", meaning they are not associated with any particular script. That includes the fullwidth and halfwidth variants mentioned above. Also, Unicode includes characters whose only purpose is to combine with preceding characters or to represent some kind of typographical effect; such characters have the script designation "Inherited", meaning that they inherit the script of some nearby character. (For combining characters, the script designation is that of the preceding base character, but there are other "Inherited characters", such as the zero-width joiner and non-joiner characters (U+200C and U+200D), for which there are other rules.)

The fullwidth variants have codes U+FF01..U+FF60 and U+FFE0..U+FFE6; the rest of the U+FF00 block contains halfwidth variants and a few unassigned code points. See the code chart.

By searching the Scripts UCD table, I also found the following characters whose names include the abbreviation "CJK":

31C0..31E3    ; Common # So  [36] CJK STROKE T..CJK STROKE Q
1F210..1F23B  ; Common # So  [44] SQUARED CJK UNIFIED IDEOGRAPH-624B..SQUARED CJK UNIFIED IDEOGRAPH-914D
1F240..1F248  ; Common # So   [9] TORTOISE SHELL BRACKETED CJK UNIFIED IDEOGRAPH-672C..TORTOISE SHELL BRACKETED CJK UNIFIED IDEOGRAPH-6557

But I don't think that is yet an exhaustive list; there are also characters which represent Mah Jong tiles, for example, which might be considered "Chinese". Usually, all characters designated as "Common" will be considered valid in any script, but you're free to be more or less discriminating, as you wish.

For the complete discussion of the Script and Script_Extension properties, see the Unicode Standard Annex 24, the Unicode Script Property.

rici
  • 234,347
  • 28
  • 237
  • 341