Remove non-ASCII characters from String in Java

Question

I have a URI that contains non-ASCII characters like :

http://www.abc.de/qq/qq.ww?MIval=typo3_bsl_int_Smtliste&p_smtbez=Schmalbl�ttrigeSomerzischeruchtanb

How can I remove "�" from this URI

The set of possible characters is large compared to the set of characters allowed in the [query part of a URI](http://illegalargumentexception.blogspot.co.uk/2009/12/java-safe-character-handling-and-url.html#URI2009_HTML). To delete all non-English text would exclude many languages. Is this what you want? Or do you want to percent-encode the text? Or do you want to [transliterate](http://en.wikipedia.org/wiki/Transliteration) the text to English? — McDowell, May 13 '12 at 18:57

score 41 · Accepted Answer · edited Mar 11 '21 at 00:34

41

I'm guessing that the source of the URL is more at fault. Perhaps you're fixing the wrong problem? Removing "strange" characters from a URI might give it an entirely different meaning.

With that said, you may be able to remove all of the non-ASCII characters with a simple string replacement:

String fixed = original.replaceAll("[^\\x20-\\x7e]", "");

Or you can extend that to all non-four-byte-UTF-8 characters if that doesn't cover the "�" character:

String fixed = original.replaceAll("[^\\u0000-\\uFFFF]", "");

edited Mar 11 '21 at 00:34

Vijin Paulraj

4,469
5
39
54

answered May 13 '12 at 18:49

Cᴏʀʏ

105,112
20
162
194

is `\u0000 - \uFFFF` 4 bytes? or 2 bytes. – JSixface Mar 27 '19 at 20:07
@JSixface: a Unicode character in Java is 16 bits, or 2 bytes. But, in the context of the ranges specified within this regex string, that question doesn't make sense to me. – Cᴏʀʏ Mar 28 '19 at 00:59
[^\x00-\x7F] is the correct range for ascii chars, if you put x20 as starting range you will remove also new line chars. – Massimo Fazzolari Aug 20 '19 at 09:46

score 21 · Answer 2 · answered Dec 08 '13 at 12:34

21

yourstring=yourstring.replaceAll("[^\\p{ASCII}]", "");

answered Dec 08 '13 at 12:34

daneshkohan

336
2
5

score 7 · Answer 3 · edited Dec 29 '15 at 12:48

7

No no no no no, this is not ASCII ... [^\x20-\x7E]

This is real ascii: [^\x00-\x7F]

Otherwise it will trim out newlines and other special characters that are part of ascii table!

edited Dec 29 '15 at 12:48

Riduidel

22,052
14
85
185

answered Nov 19 '12 at 08:39

Peter L

101
1
1

Thank you! I was wondering where my newlines went!! – Louie Bafford May 28 '21 at 15:37

score 6 · Answer 4 · edited Nov 15 '18 at 18:41

6

To remove the Non- ASCII characters from String, below code worked for me.

String str="<UPC>616043287409ÂÂÂÂ</UPC>";

str = str.replaceAll("[^\\p{ASCII}]", "");

Output:

<UPC>616043287409</UPC>

edited Nov 15 '18 at 18:41

Pritam Banerjee

17,953
10
93
108

answered Aug 08 '16 at 14:09

Yellesh Chaparthi

413
1
7
8

1

Please try to avoid just dumping code as an answer and try to explain what it does and why. Your code might not be obvious for people who do not have the relevant coding experience. – Frits Aug 08 '16 at 14:36

score 4 · Answer 5 · answered Nov 02 '17 at 17:32

4

Use Guava CharMatcher

String onlyAscii = CharMatcher.ascii().retainFrom(original)

answered Nov 02 '17 at 17:32

Juan Rada

3,513
1
26
26

Remove non-ASCII characters from String in Java

5 Answers5

Linked