How do i split a string using delimiters in java?

Question

What regex pattern would i need to pass to String.split() method to split a string into an array of sub strings using the white space as well as the following characters as delimiters. (" ! ", " , " , " ? " , " . " , " \ " , " _ " , " @ " , " ' " ) and it can also be the combination of the above characters with whitespace. I've tried something like this:

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.util.*;
class StringWordCount {
    public static void main(String[] args) throws IOException {

      BufferedReader bufferedReader = new BufferedReader(new IputStreamReader(System.in));
      String string = bufferedReader.readLine();
      String delimiter = "[,\\s]+|\\[!\\s]+|\\[?\\s]+|\\[.\\s]+|\\[_\\s]+|\\[_\\s]+|\\['\\s]+|\\[@\\s]+|\\!|\\,|\\?|\\.|\\_|\\'|\\@";
      String[] words = string.split(delimiter);
      System.out.println(words.length);
      for(int i = 0; i<words.length; i++) {
         System.out.println(words[i]);
      }
}

}

The above code only generates correct output for some testcases, in other cases, it won't generate the correct one.For example, Consider the below string where it failed to get the expected output.

It generates the output:

23
Hello
thanks
for
attempting
this
problem

Hope
it
will
help
you
to
learn
java

Good
luck
and
have
a
nice
day

Instead of this one:

21
Hello
thanks
for
attempting
this
problem
Hope
it
will
help
you
to
learn
java
Good
luck
and
have
a
nice
day

As you can see in the first output, its leaving a space on the combination of " ! " and [space] and the delimiter for the above combination is \\[!\\s], right?

Answer 1

You can try this one:

String str = "Hello, thanks for attempting this problem! Hope it will help you to learn java! Good luck and have a nice day!";
//String[] split = str.split("[\\p{Punct}\\s+]");
String[] split = str.split("[\\p{Punct}\\p{Blank}]+");
System.out.println("Arrays.toString(split) = " + Arrays.toString(split));

Result is:

Arrays.toString(split) = [Hello, thanks, for, attempting, this, problem, Hope, it, will, help, you, to, learn, java, Good, luck, and, have, a, nice, day]

Eited: edited line below

String[] split = str.split("[\\p{Punct}\\p{Blank}]+");

Answer 2

In this line:

String delimiter = "[,\\s]+|\\[!\\s]+|\\[?\\s]+|\\[.\\s]+|\\[_\\s]+|\\[_\\s]+|\\['\\s]+|\\[@\\s]+|\\!|\\,|\\?|\\.|\\_|\\'|\\@";

you have \\[ in the string literal, which means the pattern has two characters \[ in it. In the pattern matcher, this causes the matcher to look for the [ character. This isn't what you want.

When a \ character appears in a pattern string:

1. If the following character is a letter or digit, the combination has some special meaning (for example, you're using \s in the string to mean whitespace), but:
2. If the following character is something other than a letter or a digit, this means to treat the following character as itself. Any special meaning the character may have had is canceled.

It looks like you're trying to use [!\s]+ (in the pattern; of course you had to double the backslash in the string literal) to match one or more characters in the set of ! and whitespace. Here, [ and ] have a special meaning, to match any character in a set. But putting \ before the [ cancels the special meaning of [, and causes the matcher to look for a [ in the input, which it doesn't find.

See this javadoc for more information.

I'm not sure, but I think getting rid of all the \\ before each [ will make things work. The pattern will still be more complicated than necessary (and I'm not 100% clear on what the requirements are, so it's hard for me to suggest an improvement).

Answer 3

Just do matching instead of splitting..

ArrayList<String> lst = new ArrayList<String>();
Matcher m = Pattern.compile("\\w+").matcher(s);
while(m.find()) {
    lst.add(m.group());
  }