You can use this regex:
String input = "5,(5,5),C'A,B','A,B',',B','A,',\"A,B\",C\"A,B\"";
String[] toks = input.split(
",(?=(([^']*'){2})*[^']*$)(?=(([^\"]*\"){2})*[^\"]*$)(?![^()]*\\))" );
for (String tok: toks)
System.out.printf("<%s>%n", tok);
Output:
<5>
<(5,5)>
<C'A,B'>
<'A,B'>
<',B'>
<'A,'>
<"A,B">
<C"A,B">
Explanation:
, # Match literal comma
(?=(([^']*'){2})*[^']*$) # Lookahead to ensure comma is followed by even number of '
(?=(([^"]*"){2})*[^"]*$) # Lookahead to ensure comma is followed by even number of "
(?![^()]*\\)) # Negative lookahead to ensure ) is not followed by matching
# all non [()] characters in between