In your example strings you have either a part between single quotes or a part starting with word characters followed by parenthesis.
As your tried patterns start with abc, you might use:
(?:abc\s*\(\s*|\G(?!^),)('[^',]*'|\w+\([^()]*\))
Explanation
(?:
Non capture grouop
abc\s*\(\s*
Match abc
a space followed by (
between optional whitepace chars
|
Or
\G(?!^),
Assert the position at the end of the previous match, but not at the start of the string to get repetitive matches. Following by matching a comma.
)
Close the non capture group
(
Capture group 1
'[^',]*'
Match from '...'
without matching '
or a comma in between
|
Or
\w+\([^()]*\)
Match 1+ word chars followed by (...)
)
Close group 1
Regex demo | Java demo
Example code getting the group 1 value:
String regex = "(?:abc\\s*\\(\\s*|\\G(?!^),)('[^',]*'|\\w+\\([^()]*\\))";
String string = "and abc ( xyz(d.e),'f','g','h','i',abc('p/q'),'r') = u";
Pattern pattern = Pattern.compile(regex);
Matcher matcher = pattern.matcher(string);
while (matcher.find()) {
if (null != matcher.group(1)) {
System.out.println(matcher.group(1));
}
}
Output
xyz(d.e)
'f'
'g'
'h'
'i'
abc('p/q')
'r'