I want to split strings in a tensor into parts separated by punctuation or whitespace (sometimes there are multiple consecutive whitespace characters). I've tried t = tf.strings.split(t, sep = "\W-_.,") but it doesn't work the way I want. How to do it properly?
Asked
Active
Viewed 29 times
0
Brzoskwinia
- 371
- 2
- 11
-
1You did not use the square brackets, `sep = "[\W-_.,]"`, but in fact, it is just the same as `sep = r"[\W_]"` (`\W` does not match only `_`) – Wiktor Stribiżew Aug 13 '21 at 10:20
-
Define "punctuation". – Wais Kamal Aug 13 '21 at 10:21